Date   

Re: Driver health : Posting message in MessageList

EDK
 

+ Mike

On Tue, Mar 30, 2021 at 1:56 AM EDK <edk.dan@gmail.com> wrote:

Hi ,

For our products, we have UEFI Driver and HII Driver as two separate
images (this is a requirement for our product because we want to keep UEFI
Driver as thin as possible).

- Main role of UEFI Driver is providing boot services,
reporting/solving health issues, posting messages in message list, EXT SPT
etc.

FYI: UEFI Driver will install HII Config access protocol (*CAP*) and
driver health formset on the controller handle if the controller has health
issue; at this time, HII image won't be in picture (HII image come into
play ONLY when controller is healthy); if UEFI Driver detects controller is
healthy then it will not install HII Config access protocol (if it had
installed, it would uninstall before loading HII image from controller
flash to host)

Note: We have our own API to load the HII Image from controller flash to
host

- Role of HII is, installing HII config access protocol and Management
formset on the controller handle and allowing the user to view
controller/PD/VD/other hardware properties, perform various
controller/PD/VD level operations etc.



Requirement (need to see whether this is feasible or not): There are cases
where our controller is healthy but we would like to post certain
informational messages in messageList.

How do I achieve this? That is I want UEFI Driver to have the capability
to post the message in messageList and our HII Image also to be
loaded/present.



Can UEFI Driver and HII driver install HII Config access protocol on the
same handle? AFAIK, the answer is “no”.

I experimented just adding the driver health formset to the package via
the HiiAddPackages() on the controller handle (that is from UEFI Driver I
was not installing HII CAP but just installed the formset on controller
handle) and it is not working. In this case HII image is also loaded (this
means HII image would have installed HII CAP and its own formset on the
same controller handle).



Appreciate your input.



Thanks,

Daniel

Backup:

*Excerpt from UEFI Spec: *

*MessageList *A pointer to an array of warning or error messages asso
ciated with

the controller specified by *ControllerHandle *and *ChildHandle*.

This is an optional parameter that may be *NULL*. *M**essageList *is

allocated by this function with the EFI Boot Service

*AllocatePool()*, and it is the caller’s responsibility to free

*MessageList *with the EFI Boot Service *F**reePool()*. Each

message is specified by tuple of an *EFI_HII_HANDLE *and an

*EFI_STRING_ID*. The array of messages is terminated by tuple

containing a *EFI_HII_HANDLE *with a value of *NULL*. The

*EFI_HII_STRING_PROTOCOL.GetString() *function can be used

to retrieve the warning or error message as a Null-terminated string

in a specific language. Messages may be returned for any of the

*HealthStatus *values except

*EfiDriverHealthStatusReconnectRequired *and

*EfiDriverHealthStatusRebootRequired*.

*FormHiiHandle *A pointer to the HII handle containing the HII form
used when configuration is required. The HII handle is associated with the
controller specified by *ControllerHandle *and

*C**hildHandle*. If this is *N**ULL*, then no HII form is available. An
HII handle will only be returned with a *HealthStatus *value of

*E**fiDriverHealthStatusConfigurationRequired*.











If *MessageList *is not *NULL*, and *HealthStatu**s *is *EfiDriverHealthStatusReconnectRequired
*or *EfiDriverHealthStatusRebootRequired *then no messages are returned
and **MessageList *must be set to *NULL*.

If *MessageList *is not *NULL*, and there are no warning or error
messages associated with the controller specified by *ControllerHandle *and
*ChildHandle*, then **MessageList *must be set to *NULL*.

If *MessageList *is not *NULL*, and there are one or more warning or
error messages associated with the controller specified by *ControllerHandle
*and *ChildHandle*, then **MessageList *must point to a buffer allocated
with the EFI Boot Service *AllocatePool()*. The number of *EFI_DRIVER_HEALTH_HII_MESSAGE
*structures allocated in the buffer must be one more than the total
number of warning or error messages, and the *HiiHandle *field of the
last *EFI_DRIVER_HEALTH_HII_MESSAGE *structure must be set to *NULL *to
terminate the list of messages. It is the caller’s responsibility to free
the buffer returned in **MessageList *using the EFI Boot Service
*FreePool()*. Each message is specified by an *EFI_HII_HANDLE *and an
*EFI_STRING_ID*. The caller may use the *EFI_HII_STRING_PROTOCOL.GetString()
*function to convert each message into a Null-terminated string that can
may be displayed on a console device.

If *FormHiiHandle *is *NULL*, then no forms are returned from this
function.

If *FormHiiHandle *is not *NULL*, and *HealthStatus *is not
*EfiDriverHealthStatusConfigurationRequired*, then no forms are returned
and **FormHiiHandle *must be set to *NULL*.

If *FormHiiHandle *is not *NULL*, and *FormSetGuid *is not *NULL*, and *HealthStatus
*is *EfiDriverHealthStatusConfigurationRequired*, then *FormHiiHandle *is
assigned to the HII handle which contains the HII form required to perform
the configuration operation.





*//********************************************************

*// EFI_DRIVER_HEALTH_HII_MESSAGE*

*//********************************************************

*typedef struct { EFI_HII_HANDLE **H**iiHandle**; **EF**I_STRING_ID **S*
*tringId**; **UI**NT64 **MessageCode;*

*} EFI_DRIVER_HEALTH_HII_MESSAGE;*

*HiiHandle *The *EFI_HII_HANDLE *that was returned by *EFI_HII_DATABASE_PROTOCOL.NewPackageList()
*when the string pack containing StringId was registered with the HII Data
base.

*StringId *The identifier for a single string token in the string
pack associated with *HiiHandle.*

*MessageCode *64-bit numeric value of the warning/error specified by
this message.

A value of *0x0000000000000000 *is used to indicate that

*MessageCode *is not specified.

The values *0**x0000000000000001 *to *0x0fffffffffffffff *are reserved for
allocation by the UEFI Specification.

The values *0**x1000000000000000 *to *0x1fffffffffffffff *are reserved for
IHV-developed drivers.

The values *0**x8000000000000000 *to *0x8fffffffffffffff *is reserved for
platform/OEM drivers.

All other values are reserved and should not be used.


Driver health : Posting message in MessageList

EDK
 

Hi ,

For our products, we have UEFI Driver and HII Driver as two separate images
(this is a requirement for our product because we want to keep UEFI Driver
as thin as possible).

- Main role of UEFI Driver is providing boot services, reporting/solving
health issues, posting messages in message list, EXT SPT etc.

FYI: UEFI Driver will install HII Config access protocol (*CAP*) and driver
health formset on the controller handle if the controller has health issue;
at this time, HII image won't be in picture (HII image come into play ONLY
when controller is healthy); if UEFI Driver detects controller is healthy
then it will not install HII Config access protocol (if it had installed,
it would uninstall before loading HII image from controller flash to host)

Note: We have our own API to load the HII Image from controller flash to
host

- Role of HII is, installing HII config access protocol and Management
formset on the controller handle and allowing the user to view
controller/PD/VD/other hardware properties, perform various
controller/PD/VD level operations etc.



Requirement (need to see whether this is feasible or not): There are cases
where our controller is healthy but we would like to post certain
informational messages in messageList.

How do I achieve this? That is I want UEFI Driver to have the capability to
post the message in messageList and our HII Image also to be
loaded/present.



Can UEFI Driver and HII driver install HII Config access protocol on the
same handle? AFAIK, the answer is “no”.

I experimented just adding the driver health formset to the package via the
HiiAddPackages() on the controller handle (that is from UEFI Driver I was
not installing HII CAP but just installed the formset on controller handle)
and it is not working. In this case HII image is also loaded (this means
HII image would have installed HII CAP and its own formset on the same
controller handle).



Appreciate your input.



Thanks,

Daniel

Backup:

*Excerpt from UEFI Spec: *

*MessageList *A pointer to an array of warning or error messages associ
ated with

the controller specified by *ControllerHandle *and *ChildHandle*.

This is an optional parameter that may be *NULL*. *M**essageList *is

allocated by this function with the EFI Boot Service

*AllocatePool()*, and it is the caller’s responsibility to free

*MessageList *with the EFI Boot Service *F**reePool()*. Each

message is specified by tuple of an *EFI_HII_HANDLE *and an

*EFI_STRING_ID*. The array of messages is terminated by tuple

containing a *EFI_HII_HANDLE *with a value of *NULL*. The

*EFI_HII_STRING_PROTOCOL.GetString() *function can be used

to retrieve the warning or error message as a Null-terminated string

in a specific language. Messages may be returned for any of the

*HealthStatus *values except

*EfiDriverHealthStatusReconnectRequired *and

*EfiDriverHealthStatusRebootRequired*.

*FormHiiHandle *A pointer to the HII handle containing the HII form used
when configuration is required. The HII handle is associated with the
controller specified by *ControllerHandle *and

*C**hildHandle*. If this is *N**ULL*, then no HII form is available. An HII
handle will only be returned with a *HealthStatus *value of

*E**fiDriverHealthStatusConfigurationRequired*.











If *MessageList *is not *NULL*, and *HealthStatu**s *is
*EfiDriverHealthStatusReconnectRequired
*or *EfiDriverHealthStatusRebootRequired *then no messages are
returned and **MessageList
*must be set to *NULL*.

If *MessageList *is not *NULL*, and there are no warning or error messages
associated with the controller specified by *ControllerHandle *and
*ChildHandle*, then **MessageList *must be set to *NULL*.

If *MessageList *is not *NULL*, and there are one or more warning or error
messages associated with the controller specified by *ControllerHandle *and
*ChildHandle*, then **MessageList *must point to a buffer allocated with
the EFI Boot Service *AllocatePool()*. The number of
*EFI_DRIVER_HEALTH_HII_MESSAGE
*structures allocated in the buffer must be one more than the total number
of warning or error messages, and the *HiiHandle *field of the last
*EFI_DRIVER_HEALTH_HII_MESSAGE
*structure must be set to *NULL *to terminate the list of messages. It is
the caller’s responsibility to free the buffer returned in **MessageList *using
the EFI Boot Service *FreePool()*. Each message is specified by an
*EFI_HII_HANDLE
*and an *EFI_STRING_ID*. The caller may use the
*EFI_HII_STRING_PROTOCOL.GetString()
*function to convert each message into a Null-terminated string that can
may be displayed on a console device.

If *FormHiiHandle *is *NULL*, then no forms are returned from this function.

If *FormHiiHandle *is not *NULL*, and *HealthStatus *is not
*EfiDriverHealthStatusConfigurationRequired*, then no forms are returned
and **FormHiiHandle *must be set to *NULL*.

If *FormHiiHandle *is not *NULL*, and *FormSetGuid *is not *NULL*, and
*HealthStatus
*is *EfiDriverHealthStatusConfigurationRequired*, then *FormHiiHandle *is
assigned to the HII handle which contains the HII form required to perform
the configuration operation.





*//********************************************************

*// EFI_DRIVER_HEALTH_HII_MESSAGE*

*//********************************************************

*typedef struct { EFI_HII_HANDLE **H**iiHandle**; **EF**I_STRING_ID **S*
*tringId**; **UI**NT64 **MessageCode;*

*} EFI_DRIVER_HEALTH_HII_MESSAGE;*

*HiiHandle *The *EFI_HII_HANDLE *that was returned by
*EFI_HII_DATABASE_PROTOCOL.NewPackageList()
*when the string pack containing StringId was registered with the HII Databa
se.

*StringId *The identifier for a single string token in the string
pack associated with *HiiHandle.*

*MessageCode *64-bit numeric value of the warning/error specified by
this message.

A value of *0x0000000000000000 *is used to indicate that

*MessageCode *is not specified.

The values *0**x0000000000000001 *to *0x0fffffffffffffff *are reserved for
allocation by the UEFI Specification.

The values *0**x1000000000000000 *to *0x1fffffffffffffff *are reserved for
IHV-developed drivers.

The values *0**x8000000000000000 *to *0x8fffffffffffffff *is reserved for
platform/OEM drivers.

All other values are reserved and should not be used.


Re: Google Summer of Code 2021 interested student

Nate DeSimone
 

Hi Ayush,

I suspect the comment about Arch Linux was more to the effect that no one in the TianoCore project has really been using Arch and as you noticed getting the needed dependencies installed is somewhat painful. If you would like to use Arch we are all absolutely fine with it, but just keep in mind that you are straying from the most commonly travelled path and you might run into issues that the rest of us do not.

Thanks,
Nate

On 3/23/21, 12:04 AM, "discuss@edk2.groups.io on behalf of Ayush Dwivedi" <discuss@edk2.groups.io on behalf of 21cencturyayush@gmail.com> wrote:

I have one more question. Actually I am using Arch Linux 64 bit so finding dependencies took some time and in the documentation it says "Arch Linux is not officially supported or tested by the edk2 project at this time". Is it going to be a problem because of that?


Re: Google Summer of Code 2021 interested student

Nate DeSimone
 

Hi Ayush,

Great to hear you got it working! So, the next logical step would be to use a SF-600 (https://www.dediprog.com/product/SF600) to program that image you just compiled on to an Intel Tiger Lake RVP (Reference and Validation Platform) and try booting it. Unfortunately, very few Tiger Lake reference boards are in existence and they are very expensive, so we won't be able to go that far.

To address this situation, we have been adding support for other x86 boards that are easy to find and buy as we come across good ones. For example, we have support for the Up Xtreme board (https://up-shop.org/up-xtreme-series.html) in WhiskeyLakeOpenBoardPkg. There is a new version of that board coming out soon called the Up Xtreme i11 (https://up-shop.org/up-xtreme-i11-boards-series.html) that upgrades from Whiskey Lake to Tiger Lake. The problem is the starting price for that board is $400 USD. Add in the $285 for the DediProg and I don't think it would be reasonable to expect a student to spend that much money on hardware for a GSoC project. That is the nice thing about the Qemu OpenBoard project is since it is all emulation there is no expensive and special hardware needed.

So, what to do next? First of all, I recommend that you work on your GSoC application(s) of course ☺. I also recommend building OVMF, since that works in the Qemu emulator you can actually run the image you compile, read the debug messages as the firmware is booting and start to get a feel for how the boot process works. You mentioned that you were interested in learning more about how UEFI is different from BIOS. For a very detailed answer to that question, I recommend that you read "Beyond BIOS". Beyond BIOS also does a really good job of describing the boot process and the driver model and gives a good baseline knowledge for how UEFI firmware works. Hope that helps!

Thanks,
Nate

On 3/22/21, 10:04 PM, "discuss@edk2.groups.io on behalf of Ayush Dwivedi" <discuss@edk2.groups.io on behalf of 21cencturyayush@gmail.com> wrote:

Thank you Nate.
I installed edkrepo using the instructions you gave. I ran edkrepo clone min Intel-MinPlatform and it setup my workspace. Since I had already setup the EDK2 build environment and installed the dependencies(such as gcc-5) I jumped to Board Builds section and finally used python build_bios.py -p TigerlakeURvp. I chose the intel board TigerlakeURvp which was given in the list of MinPlatform supported boards. When the build was completed it yielded TIGERLAKEURVP.fd in the Build directory. I have attached one screenshot as well. What should I do next?


Use of DebugLib in StandaloneMmCoreEntryPoint for AArch64 platforms

Thomas Abraham
 

Hi All,

The StandaloneMmCoreEntryPoint library for AArch64 platforms uses DebugLib APIs in the code path starting from '_ModuleEntryPoint' to the point at which 'ProcessModuleEntryPointList' is called. In this code path, the DebugLib APIs are invoked even before the DebugLib's constructor has executed. The DebugLib's constructor executes only after the execution reaches 'StandaloneMmMain' and 'ProcessLibraryConstructorList' is executed.

Due to this, on AArch64 platforms, the calls to DebugLib APIs in the above mentioned code path is incorrect because the constructor of the DebugLib has not executed. As an example, on AArch64 platforms that use 'BaseDebugLibSerialPort' as the DebugLib implementation, the serial port is not initialized prior to the call to DebugLib APIs causing serial port writes to fail and stall the execution.

Some solutions for this issue but not preferred ones are -


* Remove the DebugLib calls in the above mentioned code path (no DebugLib calls to be used until its constructor is called).
* Ensure that a previous boot stage has configured required hardware (serial port in case of BaseDebugLibSerialPort).

Are there better solutions for this particular problem?

Thanks,
Thomas.


Re: GccBase.lds

Andrew Fish
 

On Mar 26, 2021, at 8:43 AM, Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com> wrote:

Thanks,
I didn’t notice that the path was stored in the debug directory entry.
All the detailed information was very helpful.
I was able to confirm that it is there.

We have script that would parse for the PE32 header in memory, find the PDB path and then load the symbols.
But it doesn’t work when the binary is compiled with GCC.
Since the information is there, we just need to fix it to deal with the deltas.

It only needs the base address and the path to the binary with full symbols.
I confirmed I could manually load it so the error must be in how it finds the path.
Whoot! Glad you got it working and thanks for sharing maybe that will inspire others to give this a try. I dumped out a lot of info as I thought other folks may find it useful too.

Thanks,

Andrew Fish

Thanks again

Jose Miguel Hernandez Miramontes
BIOS Engineer
jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>
Intel Corporation

From: Andrew Fish <afish@apple.com>
Sent: Thursday, March 25, 2021 3:40 PM
To: discuss@edk2.groups.io; Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com>
Cc: Ard Biesheuvel <ardb@kernel.org>; Laszlo Ersek <lersek@redhat.com>; Feng, Bob C <bob.c.feng@intel.com>; Saucedo Tejada, Genaro <genaro.saucedo.tejada@intel.com>; Ortiz Garcia, Jorge E <jorge.e.ortiz.garcia@intel.com>; Ard Biesheuvel (TianoCore) <ardb+tianocore@kernel.org>
Subject: Re: [edk2-discuss] GccBase.lds




On Mar 24, 2021, at 8:56 AM, Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>> wrote:

yes, but if the section is not removed debugger would be able to find the debug symbols. Like how it happens with PDB files.

But gdb has no idea that the PE/COFF image exists in some random memory location. So you are going to have to do an ‘add-symbol-file <filename> -o offset` [1] to even let gdb know about your image. If you are not planning on using add-symbol-file what are you planning to use? If the gdb was PE/COFF aware it could load symbols its self if you point it at the PE/COFF?


We could leave the symbols in the binaries, but it will consume more space. Space in the SPI is usually a constrain


In PE/COFF the debug directory entry points to the external symbol file and with works for Windows/Linux/macOS. Conceptually if gdb knows about PE/COFF you should be able to just point gdb start of the image and it could load symbols. But I don’t know of a gdb command that can do that.


I am just wondering if it is possible to fix this so that the section is there in the final efi.
If I were to try it, where would I start?


GenFw is converting ELF to PE/COFF and it is quite brittle and intertwined. I’d insert any new section at the end as you don’t want to break relative offsets between text and data etc. If GenFw sees the image is an ELF file it will automatically attempt to convert it to PE/COFF [1]. That calls ConvertElf() [3] and then that calls the Elf32 or Elf64 functions. As I mentioned if you look closely [4] there are a lot of intertwined assumptions, so if you try to add a new section best to do it to the end of the PE/COFF.

If you have a better strategy than add-symbol-file please share with the community. I ask as it kind of seems if gdb knows enough about PE/COFF to find your magic section, it would know enough to parse the standard PE/COFF Debug Directory Entry. Regardless just having to point at the PE/COFF load address would be somewhat simpler than also having to decode the Debug Directory Entry in a script.

I don’t know much about gdb and your magic section, but I’ve got clang working with mach-O via lldb so I can help answer generic edk2 or PE/COFF questions about how to write debugger scripts, so feel free to ask questions on the list.

[1] https://2018.osfc.io/uploads/talk/paper/15/Debugging_UEFI_Firmware_Linux_Workshop_OSFC_19.pdf
https://retrage.github.io/2019/12/05/debugging-ovmf-en.html

[2] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/GenFw.c#L2058

[3] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/ElfConvert.c#L167

[4] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/Elf64Convert.c#L762

Thanks,

Andrew Fish


Jose Miguel Hernandez Miramontes
BIOS Engineer
jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>
+1 (512) 362-1230
Intel Corporation

-----Original Message-----
From: Ard Biesheuvel <ardb@kernel.org <mailto:ardb@kernel.org>>
Sent: Wednesday, March 24, 2021 10:26 AM
To: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>>
Cc: discuss@edk2.groups.io <mailto:discuss@edk2.groups.io>; Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>>; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com>>; Saucedo Tejada, Genaro <genaro.saucedo.tejada@intel.com <mailto:genaro.saucedo.tejada@intel.com>>; Ortiz Garcia, Jorge E <jorge.e.ortiz.garcia@intel.com <mailto:jorge.e.ortiz.garcia@intel.com>>; Ard Biesheuvel (TianoCore) <ardb+tianocore@kernel.org <mailto:ardb+tianocore@kernel.org>>
Subject: Re: [edk2-discuss] GccBase.lds

On Wed, 24 Mar 2021 at 16:18, Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>> wrote:


(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:

Hi

Is anyone familiar with this file?
(Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.

git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).


The build-id is used by Linux distros when they ship debug symbols with production shared libraries. The build-id permits the loader to look up the correct file, and confirm that the versions match.

In EDK2, the ELF binary is only an intermediate artifact, and so I fail to see why we would need build IDs here. If the ELF binary was built with debug symbols, they will be in the binary itself, not in a side loaded symbol file.



Re: 答复: [edk2-discuss] lost stderr in console

wenyi,xie
 

Hi, bob,

The BZ is filed, here's the link
https://bugzilla.tianocore.org/show_bug.cgi?id=3278

Thanks
Wenyi

-----邮件原件-----
发件人: Feng, Bob C [mailto:bob.c.feng@intel.com]
发送时间: 2021年3月25日 22:42
收件人: discuss@edk2.groups.io; xiewenyi (A) <xiewenyi2@huawei.com>; gaoliming <gaoliming@byosoft.com.cn>
主题: RE: [edk2-discuss] lost stderr in console

Wenyi,

As the description in Commit 01b6090b, the change was to fix the incremental build issue in the MSVC toolchain. But now it‘s found that impacts the usage of the GCC toolchain.

Please file a BZ for this issue. And I'll fix it.

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of wenyi,xie via groups.io
Sent: Thursday, March 25, 2021 10:18 AM
To: gaoliming <gaoliming@byosoft.com.cn>; discuss@edk2.groups.io
Subject: 答复: [edk2-discuss] lost stderr in console

Hi, liming,

Yes, my command using 2>&1 to redirect strerr to stdout, and after updating to version 202011 the stderr are not shown in the screen but only written in xxx.log.
According to Laszlo's help and checking the commit log, I find the problem relate to this commit:
https://github.com/tianocore/edk2/commit/01b6090b75922bc72604c334bd3dc331490af3bb

Proc = MakeSubProc(Command, stdout=PIPE, stderr=STDOUT, env=os.environ, cwd=WorkingDir, bufsize=-1, shell=True)

As the stdout and stderr are combined togetherin since the commit, so in version 201903 my command can work, but in version 202011, after this commit merged, it can't work well.

-----邮件原件-----
发件人: gaoliming [mailto:gaoliming@byosoft.com.cn]
发送时间: 2021年3月25日 9:27
收件人: discuss@edk2.groups.io; xiewenyi (A) <xiewenyi2@huawei.com>
主题: 回复: [edk2-discuss] lost stderr in console

Wenyi:
Do you use build command to build your platform and find the message from stderr are not shown in the screen?

Thanks
Liming
-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie via
groups.io
发送时间: 2021年3月17日 11:23
收件人: discuss@edk2.groups.io
主题: [edk2-discuss] lost stderr in console

Hi,everyone,

I used to set makeflags like below,so that only message from stderr
will be shown in console while compiling, and at the same time message
from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >>
${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log

But after updating edk2 from 201903 to 202011, the message from stderr
is not shown in console any more. In xxx.log, the message from stderr
and stdout are still saved.
Do anything change in edk2 may affect on it ?




回复: [edk2-discuss] Interested in contributing

gaoliming
 

Monami:
CLANG/LLVM tool chain has been enabled in Edk2 project for Windows/Linux/Mac OS.
You can see more detail from https://github.com/tianocore/tianocore.github.io/wiki/CLANG9-Tools-Chain

But on Windows, CLANG tool chain still depends on Visual Studio (VS) tool. There are two dependency here.
One is that BaseTools C tools requires to be compiled by VS tool to windows exe, another VS nmake is required to build source file.

Can you evaluate whether CLANG/LLVM compiler generates Windows EXE without VS tool?

Thanks
Liming

-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表
monamidg@gmail.com
发送时间: 2021年3月25日 5:58
收件人: discuss@edk2.groups.io
主题: [edk2-discuss] Interested in contributing

Hi there,

I'm Monami - a Master's in Computer Engineering student at Virginia Tech, US.
This is my first semester here as an international student, so I don't have CPT
authorization to get paid for working this summer.
But I'm really interested in contributing here - especially the project "Enable
Clang/LLVM Build for Microsoft Windows", or some other similar project.

This semester I've taken the courses Compiler Optimizations and OS &
Virtualization where I've implemented an iterative dataflow framework to
analyze passes. I've also worked on UEFI by building a bootloader and setting
up virtual address spaces for user applications amongst other things. I am not
able to share those projects yet as public on my GitHub page because I'm still
only halfway through the semester and not done with the courses, but I can
share it separately to an email-id if required as proof of work. By the end of
the semester, I'll have implemented my course projects as well, and I feel the
project listed here is the perfect fit for me to expand on my skills and help the
community at the same time. I have experience with C as well as Python and
I'm a quick study and work well under minimal supervision.

I was wondering if I could still contribute here, even if I am not potentially
eligible yet to enroll as a Google Summer of Code student because of CPT
authorization issues.
Please let me know how I can help!

Regards,
Monami Dutta Gupta




Re: GccBase.lds

Andrew Fish
 

On Mar 24, 2021, at 8:56 AM, Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com> wrote:

yes, but if the section is not removed debugger would be able to find the debug symbols. Like how it happens with PDB files.
But gdb has no idea that the PE/COFF image exists in some random memory location. So you are going to have to do an ‘add-symbol-file <filename> -o offset` [1] to even let gdb know about your image. If you are not planning on using add-symbol-file what are you planning to use? If the gdb was PE/COFF aware it could load symbols its self if you point it at the PE/COFF?

We could leave the symbols in the binaries, but it will consume more space. Space in the SPI is usually a constrain
In PE/COFF the debug directory entry points to the external symbol file and with works for Windows/Linux/macOS. Conceptually if gdb knows about PE/COFF you should be able to just point gdb start of the image and it could load symbols. But I don’t know of a gdb command that can do that.

I am just wondering if it is possible to fix this so that the section is there in the final efi.
If I were to try it, where would I start?
GenFw is converting ELF to PE/COFF and it is quite brittle and intertwined. I’d insert any new section at the end as you don’t want to break relative offsets between text and data etc. If GenFw sees the image is an ELF file it will automatically attempt to convert it to PE/COFF [1]. That calls ConvertElf() [3] and then that calls the Elf32 or Elf64 functions. As I mentioned if you look closely [4] there are a lot of intertwined assumptions, so if you try to add a new section best to do it to the end of the PE/COFF.

If you have a better strategy than add-symbol-file please share with the community. I ask as it kind of seems if gdb knows enough about PE/COFF to find your magic section, it would know enough to parse the standard PE/COFF Debug Directory Entry. Regardless just having to point at the PE/COFF load address would be somewhat simpler than also having to decode the Debug Directory Entry in a script.

I don’t know much about gdb and your magic section, but I’ve got clang working with mach-O via lldb so I can help answer generic edk2 or PE/COFF questions about how to write debugger scripts, so feel free to ask questions on the list.

[1] https://2018.osfc.io/uploads/talk/paper/15/Debugging_UEFI_Firmware_Linux_Workshop_OSFC_19.pdf
https://retrage.github.io/2019/12/05/debugging-ovmf-en.html

[2] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/GenFw.c#L2058

[3] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/ElfConvert.c#L167

[4] https://github.com/tianocore/edk2/blob/master/BaseTools/Source/C/GenFw/Elf64Convert.c#L762

Thanks,

Andrew Fish

Jose Miguel Hernandez Miramontes
BIOS Engineer
jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>
+1 (512) 362-1230
Intel Corporation

-----Original Message-----
From: Ard Biesheuvel <ardb@kernel.org <mailto:ardb@kernel.org>>
Sent: Wednesday, March 24, 2021 10:26 AM
To: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>>
Cc: discuss@edk2.groups.io <mailto:discuss@edk2.groups.io>; Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com <mailto:jose.miguel.hernandez.miramontes@intel.com>>; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com>>; Saucedo Tejada, Genaro <genaro.saucedo.tejada@intel.com <mailto:genaro.saucedo.tejada@intel.com>>; Ortiz Garcia, Jorge E <jorge.e.ortiz.garcia@intel.com <mailto:jorge.e.ortiz.garcia@intel.com>>; Ard Biesheuvel (TianoCore) <ardb+tianocore@kernel.org <mailto:ardb+tianocore@kernel.org>>
Subject: Re: [edk2-discuss] GccBase.lds

On Wed, 24 Mar 2021 at 16:18, Laszlo Ersek <lersek@redhat.com> wrote:

(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:
Hi

Is anyone familiar with this file?
(Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.
git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).
The build-id is used by Linux distros when they ship debug symbols with production shared libraries. The build-id permits the loader to look up the correct file, and confirm that the versions match.

In EDK2, the ELF binary is only an intermediate artifact, and so I fail to see why we would need build IDs here. If the ELF binary was built with debug symbols, they will be in the binary itself, not in a side loaded symbol file.



Interested in contributing

monamidg@...
 

Hi there,

I'm Monami - a Master's in Computer Engineering student at Virginia Tech, US. This is my first semester here as an international student, so I don't have CPT authorization to get paid for working this summer.
But I'm really interested in contributing here - especially the project "Enable Clang/LLVM Build for Microsoft Windows", or some other similar project.

This semester I've taken the courses Compiler Optimizations and OS & Virtualization where I've implemented an iterative dataflow framework to analyze passes. I've also worked on UEFI by building a bootloader and setting up virtual address spaces for user applications amongst other things. I am not able to share those projects yet as public on my GitHub page because I'm still only halfway through the semester and not done with the courses, but I can share it separately to an email-id if required as proof of work. By the end of the semester, I'll have implemented my course projects as well, and I feel the project listed here is the perfect fit for me to expand on my skills and help the community at the same time. I have experience with C as well as Python and I'm a quick study and work well under minimal supervision.

I was wondering if I could still contribute here, even if I am not potentially eligible yet to enroll as a Google Summer of Code student because of CPT authorization issues.
Please let me know how I can help!

Regards,
Monami Dutta Gupta


Re: GccBase.lds

Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@...>
 

yes, but if the section is not removed debugger would be able to find the debug symbols. Like how it happens with PDB files.
We could leave the symbols in the binaries, but it will consume more space. Space in the SPI is usually a constrain

I am just wondering if it is possible to fix this so that the section is there in the final efi.
If I were to try it, where would I start?

Jose Miguel Hernandez Miramontes
BIOS Engineer
jose.miguel.hernandez.miramontes@intel.com
+1 (512) 362-1230
Intel Corporation

-----Original Message-----
From: Ard Biesheuvel <ardb@kernel.org>
Sent: Wednesday, March 24, 2021 10:26 AM
To: Laszlo Ersek <lersek@redhat.com>
Cc: discuss@edk2.groups.io; Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Saucedo Tejada, Genaro <genaro.saucedo.tejada@intel.com>; Ortiz Garcia, Jorge E <jorge.e.ortiz.garcia@intel.com>; Ard Biesheuvel (TianoCore) <ardb+tianocore@kernel.org>
Subject: Re: [edk2-discuss] GccBase.lds

On Wed, 24 Mar 2021 at 16:18, Laszlo Ersek <lersek@redhat.com> wrote:

(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:
Hi

Is anyone familiar with this file?
(Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.
git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).
The build-id is used by Linux distros when they ship debug symbols with production shared libraries. The build-id permits the loader to look up the correct file, and confirm that the versions match.

In EDK2, the ELF binary is only an intermediate artifact, and so I fail to see why we would need build IDs here. If the ELF binary was built with debug symbols, they will be in the binary itself, not in a side loaded symbol file.


Re: lost stderr in console

Bob Feng
 

Wenyi,

As the description in Commit 01b6090b, the change was to fix the incremental build issue in the MSVC toolchain. But now it‘s found that impacts the usage of the GCC toolchain.

Please file a BZ for this issue. And I'll fix it.

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of wenyi,xie via groups.io
Sent: Thursday, March 25, 2021 10:18 AM
To: gaoliming <gaoliming@byosoft.com.cn>; discuss@edk2.groups.io
Subject: 答复: [edk2-discuss] lost stderr in console

Hi, liming,

Yes, my command using 2>&1 to redirect strerr to stdout, and after updating to version 202011 the stderr are not shown in the screen but only written in xxx.log.
According to Laszlo's help and checking the commit log, I find the problem relate to this commit:
https://github.com/tianocore/edk2/commit/01b6090b75922bc72604c334bd3dc331490af3bb

Proc = MakeSubProc(Command, stdout=PIPE, stderr=STDOUT, env=os.environ, cwd=WorkingDir, bufsize=-1, shell=True)

As the stdout and stderr are combined togetherin since the commit, so in version 201903 my command can work, but in version 202011, after this commit merged, it can't work well.

-----邮件原件-----
发件人: gaoliming [mailto:gaoliming@byosoft.com.cn]
发送时间: 2021年3月25日 9:27
收件人: discuss@edk2.groups.io; xiewenyi (A) <xiewenyi2@huawei.com>
主题: 回复: [edk2-discuss] lost stderr in console

Wenyi:
Do you use build command to build your platform and find the message from stderr are not shown in the screen?

Thanks
Liming
-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie via
groups.io
发送时间: 2021年3月17日 11:23
收件人: discuss@edk2.groups.io
主题: [edk2-discuss] lost stderr in console

Hi,everyone,

I used to set makeflags like below,so that only message from stderr
will be shown in console while compiling, and at the same time message
from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >>
${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log

But after updating edk2 from 201903 to 202011, the message from stderr
is not shown in console any more. In xxx.log, the message from stderr
and stdout are still saved.
Do anything change in edk2 may affect on it ?




Re: 答复: [edk2-discuss] lost stderr in console

wenyi,xie
 

Hi, liming,

Yes, my command using 2>&1 to redirect strerr to stdout, and after updating to version 202011 the stderr are not shown in the screen but only written in xxx.log.
According to Laszlo's help and checking the commit log, I find the problem relate to this commit:
https://github.com/tianocore/edk2/commit/01b6090b75922bc72604c334bd3dc331490af3bb

Proc = MakeSubProc(Command, stdout=PIPE, stderr=STDOUT, env=os.environ, cwd=WorkingDir, bufsize=-1, shell=True)

As the stdout and stderr are combined togetherin since the commit, so in version 201903 my command can work, but in version 202011, after this commit merged, it can't work well.

-----邮件原件-----
发件人: gaoliming [mailto:gaoliming@byosoft.com.cn]
发送时间: 2021年3月25日 9:27
收件人: discuss@edk2.groups.io; xiewenyi (A) <xiewenyi2@huawei.com>
主题: 回复: [edk2-discuss] lost stderr in console

Wenyi:
Do you use build command to build your platform and find the message from stderr are not shown in the screen?

Thanks
Liming

-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie via
groups.io
发送时间: 2021年3月17日 11:23
收件人: discuss@edk2.groups.io
主题: [edk2-discuss] lost stderr in console

Hi,everyone,

I used to set makeflags like below,so that only message from stderr
will be shown in console while compiling, and at the same time message
from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >>
${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log

But after updating edk2 from 201903 to 202011, the message from stderr
is not shown in console any more. In xxx.log, the message from stderr
and stdout are still saved.
Do anything change in edk2 may affect on it ?




Re: 答复: [edk2-discuss] lost stderr in console

wenyi,xie
 

Hi, Laszlo,

Thank you very much for such detailed explanation. I know the root cause of the problem now.

-----邮件原件-----
发件人: Laszlo Ersek [mailto:lersek@redhat.com]
发送时间: 2021年3月18日 4:31
收件人: discuss@edk2.groups.io; xiewenyi (A) <xiewenyi2@huawei.com>
主题: Re: [edk2-discuss] lost stderr in console

On 03/17/21 04:23, wenyi,xie via groups.io wrote:
Hi,everyone,

I used to set makeflags like below,so that only message from stderr will be shown in console while compiling, and at the same time message from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >>
${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log
Redirections are processed in the following order:
- pipe first (redirects stdout)
- then left to right, as seen on the command line

So what we have for "xxx.sh" is:

step#1: file descriptor 1 refers to the (nameless) pipe's file description

step#2: file descriptor 2 refers to the same (nameless) pipe's file description (i.e., to the file description that file descriptor 1
*currently* refers to)

step#3: file descriptor 1 now refers to a file description that refers to the inode ("file") originally looked up by the name "xxx.log". At the file description level, the O_APPEND status flag is set.

So "tee" will only see stderr from "xxx.sh". Furthermore, the stdout of "xxx.sh" will only go to the log file ("xxx.log").


Let's consider "tee" then: "tee" opens the inode (the "file") looked up by the name "xxx.log" separately from when the shell opens "xxx.log", for the "xxx.sh" redirection. This means that, in the kernel, a separate file description exists, for the "xxx.log" inode. This file description also has the O_APPEND status flag set, but it doesn't matter -- the file description that "xxx.sh" writes through, and the file description that "tee" writes through, are independent. The "file offset" property is at the file description level. Therefore "tee" and "xxx.sh" do not share the file offset (and O_APPEND is useless, in both file descriptions), and they will mutually overwrite parts of each other's output.

In other words, your command line is buggy.


In general:

file descriptor --> file description --> file (inode)

When you open() the same file by name, you get this:

file descriptor --> file description \
--> file (inode)
file descriptor --> file description /

Whereas, if you use fork() or dup(), this is what you get:

file descriptor \
--> file description --> file (inode)
file descriptor /

O_APPEND and the file offset both exist in the *file description* object. So in the first case, you get no coordination from the kernel, and in the second case, you do.

Note that even in the second case, that is, when both file descriptors refer to the same file description, it is not guaranteed that
*concurrent* writes will not be *interleaved*. No data will be *overwritten*, for sure, but the granularity of "atomic" writes is not an easy question. If the file description refers to a pipe, then there are some guarantees from POSIX, as long as the writes are "small enough"
(PIPE_BUF):
<https://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html>.


So... what you want to do is actually difficult. There are two approaches, but both are broken.

Approach (1): fuse stdout and stderr into a single stream, capture the common stream in the log file, and print the "diagnostic" lines to the
console:

sh xxx.sh 2>&1 | tee -- xxx.log | grep -- 'what exacty?'

This is broken because you cannot identify the diagnostic-only lines (the original stderr) by content.

Approach (2): duplicate the original stderr into two streams. The first instance will go to the console. The second instance, together with the script's stdout, will be written to the log file.

# fd 1: original stdout
# fd 2: original stderr
# make fd 3 point to the original stdout as well

exec 3>&1

(
# fd 1: pipe leading to the log file
# fd 2: original stderr
# fd 3: original stdout
# make fd 4 too point to the pipe leading to the log file

exec 4>&1

(
# fd 1: pipe carrying the main script's stderr
# fd 2: original stderr
# fd 3: original stdout
# fd 4: pipe leading to the log file

# main script's stderr goes to the duplicator pipe
# main script's stdout goes to the log file

xxx.sh 2>&1 >&4
) | (

# fd 0: pipe carrying the main script's stderr
# fd 1: pipe leading to the log file
# fd 2: original stderr
# fd 3: original stdout
# fd 4: pipe leading to the log file

# duplicate the main script's stderr to the original stdout (3)
# and to the log file (1)

tee /dev/fd/3
)
) | cat > xxx.log


In short form (no comments and no useless subshells):

exec 3>&1
(
exec 4>&1
xxx.sh 2>&1 >&4 | tee /dev/fd/3
) | cat > xxx.log

This will not lose messages. It will also not interleave write() syscalls that do not exceed PIPE_BUF (usually 4KB) individually -- this is the justification for the outermost pipe, and "cat".

However, it is still broken: dependent on process scheduling between "xxx.sh" and "tee", it's now possible that stdout and stderr lines from "xxx.sh" will be reordered, relative to each other. Let's say line#1 is a diagnostic message, while line#2 is a normal message. "xxx.sh" writes them in line#1, line#2 order. With the above, line#1 goes from "xxx.sh"
to "tee" to "cat", while line#2 goes from "xxx.sh" to "cat". If "tee" is "slow", then "cat" could see the messages in line#2, line#1 order.

Thanks
Laszlo


回复: [edk2-discuss] lost stderr in console

gaoliming
 

Wenyi:
Do you use build command to build your platform and find the message from stderr are not shown in the screen?

Thanks
Liming

-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie
via groups.io
发送时间: 2021年3月17日 11:23
收件人: discuss@edk2.groups.io
主题: [edk2-discuss] lost stderr in console

Hi,everyone,

I used to set makeflags like below,so that only message from stderr will be
shown in console while compiling, and at the same time message from stderr
and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >>
${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log

But after updating edk2 from 201903 to 202011, the message from stderr is
not shown in console any more. In xxx.log, the message from stderr and
stdout are still saved.
Do anything change in edk2 may affect on it ?




Re: GccBase.lds

Andrew Fish
 

Jose,

Caveat Emptor as Xcode clang is my daily driver, so I don’t really use GCC.

As Ard mentioned the ELF image is converted to a PE/COFF executable for EFI, and the symbols are stripped so the build ID is not going to help you. In PE/COFF there is something called a Debug Directory Entry[1]. It can point to a .debug section, or info in any section. The Debug Directory Entry starts with a 4 byte structure that defines its layout [1].

The 2nd issue you will have is the PE/COFF images are relocatable executables so they are not loaded at their linked address. Thus to load symbols you need to know the slide (delta of load address to link address) which you can derive from the load address of the image, and a path to the external symbols file. PE/COFF has the property that all the PE/COFF headers are loaded into memory when a PE/COFF image is loaded, this will be important later.

When you get past early PEI[2] and code is no longer XIP (eXecute In Place) you need to know the load address of the code to load symbols. So you end up with these options:
1) Load symbols for PC (This works for XIP too).
2) Load symbols for frame. Really just 1) for each frame PC.
3) Load symbols based on some table that has all the load info. This is the example from the EFI Spec [3].

To implement 1) you start with the PC and walk backwards in memory looking for the PE/COFF (or DOS) header. That header is the load address of the module. From the headers you can find the debug directory entry. The Debug Directory entry can then be decoded and that gives you (maybe) a unique identifier and the file name of the symbols the debugger needs to load. At that point you can tell the debugger this debug info matches the module loaded at this address. If your debugger knows PE/COFF you just have to point at the DOS or PE/COFF header.

So given I use lldb and the macOS flavor of lldb does not parse PE/COFF we had to write lldb Python to do all of the above and some more. So above is the how to do everything yourself, or theory of operation.

There is also the primitive look at the serial logs and load symbols by hand using debugger commands.

In terms of `git grep` in the edk2 look for EFI_IMAGE_DEBUG_DIRECTORY_ENTRY. There is a lib function called PeCoffLoaderGetPdbPointer() that returns a pointer to the PE/COFF Debug Directory Entry, it has the name pdb as that is the name of the windows symbols file that the Debug Directory Entry generally points to for VC++. For Xcode/clang the Debug Directory Entry entry has the UUID of mach-O executable (that got converted into a PE/COFF) and lldb can look up the symbols that way, other toolchains need the path to the debug file to load. So for example the OVMF exception handler [4] will dump out info about the faulting PC.

Sorry for the very generic answer, but I though it might be helpful for other folks too. You might want to ask a question about your target architecture, toolchain, and debugger on the mailing to find out what of the above is available for your toolchain with the edk2 for free. If you are interested in adding more features above is an outline of how it works and you can always ask more detailed questions on the mailing list.

[1] https://github.com/tianocore/edk2/blob/master/MdePkg/Include/IndustryStandard/PeImage.h#L609

[2] The build kicks out an <FvName>.Fv.map file that contains the info you need to load symbols.
For example: Build/OvmfX64/DEBUG_XCODE5/FV/PEIFV.Fv.map
EFI_FV_TOTAL_SIZE = 0xe0000
EFI_FV_TAKEN_SIZE = 0x43728
EFI_FV_SPACE_SIZE = 0x9c8d8

PeiCore (Fixed Flash Address, BaseAddress=0x0000820120, EntryPoint=0x000082b012)
(GUID=52C05B14-0B98-496C-BC3B-04B50211D680 .textbaseaddress=0x0000820360 .databaseaddress=0x000082e420)



PcdPeim (Fixed Flash Address, BaseAddress=0x000082ed00, EntryPoint=0x0000831a62)
(GUID=9B3ADA4F-AE56-4C24-8DEA-F03B7558AE50 .textbaseaddress=0x000082ef40 .databaseaddress=0x0000834ac0)
...

[3] https://github.com/tianocore/edk2/blob/master/MdePkg/Include/Guid/DebugImageInfoTable.h

[4] UefiCpuPkg/Library/CpuExceptionHandlerLib/CpuExceptionCommon.c:130: PdbPointer = PeCoffLoaderGetPdbPointer ((VOID *) Pe32Data);

Thanks,

Andrew Fish

On Mar 24, 2021, at 8:25 AM, Ard Biesheuvel <ardb@kernel.org> wrote:

On Wed, 24 Mar 2021 at 16:18, Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>> wrote:

(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:
Hi

Is anyone familiar with this file? (Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.
git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).
The build-id is used by Linux distros when they ship debug symbols
with production shared libraries. The build-id permits the loader to
look up the correct file, and confirm that the versions match.

In EDK2, the ELF binary is only an intermediate artifact, and so I
fail to see why we would need build IDs here. If the ELF binary was
built with debug symbols, they will be in the binary itself, not in a
side loaded symbol file.



Re: GccBase.lds

Ard Biesheuvel
 

On Wed, 24 Mar 2021 at 16:56, Hernandez Miramontes, Jose Miguel
<jose.miguel.hernandez.miramontes@intel.com> wrote:

yes, but if the section is not removed debugger would be able to find the debug symbols. Like how it happens with PDB files.
We could leave the symbols in the binaries, but it will consume more space. Space in the SPI is usually a constrain
I am not sure how your build environment works exactly, but when using
GCC to build for ARM, we already have this feature: the CodeView NB10
entry in the debug table in the PE/COFF metadata is populated with the
path to the .dll on the host from which the EFI executable was
generated. The debugger extracts this data to locate the file on the
build machine, and loads the symbols.

I am just wondering if it is possible to fix this so that the section is there in the final efi.
If I were to try it, where would I start?
I wouldn't be in favor of hacking this up: the ELF to PE/COFF
conversion in EDK2 is already highly dubious, and it would be much
better to stick with the PE/COFF semantics for carrying such metadata,
like the CodeView NB10 entry does.


Re: GccBase.lds

Ard Biesheuvel
 

On Wed, 24 Mar 2021 at 16:18, Laszlo Ersek <lersek@redhat.com> wrote:

(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:
Hi

Is anyone familiar with this file? (Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.
git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).
The build-id is used by Linux distros when they ship debug symbols
with production shared libraries. The build-id permits the loader to
look up the correct file, and confirm that the versions match.

In EDK2, the ELF binary is only an intermediate artifact, and so I
fail to see why we would need build IDs here. If the ELF binary was
built with debug symbols, they will be in the binary itself, not in a
side loaded symbol file.


Re: GccBase.lds

Laszlo Ersek
 

(+Ard)

On 03/24/21 02:49, Hernandez Miramontes, Jose Miguel wrote:
Hi

Is anyone familiar with this file? (Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.
git-blame is your friend. It leads to commit 7fd5d619806d ("BaseTools
GCC: drop GNU notes section from EFI image", 2016-08-02).

Thanks
Laszlo

What would it take to preserve it?

When reading this, https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html,
I understood that this section is needed so GDB can find the symbols when they are extracted from the binary.


objdump -fh PeiCore.dll

PeiCore.dll: file format elf32-i386
architecture: i386, flags 0x00000013:
HAS_RELOC, EXEC_P, HAS_SYMS
start address 0x00006089

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 000060bc 00000220 00000220 00000080 2**5
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
1 .data 00000348 000062e0 000062e0 00006140 2**5
CONTENTS, ALLOC, LOAD, RELOC, DATA
2 .build-id 00000024 00006628 00006628 00006488 2**2
CONTENTS, READONLY

$objdump -fh PeiCore.efi

PeiCore.efi: file format pei-i386
architecture: i386, flags 0x0000010b:
HAS_RELOC, EXEC_P, HAS_DEBUG, D_PAGED
start address 0x00006089

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 000060c0 00000220 00000220 00000220 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000400 000062e0 000062e0 000062e0 2**2
CONTENTS, ALLOC, LOAD, DATA
2 .reloc 000001a0 000066e0 000066e0 000066e0 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
Jose Miguel Hernandez Miramontes
BIOS Engineer
jose.miguel.hernandez.miramontes@intel.com
+1 (512) 362-1230
Intel Corporation







GccBase.lds

Hernandez Miramontes, Jose Miguel <jose.miguel.hernandez.miramontes@...>
 

Hi

 

Is anyone familiar with this file? (Edk2\BaseTools\Scripts\GccBase.lds)
I would like to understand more what it does and why it is needed.

When looking at the .efi output of genfw, it seems the .build-id section generated by gcc is discarded.

What would it take to preserve it?

 

When reading this, https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html,

I understood that this section is needed so GDB can find the symbols when they are extracted from the binary.



objdump -fh PeiCore.dll

 

PeiCore.dll:     file format elf32-i386

architecture: i386, flags 0x00000013:

HAS_RELOC, EXEC_P, HAS_SYMS

start address 0x00006089

 

Sections:

Idx Name          Size      VMA       LMA       File off  Algn

  0 .text         000060bc  00000220  00000220  00000080  2**5

                  CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE

  1 .data         00000348  000062e0  000062e0  00006140  2**5

                  CONTENTS, ALLOC, LOAD, RELOC, DATA

  2 .build-id     00000024  00006628  00006628  00006488  2**2

                  CONTENTS, READONLY

 

$objdump -fh PeiCore.efi

 

PeiCore.efi:     file format pei-i386

architecture: i386, flags 0x0000010b:

HAS_RELOC, EXEC_P, HAS_DEBUG, D_PAGED

start address 0x00006089

 

Sections:

Idx Name          Size      VMA       LMA       File off  Algn

  0 .text         000060c0  00000220  00000220  00000220  2**2

                  CONTENTS, ALLOC, LOAD, READONLY, CODE

  1 .data         00000400  000062e0  000062e0  000062e0  2**2

                  CONTENTS, ALLOC, LOAD, DATA

  2 .reloc        000001a0  000066e0  000066e0  000066e0  2**2

                  CONTENTS, ALLOC, LOAD, READONLY, DATA

Jose Miguel Hernandez Miramontes

BIOS  Engineer

jose.miguel.hernandez.miramontes@...

+1 (512) 362-1230

Intel Corporation

 

181 - 200 of 862