Build Error - ModuleNotFoundError with build.py

Satoshi Tanda

Hi,

I am unable to complete the "build" command successfully due to the import
error. Can someone point me where to look into?

C:\edk2>build
Traceback (most recent call last):
File "C:\edk2\BaseTools\Source\Python\build\build.py", line 29, in
<module>
from Common.buildoptions import BuildOption,BuildTarget
ModuleNotFoundError: No module named 'Common'

I am trying to follow the build instructions for Windows here, and the
commit I try to work on is edk2-stable201911.
https://github.com/tianocore/tianocore.github.io/wiki/Windows-systems

>git log -1
edk2-stable201911, tag: edk2-stable201911)

Required software and Conf\target.txt are setup like this.

>set NASM_PREFIX
NASM_PREFIX=C:\Users\tanda\AppData\Local\bin\NASM\

>C:\ASL\iasl.exe -v
...
ASL+ Optimizing Compiler/Disassembler version 20191018

>set PYTHON_HOME
PYTHON_HOME=C:\Python
>python --version
Python 3.8.1

---
ACTIVE_PLATFORM = MdeModulePkg/MdeModulePkg.dsc
TOOL_CHAIN_TAG = VS2019
---

Then, I run

>edksetup.bat Rebuild
>edksetup.bat
>build

Both edksetup runs end without any errors.

At some point before I was able to build things without the issues. I
suspect some Python setup changes since then caused this but cannot spot
what the cause and how to resolve this.

Best,
Satoshi

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Nope, I’m building for X64 in my targets.txt file. Running objdump on the file returns "craftos-efi/CraftOS.efi: file format pei-x86-64" so it’s definitely an X64 image.

On Jan 31, 2020, at 10:34 AM, Laszlo Ersek <lersek@redhat.com> wrote:

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

Laszlo Ersek

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Alright, I’ve done some testing on using different build configurations. Here is what I have done so far:
I reverted my build system back to UDK 2015.
I tried building my version of Lua (5.1.5) using the same configuration files as are in the Lua included with StdLib. This worked fine.
I removed all #includes from LibC that were not in the Lua source. These included dirent.h, libgen.h, and stat.h.
I moved the Lua dependency into its own library file rather than compiling it with the original source.
I used the EFI Shell 2.1 that ships with UDK 2015 instead of the shell that comes with the latest EDK. I also tried the shell built into rEFInd.
I built the application using Visual Studio 2019 on Windows instead of my GCC cross-compiler.
None of the steps I have taken have fixed the issue. Also, something to note is that I’m overriding the ShellCEntryLib function (https://github.com/MCJack123/craftos-efi/blob/master/efi.c) because my application needs access to the raw ConIn/ConOut handles. While doing this I added a simple print statement at the start of the function. This code does not run at all: it just errors out before the entry point, it seems. I would like to be able to switch to using the actual ShellCEntryLib library, but the SystemTable is an absolute necessity to the functioning of the program. I’m thinking this may be some internal issue inside EDK since it doesn’t even get to the start function before crashing immediately. Maybe it could be something with the executable format not being built correctly?

If anyone has the time to look over my code, it is currently available at https://github.com/MCJack123/craftos-efi as I mentioned in the OP. A downloadable EFI file & disk image are available in the Releases tab for testing (though this is an older build without the aforementioned modifications).

On Jan 28, 2020, at 12:56 AM, JackMacWindows <jackmacwindowslinux@gmail.com> wrote:

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@intel.com> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Saving data structure at Pre EFI Initialization phase in memory to use it at DXE phase by some UEFI application?

sergestus@...

Is it possible to save SYSHOST data structure at PEI phase in memory at some default location to use it at DXE phase by some UEFI application?

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@intel.com> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

alejandro.estay@...

Taking Nate DeSimone point. try to put some assertions to check what protocol is missing. Also loading a different shell, or loading shell instance as necessary can be a way to check what is actually happening. Check LibC code to see what protocols are added. Put some prints there. I know this can be a exahustive task, but apparently there's no other way you can check from where that "EFI_UNSUPPORTED" is coming, except perhaps a source level debug instance, but I don{t know how this can be done in a non-tianocore machine.

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

Nate DeSimone

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

jackmacwindowslinux@...

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Re: dirent always returns NULL when opening a directory, errno is 0

Laszlo Ersek

On 01/24/20 09:52, Jack Bruienne wrote:
I’m testing out running some code under the EFI environment to learn about writing EFI applications. I’m trying to list files in a directory on the filesystem using the dirent API provided with LibStd in EDK II. I have successfully compiled the code and linked with all required libraries, but I keep getting an issue where opendir() is always returning NULL. When checking the value of errno, it’s still EOK anyway. Here’s the code I’m using:

struct dirent *dir;
char * path = fixpath(old_path); // converts "/path/to/dir" to "FS0:\path\to\dir"
DIR * d = opendir(path);
if (d) {
for (int i = 0; (dir = readdir(d)) != NULL; i++) {
// process file entries
}
closedir(d);
} else err(path, "Not a directory");

When this code is run, it ends up calling err() and showing a message on the screen ("FS0:\: Not a directory (0)") instead of processing the file entries in the dirent structure. Are there any known bugs with the EDK II implementation of dirent? If so, how can I rewrite my code to use native EFI calls instead of dirent?

I am using the stable201911 version of EDK II with the latest edk2-libc, and I'm compiling under macOS using a Linux cross-compilation toolchain. The built EFI application is run under QEMU x86_64 with an OVMF firmware built with the same toolchain.
I think the fixpath() call is superfluous and wrong. As far as I
remember, when programming against edk2-libc, edk2-libc places some
directory structure requirements on the underlying UEFI filesystem.
Therefore manual conversion of pathnames should not be necessary;
instead it is on the provider of the underlying UEFI filesystem that
needs to place files in the right spots.

Let me see... Yes, please refer to section "TARGET-SYSTEM INSTALLATION"
in "StdLib/ReadMe.txt", in the edk2-libc project (with master @
61687168fe02).

It seems that, using StdLib, you can only refer to files that are on the
same filesystem that the executable was loaded from.

Thanks
Laszlo

dirent always returns NULL when opening a directory, errno is 0

Jack Bruienne

I’m testing out running some code under the EFI environment to learn about writing EFI applications. I’m trying to list files in a directory on the filesystem using the dirent API provided with LibStd in EDK II. I have successfully compiled the code and linked with all required libraries, but I keep getting an issue where opendir() is always returning NULL. When checking the value of errno, it’s still EOK anyway. Here’s the code I’m using:

struct dirent *dir;
char * path = fixpath(old_path); // converts "/path/to/dir" to "FS0:\path\to\dir"
DIR * d = opendir(path);
if (d) {
for (int i = 0; (dir = readdir(d)) != NULL; i++) {
// process file entries
}
closedir(d);
} else err(path, "Not a directory");

When this code is run, it ends up calling err() and showing a message on the screen ("FS0:\: Not a directory (0)") instead of processing the file entries in the dirent structure. Are there any known bugs with the EDK II implementation of dirent? If so, how can I rewrite my code to use native EFI calls instead of dirent?

I am using the stable201911 version of EDK II with the latest edk2-libc, and I'm compiling under macOS using a Linux cross-compilation toolchain. The built EFI application is run under QEMU x86_64 with an OVMF firmware built with the same toolchain.

Re: EFI Group Event Callback Order

Li, Walon

Thanks for your explanation, it's very clear.
Is it possible to update spec that makes unspecified behavior to specified? I know the implementation is in accordance with spec but add some comments can make programmer use this mechanism flexible.

Thanks!
Walon

-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Saturday, January 18, 2020 2:19 AM
To: Li, Walon <walon.li@hpe.com>; discuss@edk2.groups.io
Cc: Wei, Kent (HPS SW) <kent.wei@hpe.com>; Wang, Sunny (HPS SW) <sunnywang@hpe.com>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; Spottswood, Jason <jason.spottswood@hpe.com>; Shifflett, Joseph <joseph.shifflett@hpe.com>; Haskell, Darrell <darrell.haskell@hpe.com>; Wiginton, Scott <scott.wiginton@hpe.com>
Subject: Re: [edk2-discuss] EFI Group Event Callback Order

On 01/17/20 14:24, Li, Walon wrote:
Laszlo,

Yes, we can specific TPL to adjust callback order. But sometimes, an
event has many callbacks like as ReadyToBoot and TPL level may not
satisfy purpose. In UEFI spec CreateEvent chapter, also mentioned "The
functions in these queues are invoked in FIFO order". So, if it define
clearly in UEFI spec, LIFO or FIFO, we may use driver dependency to
decide which driver's event registration priority and specific
callback order indirectly.
The FIFO order refers to something else. Here's a larger citation:

When the event is signaled, firmware changes its state to "signaled"
and, if EVT_NOTIFY_SIGNAL is specified, places a call to its
notification function in a FIFO queue. There is a queue for each of
the "basic" task priority levels defined in Section 7.1
(TPL_CALLBACK, and TPL_NOTIFY). The functions in these queues are
invoked in FIFO order, starting with the highest priority level
queue and proceeding to the lowest priority queue that is unmasked
by the current TPL.

It means that, within a given TPL, the notification function invocation order will reflect the enqueueing (= signaling) order.

But the spec does not specify the *signaling* order for such events that belong to the same event group, when one of those events is signaled.
The spec says,

Event groups are collections of events identified by a shared
EFI_GUID where, when one member event is signaled, all other events
are signaled and their individual notification actions are taken (as
described in CreateEvent). All events are guaranteed to be signaled
functions will be executed in the order specified by their
NotifyTpl.

So if you have two events, A and B, and they belong to the same event group G, and NotifyTpl is the same for both, then if you signal either A or B, then the notification functions may be queued, *and invoked*, in either (A, B), or (B, A), order. The invocation order will reflect the queueing order, yes, but the latter is unspecified, when you go through an event group.

Thanks
Laszlo

Re: EFI Group Event Callback Order

Laszlo Ersek

On 01/17/20 14:24, Li, Walon wrote:
Laszlo,

Yes, we can specific TPL to adjust callback order. But sometimes, an
event has many callbacks like as ReadyToBoot and TPL level may not
satisfy purpose. In UEFI spec CreateEvent chapter, also mentioned
"The functions in these queues are invoked in FIFO order". So, if it
define clearly in UEFI spec, LIFO or FIFO, we may use driver
dependency to decide which driver's event registration priority and
specific callback order indirectly.
The FIFO order refers to something else. Here's a larger citation:

When the event is signaled, firmware changes its state to “signaled”
and, if EVT_NOTIFY_SIGNAL is specified, places a call to its
notification function in a FIFO queue. There is a queue for each of
the “basic” task priority levels defined in Section 7.1
(TPL_CALLBACK, and TPL_NOTIFY). The functions in these queues are
invoked in FIFO order, starting with the highest priority level
queue and proceeding to the lowest priority queue that is unmasked
by the current TPL.

It means that, within a given TPL, the notification function invocation
order will reflect the enqueueing (= signaling) order.

But the spec does not specify the *signaling* order for such events that
belong to the same event group, when one of those events is signaled.
The spec says,

Event groups are collections of events identified by a shared
EFI_GUID where, when one member event is signaled, all other events
are signaled and their individual notification actions are taken (as
described in CreateEvent). All events are guaranteed to be signaled
functions will be executed in the order specified by their
NotifyTpl.

So if you have two events, A and B, and they belong to the same event
group G, and NotifyTpl is the same for both, then if you signal either A
or B, then the notification functions may be queued, *and invoked*, in
either (A, B), or (B, A), order. The invocation order will reflect the
queueing order, yes, but the latter is unspecified, when you go through
an event group.

Thanks
Laszlo

Re: EFI Group Event Callback Order

Tomas Pilar (tpilar)

I had to solve this for myself by creating a PostReadyToBoot event that is signalled by my ReadyToBoot callback. That meant that my PostReadyToBoot callbacks got executed after all the ReadyToBoot event functions.

Cheers,
Tom
________________________________________
From: discuss@edk2.groups.io <discuss@edk2.groups.io> on behalf of Li, Walon <walon.li@hpe.com>
Sent: 17 January 2020 13:24
To: discuss@edk2.groups.io; Laszlo Ersek
Cc: Wei, Kent (HPS SW); Wang, Sunny (HPS SW); Chang, Abner (HPS SW/FW Technologist); Spottswood, Jason; Shifflett, Joseph; Haskell, Darrell; Wiginton, Scott
Subject: Re: [edk2-discuss] EFI Group Event Callback Order

Laszlo,

Yes, we can specific TPL to adjust callback order. But sometimes, an event has many callbacks like as ReadyToBoot and TPL level may not satisfy purpose.
In UEFI spec CreateEvent chapter, also mentioned "The functions in these queues are invoked in FIFO order". So, if it define clearly in UEFI spec, LIFO or FIFO, we may use driver dependency to decide which driver's event registration priority and specific callback order indirectly.

Thank you,
Walon

-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Friday, January 17, 2020 4:09 PM
To: discuss@edk2.groups.io; Li, Walon <walon.li@hpe.com>
Subject: Re: [edk2-discuss] EFI Group Event Callback Order

On 01/16/20 09:32, Li, Walon wrote:
Hi edk2,

As UEFI event group mechanism, we can register callbacks under event group. For example, register two event callbacks and will be signal in ReadyToBoot.
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback1,
NULL,
);
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback2,
NULL,
);
I'm curious the order of callback. In this case, the order is LIFO and TestCallback2 will be executed than TestCallback1.
As the UEFI spec page 152, only said "If the supplied Event is a part of an event group, then all of the events in the event group are also signaled and their notification functions are scheduled." It doesn't define order clearly.

However, edk2 has different implementation in EVT_RUNTIME / EVT_NOTIFY_SIGNAL attribute of group event. In Event.c, it inserts new event to Head in EVT_NOTIFY_SIGNAL queue (LIFO) and inserts to Tail in EVT_RUNTIME queue (FIFO).
I know the programmer shouldn't assume any order but want to know why is different implementation in group event. Have any history reason or limitation?

if ((Type & EVT_RUNTIME) != 0) {
//
// Keep a list of all RT events so we can tell the RT AP.
//
IEvent->RuntimeData.Type = Type;
IEvent->RuntimeData.NotifyTpl = NotifyTpl;
IEvent->RuntimeData.NotifyFunction = NotifyFunction;
IEvent->RuntimeData.NotifyContext = (VOID *) NotifyContext;
//
// Work around the bug in the Platform Init specification (v1.7), reported
// as Mantis#2017: "EFI_RUNTIME_EVENT_ENTRY.Event" should have type
// EFI_EVENT, not (EFI_EVENT*). The PI spec documents the field correctly
// as "The EFI_EVENT returned by CreateEvent()", but the type of the field
// doesn't match the natural language description. Therefore we need an
// explicit cast here.
//
IEvent->RuntimeData.Event = (EFI_EVENT *) IEvent;
}

CoreAcquireEventLock ();

if ((Type & EVT_NOTIFY_SIGNAL) != 0x00000000) {
//
// The Event's NotifyFunction must be queued whenever the event is signaled
//
}

Thank you,
Walon

It's unspecified. Implementations may differ, and an implementation isn't even required to be self-consistent wrt. the order.

Ordering is only specified in terms of TPLs.

Thanks
Laszlo

The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.

Re: EFI Group Event Callback Order

Li, Walon

Laszlo,

Yes, we can specific TPL to adjust callback order. But sometimes, an event has many callbacks like as ReadyToBoot and TPL level may not satisfy purpose.
In UEFI spec CreateEvent chapter, also mentioned "The functions in these queues are invoked in FIFO order". So, if it define clearly in UEFI spec, LIFO or FIFO, we may use driver dependency to decide which driver's event registration priority and specific callback order indirectly.

Thank you,
Walon

-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Friday, January 17, 2020 4:09 PM
To: discuss@edk2.groups.io; Li, Walon <walon.li@hpe.com>
Subject: Re: [edk2-discuss] EFI Group Event Callback Order

On 01/16/20 09:32, Li, Walon wrote:
Hi edk2,

As UEFI event group mechanism, we can register callbacks under event group. For example, register two event callbacks and will be signal in ReadyToBoot.
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback1,
NULL,
);
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback2,
NULL,
);
I'm curious the order of callback. In this case, the order is LIFO and TestCallback2 will be executed than TestCallback1.
As the UEFI spec page 152, only said "If the supplied Event is a part of an event group, then all of the events in the event group are also signaled and their notification functions are scheduled." It doesn't define order clearly.

However, edk2 has different implementation in EVT_RUNTIME / EVT_NOTIFY_SIGNAL attribute of group event. In Event.c, it inserts new event to Head in EVT_NOTIFY_SIGNAL queue (LIFO) and inserts to Tail in EVT_RUNTIME queue (FIFO).
I know the programmer shouldn't assume any order but want to know why is different implementation in group event. Have any history reason or limitation?

if ((Type & EVT_RUNTIME) != 0) {
//
// Keep a list of all RT events so we can tell the RT AP.
//
IEvent->RuntimeData.Type = Type;
IEvent->RuntimeData.NotifyTpl = NotifyTpl;
IEvent->RuntimeData.NotifyFunction = NotifyFunction;
IEvent->RuntimeData.NotifyContext = (VOID *) NotifyContext;
//
// Work around the bug in the Platform Init specification (v1.7), reported
// as Mantis#2017: "EFI_RUNTIME_EVENT_ENTRY.Event" should have type
// EFI_EVENT, not (EFI_EVENT*). The PI spec documents the field correctly
// as "The EFI_EVENT returned by CreateEvent()", but the type of the field
// doesn't match the natural language description. Therefore we need an
// explicit cast here.
//
IEvent->RuntimeData.Event = (EFI_EVENT *) IEvent;
}

CoreAcquireEventLock ();

if ((Type & EVT_NOTIFY_SIGNAL) != 0x00000000) {
//
// The Event's NotifyFunction must be queued whenever the event is signaled
//
}

Thank you,
Walon

It's unspecified. Implementations may differ, and an implementation isn't even required to be self-consistent wrt. the order.

Ordering is only specified in terms of TPLs.

Thanks
Laszlo

Re: EFI Group Event Callback Order

Laszlo Ersek

On 01/16/20 09:32, Li, Walon wrote:
Hi edk2,

As UEFI event group mechanism, we can register callbacks under event group. For example, register two event callbacks and will be signal in ReadyToBoot.
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback1,
NULL,
);
Status = gBS->CreateEventEx (
EVT_NOTIFY_SIGNAL,
TPL_CALLBACK,
TestCallback2,
NULL,
);
I'm curious the order of callback.
It's unspecified. Implementations may differ, and an implementation
isn't even required to be self-consistent wrt. the order.

Ordering is only specified in terms of TPLs.

Thanks
Laszlo

Re: Is possible to obtain assembly output from modules?

alejandro.estay@...

That did exactly I wanted. Thanks Laszlo.

EFI Group Event Callback Order

Li, Walon <walon.li@...>

Hi edk2,

As UEFI event group mechanism, we can register callbacks under event group. For example, register two event callbacks and will be signal in ReadyToBoot.

Status = gBS->CreateEventEx (

EVT_NOTIFY_SIGNAL,

TPL_CALLBACK,

TestCallback1,

NULL,

);

Status = gBS->CreateEventEx (

EVT_NOTIFY_SIGNAL,

TPL_CALLBACK,

TestCallback2,

NULL,

);

I'm curious the order of callback. In this case, the order is LIFO and TestCallback2 will be executed than TestCallback1.

As the UEFI spec page 152, only said "If the supplied Event is a part of an event group, then all of the events in the event group are also signaled and their notification functions are scheduled." It doesn't define order clearly.

However, edk2 has different implementation in EVT_RUNTIME / EVT_NOTIFY_SIGNAL attribute of group event. In Event.c, it inserts new event to Head in EVT_NOTIFY_SIGNAL queue (LIFO) and inserts to Tail in EVT_RUNTIME queue (FIFO).

I know the programmer shouldn't assume any order but want to know why is different implementation in group event. Have any history reason or limitation?

if ((Type & EVT_RUNTIME) != 0) {

//

// Keep a list of all RT events so we can tell the RT AP.

//

IEvent->RuntimeData.Type           = Type;

IEvent->RuntimeData.NotifyTpl      = NotifyTpl;

IEvent->RuntimeData.NotifyFunction = NotifyFunction;

IEvent->RuntimeData.NotifyContext  = (VOID *) NotifyContext;

//

// Work around the bug in the Platform Init specification (v1.7), reported

// as Mantis#2017: "EFI_RUNTIME_EVENT_ENTRY.Event" should have type

// EFI_EVENT, not (EFI_EVENT*). The PI spec documents the field correctly

// as "The EFI_EVENT returned by CreateEvent()", but the type of the field

// doesn't match the natural language description. Therefore we need an

// explicit cast here.

//

IEvent->RuntimeData.Event          = (EFI_EVENT *) IEvent;

}

CoreAcquireEventLock ();

if ((Type & EVT_NOTIFY_SIGNAL) != 0x00000000) {

//

// The Event's NotifyFunction must be queued whenever the event is signaled

//

}

Thank you,

Walon

Re: Analogous to Print() but for keyboard input?

Laszlo Ersek

On 01/13/20 05:34, alejandro.estay@gmail.com wrote:
Is there some library like UefiLib, with some function working like
scanf()? For example, print() shares some things with printf(), but I
can't find any related to input working that way.
I don't really understand your question.

If you are looking for functions that parse string representations of
integers into integer variables, "MdePkg/Include/Library/BaseLib.h" has:

(Ascii)?Str(Decimal|Hex)ToUint(64|n)S?

Before exposing any such function to untrusted input, I'd strongly
recommend double-checking the implementation.

For regular expressions, you could include RegularExpressionDxe in your
platform build, and use EFI_REGULAR_EXPRESSION_PROTOCOL.

HTH
Laszlo

Re: Is possible to obtain assembly output from modules?

Laszlo Ersek

On 01/14/20 04:41, alejandro.estay@gmail.com wrote:
Hi
I want to check some .efi and call format stuff. However I don't know exactly how to produce assembly output from edk2 gcc. I was trying to tweak .dsc. The problem is that the last file is compile overwrites the rrest for every module.
I pasted this at the end of MdeModulePkg last line

However asmdump.asm only contains listing for the last file compiled in the module Is there a better way to do this?
The way I generally do this (for OVMF) is as follows:

- run a complete NOOPT build

- run a command like:

objdump -S Build/Ovmf3264/NOOPT_GCC48/X64/MdeModulePkg/Core/PiSmmCore/PiSmmCore/DEBUG/PiSmmCore.debug \
| less

This is going to give you a disassembly intermixed with C source code, for the "MdeModulePkg/Core/PiSmmCore/PiSmmCore.c" file.

Update the pathname as necessary.

Thanks
Laszlo

 761 - 780 of 888