Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM


jackmacwindowslinux@...
 

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.


Nate DeSimone
 

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.


alejandro.estay@...
 

Taking Nate DeSimone point. try to put some assertions to check what protocol is missing. Also loading a different shell, or loading shell instance as necessary can be a way to check what is actually happening. Check LibC code to see what protocols are added. Put some prints there. I know this can be a exahustive task, but apparently there's no other way you can check from where that "EFI_UNSUPPORTED" is coming, except perhaps a source level debug instance, but I don{t know how this can be done in a non-tianocore machine.


JackMacWindows <jackmacwindowslinux@...>
 

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@...> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.






JackMacWindows <jackmacwindowslinux@...>
 

Alright, I’ve done some testing on using different build configurations. Here is what I have done so far:
I reverted my build system back to UDK 2015.
I tried building my version of Lua (5.1.5) using the same configuration files as are in the Lua included with StdLib. This worked fine.
I removed all #includes from LibC that were not in the Lua source. These included dirent.h, libgen.h, and stat.h.
I moved the Lua dependency into its own library file rather than compiling it with the original source.
I used the EFI Shell 2.1 that ships with UDK 2015 instead of the shell that comes with the latest EDK. I also tried the shell built into rEFInd.
I built the application using Visual Studio 2019 on Windows instead of my GCC cross-compiler.
None of the steps I have taken have fixed the issue. Also, something to note is that I’m overriding the ShellCEntryLib function (https://github.com/MCJack123/craftos-efi/blob/master/efi.c) because my application needs access to the raw ConIn/ConOut handles. While doing this I added a simple print statement at the start of the function. This code does not run at all: it just errors out before the entry point, it seems. I would like to be able to switch to using the actual ShellCEntryLib library, but the SystemTable is an absolute necessity to the functioning of the program. I’m thinking this may be some internal issue inside EDK since it doesn’t even get to the start function before crashing immediately. Maybe it could be something with the executable format not being built correctly?

If anyone has the time to look over my code, it is currently available at https://github.com/MCJack123/craftos-efi as I mentioned in the OP. A downloadable EFI file & disk image are available in the Releases tab for testing (though this is an older build without the aforementioned modifications).

On Jan 28, 2020, at 12:56 AM, JackMacWindows <jackmacwindowslinux@...> wrote:

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@...> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.


Laszlo Ersek
 

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo


JackMacWindows <jackmacwindowslinux@...>
 

Nope, I’m building for X64 in my targets.txt file. Running objdump on the file returns "craftos-efi/CraftOS.efi: file format pei-x86-64" so it’s definitely an X64 image.

On Jan 31, 2020, at 10:34 AM, Laszlo Ersek <lersek@...> wrote:

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo