Unit Test Framework for UEFI driver of RAID controller

sanket.goswami@...

Hi,
I'm trying to create Unit test framework for UEFI driver of PCI RAID controller and looking for a suitable tool for this.
Has anyone used googlemock framework with UEFI? I wish to know if it's possible to integrate both in EDK build.
Please let me know if anyone has used other similar tools for unit test framework for UEFI.

Thanks,
Sanket

Re: A problem with live migration of UEFI virtual machines

Alex Bennée <alex.bennee@...>

wuchenye1995 <wuchenye1995@...> writes:

Hi all,
We found a problem with live migration of UEFI virtual machines due to size of OVMF.fd changes.
Specifically, the size of OVMF.fd in edk with low version such as edk-2.0-25 is 2MB while the size of it in higher version such as edk-2.0-30 is 4MB.
When we migrate a UEFI virtual machine from the host with low version of edk2 to the host with higher one, qemu component will report an error in function qemu_ram_resize while
checking size of ovmf_pcbios: Length mismatch: pc.bios: 0x200000 in != 0x400000: Invalid argument.
We want to know how to solve this problem after updating the
version of edk2.
You can only migrate a machine that is identical - so instantiating a
empty machine with a different EDK image is bound to cause a problem
because the machines don't match.

--
Alex Bennée

A problem with live migration of UEFI virtual machines

"wuchenye1995

Hi all,
We found a problem with live migration of UEFI virtual machines due to size of OVMF.fd changes.
Specifically, the size of OVMF.fd in edk with low version such as edk-2.0-25 is *2MB* while the size of it in higher version such as edk-2.0-30 is *4MB*.
When we migrate a UEFI virtual machine from the host with low version of edk2 to the host with higher one, qemu component will report an error in function *qemu_ram_resize* while
checking size of ovmf_pcbios: *Length mismatch: pc.bios: 0x200000 in != 0x400000: Invalid argument.*
** We want to know how to solve this problem after updating the version of edk2.
Thank you.

Chenye Wu                                                                                                                                                                                                                                                             2020.2.12

Re: Filter non-UEFI drives

Tim Crawford

Thanks Ashish, that looks useful for keeping some of our custom logic out of MdeModulePkg. I've added it to my list of things to do for rebasing on edk2 master.

Re: Lock BootOrder variable

Bret Barkelew <bret.barkelew@...>

Sunny,

Have you seen the work that we've proposed on the VariablePolicy RFC (in the devel and rfc mailing groups)? Would that work for your purposes?

Thanks!
- Bret

Re: Lock BootOrder variable

Wang, Sunny (HPS SW)

Hi Lazlo,

Thanks for the comments and sorry for the delay. I was just back from my long vacation and was busy with some other stuff.

Yeah, locking BootOrder is not acceptable, so we would like to override or ignore the BootOrder change made by OS or non-trusted software at the next boot to protect and recover the BootOrder. In other words, locking variables will definitely not be used as a solution. We just want OS vendors to gracefully deal with all the error status from the runtime variable service. However, if you think RuntimeServicesSupported is good and clear enough for OS vendors to gracefully deal with this, I'm totally fine with not adding another description into UEFI specification.

As for the solution, you're also right about that it should be implemented as a platform code. However, since the system may have more UEFI variables (not only BootOrder) that are identified as critical data and needed to be protected as well, we would like to propose a platform variable hook library to add the hook function calls into the variable driver for IBV/OEM like us to implement the platform protection code.

Moreover, before submitting my patches, I think it would be better to further discuss this at the design meeting, so I will check this with Ray to find an available time slot and then present the details to you guys. The purpose of our proposal will be to have platform libraries for IBV/OEM to implement their code for complying with the NIST 800-193 and other security guidelines mentioned in the previous emails (Supporting the UEFI Variable Resiliency (protection, detection, and recovery)).

Regards,
Sunny Wang

-----Original Message-----
From: discuss@edk2.groups.io [mailto:discuss@edk2.groups.io] On Behalf Of Laszlo Ersek
Sent: Saturday, January 4, 2020 1:48 AM
To: discuss@edk2.groups.io; Wang, Sunny (HPS SW) <sunnywang@...>; sean.brogan@...; tim.lewis@...; phlamorim@...; Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@...>
Cc: Spottswood, Jason <jason.spottswood@...>; Haskell, Darrell <darrell.haskell@...>; Wiginton, Scott <scott.wiginton@...>; Javier Martinez Canillas <javierm@...>; Peter Jones <pjones@...>
Subject: Re: [edk2-discuss] Lock BootOrder variable

On 12/18/19 11:03, Wang, Sunny (HPS SW) wrote:
Hi Sean,

Thanks for checking this further and bringing this to the Microsoft OS team.

Yeah, your guess is part of the solution that I'm working on. I'm making a spec'd way to fix this and will bring it to USWG and EDK2 design meeting to further discuss with you guys.

Moreover, I tried RHEL and SLES as well, and ran into the same problem
(installation failure). Therefore, it will be good if some people from
Linux OS vendors in this email list can also help drive this for Linux
OS. Of course, I will still bring this to USWG for checking if we need
to gracefully deal with the locked variable (and error
Locking the boot order by way of causing SetVariable to fail for the OS seems wrong to me.

BDS is platform policy. A platform BDS implementation can simply re-generate all Boot#### variables, and the BootOrder variable, from scratch, every time the platform boots. Why is that not sufficient? (For example, platform BDS could base that boot order re-generation on a set of boot-time only variables.)

Furthermore, we already have RuntimeServicesSupported, for exposing (among other things) that SetVariable doesn't work, "en bloc". Punching further holes into previously promised interfaces, piece-meal, is becoming quite baroque.

I don't understand how "security" is improved by locking down boot order. We already have Secure Boot for executing trusted binaries only, and we already have TCG2 / TPM2 for tying secrets to any and all aspects of the boot path (such as executables launched, the order they are launched in, their config files, and so on).

Anyway, I've CC'd Javier and Peter from the rhboot team.

Laszlo

-----Original Message-----
From: discuss@edk2.groups.io [mailto:discuss@edk2.groups.io] On Behalf
Of Sean via Groups.Io
Sent: Wednesday, December 18, 2019 4:55 AM
To: Wang, Sunny (HPS SW) <sunnywang@...>; tim.lewis@...;
discuss@edk2.groups.io; phlamorim@...; Samer El-Haj-Mahmoud
<Samer.El-Haj-Mahmoud@...>
Cc: Spottswood, Jason <jason.spottswood@...>; Haskell, Darrell
Subject: Re: [edk2-discuss] Lock BootOrder variable

Sunny,

There isn't UEFI code that is going to resolve the OS failure. I guess you could let the OS write the values and just not use those values but that is probably a bad pattern and would lead to new issues. We have talked with the Microsoft OS team about fixing this but it hasn't been a high priority. Our general guidance is don't lock the boot order until after your have deployed your OS image but there are some servicing issues that can occur.

I'll talk with the boot manager team and see if we can start to fix this in Windows. I'll update this thread if I get an answer.
Does anyone know what the different Linux loader/install process does?

Thanks
Sean

-----Original Message-----
From: Wang, Sunny (HPS SW) <sunnywang@...>
Sent: Friday, December 13, 2019 12:55 AM
To: tim.lewis@...; discuss@edk2.groups.io; Sean Brogan
<sean.brogan@...>; phlamorim@...; Samer
El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@...>
Cc: Spottswood, Jason <jason.spottswood@...>; Haskell, Darrell
Wang, Sunny (HPS SW) <sunnywang@...>
Subject: [EXTERNAL] RE: [edk2-discuss] Lock BootOrder variable

Thanks, guys.

Hi Samer,
Thanks for checking this and giving great information. I just knew that NIST 800-193 has some guidelines for locking down the UEFI boot order. The requirements and guidelines you added further proved the need of locking boot order.

Hi Sean,
Actually, I did check the Project Mu VariablePolicy protocol when you guys proposed this in the design meeting. This is definitely a good enhancement because I also ran into the problems mentioned in slides.
-
INVALID URI REMOVED
rotection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fedk2.groups.io-252F
g-252Fdevel-252Ffiles-252FDesigns-252F2019-252F0516-26amp-3Bdata-3D02-
257C01-257Csean.brogan-2540microsoft.com-257C7a27e3cc7d754208afd108d77
faa2a64-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C637118241
153980473-26amp-3Bsdata-3DxjJYipCUkrC19mz9D0eRqAJYAOB0DGBVK7kgn7fTXuQ-
253D-26amp-3Breserved-3D0&d=DwICaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=Z9cLgEMd
GZCI1_R0bW6KqOAGzCXLJUR24f8N3205AYw&m=uWkd1_Wtepp1js5IwHPeUBVeN78J71Ii
_NieuwsJZk8&s=6O-ZcjoIEhjvpzBGydrRy4RM6BCiLwpTVRmAlkphX-o&e=
However, It looks like we would still run into the OS installation failure with the current VariablePolicy protocol implementation. I didn't deeply look into the VariablePolicy protocol or give it a try, so I may overlook the solution in your implementation. Could you help point out where the code for solving OS installation failure is? The problem here is that OS doesn't gracefully deal with the locked UEFI variable, so we may not be able to return EFI_WRITE_PROTECTED to OS for the writes to BootOrder.

Hi All,
I think we all agree with the following points:
1. There is a need to lock BootOrder like the information brought by Samer and NIST 800-193 guidelines.
2. OS needs to gracefully deal with the locked BootOrder without causing failure during the installation and operations.
3. We will at least need to update the UEFI specification for asking OS to gracefully deal with the locked variable. If no one is currently working on this, I will bring it to USWG and work on it.

Regards,
Sunny Wang

-----Original Message-----
From: tim.lewis@... [mailto:tim.lewis@...]
Sent: Friday, December 13, 2019 10:26 AM
To: discuss@edk2.groups.io; sean.brogan@...;
phlamorim@...; Wang, Sunny (HPS SW) <sunnywang@...>
Subject: RE: [edk2-discuss] Lock BootOrder variable

Sean --

Since you already have published code and it is already publicly available, then I suggest that you bring it to USWG now, rather than later.

Thanks,

Tim

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
Sean via Groups.Io
Sent: Thursday, December 12, 2019 5:31 PM
To: discuss@edk2.groups.io; phlamorim@...; sunnywang@...
Subject: Re: [edk2-discuss] Lock BootOrder variable

Sunny,

There are two other public, non-uefi spec solutions I am aware of.

1. Edk2 VariableLock protocol:
INVALID URI REMOVED
rotection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Ftian
ocore-252Fedk2-252Fblob-252Fmaster-252FMdeModulePkg-252FInclude-252FPr
otocol-252FVariableLock.h-26amp-3Bdata-3D02-257C01-257Csean.brogan-254
0microsoft.com-257C7a27e3cc7d754208afd108d77faa2a64-257C72f988bf86f141
af91ab2d7cd011db47-257C1-257C0-257C637118241153980473-26amp-3Bsdata-3D
avZwx8B1gRbULoRQuFhAldnvYAe20QFzyhG802LRzcg-253D-26amp-3Breserved-3D0&
d=DwICaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=Z9cLgEMdGZCI1_R0bW6KqOAGzCXLJUR24f
8N3205AYw&m=uWkd1_Wtepp1js5IwHPeUBVeN78J71Ii_NieuwsJZk8&s=ACauqOvz9A5G
8S3vH-bALwc4RPyfS1u8O2sza1_9Hbw&e=
A relatively limited solution with hard coded lock points tied to edk2 SMM variable store.

2. Project Mu VariablePolicy protocol:
INVALID URI REMOVED
rotection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Fmicr
osoft-252Fmu-5Fbasecore-252Fblob-252Frelease-252F201911-252FMdeModuleP
kg-252FInclude-252FProtocol-252FVariablePolicy.h-26amp-3Bdata-3D02-257
C01-257Csean.brogan-2540microsoft.com-257C7a27e3cc7d754208afd108d77faa
2a64-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C637118241153
980473-26amp-3Bsdata-3D14Wv-252B8VbDcwwAcuDQ2rG0V8g561YSDw9By26gIFIr8c
-253D-26amp-3Breserved-3D0&d=DwICaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=Z9cLgEM
dGZCI1_R0bW6KqOAGzCXLJUR24f8N3205AYw&m=uWkd1_Wtepp1js5IwHPeUBVeN78J71I
i_NieuwsJZk8&s=xjXKoZxYjZwyWJrEHGS5jayMp6HNXhombPuvq7NNM8g&e=
Flexible policy based locking that can be implemented in various hardware architectures.

My team will be proposing the VariablePolicy protocol (potentially as a "code-first" effort) in the coming months and working to upstream this feature into edk2. The reality is some users and use cases want higher assurance for their platform settings and this can include the boot order. Doing this thru a well-defined and auditable protocol is better than an ad-hoc solutions. As you know locking some variables may break assumptions (or spec definition) that other code may have but that tradeoff is best evaluated by the use case.

Thanks
Sean

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
Paulo Henrique Lacerda de Amorim via Groups.Io
Sent: Thursday, December 12, 2019 12:53 PM
To: discuss@edk2.groups.io; sunnywang@...
Subject: [EXTERNAL] Re: [edk2-discuss] Lock BootOrder variable

The UEFI define the BootOrder variable with NV+BS+RT attributes, so its not possible to lock this variable, you can try to delete the BootOrder variable and then set the variable with AT attribute, which will probably will result in an undefined behavior. OVMF just recreates the the BootOrder with NV+BS+RT again, then any code which can call Runtime Services will be able to change BootOrder again.

A possible method is too use a signed loader which have your own 'BootOrder' hardcoded.

-----Original Message-----
From: Samer El-Haj-Mahmoud [mailto:Samer.El-Haj-Mahmoud@...]
Sent: Friday, December 13, 2019 12:30 AM
To: discuss@edk2.groups.io; Wang, Sunny (HPS SW) <sunnywang@...>
Cc: Spottswood, Jason <jason.spottswood@...>; Wiginton, Scott
<scott.wiginton@...>; Bodner, James <james.bodner@...>;
Subject: RE: [edk2-discuss] Lock BootOrder variable

Sunny,

Looks like this is a Windows Hardware Compatibility Specification
Systems.pdf, under System.Fundamentals.Security.DGCG.DeviceGuard -
Firmware BIOS lockdown

I am aware of some proprietary implementations of "Lock Boot Order" as a firmware/UEFI setting, but not a standard method defined in the spec.

Also, NSA has some guidelines on locking down UEFI boot order using whatever firmware settings to disable any undesired boot sources (such as externally available USB or network ports): https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__www.nsa.gov_Portals_70_documents_what-2Dwe-2Ddo_cybersecurity_professional-2Dresources_csi-2Duefi-2Dlockdown.pdf%26d%3DDwIFAg%26c%3DC5b8zRQO1miGmBeVZ2LFWg%26r%3DZ9cLgEMdGZCI1_R0bW6KqOAGzCXLJUR24f8N3205AYw%26m%3DasfixhAVUFNND_WEH4KyE0iVEY8HgOOvdPb6NrNwOUQ%26s%3D-rAbAKmK6iJFSGCadTldga_88zXzHJY2rz7Zmyh_lSM%26e&;data=02%7C01%7Csean.brogan%40microsoft.com%7C7a27e3cc7d754208afd108d77faa2a64%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637118241153980473&amp;sdata=3F6hFgqkkXYyeqS%2BiHp2TfYNGK9J82AZdWwX1mZNc5w%3D&amp;reserved=0= .

Thanks,

Em 12/12/2019 05:49, Wang, Sunny (HPS SW) escreveu:
Hi All,

Is there any spec'd way that we can use to lock some UEFI variables like BootOrder without breaking OS installation and OS functionalities?

For some security reasons and customer use cases, we need to let system firmware completely own some UEFI variables like BootOrder. In other words, we don't want some UEFI variables to be controlled by the OS using the UEFI runtime service SetVariable. In addition, we tried to lock the BootOrder variable, but it would break OS installation or some OS functionalities.

By the way, we will bring this need to USWG if there is no existing spec'd way for satisfying this need.

Regards,
Sunny Wang

Re: Filter non-UEFI drives

Ashish Singhal

Hello Tim,

I had a similar issue some time back and after discussion with edk2 community, I added a new protocol to MdeModulePkg that allows platform-specific hooks to take care of this. Please refer to commit 972d88726410e21b1fff1a528854202c67e97ef1 to see what has been added.

In your implementation of RefreshAllBootOptions api, you can parse the auto enumerated boot options (generated by BmEnumerateBootOptions()) and return back only the ones which you want to use.

Thanks
Ashish

On Wed, Feb 5, 2020 at 02:59 PM, Tim Crawford wrote:

Hi all,

I'm looking for a way to filter boot options to only those that contain EFI
volumes. Right now, testing in QEMU, I see things like empty CD-ROM drives and
blank USB drives as boot options.

In IntelFrameworkModulePkg, I accomplished this by adding a call to
BdsLibGetBootableHandle() in BdsLibEnumerateAllBootOption(). If I didn't get
a handle from the device, I skipped building a boot option for it.

In MdeModulePkg, this seems to correspond to BmEnumerateBootOptions(). But I'm
unsure of how to filter out non-EFI drives here.
in BmLoadOption.c sounds promising for what I want.)

Thanks,
Tim

Filter non-UEFI drives

Tim Crawford

Hi all,

I'm looking for a way to filter boot options to only those that contain EFI
volumes. Right now, testing in QEMU, I see things like empty CD-ROM drives and
blank USB drives as boot options.

In IntelFrameworkModulePkg, I accomplished this by adding a call to
BdsLibGetBootableHandle() in BdsLibEnumerateAllBootOption(). If I didn't get
a handle from the device, I skipped building a boot option for it.

In MdeModulePkg, this seems to correspond to BmEnumerateBootOptions(). But I'm
in BmLoadOption.c sounds promising for what I want.)

Thanks,
Tim

Build Error - ModuleNotFoundError with build.py

Satoshi Tanda

Hi,

I am unable to complete the "build" command successfully due to the import
error. Can someone point me where to look into?

C:\edk2>build
Traceback (most recent call last):
File "C:\edk2\BaseTools\Source\Python\build\build.py", line 29, in
<module>
from Common.buildoptions import BuildOption,BuildTarget
ModuleNotFoundError: No module named 'Common'

I am trying to follow the build instructions for Windows here, and the
commit I try to work on is edk2-stable201911.
https://github.com/tianocore/tianocore.github.io/wiki/Windows-systems

>git log -1
edk2-stable201911, tag: edk2-stable201911)

Required software and Conf\target.txt are setup like this.

>set NASM_PREFIX
NASM_PREFIX=C:\Users\tanda\AppData\Local\bin\NASM\

>C:\ASL\iasl.exe -v
...
ASL+ Optimizing Compiler/Disassembler version 20191018

>set PYTHON_HOME
PYTHON_HOME=C:\Python
>python --version
Python 3.8.1

---
ACTIVE_PLATFORM = MdeModulePkg/MdeModulePkg.dsc
TOOL_CHAIN_TAG = VS2019
---

Then, I run

>edksetup.bat Rebuild
>edksetup.bat
>build

Both edksetup runs end without any errors.

At some point before I was able to build things without the issues. I
suspect some Python setup changes since then caused this but cannot spot
what the cause and how to resolve this.

Best,
Satoshi

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Nope, I’m building for X64 in my targets.txt file. Running objdump on the file returns "craftos-efi/CraftOS.efi: file format pei-x86-64" so it’s definitely an X64 image.

On Jan 31, 2020, at 10:34 AM, Laszlo Ersek <lersek@...> wrote:

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

Laszlo Ersek

On 01/26/20 01:00, JackMacWindows wrote:
I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.
Dumb question but it's worth a shot: are you perchance building both
your application and OVMF for IA32, but trying to run your IA32
application binary on an X64 physical platform? That would give you
EFI_UNSUPPORTED, from the platform's LoadImage() boot service.

Laszlo

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Alright, I’ve done some testing on using different build configurations. Here is what I have done so far:
I reverted my build system back to UDK 2015.
I tried building my version of Lua (5.1.5) using the same configuration files as are in the Lua included with StdLib. This worked fine.
I removed all #includes from LibC that were not in the Lua source. These included dirent.h, libgen.h, and stat.h.
I moved the Lua dependency into its own library file rather than compiling it with the original source.
I used the EFI Shell 2.1 that ships with UDK 2015 instead of the shell that comes with the latest EDK. I also tried the shell built into rEFInd.
I built the application using Visual Studio 2019 on Windows instead of my GCC cross-compiler.
None of the steps I have taken have fixed the issue. Also, something to note is that I’m overriding the ShellCEntryLib function (https://github.com/MCJack123/craftos-efi/blob/master/efi.c) because my application needs access to the raw ConIn/ConOut handles. While doing this I added a simple print statement at the start of the function. This code does not run at all: it just errors out before the entry point, it seems. I would like to be able to switch to using the actual ShellCEntryLib library, but the SystemTable is an absolute necessity to the functioning of the program. I’m thinking this may be some internal issue inside EDK since it doesn’t even get to the start function before crashing immediately. Maybe it could be something with the executable format not being built correctly?

If anyone has the time to look over my code, it is currently available at https://github.com/MCJack123/craftos-efi as I mentioned in the OP. A downloadable EFI file & disk image are available in the Releases tab for testing (though this is an older build without the aforementioned modifications).

On Jan 28, 2020, at 12:56 AM, JackMacWindows <jackmacwindowslinux@...> wrote:

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@...> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Saving data structure at Pre EFI Initialization phase in memory to use it at DXE phase by some UEFI application?

sergestus@...

Is it possible to save SYSHOST data structure at PEI phase in memory at some default location to use it at DXE phase by some UEFI application?

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

JackMacWindows <jackmacwindowslinux@...>

Thanks for your feedback. I tried running the application using old builds of OVMF back to 2.31 (which is the same version as my testing laptop), and it still worked in the VM. Perhaps there’s some differences in OVMF that aren’t compatible with stock UEFI firmwares? Also, supposing it is an issue with missing protocols, is there any way I could figure out what it’s looking for that’s not there?

I forgot to note that I’m using the November 2019 build of EDK II to build. I’ll try using the UDK 2018 or earlier to see if that might fix it.

On Jan 27, 2020, at 4:54 PM, Nate DeSimone <nathaniel.l.desimone@...> wrote:

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

alejandro.estay@...

Taking Nate DeSimone point. try to put some assertions to check what protocol is missing. Also loading a different shell, or loading shell instance as necessary can be a way to check what is actually happening. Check LibC code to see what protocols are added. Put some prints there. I know this can be a exahustive task, but apparently there's no other way you can check from where that "EFI_UNSUPPORTED" is coming, except perhaps a source level debug instance, but I don{t know how this can be done in a non-tianocore machine.

Re: Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

Nate DeSimone

Probably the version of the UEFI shell you are using to launch the lua interpreter is newer and/or older than the version of edk2 you are using to compile the lua interpreter.

The LibC in AppPkg does a lot of runtime dynamic linking with the UEFI shell using EFI protocols. This is because several things that libc expects the OS to provide such as command line arguments (argc, argv) and relative filesystem paths do not exist in the baseline UEFI spec. The UEFI shell fills in most of the missing pieces.

My shot in the dark guess is that EFI_UNSUPPORTED is coming back because one of the EFI protocols that LibC is looking for during startup is missing.

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of JackMacWindows
Sent: Saturday, January 25, 2020 4:00 PM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Running LibC application from shell causes "Command Error Status: Unsupported" on real hardware, works in VM

jackmacwindowslinux@...

I’m writing an application for UEFI that provides a Lua shell for users to interact with. I have successfully gotten the code to run inside QEMU with OVMF, but any time I try to run it on real hardware I get "Command Error Status: Unsupported" from the shell. The testing machine I’m using is an Ivy Bridge Dell laptop with UEFI 2.31, but it also didn’t work when testing it on a Kaby Lake laptop with UEFI 2.60. The build files (.inf, .dsc) are almost exactly the same as the Lua 5.2 example listed in AppPkg in LibC. I have added print statements around the program, but none of them appear, including one at the absolute beginning of the entry point. (I overrode ShellCEntryLib to do this.) This indicates that the error is likely relating to linking/executable generation. I’m able to compile and run other applications on real hardware, including the shell as well as Main and Lua in AppPkg.

I have absolutely no idea what I can do to debug this further, as everything I have done to try to fix this has failed. Does anybody have any idea what could be going wrong? The source code can be found at https://github.com/MCJack123/craftos-efi.

Re: dirent always returns NULL when opening a directory, errno is 0

Laszlo Ersek

On 01/24/20 09:52, Jack Bruienne wrote:
I’m testing out running some code under the EFI environment to learn about writing EFI applications. I’m trying to list files in a directory on the filesystem using the dirent API provided with LibStd in EDK II. I have successfully compiled the code and linked with all required libraries, but I keep getting an issue where opendir() is always returning NULL. When checking the value of errno, it’s still EOK anyway. Here’s the code I’m using:

struct dirent *dir;
char * path = fixpath(old_path); // converts "/path/to/dir" to "FS0:\path\to\dir"
DIR * d = opendir(path);
if (d) {
for (int i = 0; (dir = readdir(d)) != NULL; i++) {
// process file entries
}
closedir(d);
} else err(path, "Not a directory");

When this code is run, it ends up calling err() and showing a message on the screen ("FS0:\: Not a directory (0)") instead of processing the file entries in the dirent structure. Are there any known bugs with the EDK II implementation of dirent? If so, how can I rewrite my code to use native EFI calls instead of dirent?

I am using the stable201911 version of EDK II with the latest edk2-libc, and I'm compiling under macOS using a Linux cross-compilation toolchain. The built EFI application is run under QEMU x86_64 with an OVMF firmware built with the same toolchain.
I think the fixpath() call is superfluous and wrong. As far as I
remember, when programming against edk2-libc, edk2-libc places some
directory structure requirements on the underlying UEFI filesystem.
Therefore manual conversion of pathnames should not be necessary;
instead it is on the provider of the underlying UEFI filesystem that
needs to place files in the right spots.

Let me see... Yes, please refer to section "TARGET-SYSTEM INSTALLATION"
in "StdLib/ReadMe.txt", in the edk2-libc project (with master @
61687168fe02).

It seems that, using StdLib, you can only refer to files that are on the
same filesystem that the executable was loaded from.

Thanks
Laszlo

dirent always returns NULL when opening a directory, errno is 0

Jack Bruienne

I’m testing out running some code under the EFI environment to learn about writing EFI applications. I’m trying to list files in a directory on the filesystem using the dirent API provided with LibStd in EDK II. I have successfully compiled the code and linked with all required libraries, but I keep getting an issue where opendir() is always returning NULL. When checking the value of errno, it’s still EOK anyway. Here’s the code I’m using:

struct dirent *dir;
char * path = fixpath(old_path); // converts "/path/to/dir" to "FS0:\path\to\dir"
DIR * d = opendir(path);
if (d) {
for (int i = 0; (dir = readdir(d)) != NULL; i++) {
// process file entries
}
closedir(d);
} else err(path, "Not a directory");

When this code is run, it ends up calling err() and showing a message on the screen ("FS0:\: Not a directory (0)") instead of processing the file entries in the dirent structure. Are there any known bugs with the EDK II implementation of dirent? If so, how can I rewrite my code to use native EFI calls instead of dirent?

I am using the stable201911 version of EDK II with the latest edk2-libc, and I'm compiling under macOS using a Linux cross-compilation toolchain. The built EFI application is run under QEMU x86_64 with an OVMF firmware built with the same toolchain.

Re: EFI Group Event Callback Order

Li, Walon

Thanks for your explanation, it's very clear.
Is it possible to update spec that makes unspecified behavior to specified? I know the implementation is in accordance with spec but add some comments can make programmer use this mechanism flexible.

Thanks!
Walon

-----Original Message-----
From: Laszlo Ersek [mailto:lersek@...]
Sent: Saturday, January 18, 2020 2:19 AM
To: Li, Walon <walon.li@...>; discuss@edk2.groups.io
Cc: Wei, Kent (HPS SW) <kent.wei@...>; Wang, Sunny (HPS SW) <sunnywang@...>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@...>; Spottswood, Jason <jason.spottswood@...>; Shifflett, Joseph <joseph.shifflett@...>; Haskell, Darrell <darrell.haskell@...>; Wiginton, Scott <scott.wiginton@...>
Subject: Re: [edk2-discuss] EFI Group Event Callback Order

On 01/17/20 14:24, Li, Walon wrote:
Laszlo,

Yes, we can specific TPL to adjust callback order. But sometimes, an
event has many callbacks like as ReadyToBoot and TPL level may not
satisfy purpose. In UEFI spec CreateEvent chapter, also mentioned "The
functions in these queues are invoked in FIFO order". So, if it define
clearly in UEFI spec, LIFO or FIFO, we may use driver dependency to
decide which driver's event registration priority and specific
callback order indirectly.
The FIFO order refers to something else. Here's a larger citation:

When the event is signaled, firmware changes its state to "signaled"
and, if EVT_NOTIFY_SIGNAL is specified, places a call to its
notification function in a FIFO queue. There is a queue for each of
the "basic" task priority levels defined in Section 7.1
(TPL_CALLBACK, and TPL_NOTIFY). The functions in these queues are
invoked in FIFO order, starting with the highest priority level
queue and proceeding to the lowest priority queue that is unmasked
by the current TPL.

It means that, within a given TPL, the notification function invocation order will reflect the enqueueing (= signaling) order.

But the spec does not specify the *signaling* order for such events that belong to the same event group, when one of those events is signaled.
The spec says,

Event groups are collections of events identified by a shared
EFI_GUID where, when one member event is signaled, all other events
are signaled and their individual notification actions are taken (as
described in CreateEvent). All events are guaranteed to be signaled