Date   

Re: Can DXE drivers be included in executable EFI binary?

joseph@...
 

It was very helpful.
Thank you for your kindness!


Re: Can DXE drivers be included in executable EFI binary?

Laszlo Ersek
 

On 02/23/21 02:22, joseph via [] wrote:
Hello Laszlo,

It has not been checked in detail, but my predictions are this.
1. When a driver is loaded, it calls "InstallMultipleProtocolInterfaces" within that driver.
This applies only to drivers that do not follow the UEFI Driver Model
(considering the protocols that expose the driver's actual business).

NvmExpressDxe is a UEFI_DRIVER that follows the UEFI Driver Model. In
its entry point (i.e., when it is loaded), it does not install BlockIo.
It installs BlockIo only when it binds PciIo, generally due to some
ConnectController() calls.

2. In this process, "CoreNotifyProtocolEntry" is finally called.
Protocol installations always have to trigger notifications that have
been registered with the RegisterProtocolNotify() boot service, but
RegisterProtocolNotify() is again something that a driver following the
UEFI Driver Model would not use. RegisterProtocolNotify() is generally
used by DXE_DRIVER modules and UEFI_DRIVERs that don't follow the UEFI
Driver Model.

I thought this was the cause of the problem solve, so I thought I should "Notify" the filesystem protocol.


The current situation is like this. This is the mapping table in the EFI Shell right after booting:
FS0: ... PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)/HD(1,MBR,...)
BLK0: PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)
BLK1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)
After `load (anything filesystem driver).efi` and `map -u`:
FS0: ... PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)/HD(1,MBR,...)
BLK0: PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)
BLK1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)
FS1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)/HD(1,GPT,.....) <= The newly detected one.
Please see the shell spec on the "load" command:

This command loads an driver into memory. It can load multiple files
at one time, and the file name supports wildcards.

If the -nc flag is not specified, this command will try to connect
the driver to a proper device; it may also cause already loaded
drivers be connected to their corresponding devices.

So my impression is that your BDS might not connect the partition driver
(PartitionDxe) to the SATA disk's BlockIo / DiskIo protocol interfaces
[*]. But when you load a filesystem driver, even that kind of connection
could be made.

[*] see e.g. commit b82802b83f069 for the SATA-related protocol stack,
up to and including BlockIo

Laszlo


Re: Compile EDK2 to set boot order to PXE

Werner Buck <wernerbuck@...>
 

I believe the way Raspberry Pi defaults to network boot is actually by
updating the boot options and issuing a reboot, per:
https://github.com/tianocore/edk2-platforms/commit/8dd78ea11a38d2d7e8031c4783e9f2ca5569956b

That was indeed the problem. I changed the RPI4 platform code to always
connect to devices and regenerate boot-options on every boot, and not only
when boot options have failed:
https://github.com/tianocore/edk2-platforms/commit/f88bff4d000c69b1135969dbeafccaeb782dcdf3

@Samer, could this option be the default? Or can this be a compile flag? I
imagine the reasons for the current workflow were to speed up booting and
not process four networkboot options until finally hitting the SD card.

In any case, the goal is complete, there is full headless automated
provisiong of RPI with UEFI firmware.
Thank you all for your assistance.

Kind regards,

Werner Buck

Op za 6 feb. 2021 om 23:59 schreef Pete Batard <pete@akeo.ie>:

On 2021.02.06 16:01, Werner Buck wrote:
It is exactly as Micheal Brown put it. Thank you for all your thoughts
into this!

> 1. UEFI firmware defaults to attempting a network boot if no SD card
is
present. This would allow the UEFI firmware to load iPXE via TFTP.
Actually this is what is surprising to me. When RPI4 UEFI is running it
tries to boot `efi/boot/bootaa64.efi` as a first option but as the uefi
is running from network boot is unable to find it. I would expect the
firmware to then choose network boot but the pi actually does a reboot
loop.

I believe the way Raspberry Pi defaults to network boot is actually by
updating the boot options and issuing a reboot, per:

https://github.com/tianocore/edk2-platforms/commit/8dd78ea11a38d2d7e8031c4783e9f2ca5569956b

In other words, we currently rely on hitting
PlatformBootManagerUnableToBoot() once, so that we update the boot
options, and then reboot the system into an environment that now has
network boot as default.

You can validate this by simply extracting a newly built UEFI firmware
(such as the one from the official archives) on USB/SD and powering up.
You'll see that it takes one reboot, before it starts to look for
network files.

But of course, that only works if you can save variables... which you
can't do in this scenario. If you are downloading the same "pristine"
UEFI firmware over and over, you will indeed hit a boot loop.

CCing Samer, who might have looked at alternatives to enable network
boot by default, and could have some ideas of how we may achieve this,
in a scenario where we can't save boot variables...

Regards,

/Pete

Mind that this is all without an SD card present currently. Should I
raise a bug for this at the RPI4 platform side?

I am actually doing triple network boot here. And have actually had 0
reliability problems with network booting at least on the RPI4 with DHCP
ISC server with the below config and having dnsmasq serve an tftp server.
Just for transparency this is what im running in an isc dhcp server:

```
# network boot coming in
if substring (option vendor-class-identifier,0,9) = "PXEClient" {
option tftp-server-name "10.10.10.1";
# raspberry pi defaults to loading bootcode.bin from tftp and i
have the rpi4 uefi firmware just extracted on the tftproot.
# when booted using uefi and starting network boot again
architecture is 00:0b (uefi arm64) and we download iPXE
if option arch = 00:0b {
log(info, "This is UEFI ARM boot (raspberry pi), booting
ipxe..");
#specially compiled ipxe with 8gb
filename "ipxe-pi.efi";
}
}
# Detecting ipxe https flag so we point filename to the boot.ipxe
filename.
if exists ipxe.https {
log (info, "ipxe enabled");
# matchbox server
filename "http://10.10.10.1:8085/boot.ipxe
<http://10.10.10.1:8085/boot.ipxe>";
}
```

Packing iPXE in some way and tricking UEFI with the PCD file GUID is
what im going to try this weekend!

Thanks all I will keep you posted.

Werner Buck


Op do 4 feb. 2021 om 13:54 schreef Laszlo Ersek <lersek@redhat.com
<mailto:lersek@redhat.com>>:

On 02/04/21 13:35, Michael Brown wrote:
> On 04/02/2021 08:56, Laszlo Ersek wrote:
>> I've attempted to wrap my brain around the initial report in this
>> thread, and honestly I'm lost. My standard answer would be "BDS
behavior
>> is platform policy, so look into whatever PlatformBootManagerLib
>> instance is used by the RPi firmware" (and I understand that
Ard's
>> response is somehow consistent with this). But honestly I can't
figure
>> out whether the problem is related to the part of the RPi
firmware that
>> runs *before* TianoCore. What exactly is PXE-booted before what
else?...
>> I think we'd need a much more detailed issue report here.
>
> My best guess (based on having tried to do something similar
before) is:
>
> - Pi has no local storage.
>
> - Pi uses its builtin network boot (running on the VC4 GPU, not
the CPU)
> to download bootcode.bin, config.txt, and an appropriate
start*.elf via
> TFTP.
>
> - The downloaded config.txt includes "armstub=RPI_EFI.fd". The
VC4 GPU
> (now executing start*.elf, as far as I am aware) downloads
RPI_EFI.fd
> and boots the ARM CPU into this.
>
> - At this point, we have the CPU running the RPi UEFI firmware.
>
> There is no non-volatile storage available

O_o

Thank you for the explanation. Not that I can help Werner in any way,
but now I understand the issue at least.

Thanks,
Laszlo

> and so no way to use EFI
> variables to control the boot order. The boot device will be
whatever
> is the *compile-time* default for RPI_EFI.fd.
>
> The intention is that the UEFI firmware should then attempt a
network
> boot. This could be done in at least three ways:
>
> 1. UEFI firmware defaults to attempting a network boot if no SD
card is
> present. This would allow the UEFI firmware to load iPXE via
TFTP.
>
> 2. iPXE as a driver (specifically bin-arm64-efi/rpi.efidrv) is
embedded
> within the RPI_EFI.fd image, and UEFI firmware defaults to
attempting a
> network boot if no SD card is present.
>
> 3. iPXE as an application (specifically bin-arm64-efi/rpi.efi) is
> embedded within the RPI_EFI.fd image, and UEFI firmware somehow
defaults
> to running this application (e.g. via the PCD file GUID trick
suggested
> by Ard).
>
> All three of these options would result in the RPi running UEFI
iPXE as
> requested.
>
> Werner: please correct me if I have misunderstood what you are
trying to
> do.
>
> Thanks,
>
> Michael
>


Re: Can DXE drivers be included in executable EFI binary?

joseph@...
 

Hello Laszlo,

Thank you sincerely!
It works just as you said!

Snippet: https://gist.github.com/jc-lab/e9c019b8f5c220d06fe312c9bf3af350

Thanks
Jospeh.


Re: Can DXE drivers be included in executable EFI binary?

joseph@...
 

Hello Laszlo,

It has not been checked in detail, but my predictions are this.
1. When a driver is loaded, it calls "InstallMultipleProtocolInterfaces" within that driver.
2. In this process, "CoreNotifyProtocolEntry" is finally called.
I thought this was the cause of the problem solve, so I thought I should "Notify" the filesystem protocol.


The current situation is like this. This is the mapping table in the EFI Shell right after booting:
FS0: ... PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)/HD(1,MBR,...)
BLK0: PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)
BLK1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)
After `load (anything filesystem driver).efi` and `map -u`:
FS0: ... PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)/HD(1,MBR,...)
BLK0: PciRoot(0x0)/Pci(0x14,0x0)/USB(0x1,0x0)
BLK1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)
FS1: PciRoot(0x0)/Pci(0x17,0x0)/Sata(0x0,0x0,0x0)/HD(1,GPT,.....) <= The newly detected one.
Thanks
Joseph.


Re: Can DXE drivers be included in executable EFI binary?

Laszlo Ersek
 

On 02/22/21 12:12, joseph via [] wrote:
Hello Laszlo,

Thanks for your answer.

I looked at this a bit more today.
The cause is not that there is no driver, but it seems that Notify does not work properly for the protocol after installing the driver.
I don't understand "installing the driver".

Also, what is "notify" in this context? Do you mean the
RegisterProtocolNotify() boot service? In what context are you trying to
use it?

Immediately after booting, the EFI Shell recognizes NVMe devices but not partitions and filesystems.
If run `map -u` after loading anything filesystem driver (e.g. ext4) the NVMe partition and filesystems are recognized.
The BDS phase is responsible for connecting devices.

BDS is platform policy though. Even if you have a correctly loaded &
functioning NVMe driver, if BDS does not connect NVMe devices for you,
your application will not see them out of the box.

Your application may call ConnectController(). You can use whatever
methods to locate the suitable handles (you can collect PciIo interfaces
and check them for various signs, or you can use LocateDevicePath()
etc). Once you have some candidate handles, you could call
ConnectController().

When you enter the UEFI shell, there's usually a "connect all devices to
all drivers" step. That may be expensive, so it's usually not done for a
normal boot.

Do you know why?
And how can I Notify to gEfiSimpleFileSystemProtocolGuid?
I don't understand this.

Please describe the symptoms better, and also what you're trying to achieve.

Thanks
Laszlo


Immediately after booting,
GetHandleListByProtocol(&gEfiSimpleFileSystemProtocolGuid);
Has a value of 2.

After installing any filesystem driver, its value will be 3.

Thanks.


Re: Can DXE drivers be included in executable EFI binary?

joseph@...
 

Hello Laszlo,

Thanks for your answer.

I looked at this a bit more today.
The cause is not that there is no driver, but it seems that Notify does not work properly for the protocol after installing the driver.

Immediately after booting, the EFI Shell recognizes NVMe devices but not partitions and filesystems.
If run `map -u` after loading anything filesystem driver (e.g. ext4) the NVMe partition and filesystems are recognized.

Do you know why?
And how can I Notify to gEfiSimpleFileSystemProtocolGuid?

Immediately after booting,
GetHandleListByProtocol(&gEfiSimpleFileSystemProtocolGuid);
Has a value of 2.

After installing any filesystem driver, its value will be 3.

Thanks.


Re: Can DXE drivers be included in executable EFI binary?

Laszlo Ersek
 

Dropping <joseph@zeronsoftn.com> from the address list, because the RH
SMTP server keeps telling me that the domain for that email address
(i.e., "zeronsoftn.com") is unidentifiable. Comments below.

On 02/21/21 07:55, joseph via groups.io wrote:
Hello,

Some computers (what I checked was an LG laptop) do not recognize the NVMe SSD as a block device in the EFI program.
Of course, the OS on the NVMe SSD is bootable.
But when I boot with Shellx64.efi in USB flash memory I can't find that disk.
In this case, load NvmExpressDxe and will be able to find the SSD.

So, I want to include NvmExpressDxe in my executable EFI Binary.
Or, after making the multiple files into FV, can make the FV into an executable EFI file?
These are really bad ideas.

"MdeModulePkg/Bus/Pci/NvmExpressDxe/NvmExpressDxe.inf" is a UEFI_DRIVER.
It is supposed to stay resident, and to provide various protocol services.

Your program is (AIUI) a UEFI_APPLICATION module. When it exits, it's gone.

This shows that, what you're considering, would fundamentally break the
UEFI driver model.

If your platform does not provide an NVMe driver, consider distributing
an NvmExpressDxe driver binary (in accordance with its license) on the
same USB disk as your own application. Users can load it or start it
manually, or they can create a Driver#### option too, for auto-loading it.

Laszlo


Can DXE drivers be included in executable EFI binary?

joseph@...
 

Hello,

Some computers (what I checked was an LG laptop) do not recognize the NVMe SSD as a block device in the EFI program.
Of course, the OS on the NVMe SSD is bootable.
But when I boot with Shellx64.efi in USB flash memory I can't find that disk.
In this case, load NvmExpressDxe and will be able to find the SSD.

So, I want to include NvmExpressDxe in my executable EFI Binary.
Or, after making the multiple files into FV, can make the FV into an executable EFI file?

Thanks.


Re: Questions about unusual PCI location when using EFI_PCI_IO_PROTOCOL in ARM system

gustavo.marcello@...
 

Hi Laszlo Ersek,
Thanks for the style comments, I appreciate it, I will do these changes.

And, for the questions, I agree, for the first question, that seems to be a platform driver bug and, for the second one, it seems something related to the platform as you said. I will try to contact the manufacturer to clarify if it's correct.

Thanks so much for your reply, it was very helpful!


Re: Questions about unusual PCI location when using EFI_PCI_IO_PROTOCOL in ARM system

Laszlo Ersek
 

On 02/16/21 14:41, gustavo.marcello via groups.io wrote:
Hello,

I am learning about PCI and trying to write an application that lists PCI controllers in UEFI environment, using EFI_PCI_IO_PROTOCOL and EDK2. I want to list some information from Configuration Space Header and the Bus/Device/Function location for each instance.

The system that I am executing the application is an ARM, which has a Qualcomm
Snapdragon SoC. The code simply locates all handles for EFI_PCI_IO_PROTOCOL and reads some information from each one, as shown in this code snippet:

EFI_GUID gPciIo = EFI_PCI_IO_PROTOCOL_GUID;
EFI_PCI_IO_PROTOCOL *PciIo;
...

// Locate all handles with PciIo protocol
gBS->LocateHandleBuffer(ByProtocol, &gPciIo, NULL, &NoHandles, &Buffer);
Style comment: you should remove "gPciIo", and use the auto-generated
"gEfiPciIoProtocolGuid" variable instead.


// Scan all found handles
for (Index = 0, Count = 0; Index < NoHandles; Index++) {
// Get PciIo protocol for current handle
gBS->HandleProtocol (Buffer[Index], &gPciIo, (VOID**) &PciIo);
// Get some PCI information
PciIo->Pci.Read (PciIo, EfiPciIoWidthUint8, 0x00, 2, &VendorId);
Why not read it with a single EfiPciIoWidthUint16 operation?

Also, for the offset, PCI_VENDOR_ID_OFFSET would be more idiomatic.

...
// Check if VendorId is different from 0xFFFF
...

// Get location
PciIo->GetLocation (PciIo, &Segment, &Bus, &Device, &Function);

// Show PCI information
...
}

The information retrieved from EFI_PCI_IO_PROTOCOL seems to be OK, except the information from GetLocation that some USB controller handles are returning, as follows:

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x80
Function: 0x00

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x81
Function: 0x00

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x82
Function: 0x00

My questions are:

1 - I am not familiar with PCI, but, as far as I know, maximum device number is 32, since it comes from a 5 bits field, am I correct? If so, how can it be 0x80, 0x81 or 0x82, maybe a bug in EFI_PCI_IO_PROTOCOL code or am I missing something?
Yes, I would hazard that this is a bug in your platform's PCI host
bridge driver. (I assume they use the PCI bus driver from edk2 without
any modifications.)

Or maybe the MMCONFIG range is misconfigured somehow; not sure.

(I'm reminded of ARI too, which I don't have any practical experience
with, however. ARI seems to permit up to 256 functions, combining the 5
bits from the device number with the 3 bits from the function number.
But a *device* number as high as 0x80 -- and not a function number that
high -- looks a bit fishy still. ARI seems to eliminate the device
number completely.)


2 - Usually , the bus numbering use sequential numbers starting from 0 (afaik again), and since it's a simple small system with few PCI components (it does not have 255 buses), what does the number 0xFF for Bus indicates?
Assuming the bus number refers to a root bridge (and not a bridge behind
a root port, or the downstream port of a switch), the bus number is not
assigned by the PCI bus driver. It's just a platform trait. I don't
think it's necessarily a bug.

Hope this helps; I could only make some guesses.
Laszlo



PS: "pci" command in UEFI Shell do not show these USB controllers but "devtree" command shows these USB contollers with the same Device/Funcion numbers.

Thanks so much!





Questions about unusual PCI location when using EFI_PCI_IO_PROTOCOL in ARM system

gustavo.marcello@...
 

Hello,

I am learning about PCI and trying to write an application that lists PCI controllers in UEFI environment, using EFI_PCI_IO_PROTOCOL and EDK2. I want to list some information from Configuration Space Header and the Bus/Device/Function location for each instance.

The system that I am executing the application is an ARM, which has a Qualcomm
Snapdragon SoC. The code simply locates all handles for EFI_PCI_IO_PROTOCOL and reads some information from each one, as shown in this code snippet:

EFI_GUID gPciIo = EFI_PCI_IO_PROTOCOL_GUID;
EFI_PCI_IO_PROTOCOL *PciIo;
...

// Locate all handles with PciIo protocol
gBS->LocateHandleBuffer(ByProtocol, &gPciIo, NULL, &NoHandles, &Buffer);

// Scan all found handles
for (Index = 0, Count = 0; Index < NoHandles; Index++) {
// Get PciIo protocol for current handle
gBS->HandleProtocol (Buffer[Index], &gPciIo, (VOID**) &PciIo);
// Get some PCI information
PciIo->Pci.Read (PciIo, EfiPciIoWidthUint8, 0x00, 2, &VendorId);
...
// Check if VendorId is different from 0xFFFF
...

// Get location
PciIo->GetLocation (PciIo, &Segment, &Bus, &Device, &Function);

// Show PCI information
...
}

The information retrieved from EFI_PCI_IO_PROTOCOL seems to be OK, except the information from GetLocation that some USB controller handles are returning, as follows:

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x80
Function: 0x00

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x81
Function: 0x00

Class: Serial Bus Controller
Subclass: USB Controller
Bus: 0xFF
Device: 0x82
Function: 0x00

My questions are:

1 - I am not familiar with PCI, but, as far as I know, maximum device number is 32, since it comes from a 5 bits field, am I correct? If so, how can it be 0x80, 0x81 or 0x82, maybe a bug in EFI_PCI_IO_PROTOCOL code or am I missing something?

2 - Usually , the bus numbering use sequential numbers starting from 0 (afaik again), and since it's a simple small system with few PCI components (it does not have 255 buses), what does the number 0xFF for Bus indicates?

PS: "pci" command in UEFI Shell do not show these USB controllers but "devtree" command shows these USB contollers with the same Device/Funcion numbers.

Thanks so much!


UEFI-SCT: Support for UEFI-SCT for X86 architecture.

gauravpandya321@...
 

Hi,

I was trying to build the UEFI-SCT for EDK2 on my windows box running cygwin.
I follow edk2-test/uefi-sct/HowToBuild/How to build SCT.txt for build instruction. It says X64 is unsupported target architecture.

Also on CYGWIN i had to mention RVCT as tool chain and RVCT seems to support only ARM.
How can i build UEFI-SCT for EDK2 ?

There are some instruction to build UEFI-SCT for UDK2017:
edk2-test/uefi-sct/HowToBuild/How to build SCT in UDK2017.txt

But i want to build and test EDK2.

Thanks.


Re: Potentially missing CloseProtocol() call

Laszlo Ersek
 

On 02/09/21 13:33, Michael Brown wrote:
On 09/02/2021 11:54, Laszlo Ersek wrote:
On 02/08/21 16:28, Michael Brown wrote:
This reminds me of a very longstanding question I've had around
OpenProtocol(): is there any defined way for something that is an
ordinary consumer (rather than a driver or child controller) to obtain a
long-lived pointer to a protocol interface?
The CloseProtocol() specification has parts as follows (relevant passage
is the last one below):

     The caller is responsible for ensuring that there are no references
     to a protocol interface that has been removed. In some cases,
     outstanding reference information is not available in the protocol,
     so the protocol, once added, cannot be removed. Examples include
     Console I/O, Block I/O, Disk I/O, and (in general) handles to device
     protocols.

<snip>
Thank you!  I see that text in the documentation for the deprecated call
UninstallProtocolInterface(), which explains why I hadn't previously
noticed it.
Right, my apologies. I'm not sure why I wrote CloseProtocol(). Perhaps a
typo, or maybe I lost track of where I was in the PDF with "evince".


My comments:

(1) I don't really understand what the last quoted passage means by
"closing an agent".
Based on what the EDK2 implementation actually does, I'm pretty sure
this means that any opens recorded with attributes of
BY_HANDLE_PROTOCOL, GET_PROTOCOL, or TEST_PROTOCOL are simply discarded:

https://github.com/tianocore/edk2/blob/stable/202011/MdeModulePkg/Core/Dxe/Hand/Handle.c#L659-L674


The code that performed the BY_HANDLE_PROTOCOL, GET_PROTOCOL, or
TEST_PROTOCOL open is not notified in any way.

(2) I've never tested this, nor attempted to verify it from the edk2
source, myself. It does suggest however that GET_PROTOCOL opens are
tracked too, and thus, *not* closing a GET_PROTOCOL open is technically
still a leak.
They are tracked, but as far as I can tell only for informative purposes
(e.g. debugging using the EFI shell "openinfo" command, or with iPXE's
DBG_EFI_OPENERS() call).

So: technically a leak, but a leak that is arguably useful from the
perspective of someone (e.g. quite often me) trying to debug what code
has tried to do something with a particular handle.  And a leak that
will get automatically cleaned up when the protocol is uninstalled.


But this still leaves my original problem: there is no way for code to
ever know that a protocol it opened with GET_PROTOCOL has ceased to be
valid, and nothing that stops said code from dereferencing the
potentially invalid pointer.
That seems to be the case, yes.

As far as I can tell, the only way to be notified when a protocol
pointer ceases to be valid is to open BY_DRIVER, and we can't open some
protocols (such as EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL) with BY_DRIVER
attributes because there will be multiple openers of that protocol.
Agreed.

Laszlo


Re: Potentially missing CloseProtocol() call

Michael Brown
 

On 09/02/2021 11:54, Laszlo Ersek wrote:
On 02/08/21 16:28, Michael Brown wrote:
This reminds me of a very longstanding question I've had around
OpenProtocol(): is there any defined way for something that is an
ordinary consumer (rather than a driver or child controller) to obtain a
long-lived pointer to a protocol interface?
The CloseProtocol() specification has parts as follows (relevant passage
is the last one below):
The caller is responsible for ensuring that there are no references
to a protocol interface that has been removed. In some cases,
outstanding reference information is not available in the protocol,
so the protocol, once added, cannot be removed. Examples include
Console I/O, Block I/O, Disk I/O, and (in general) handles to device
protocols.
<snip>
Thank you! I see that text in the documentation for the deprecated call UninstallProtocolInterface(), which explains why I hadn't previously noticed it.

My comments:
(1) I don't really understand what the last quoted passage means by
"closing an agent".
Based on what the EDK2 implementation actually does, I'm pretty sure this means that any opens recorded with attributes of BY_HANDLE_PROTOCOL, GET_PROTOCOL, or TEST_PROTOCOL are simply discarded:

https://github.com/tianocore/edk2/blob/stable/202011/MdeModulePkg/Core/Dxe/Hand/Handle.c#L659-L674

The code that performed the BY_HANDLE_PROTOCOL, GET_PROTOCOL, or TEST_PROTOCOL open is not notified in any way.

(2) I've never tested this, nor attempted to verify it from the edk2
source, myself. It does suggest however that GET_PROTOCOL opens are
tracked too, and thus, *not* closing a GET_PROTOCOL open is technically
still a leak.
They are tracked, but as far as I can tell only for informative purposes (e.g. debugging using the EFI shell "openinfo" command, or with iPXE's DBG_EFI_OPENERS() call).

So: technically a leak, but a leak that is arguably useful from the perspective of someone (e.g. quite often me) trying to debug what code has tried to do something with a particular handle. And a leak that will get automatically cleaned up when the protocol is uninstalled.


But this still leaves my original problem: there is no way for code to ever know that a protocol it opened with GET_PROTOCOL has ceased to be valid, and nothing that stops said code from dereferencing the potentially invalid pointer.

As far as I can tell, the only way to be notified when a protocol pointer ceases to be valid is to open BY_DRIVER, and we can't open some protocols (such as EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL) with BY_DRIVER attributes because there will be multiple openers of that protocol.

Michael


Re: Potentially missing CloseProtocol() call

Laszlo Ersek
 

On 02/08/21 16:28, Michael Brown wrote:
On 08/02/2021 14:22, Laszlo Ersek wrote:
gBS->OpenProtocol() calls that pass the EFI_OPEN_PROTOCOL_GET_PROTOCOL
argument for the "Attributes" parameter *need not* be mirrored with
gBS->CloseProtocol() calls.

Please refer to the UEFI spec on the OpenProtocol() boot service,
Attributes=EFI_OPEN_PROTOCOL_GET_PROTOCOL:

     [...] The caller is also not required to close the protocol
     interface with EFI_BOOT_SERVICES.CloseProtocol().

So you *can* call CloseProtocol(), but you don't have to.
This reminds me of a very longstanding question I've had around
OpenProtocol(): is there any defined way for something that is an
ordinary consumer (rather than a driver or child controller) to obtain a
long-lived pointer to a protocol interface?

For example:
ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/MainTextEditor.c seems
to open EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL using just HandleProtocol()
and then continues to use the pointer for the lifetime of the
application.  But, as far as I can tell, there's nothing that guarantees
that this pointer remains valid?
The CloseProtocol() specification has parts as follows (relevant passage
is the last one below):

The caller is responsible for ensuring that there are no references
to a protocol interface that has been removed. In some cases,
outstanding reference information is not available in the protocol,
so the protocol, once added, cannot be removed. Examples include
Console I/O, Block I/O, Disk I/O, and (in general) handles to device
protocols.

[...]

EFI 1.10 Extension

The extension to this service directly addresses the limitations
described in the section above. There may be some drivers that are
currently consuming the protocol interface that needs to be
uninstalled, so it may be dangerous to just blindly remove a
protocol interface from the system. Since the usage of protocol
interfaces is now being tracked for components that use the
EFI_BOOT_SERVICES.OpenProtocol() and
EFI_BOOT_SERVICES.CloseProtocol() boot services, a safe version of
this function can be implemented.

[...]

Lastly, all of the agents that have the protocol interface open with
an attribute of EFI_OPEN_PROTOCOL_BY_HANDLE_PROTOCOL,
EFI_OPEN_PROTOCOL_GET_PROTOCOL, or EFI_OPEN_PROTOCOL_TEST_PROTOCOL
are closed. If there are any agents remaining that still have the
protocol interface open, the protocol interface is not removed from
the handle and EFI_ACCESS_DENIED is returned. In addition, all of
the drivers that were disconnected with the boot service
DisconnectController() earlier, are reconnected with the boot
service EFI_BOOT_SERVICES.ConnectController(). If there are no
agents remaining that are consuming the protocol interface, then the
protocol interface is removed from the handle as described above.

My comments:

(1) I don't really understand what the last quoted passage means by
"closing an agent".

(2) I've never tested this, nor attempted to verify it from the edk2
source, myself. It does suggest however that GET_PROTOCOL opens are
tracked too, and thus, *not* closing a GET_PROTOCOL open is technically
still a leak.

Thanks
Laszlo


Re: Potentially missing CloseProtocol() call

Satoshi Tanda
 

Hi Laszlo,

I was almost exclusively checking edk2 headers but should have checked with
the specs. Thank you for clarifying it for me.

Best,
Satoshi

On Mon, Feb 8, 2021 at 7:28 AM Michael Brown <mcb30@ipxe.org> wrote:

On 08/02/2021 14:22, Laszlo Ersek wrote:
gBS->OpenProtocol() calls that pass the EFI_OPEN_PROTOCOL_GET_PROTOCOL
argument for the "Attributes" parameter *need not* be mirrored with
gBS->CloseProtocol() calls.

Please refer to the UEFI spec on the OpenProtocol() boot service,
Attributes=EFI_OPEN_PROTOCOL_GET_PROTOCOL:

[...] The caller is also not required to close the protocol
interface with EFI_BOOT_SERVICES.CloseProtocol().

So you *can* call CloseProtocol(), but you don't have to.
This reminds me of a very longstanding question I've had around
OpenProtocol(): is there any defined way for something that is an
ordinary consumer (rather than a driver or child controller) to obtain a
long-lived pointer to a protocol interface?

For example:
ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/MainTextEditor.c seems
to open EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL using just HandleProtocol()
and then continues to use the pointer for the lifetime of the
application. But, as far as I can tell, there's nothing that guarantees
that this pointer remains valid?

Michael


Re: Potentially missing CloseProtocol() call

Michael Brown
 

On 08/02/2021 14:22, Laszlo Ersek wrote:
gBS->OpenProtocol() calls that pass the EFI_OPEN_PROTOCOL_GET_PROTOCOL
argument for the "Attributes" parameter *need not* be mirrored with
gBS->CloseProtocol() calls.
Please refer to the UEFI spec on the OpenProtocol() boot service,
Attributes=EFI_OPEN_PROTOCOL_GET_PROTOCOL:
[...] The caller is also not required to close the protocol
interface with EFI_BOOT_SERVICES.CloseProtocol().
So you *can* call CloseProtocol(), but you don't have to.
This reminds me of a very longstanding question I've had around OpenProtocol(): is there any defined way for something that is an ordinary consumer (rather than a driver or child controller) to obtain a long-lived pointer to a protocol interface?

For example: ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/MainTextEditor.c seems to open EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL using just HandleProtocol() and then continues to use the pointer for the lifetime of the application. But, as far as I can tell, there's nothing that guarantees that this pointer remains valid?

Michael


Re: Potentially missing CloseProtocol() call

Laszlo Ersek
 

Hi,

On 02/06/21 19:56, Satoshi Tanda wrote:
There are a number of places that do not call CloseProtocol() while it
appears to be required, in EDK2. Can someone confirm if (some of) those are
indeed errors, or there are actually cases where skipping CloseProtocol()
after OpenProtocol() is appropriate?

Here are the places I would expect a CloseProtocol() call but apparently
missing it.

MdeModulePkg/Universal/DebugSupportDxe/DebugSupport.c
- InitializeDebugSupportDriver().

ShellPkg/Library/UefiShellDriver1CommandsLib/Dh.c
- GetDriverName() / GetDriverImageName()

ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.c
- *ProtocolDumpInformation(). DevicePathProtocolDumpInformationEx() does
call it. So lack of call seems like an error to me.

ShellPkg/Library/UefiShellDriver1CommandsLib/DevTree.c
- DoDevTreeForHandle()

I am new to UEFI and trying to learn basics like how to use OpenProtocol().
I have not observed or encountered any concrete issue.
gBS->OpenProtocol() calls that pass the EFI_OPEN_PROTOCOL_GET_PROTOCOL
argument for the "Attributes" parameter *need not* be mirrored with
gBS->CloseProtocol() calls.

Please refer to the UEFI spec on the OpenProtocol() boot service,
Attributes=EFI_OPEN_PROTOCOL_GET_PROTOCOL:

[...] The caller is also not required to close the protocol
interface with EFI_BOOT_SERVICES.CloseProtocol().

So you *can* call CloseProtocol(), but you don't have to.

Thanks
Laszlo


Re: UEFI Payload Issue

Christian Walter
 

Hi,

we do have a updated fork here: github.com/9elements/edk2

This should be more stable than the current one - also we have at least build testing for the UEFIPayload branch.

Best,

Chris

On 2/6/21 9:56 AM, You, Benjamin wrote:
Hi,

If HPET works but 8254 does not work, further investigation could be on 8254
settings made by Coreboot.

Thanks,

- ben

-----Original Message-----
From: You, Benjamin
Sent: Saturday, February 6, 2021 4:41 PM
To: Ma, Maurice <maurice.ma@intel.com>; Laszlo Ersek <lersek@redhat.com>;
discuss@edk2.groups.io; sent888@gmail.com
Cc: Dong, Guo <guo.dong@intel.com>
Subject: RE: [edk2-discuss] UEFI Payload Issue

Hi,

Yes, timer could be a cause for this hang. Old Payload has the option of
USE_HPET_TIMER - you could try setting this to TRUE to use HPET instead of
8254

Thanks,

- ben

-----Original Message-----
From: Ma, Maurice <maurice.ma@intel.com>
Sent: Thursday, February 4, 2021 11:26 PM
To: Laszlo Ersek <lersek@redhat.com>; discuss@edk2.groups.io;
sent888@gmail.com
Cc: Dong, Guo <guo.dong@intel.com>; You, Benjamin
<benjamin.you@intel.com>
Subject: RE: [edk2-discuss] UEFI Payload Issue

UEFI Payload from EDK2 2017 seems to be pretty old.

Below are some of my recommendations:
- Enable serial debug console in UEFI payload so that you don't depend on USB
KB for input. Once you booted to Shell using serial console, further info can be
collected to root cause USB issue.
- If you saw the hang at "[Bds]BdsWait(3)..Zzzz...", it could be timer related
issue.
I saw some instances where the ACPI timer base was reported incorrectly to
UEFI payload and ACPI timer base was wrong. That would cause the dead
loop
in delay function.
- Try out the latest UefiPayload in EDK2 to see if it works for you.

Regards
Maurice

-----Original Message-----
From: Laszlo Ersek <lersek@redhat.com>
Sent: Thursday, February 4, 2021 1:58
To: discuss@edk2.groups.io; sent888@gmail.com
Cc: Ma, Maurice <maurice.ma@intel.com>; Dong, Guo
<guo.dong@intel.com>; You, Benjamin <benjamin.you@intel.com>
Subject: Re: [edk2-discuss] UEFI Payload Issue

On 02/02/21 13:16, sent888@gmail.com wrote:
Hi,

I have generated a payload from edk2 2017. When i booting with payload i
got struck after the below message. The keyboard is not detecting so i can't
press any key. Mouse got powered up. Even i exchanged the usb ports but
still keyboard not detecting. I am using coreboot and edk2 payload. I am
using Coffeelake processor. If i use the uefi payload binary from Intel.
Everything is working and able to select device in the boot manager and load
OS.
LastBlock : 4FFFFF
[0m [37m [40m
F2 or Down to enter Boot Manager Menu.
ENTER to boot directly.

[Bds]OsIndication: 0000000000000000
[Bds]=============Begin Load Options Dumping ...=============
Driver
Options:
SysPrep Options:
Boot Options:
Boot0000: UiApp 0x0109
Boot0001: UEFI SM659GXC CDZ A0519022617060000001 0x0001
Boot0002: UEFI Shell 0x0001
PlatformRecovery Options:
PlatformRecovery0000: Default PlatformRecovery 0x0001
[Bds]=============End Load Options Dumping=============
[Bds]BdsWait
...Zzzzzzzzzzzz...
[Bds]BdsWait(3)..Zzzz...
Adding the UefiPayloadPkg folks to the CC list.

Thanks
Laszlo


--
*Christian Walter*
*Head of Firmware Development / Cyber Security *



9elements GmbH, Kortumstraße 19-21, 44787 Bochum, Germany
Email: christian.walter@9elements.com
Phone: _+49 234 68 94 188 <tel:+492346894188>_
Mobile: _+49 176 70845047 <tel:+4917670845047>_

Sitz der Gesellschaft: Bochum
Handelsregister: Amtsgericht Bochum, HRB 17519
Geschäftsführung: Sebastian Deutsch, Eray Basar

Datenschutzhinweise nach Art. 13 DSGVO <https://9elements.com/privacy>

281 - 300 of 834