Date   

Re: 回复: [edk2-discuss] compile error

wenyi,xie
 

Hi,Liming:

Our project has several components, EDK2 is one of them. This project has a top makefile, and this top makefile will call sub makefile to build every component.
In the top makefile, it uses make -j4 to build every component include EDK2. So even if we add "option -n 1" to EDK2 build command, it doesn't work.
But if changing -j4 to -j1 in the top makefile, the error will not occur again.

And there's a warning when using make -j4.
Copying $EDK_TOOLS_PATH/Conf/target.template
to /home/phisik3/bios_tt/bios/output/edk2/BaseTools/Conf/target.txt
make[7]:warning: jobserver unvaliable: using -j1. Add '+' to parent make rule.
make[7]:Entering directory '/home/phisik3/bios_tt/bios'

Thanks
Wenyi

On 2021/5/14 10:40, gaoliming wrote:
Wenyi:

Build command has option -n. Can you try build -n 1 to disable multiple thread?

-n THREADNUMBER Build the platform using multi-threaded compiler. The
value overrides target.txt's
MAX_CONCURRENT_THREAD_NUMBER. When value is set to 0,
tool automatically detect number of processor threads,
set value to 1 means disable multi-thread build, and
set value to more than 1 means user specify the

Thanks
Liming
-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie
via groups.io
发送时间: 2021年5月13日 20:05
收件人: Feng, Bob C <bob.c.feng@intel.com>; discuss@edk2.groups.io;
Andrew (EFI) Fish <afish@apple.com>
主题: Re: [edk2-discuss] compile error

The environment is a docker, itself has no kernel, it depends on the kernel of
host machine.

On 2021/5/13 19:29, Feng, Bob C wrote:
Could you try the python3?

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
wenyi,xie via groups.io
Sent: Thursday, May 13, 2021 7:20 PM
To: Andrew (EFI) Fish <afish@apple.com>; discuss@edk2.groups.io
Subject: Re: [edk2-discuss] compile error

And this error only occurred in a specific environment. We trace the commit
log and find this error relate to the feature "BaseTools Enable multiple-thread
autogen".
Maybe this environment doesn't support multi thread well, the root cause is
still unknown.
May I ask whether there's a way to disable this feature, or any other
workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io
<xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a
compile error, the error message like below. we roll back the EDK2 version
and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4
4.15.0-45-generic #86-Ubuntu SMP X86_64 python 2.7.17 Does anyone
know how to solve this compile error.

Building ...
/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/e
dk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf
[AARCH64] Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File
"/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/
edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.







.


回复: [edk2-discuss] compile error

gaoliming
 

Wenyi:

Build command has option -n. Can you try build -n 1 to disable multiple thread?

-n THREADNUMBER Build the platform using multi-threaded compiler. The
value overrides target.txt's
MAX_CONCURRENT_THREAD_NUMBER. When value is set to 0,
tool automatically detect number of processor threads,
set value to 1 means disable multi-thread build, and
set value to more than 1 means user specify the

Thanks
Liming

-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 wenyi,xie
via groups.io
发送时间: 2021年5月13日 20:05
收件人: Feng, Bob C <bob.c.feng@intel.com>; discuss@edk2.groups.io;
Andrew (EFI) Fish <afish@apple.com>
主题: Re: [edk2-discuss] compile error

The environment is a docker, itself has no kernel, it depends on the kernel of
host machine.

On 2021/5/13 19:29, Feng, Bob C wrote:
Could you try the python3?

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
wenyi,xie via groups.io
Sent: Thursday, May 13, 2021 7:20 PM
To: Andrew (EFI) Fish <afish@apple.com>; discuss@edk2.groups.io
Subject: Re: [edk2-discuss] compile error

And this error only occurred in a specific environment. We trace the commit
log and find this error relate to the feature "BaseTools Enable multiple-thread
autogen".
Maybe this environment doesn't support multi thread well, the root cause is
still unknown.
May I ask whether there's a way to disable this feature, or any other
workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io
<xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a
compile error, the error message like below. we roll back the EDK2 version
and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4
4.15.0-45-generic #86-Ubuntu SMP X86_64 python 2.7.17 Does anyone
know how to solve this compile error.

Building ...
/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/e
dk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf
[AARCH64] Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File
"/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/
edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.






Re: compile error

wenyi,xie
 

The environment is a docker, itself has no kernel, it depends on the kernel of host machine.

On 2021/5/13 19:29, Feng, Bob C wrote:
Could you try the python3?

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of wenyi,xie via groups.io
Sent: Thursday, May 13, 2021 7:20 PM
To: Andrew (EFI) Fish <afish@apple.com>; discuss@edk2.groups.io
Subject: Re: [edk2-discuss] compile error

And this error only occurred in a specific environment. We trace the commit log and find this error relate to the feature "BaseTools Enable multiple-thread autogen".
Maybe this environment doesn't support multi thread well, the root cause is still unknown.
May I ask whether there's a way to disable this feature, or any other workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io <xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4
4.15.0-45-generic #86-Ubuntu SMP X86_64 python 2.7.17 Does anyone
know how to solve this compile error.

Building ...
/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64] Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.




Re: compile error

wenyi,xie
 

On 2021/5/13 19:29, Feng, Bob C wrote:
Could you try the python3?
We have tried to set the default version of python to 3.7, but the error is still there.


Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of wenyi,xie via groups.io
Sent: Thursday, May 13, 2021 7:20 PM
To: Andrew (EFI) Fish <afish@apple.com>; discuss@edk2.groups.io
Subject: Re: [edk2-discuss] compile error

And this error only occurred in a specific environment. We trace the commit log and find this error relate to the feature "BaseTools Enable multiple-thread autogen".
Maybe this environment doesn't support multi thread well, the root cause is still unknown.
May I ask whether there's a way to disable this feature, or any other workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io <xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4
4.15.0-45-generic #86-Ubuntu SMP X86_64 python 2.7.17 Does anyone
know how to solve this compile error.

Building ...
/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64] Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.




Re: compile error

Bob Feng
 

Could you try the python3?

Thanks,
Bob

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of wenyi,xie via groups.io
Sent: Thursday, May 13, 2021 7:20 PM
To: Andrew (EFI) Fish <afish@apple.com>; discuss@edk2.groups.io
Subject: Re: [edk2-discuss] compile error

And this error only occurred in a specific environment. We trace the commit log and find this error relate to the feature "BaseTools Enable multiple-thread autogen".
Maybe this environment doesn't support multi thread well, the root cause is still unknown.
May I ask whether there's a way to disable this feature, or any other workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io <xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4
4.15.0-45-generic #86-Ubuntu SMP X86_64 python 2.7.17 Does anyone
know how to solve this compile error.

Building ...
/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64] Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.


Re: compile error

wenyi,xie
 

And this error only occurred in a specific environment. We trace the commit log and find this error relate to the feature "BaseTools Enable multiple-thread autogen".
Maybe this environment doesn't support multi thread well, the root cause is still unknown.
May I ask whether there's a way to disable this feature, or any other workaround for this problem.

Thanks
Wenyi

On 2021/5/12 23:23, Andrew (EFI) Fish wrote:
That is an error in a Python build tool, not a compiler error?

Sent from my iPhone

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io <xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4 4.15.0-45-generic #86-Ubuntu SMP X86_64
python 2.7.17
Does anyone know how to solve this compile error.

Building ... /home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64]
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.




.


Re: compile error

Andrew Fish
 

That is an error in a Python build tool, not a compiler error?

On May 12, 2021, at 2:41 AM, wenyi,xie via groups.io <xiewenyi2=huawei.com@groups.io> wrote:

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4 4.15.0-45-generic #86-Ubuntu SMP X86_64
python 2.7.17
Does anyone know how to solve this compile error.

Building ... /home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64]
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.





compile error

wenyi,xie
 

Hello everyone,

When we update EDK2 version from 201903 to 202011, we meet a compile error, the error message like below. we roll back the EDK2 version and find the error starts from 201908.
Our env is
Linux job-for-start-dev-docker-210-mxzlp-5vjvk-n9nt4 4.15.0-45-generic #86-Ubuntu SMP X86_64
python 2.7.17
Does anyone know how to solve this compile error.

Building ... /home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf [AARCH64]
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/phisik3/jenkins_home/workspace/C_56173/work_code/bios/output/edk2/BaseTools/Source/Python/AutoGen/AutoGenWorker.py", line 85, in run
log_message = self.log_q.get()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
res = self._recv()
UnpicklingError: invalid load key, ''.


Re: Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver' from firmware?

Michael Brown
 

On 07/05/2021 13:08, Laszlo Ersek wrote:
On 05/07/21 09:13, alexey3nemckovich@gmail.com wrote:
I am developing efi application that should work with network.
I found, that some drivers (including 'Realtek UEFI UNDI Driver') are not loaded, when BIOS option 'Fast boot' is on.
I'v tried to load it using EFI_FIRMWARE_VOLUME2_PROTOCOL, enumerating files of type EFI_FV_FILETYPE_DRIVER and loading them, in such way
That will only load drivers that are part of the platform firmware. It
will not load drivers from the disk (which could otherwise be loaded via
Driver#### options), or from the ROM BARs of PCI cards. (The PCI bus
driver installs EFI_LOAD_FILE2_PROTOCOL on the handle that carries
EFI_PCI_IO_PROTOCOL too, for exposing the UEFI driver.) See also
EFI_PCI_IO_PROTOCOL.{RomSize,RomImage}.
... TBH I'm doubtful a UEFI application is supposed to load option ROM
drivers. That's the job of platform BDS. If "fast boot" is selected,
then the system seems to work as intended -- I'm doubtful that running
your application qualifies as "fast boot" either. I'd suggest disabling
fast boot in the first place.
There is also the possibility that the "fast boot" option is implemented by having the Realtek UEFI UNDI Driver itself check for the "fast boot" setting, and refusing to be loaded (i.e. returning an error from the entry point) if fast boot is enabled. I've seen some vendor drivers using that kind of hack.

Michael


Re: Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver' from firmware?

Sean
 

The uefi spec defined method to request the bds connect something is using https://github.com/tianocore/edk2/blob/master/MdePkg/Include/Protocol/BootManagerPolicy.h

We have been trying to move partners away from hacky solutions and to this method.

Thanks
Sean

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: discuss@edk2.groups.io <discuss@edk2.groups.io> on behalf of Laszlo Ersek <lersek@redhat.com>
Sent: Friday, May 7, 2021 9:08:17 PM
To: discuss@edk2.groups.io <discuss@edk2.groups.io>; alexey3nemckovich@gmail.com <alexey3nemckovich@gmail.com>
Subject: Re: [edk2-discuss] Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver' from firmware?

On 05/07/21 09:13, alexey3nemckovich@gmail.com wrote:
I am developing efi application that should work with network.
I found, that some drivers (including 'Realtek UEFI UNDI Driver') are not loaded, when BIOS option 'Fast boot' is on.
I'v tried to load it using EFI_FIRMWARE_VOLUME2_PROTOCOL, enumerating files of type EFI_FV_FILETYPE_DRIVER and loading them, in such way
That will only load drivers that are part of the platform firmware. It
will not load drivers from the disk (which could otherwise be loaded via
Driver#### options), or from the ROM BARs of PCI cards. (The PCI bus
driver installs EFI_LOAD_FILE2_PROTOCOL on the handle that carries
EFI_PCI_IO_PROTOCOL too, for exposing the UEFI driver.) See also
EFI_PCI_IO_PROTOCOL.{RomSize,RomImage}.

... TBH I'm doubtful a UEFI application is supposed to load option ROM
drivers. That's the job of platform BDS. If "fast boot" is selected,
then the system seems to work as intended -- I'm doubtful that running
your application qualifies as "fast boot" either. I'd suggest disabling
fast boot in the first place.

Laszlo


Re: Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver' from firmware?

Laszlo Ersek
 

On 05/07/21 09:13, alexey3nemckovich@gmail.com wrote:
I am developing efi application that should work with network.
I found, that some drivers (including 'Realtek UEFI UNDI Driver') are not loaded, when BIOS option 'Fast boot' is on.
I'v tried to load it using EFI_FIRMWARE_VOLUME2_PROTOCOL, enumerating files of type EFI_FV_FILETYPE_DRIVER and loading them, in such way
That will only load drivers that are part of the platform firmware. It
will not load drivers from the disk (which could otherwise be loaded via
Driver#### options), or from the ROM BARs of PCI cards. (The PCI bus
driver installs EFI_LOAD_FILE2_PROTOCOL on the handle that carries
EFI_PCI_IO_PROTOCOL too, for exposing the UEFI driver.) See also
EFI_PCI_IO_PROTOCOL.{RomSize,RomImage}.

... TBH I'm doubtful a UEFI application is supposed to load option ROM
drivers. That's the job of platform BDS. If "fast boot" is selected,
then the system seems to work as intended -- I'm doubtful that running
your application qualifies as "fast boot" either. I'd suggest disabling
fast boot in the first place.

Laszlo


Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver' from firmware?

alexey3nemckovich@...
 

I am developing efi application that should work with network.
I found, that some drivers (including 'Realtek UEFI UNDI Driver') are not loaded, when BIOS option 'Fast boot' is on.
I'v tried to load it using EFI_FIRMWARE_VOLUME2_PROTOCOL, enumerating files of type EFI_FV_FILETYPE_DRIVER and loading them, in such way

Status = gBS->LocateHandleBuffer(ByProtocol, &gEfiFirmwareVolumeProtocolGuid, NULL, &NoHandles, &Buffer);
if (!EFI_ERROR(Status)) {
for (Index = 0; Index < NoHandles; Index++) {
Status = gBS->HandleProtocol(Buffer[Index], &gEfiFirmwareVolumeProtocolGuid, (VOID **)&Fv);
if (!EFI_ERROR(Status)) {

for (Index2 = 0; Index2 < sizeof(FileTypes)/sizeof(EFI_FV_FILETYPE); Index2++) {

FileType = FileTypes[Index2];

Key = AllocatePool(Fv->KeySize);
ASSERT(Key != NULL);
ZeroMem(Key, Fv->KeySize);

Index3 = 0;

do {
NextStatus = Fv->GetNextFile(Fv, Key, &FileType, &NameGuid, &Attributes, &Size);

if (EFI_SUCCESS == NextStatus/* && Index3 < 50*/) {

Print(L"1\n");

UiSection = NULL;
Status = Fv->ReadSection(
Fv,
&NameGuid,
EFI_SECTION_USER_INTERFACE,
0,
(VOID **)&UiSection,
&Size,
&Authentication
);

Print(L"3\n");

if (!EFI_ERROR(Status)) {

Print(L"4\n");

Print(L"%d) Found driver image '%s'\n", Index3, UiSection);

EFI_DEVICE_PATH_PROTOCOL *DevicePath;
EFI_HANDLE ImageHandle;

DevicePath = FvFileDevicePath(Buffer[Index], &NameGuid);

Status = gBS->LoadImage(FALSE, gImageHandle, DevicePath, NULL, 0, &ImageHandle);
if (!EFI_ERROR(Status)) {
//PERF_END(NULL, "BDS", NULL, 0);

Print(L"%d) Load image success '%s'\n", Index3, UiSection);

Status = gBS->StartImage(ImageHandle, NULL, NULL);

Some drivers were loaded, but there was not 'Realtek UEFI UNDI Driver' among them.
Maybe somebody knows from where I can load this UNDI driver?


Is it possible to manually (from code) load 'Realtek UEFI UNDI Driver'?

alexey3nemckovich@...
 

I am developing efi application that should work with network.
I found, that some drivers (including 'Realtek UEFI UNDI Driver') are not loaded, when BIOS option 'Fast boot' is on.
I'v tried to load it using EFI_FIRMWARE_VOLUME2_PROTOCOL, enumerating files of type EFI_FV_FILETYPE_DRIVER and loading them, in such way

Status = gBS->LocateHandleBuffer(ByProtocol, &gEfiFirmwareVolumeProtocolGuid, NULL, &NoHandles, &Buffer);
if (!EFI_ERROR(Status)) {
for (Index = 0; Index < NoHandles; Index++) {
Status = gBS->HandleProtocol(Buffer[Index], &gEfiFirmwareVolumeProtocolGuid, (VOID **)&Fv);
if (!EFI_ERROR(Status)) {

for (Index2 = 0; Index2 < sizeof(FileTypes)/sizeof(EFI_FV_FILETYPE); Index2++) {

FileType = FileTypes[Index2];

Key = AllocatePool(Fv->KeySize);
ASSERT(Key != NULL);
ZeroMem(Key, Fv->KeySize);

Index3 = 0;

do {
NextStatus = Fv->GetNextFile(Fv, Key, &FileType, &NameGuid, &Attributes, &Size);

if (EFI_SUCCESS == NextStatus/* && Index3 < 50*/) {

Print(L"1\n");

UiSection = NULL;
Status = Fv->ReadSection(
Fv,
&NameGuid,
EFI_SECTION_USER_INTERFACE,
0,
(VOID **)&UiSection,
&Size,
&Authentication
);

Print(L"3\n");

if (!EFI_ERROR(Status)) {

Print(L"4\n");

Print(L"%d) Found driver image '%s'\n", Index3, UiSection);

EFI_DEVICE_PATH_PROTOCOL *DevicePath;
EFI_HANDLE ImageHandle;

DevicePath = FvFileDevicePath(Buffer[Index], &NameGuid);

Status = gBS->LoadImage(FALSE, gImageHandle, DevicePath, NULL, 0, &ImageHandle);
if (!EFI_ERROR(Status)) {
//PERF_END(NULL, "BDS", NULL, 0);

Print(L"%d) Load image success '%s'\n", Index3, UiSection);

Status = gBS->StartImage(ImageHandle, NULL, NULL);

Some drivers were loaded, but there was not 'Realtek UEFI UNDI Driver' among them.
Maybe somebody knows from where and how I can load this UNDI driver?


Re: Drivers that based on two underlying hardware devices

Michael Brown
 

On 30/04/2021 21:56, Jeff Brasen wrote:
Trying to determine any recommendations for best practices for drivers that need to connect to multiple pieces of hardware and the interaction with the connection sequence. For example if a driver needs to talk to a given hardware component as well as an i2c device. With my understanding of the UEFI driver model seems to work there doesn't seem to be good way to support this without either needing to have a connect all or custom connection logic. If that is what is needed it isn't unworkable but thought I would ask if anyone has alternative ideas for this sort of devices and making a simplified connection approach for devices.
There should be a working example of this in almost any UEFI USB network driver, since most USB NICs require access to two USB-level interfaces but the EFI_USB_IO_PROTOCOL model exposes a single device handle per interface.

Hope that helps,

Michael


Drivers that based on two underlying hardware devices

Jeff Brasen
 

Hi,

Trying to determine any recommendations for best practices for drivers that need to connect to multiple pieces of hardware and the interaction with the connection sequence. For example if a driver needs to talk to a given hardware component as well as an i2c device. With my understanding of the UEFI driver model seems to work there doesn't seem to be good way to support this without either needing to have a connect all or custom connection logic. If that is what is needed it isn't unworkable but thought I would ask if anyone has alternative ideas for this sort of devices and making a simplified connection approach for devices.

Thanks,
Jeff


Re: Having problems when trying to instrument all code of a specific UEFI driver (including the library code)

mick21@...
 

For ASan and MSan I just add the relevant flags to CC_FLAGS and sometimes I have to add custom flags. I must mention that I enabled Windows as a target for MSan, this wasn't the case for my LLVM version.

[BuildOptions]
*_*_*_CC_FLAGS = -fsanitize=memory -mllvm -msan-smm-tianocore=1 -fsanitize-blacklist=/mnt/part5/edk2-msan/msan_blacklist.txt
For ASan I had to remove link-time optimizations, due to errors related to comdat sections which I could not fix. The more difficult part for me is setting up the shadow memory and implementing enough of the sanitizer runtime in order for it to work.

Also, some functions have other dependencies which are out of the instrumentation still, for instance [1]:

InternalSmmBase2->GetSmstLocation (InternalSmmBase2, &gSmst);
ASSERT (gSmst != NULL);
This initializes the gSmst variable, with the function SmmBase2GetSmstLocation() [2] in PiSmmIpl.c, and will now cause MSan to error on the ASSERT() call. I still have to work out these cases. That said, it is still very much a work in progress and it is all a bit hacky.

Kind regards,

Mick

[1] https://github.com/tianocore/edk2/blob/83876950ab3cf5278d0ae7542086bd4be75059d3/MdePkg/Library/SmmServicesTableLib/SmmServicesTableLib.c#L52
[2] https://github.com/tianocore/edk2/blob/83876950ab3cf5278d0ae7542086bd4be75059d3/MdeModulePkg/Core/PiSmmCore/PiSmmIpl.c#L77


Re: Having problems when trying to instrument all code of a specific UEFI driver (including the library code)

Andrew Fish
 

On Apr 13, 2021, at 2:38 PM, mick21@live.nl wrote:

Sure, currently I add the adjusted libraries like this:

MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
PcdLib|MdePkg/Library/DxePcdLib/DxePcdLib.sanitizer.inf
TimerLib|OvmfPkg/Library/AcpiTimerLib/DxeAcpiTimerLib.sanitizer.inf
ResetSystemLib|OvmfPkg/Library/ResetSystemLib/DxeResetSystemLib.sanitizer.inf
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.sanitizer.inf
What compiler flags did you change to get the clang sanitizer working?

Thanks,

Andrew Fish

...

Then I copy all the original library.inf to library.sanitizer.inf and then append my [BuildOptions] section, depending on the instrumentation. You can then check the static_library_files.lst in the build directory for your driver to see if all libraries are replaced with the sanitized ones. This is only "one layer" though, I'm not sure whether this matters for my implementation, but libraries can use other libraries, which should also be instrumented, that is not the case with this.


Re: Having problems when trying to instrument all code of a specific UEFI driver (including the library code)

mick21@...
 

Sure, currently I add the adjusted libraries like this:

MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.inf {
<LibraryClasses>
PcdLib|MdePkg/Library/DxePcdLib/DxePcdLib.sanitizer.inf
TimerLib|OvmfPkg/Library/AcpiTimerLib/DxeAcpiTimerLib.sanitizer.inf
ResetSystemLib|OvmfPkg/Library/ResetSystemLib/DxeResetSystemLib.sanitizer.inf
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.sanitizer.inf
...

Then I copy all the original library.inf to library.sanitizer.inf and then append my [BuildOptions] section, depending on the instrumentation. You can then check the static_library_files.lst in the build directory for your driver to see if all libraries are replaced with the sanitized ones. This is only "one layer" though, I'm not sure whether this matters for my implementation, but libraries can use other libraries, which should also be instrumented, that is not the case with this.


Re: Having problems when trying to instrument all code of a specific UEFI driver (including the library code)

Andrew Fish
 

Feel free to post examples if it would help others.

On Apr 13, 2021, at 12:34 PM, mick21@live.nl wrote:

Hi Andrew,

Apologies for the late reply!

ShellPkg/Application/Shell/Shell.inf {
<LibraryClasses>
HandleParsingLib|ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.sanitize.inf
}
I think I have it working now, I'm overwriting all of the used libraries with the *.sanitizer.inf alternative, like you said, and add the BuildOptions there. I'm not sure whether this is the best option, but it is cleaner than what I had in mind and most importantly: it works :).

Thank you for your help, I really appreciate it.

Kind regards,

Mick





Re: Having problems when trying to instrument all code of a specific UEFI driver (including the library code)

mick21@...
 

Hi Andrew,

Apologies for the late reply!

ShellPkg/Application/Shell/Shell.inf {
<LibraryClasses>
HandleParsingLib|ShellPkg/Library/UefiHandleParsingLib/UefiHandleParsingLib.sanitize.inf
}
I think I have it working now, I'm overwriting all of the used libraries with the *.sanitizer.inf alternative, like you said, and add the BuildOptions there. I'm not sure whether this is the best option, but it is cleaner than what I had in mind and most importantly: it works :).

Thank you for your help, I really appreciate it.

Kind regards,

Mick

81 - 100 of 805