Date   

Windows guest fails to boot into recovery mode due to commit 5267926

annie li
 

Hello,

I ran into a windows booting failure issue(a page fault exception), and narrow down it to the following patch,
MdeModulePkg/DxeIpl: support more NX related PCDs
https://github.com/tianocore/edk2/commit/5267926134d17e86672b84fd57b438f05ffa68e1

This issue always happens after QMP is terminated by <ctrl-C> twice, see following steps.

1. Boot Windows VM up, and <ctrl-C> to exit the QMP

2. Repeat 1

3. Boot Windows VM, and this page fault issue happens. (Note: Windows should boot into recovery mode in this round, and this is due to the previous two consecutive boot failure, see https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/windows-recovery-environment--windows-re--technical-reference#entry-points-into-winre)

During above 3 windows booting procedures, the value of following variables are always the same,
PcdSetNxForStack 0
PcdDxeNxMemoryProtectionPolicy 0
PcdImageProtectionPolicy 2

However, Windows guest fails to boot up into recovery mode in the 3rd round due to the patch above(5267926). I modified the return value to "(PcdGetBool (PcdSetNxForStack)" in function "IsEnableNonExecNeeded" in MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c, this page fault issue is gone with this change. The patch(5267926) is for fixing bug https://bugzilla.tianocore.org/show_bug.cgi?id=1116, where the comments show PcdImageProtectionPolicy needs also to enable NXE. But this does cause the page fault exception in this scenario, any suggestion?

The page fault exception is pasted here,


!!!! X64 Exception Type - 0E(#PF - Page-Fault) CPU Apic ID - 00000000 !!!!
ExceptionData - 0000000000000009 I:0 R:1 U:0 W:0 P:1 PK:0 SS:0 SGX:0
RIP - 000000003E0A7C75, CS - 0000000000000038, RFLAGS - 0000000000010202
RAX - 8000000000000003, RCX - 0000000000000001, RDX - 0000000001040001
RBX - 0000000000000001, RSP - 00000000001A6AA0, RBP - 0000000001040001
RSI - 000000003F2E2010, RDI - 0000000000000001
R8 - 0000000000000000, R9 - 000000003E0AEC90, R10 - 0000FFFFFFFFF000
R11 - 00000000001A6E90, R12 - 0000000000000000, R13 - 000000003E0AEC90
R14 - 00000000001A6B28, R15 - 00000000001AB000
DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030
GS - 0000000000000030, SS - 0000000000000030
CR0 - 0000000080010033, CR2 - 000000003F2E2010, CR3 - 000000003F401000
CR4 - 0000000000040668, CR8 - 0000000000000000
DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000003F1EE698 0000000000000047, LDTR - 0000000000000000
IDTR - 000000003ECCA018 0000000000000FFF, TR - 0000000000000000
FXSAVE_STATE - 00000000001A6700
!!!! Find image based on IP(0x3E0A7C75) /builddir/build/BUILD/edk2-1.4.3/Build/OvmfX64/DEBUG_GCC48/X64/MdeModulePkg/Universal/Console/TerminalDxe/TerminalDxe/DEBUG/TerminalDxe.dll (ImageBase=000000003E0A5000, EntryPoint=000000003E0A86E8) !!!!
Thanks
Annie


Re: lost stderr in console

Laszlo Ersek
 

On 03/17/21 04:23, wenyi,xie via groups.io wrote:
Hi,everyone,

I used to set makeflags like below,so that only message from stderr will be shown in console while compiling, and at the same time message from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >> ${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log
Redirections are processed in the following order:
- pipe first (redirects stdout)
- then left to right, as seen on the command line

So what we have for "xxx.sh" is:

step#1: file descriptor 1 refers to the (nameless) pipe's file description

step#2: file descriptor 2 refers to the same (nameless) pipe's file
description (i.e., to the file description that file descriptor 1
*currently* refers to)

step#3: file descriptor 1 now refers to a file description that refers
to the inode ("file") originally looked up by the name "xxx.log". At the
file description level, the O_APPEND status flag is set.

So "tee" will only see stderr from "xxx.sh". Furthermore, the stdout of
"xxx.sh" will only go to the log file ("xxx.log").


Let's consider "tee" then: "tee" opens the inode (the "file") looked up
by the name "xxx.log" separately from when the shell opens "xxx.log",
for the "xxx.sh" redirection. This means that, in the kernel, a separate
file description exists, for the "xxx.log" inode. This file description
also has the O_APPEND status flag set, but it doesn't matter -- the file
description that "xxx.sh" writes through, and the file description that
"tee" writes through, are independent. The "file offset" property is at
the file description level. Therefore "tee" and "xxx.sh" do not share
the file offset (and O_APPEND is useless, in both file descriptions),
and they will mutually overwrite parts of each other's output.

In other words, your command line is buggy.


In general:

file descriptor --> file description --> file (inode)

When you open() the same file by name, you get this:

file descriptor --> file description \
--> file (inode)
file descriptor --> file description /

Whereas, if you use fork() or dup(), this is what you get:

file descriptor \
--> file description --> file (inode)
file descriptor /

O_APPEND and the file offset both exist in the *file description*
object. So in the first case, you get no coordination from the kernel,
and in the second case, you do.

Note that even in the second case, that is, when both file descriptors
refer to the same file description, it is not guaranteed that
*concurrent* writes will not be *interleaved*. No data will be
*overwritten*, for sure, but the granularity of "atomic" writes is not
an easy question. If the file description refers to a pipe, then there
are some guarantees from POSIX, as long as the writes are "small enough"
(PIPE_BUF):
<https://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html>.


So... what you want to do is actually difficult. There are two
approaches, but both are broken.

Approach (1): fuse stdout and stderr into a single stream, capture the
common stream in the log file, and print the "diagnostic" lines to the
console:

sh xxx.sh 2>&1 | tee -- xxx.log | grep -- 'what exacty?'

This is broken because you cannot identify the diagnostic-only lines
(the original stderr) by content.

Approach (2): duplicate the original stderr into two streams. The first
instance will go to the console. The second instance, together with the
script's stdout, will be written to the log file.

# fd 1: original stdout
# fd 2: original stderr
# make fd 3 point to the original stdout as well

exec 3>&1

(
# fd 1: pipe leading to the log file
# fd 2: original stderr
# fd 3: original stdout
# make fd 4 too point to the pipe leading to the log file

exec 4>&1

(
# fd 1: pipe carrying the main script's stderr
# fd 2: original stderr
# fd 3: original stdout
# fd 4: pipe leading to the log file

# main script's stderr goes to the duplicator pipe
# main script's stdout goes to the log file

xxx.sh 2>&1 >&4
) | (

# fd 0: pipe carrying the main script's stderr
# fd 1: pipe leading to the log file
# fd 2: original stderr
# fd 3: original stdout
# fd 4: pipe leading to the log file

# duplicate the main script's stderr to the original stdout (3)
# and to the log file (1)

tee /dev/fd/3
)
) | cat > xxx.log


In short form (no comments and no useless subshells):

exec 3>&1
(
exec 4>&1
xxx.sh 2>&1 >&4 | tee /dev/fd/3
) | cat > xxx.log

This will not lose messages. It will also not interleave write()
syscalls that do not exceed PIPE_BUF (usually 4KB) individually -- this
is the justification for the outermost pipe, and "cat".

However, it is still broken: dependent on process scheduling between
"xxx.sh" and "tee", it's now possible that stdout and stderr lines from
"xxx.sh" will be reordered, relative to each other. Let's say line#1 is
a diagnostic message, while line#2 is a normal message. "xxx.sh" writes
them in line#1, line#2 order. With the above, line#1 goes from "xxx.sh"
to "tee" to "cat", while line#2 goes from "xxx.sh" to "cat". If "tee" is
"slow", then "cat" could see the messages in line#2, line#1 order.

Thanks
Laszlo


Re: Google Summer of Code Interested Student

Laszlo Ersek
 

On 03/17/21 00:25, Desimone, Nathaniel L wrote:

The UEFI spec doesn't read on this at all, even though it describes
VT100 and VT100+ as separate modes... it doesn't say how they differ.
I agree with you that it seems reasonable for VT100 to keep character
output to strict ASCII only... that way the "+" in VT100+ actually
means something. I've updated the wiki accordingly.

I'd advocate for the default to be switched to VT_UTF8. I really
don't think you will run into many terminal emulators that don't
implement UTF-8 anymore, XTerm included. Those who want pure ASCII
output can switch to VT100.
Hmmm, OK. As long as I can permanently switch my domains, one by one, to
VT100, I guess I'll be fine.

Thanks!
Laszlo


Re: 回复: [edk2-discuss] 回复: Non-reproducible binaries generated by GenFw

Laszlo Ersek
 

On 03/17/21 03:47, gaoliming wrote:
Yes. Just RELEASE_*_*_GENFW_FLAGS = --zero

The format is defined in DSC spec. = means append, == means replacement.
And the EDK2 specifications can be found here:

https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Documentation#specifications

Thanks!
Laszlo


Thanks
Liming
-----邮件原件-----
发件人: Ross Burton <ross@burtonini.com>
发送时间: 2021年3月17日 0:27
收件人: Laszlo Ersek <lersek@redhat.com>
抄送: discuss@edk2.groups.io; gaoliming <gaoliming@byosoft.com.cn>;
bob.c.feng@intel.com; yuwei.chen@intel.com
主题: Re: [edk2-discuss] 回复: Non-reproducible binaries generated by
GenFw

On Tue, 16 Mar 2021 at 16:15, Laszlo Ersek <lersek@redhat.com> wrote:
But: if you post such a patch for OVMF (modifying *all* DSC files in
OvmfPkg), then I'll review & merge it.
Is there any documentation for the dsc format? Specifically if a dsc
already has:

!if $(SOURCE_DEBUG_ENABLE) == TRUE
INTEL:*_*_X64_GENFW_FLAGS = --keepexceptiontable
!endif

Can I just do:
RELEASE_*_*_GENFW_FLAGS = --zero

Or do I need to use += or something else?

Ross


Re: [edk2-devel] [edk2-discuss] Google Summer of Code Interested Student

Andrew Fish
 

If we are mentioning terminal types the default terminal type on a Mac is xterm-256color. So that is going to be the default when people run OVMF on a Mac. So it would be nice if we can add that. I can help out with anything xterm-256color related.

Thanks,

Andrew Fish

On Mar 16, 2021, at 8:23 AM, Laszlo Ersek <lersek@redhat.com> wrote:

Hi Nate,

(adding Leif and Ard)

On 03/13/21 03:52, Desimone, Nathaniel L wrote:
I've created a new wiki page for this task with all the information I
have gathered thus far. I've done some more experimentation and found
that there are several newer terminal emulators that don't support
DEC Special Graphics so I've reduced the number of modes where DEC
Special Graphics should be preferred. Laszlo, if you could take a
look at the terminal type matrix I created that would be very
helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
(

My background:

I settled on plain (non-UTF-8) xterm around 1998, and have been using it
ever since. Whenever something was off, I always tried to hammer the
application into conformance with my particular xterm setup, rather than
the other way around. I also have some quirky terminal settings -- for
me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
"delete" generates keycode 119, and there's no "rubout". I still don't
use UTF-8 (I use latin2).

)

* Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
setting (which means VT-100). Using that setting, I see the following
kind of "ASCII approximation" for box drawing:

/------------------------------------------------------------------------------\
| Boot Manager |
\------------------------------------------------------------------------------/

I'm really happy with this, as I don't care much for nice-looking
boxes; instead I prefer portability.

(NB: this seems to disagree with your "Current Behavior (Which is
wrong)" line for VT100, as it suggests CP437. That's not what I'm
seeing with VT100.)

TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
the default, in 2015:

http://mid.mail-archive.com/555458DB.3090602@redhat.com
http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sQf8ww@mail.gmail.com

(I'd like to CC Roy, but I can't tell if he's now working for Linaro,
Cavium, HPE, Marvell, or another company.)

* Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
looks like this:

ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄż
ł Boot Manager ł
ŔÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄŮ

Obviously I'd much prefer if I got the simple ASCII approximation here
as well.

* Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
to use DEC Special Graphics, I can't tell.

I know what my preferences are:

- the current BackSpace and Delete mappings (which work fine for me
with both VT100 and PC_ANSI, but *not* with TTY_TERM),

- and the most primitive ASCII mapping (no special graphics, no UTF-8
sequences, etc). I really like a super dumb terminal, where taking
simple "ASCII screenshots" (and pasting them into plaintext emails!)
is *trivial*.

... Looking at your "Expected Behavior" table, there is only one line
left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
TTY_TERM breaks my BackSpace / Delete settings :(

* In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
rather than PC_ANSI, by default.

Thanks!
Laszlo






Re: Google Summer of Code Interested Student

Leif Lindholm
 

+Roy (now at Marvell)

/
Leif (now at Qualcomm)

On Tue, Mar 16, 2021 at 16:23:45 +0100, Laszlo Ersek wrote:
Hi Nate,

(adding Leif and Ard)

On 03/13/21 03:52, Desimone, Nathaniel L wrote:
I've created a new wiki page for this task with all the information I
have gathered thus far. I've done some more experimentation and found
that there are several newer terminal emulators that don't support
DEC Special Graphics so I've reduced the number of modes where DEC
Special Graphics should be preferred. Laszlo, if you could take a
look at the terminal type matrix I created that would be very
helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
(

My background:

I settled on plain (non-UTF-8) xterm around 1998, and have been using it
ever since. Whenever something was off, I always tried to hammer the
application into conformance with my particular xterm setup, rather than
the other way around. I also have some quirky terminal settings -- for
me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
"delete" generates keycode 119, and there's no "rubout". I still don't
use UTF-8 (I use latin2).

)

* Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
setting (which means VT-100). Using that setting, I see the following
kind of "ASCII approximation" for box drawing:

/------------------------------------------------------------------------------\
| Boot Manager |
\------------------------------------------------------------------------------/

I'm really happy with this, as I don't care much for nice-looking
boxes; instead I prefer portability.

(NB: this seems to disagree with your "Current Behavior (Which is
wrong)" line for VT100, as it suggests CP437. That's not what I'm
seeing with VT100.)

TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
the default, in 2015:

http://mid.mail-archive.com/555458DB.3090602@redhat.com
http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sQf8ww@mail.gmail.com

(I'd like to CC Roy, but I can't tell if he's now working for Linaro,
Cavium, HPE, Marvell, or another company.)

* Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
looks like this:

ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄż
ł Boot Manager ł
ŔÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄŮ

Obviously I'd much prefer if I got the simple ASCII approximation here
as well.

* Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
to use DEC Special Graphics, I can't tell.

I know what my preferences are:

- the current BackSpace and Delete mappings (which work fine for me
with both VT100 and PC_ANSI, but *not* with TTY_TERM),

- and the most primitive ASCII mapping (no special graphics, no UTF-8
sequences, etc). I really like a super dumb terminal, where taking
simple "ASCII screenshots" (and pasting them into plaintext emails!)
is *trivial*.

... Looking at your "Expected Behavior" table, there is only one line
left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
TTY_TERM breaks my BackSpace / Delete settings :(

* In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
rather than PC_ANSI, by default.

Thanks!
Laszlo


lost stderr in console

wenyi,xie
 

Hi,everyone,

I used to set makeflags like below,so that only message from stderr will be shown in console while compiling, and at the same time message from stderr and stdout will all be saved in xxx.log.
COMMAND MAKEFLAGS= set -eo pipefail && sh xxx.sh 2>&1 >> ${LOG_FILE_DIR}/xxx.log | tee -a ${LOG_FILE_DIR}/xxx.log

But after updating edk2 from 201903 to 202011, the message from stderr is not shown in console any more. In xxx.log, the message from stderr and stdout are still saved.
Do anything change in edk2 may affect on it ?


回复: [edk2-discuss] 回复: Non-reproducible binaries generated by GenFw

gaoliming
 

Yes. Just RELEASE_*_*_GENFW_FLAGS = --zero

The format is defined in DSC spec. = means append, == means replacement.

Thanks
Liming

-----邮件原件-----
发件人: Ross Burton <ross@burtonini.com>
发送时间: 2021年3月17日 0:27
收件人: Laszlo Ersek <lersek@redhat.com>
抄送: discuss@edk2.groups.io; gaoliming <gaoliming@byosoft.com.cn>;
bob.c.feng@intel.com; yuwei.chen@intel.com
主题: Re: [edk2-discuss] 回复: Non-reproducible binaries generated by
GenFw

On Tue, 16 Mar 2021 at 16:15, Laszlo Ersek <lersek@redhat.com> wrote:
But: if you post such a patch for OVMF (modifying *all* DSC files in
OvmfPkg), then I'll review & merge it.
Is there any documentation for the dsc format? Specifically if a dsc
already has:

!if $(SOURCE_DEBUG_ENABLE) == TRUE
INTEL:*_*_X64_GENFW_FLAGS = --keepexceptiontable
!endif

Can I just do:
RELEASE_*_*_GENFW_FLAGS = --zero

Or do I need to use += or something else?

Ross


Re: GSOC 21 interested student

Nate DeSimone
 

Hi Vikas,

Great to meet you and welcome to the TianoCore project! Glad you hear you are interested! There are some existing tasks listed in the wiki for proposals. Another project which I haven't had a chance to add to the wiki yet is implementing a multi-threaded TCP/IP stack. There are been some work already done on this, that work is currently sitting here: https://github.com/tianocore/edk2-staging/tree/MpNetworkStack

I believe Maciej ran into issues with reconciling this work with the current UEFI spec and UNDI driver requirements. Though there is definitely a lot of promise here. From what I remember it increased network performance from 100 Mb/s to 1000 Mb/s. Figuring out what work is required to make this production ready and get it merged into the mainline would be a great improvement.

Let me know if you have any questions!

With Best Regards,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
vikasraist@gmail.com
Sent: Monday, March 15, 2021 11:42 AM
To: discuss@edk2.groups.io
Subject: [edk2-discuss] GSOC 21 interested student

Hi everyone, I am Vikas Kundu, a post-grad student of Computer Science in
India. I can code in Python, node.js, C and C++ and is aware of networking
concepts especially DNS. I wish to work on the UEFI Networking
Improvements part. Currently i am in the process of reading the
documentation for the same on the tianocore and nbd repos. I wish to look
forward for guidance from this community. My github profile is:
https://www.github.com/vikas-kundu.





Re: Google Summer of Code Interested Student

Nate DeSimone
 

Hi Laszlo,

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
Laszlo Ersek
Sent: Tuesday, March 16, 2021 8:24 AM
To: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>
Cc: discuss@edk2.groups.io; cadenkline9@gmail.com; edk2-devel-groups-io
<devel@edk2.groups.io>; Ard Biesheuvel (TianoCore)
<ardb+tianocore@kernel.org>; Leif Lindholm (Nuvia address)
<leif@nuviainc.com>
Subject: Re: [edk2-discuss] Google Summer of Code Interested Student

Hi Nate,

(adding Leif and Ard)

On 03/13/21 03:52, Desimone, Nathaniel L wrote:
I've created a new wiki page for this task with all the information I
have gathered thus far. I've done some more experimentation and found
that there are several newer terminal emulators that don't support DEC
Special Graphics so I've reduced the number of modes where DEC Special
Graphics should be preferred. Laszlo, if you could take a look at the
terminal type matrix I created that would be very helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-d
river-improvements
(

My background:

I settled on plain (non-UTF-8) xterm around 1998, and have been using it
ever since. Whenever something was off, I always tried to hammer the
application into conformance with my particular xterm setup, rather than the
other way around. I also have some quirky terminal settings -- for me,
"backspace" generates ^H / keycode 22 (stty sets erase to ^H), "delete"
generates keycode 119, and there's no "rubout". I still don't use UTF-8 (I use
latin2).

)

* Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
setting (which means VT-100). Using that setting, I see the following
kind of "ASCII approximation" for box drawing:

/------------------------------------------------------------------------------\
| Boot Manager |
\------------------------------------------------------------------------------/

I'm really happy with this, as I don't care much for nice-looking
boxes; instead I prefer portability.

(NB: this seems to disagree with your "Current Behavior (Which is
wrong)" line for VT100, as it suggests CP437. That's not what I'm
seeing with VT100.)
I went back and looked at this is more detail, and I missed the following critical detail:

if (TerminalDevice->TerminalType != TerminalTypePcAnsi) {
GraphicChar = AsciiChar;
}

Yes you are totally right! I've adjusted the table to reflect this behavior.


TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
the default, in 2015:

http://mid.mail-archive.com/555458DB.3090602@redhat.com
http://mid.mail-
archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sQf8w
w@mail.gmail.com

(I'd like to CC Roy, but I can't tell if he's now working for Linaro,
Cavium, HPE, Marvell, or another company.)

* Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
looks like this:


ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄż
ł Boot Manager ł

ŔÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄŮ

Obviously I'd much prefer if I got the simple ASCII approximation here
as well.

* Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially*
supposed
to use DEC Special Graphics, I can't tell.
The UEFI spec doesn't read on this at all, even though it describes VT100 and VT100+ as separate modes... it doesn't say how they differ. I agree with you that it seems reasonable for VT100 to keep character output to strict ASCII only... that way the "+" in VT100+ actually means something. I've updated the wiki accordingly.

I'd advocate for the default to be switched to VT_UTF8. I really don't think you will run into many terminal emulators that don't implement UTF-8 anymore, XTerm included. Those who want pure ASCII output can switch to VT100.


I know what my preferences are:

- the current BackSpace and Delete mappings (which work fine for me
with both VT100 and PC_ANSI, but *not* with TTY_TERM),

- and the most primitive ASCII mapping (no special graphics, no UTF-8
sequences, etc). I really like a super dumb terminal, where taking
simple "ASCII screenshots" (and pasting them into plaintext emails!)
is *trivial*.

... Looking at your "Expected Behavior" table, there is only one line
left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
TTY_TERM breaks my BackSpace / Delete settings :(

* In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
rather than PC_ANSI, by default.

Thanks!
Laszlo





GSOC 21 interested student

vikasraist@...
 

Hi everyone, I am Vikas Kundu, a post-grad student of Computer Science in India. I can code in Python, node.js, C and C++ and is aware of networking concepts especially DNS. I wish to work on the UEFI Networking Improvements part. Currently i am in the process of reading the documentation for the same on the tianocore and nbd repos. I wish to look forward for guidance from this community. My github profile is: https://www.github.com/vikas-kundu.


Re: 回复: Non-reproducible binaries generated by GenFw

Ross Burton
 

On Tue, 16 Mar 2021 at 16:15, Laszlo Ersek <lersek@redhat.com> wrote:
But: if you post such a patch for OVMF (modifying *all* DSC files in
OvmfPkg), then I'll review & merge it.
Is there any documentation for the dsc format? Specifically if a dsc
already has:

!if $(SOURCE_DEBUG_ENABLE) == TRUE
INTEL:*_*_X64_GENFW_FLAGS = --keepexceptiontable
!endif

Can I just do:
RELEASE_*_*_GENFW_FLAGS = --zero

Or do I need to use += or something else?

Ross


Re: 回复: Non-reproducible binaries generated by GenFw

Laszlo Ersek
 

On 03/15/21 18:40, Ross Burton wrote:
On Fri, 12 Mar 2021 at 18:37, Laszlo Ersek <lersek@redhat.com> wrote:
I'm failing at making this actually stick in the makefiles though...
Can you provide your platform DSC file?

What is your exact "build" command line? (I'm looking for the "-a",
"-b", "-t" options.)
Essentially, OvmfPkg/build.sh -a X86 -b RELEASE -t GCC5. It appears I
should be editing OvmfPkg/OvmfX86.dsc instead of ShellPkg.dsc.

Is it agreed that this should be fixed? Is putting
RELEASE_*_*_GENFW_FLAGS=-z all over the repository the right solution,
or is there a better solution that can be done once centrally?
I'm a bit torn myself (I can't imagine a good reason why we *wouldn't*
want to make --zero the default for GenFw in RELEASE builds).

But: if you post such a patch for OVMF (modifying *all* DSC files in
OvmfPkg), then I'll review & merge it.

OvmfPkg/AmdSev/AmdSevX64.dsc
OvmfPkg/Bhyve/BhyveX64.dsc
OvmfPkg/OvmfPkgIa32.dsc
OvmfPkg/OvmfPkgIa32X64.dsc
OvmfPkg/OvmfPkgX64.dsc
OvmfPkg/OvmfXen.dsc

Thanks
Laszlo


Re: Google Summer of Code Interested Student

Laszlo Ersek
 

Hi Nate,

(adding Leif and Ard)

On 03/13/21 03:52, Desimone, Nathaniel L wrote:
I've created a new wiki page for this task with all the information I
have gathered thus far. I've done some more experimentation and found
that there are several newer terminal emulators that don't support
DEC Special Graphics so I've reduced the number of modes where DEC
Special Graphics should be preferred. Laszlo, if you could take a
look at the terminal type matrix I created that would be very
helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements
(

My background:

I settled on plain (non-UTF-8) xterm around 1998, and have been using it
ever since. Whenever something was off, I always tried to hammer the
application into conformance with my particular xterm setup, rather than
the other way around. I also have some quirky terminal settings -- for
me, "backspace" generates ^H / keycode 22 (stty sets erase to ^H),
"delete" generates keycode 119, and there's no "rubout". I still don't
use UTF-8 (I use latin2).

)

* Regarding ArmVirtPkg, I stick with the default TTY_TERMINAL=FALSE
setting (which means VT-100). Using that setting, I see the following
kind of "ASCII approximation" for box drawing:

/------------------------------------------------------------------------------\
| Boot Manager |
\------------------------------------------------------------------------------/

I'm really happy with this, as I don't care much for nice-looking
boxes; instead I prefer portability.

(NB: this seems to disagree with your "Current Behavior (Which is
wrong)" line for VT100, as it suggests CP437. That's not what I'm
seeing with VT100.)

TTY_TERMINAL=TRUE would mainly affect backspace / delete I think -- as
far as I recall, that's why I asked Roy not to make TTY_TERMINAL=TRUE
the default, in 2015:

http://mid.mail-archive.com/555458DB.3090602@redhat.com
http://mid.mail-archive.com/CAFECyb_E+bGZt5xv7QhRqyD0jX=AzoEMw7VW_tjZr+E=sQf8ww@mail.gmail.com

(I'd like to CC Roy, but I can't tell if he's now working for Linaro,
Cavium, HPE, Marvell, or another company.)

* Regarding OvmfPkg, currently PC_ANSI is hard-coded, and for me it
looks like this:

ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄż
ł Boot Manager ł
ŔÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄŮ

Obviously I'd much prefer if I got the simple ASCII approximation here
as well.

* Whether VT100 and/or PC_ANSI and/or TTY_TERM are *officially* supposed
to use DEC Special Graphics, I can't tell.

I know what my preferences are:

- the current BackSpace and Delete mappings (which work fine for me
with both VT100 and PC_ANSI, but *not* with TTY_TERM),

- and the most primitive ASCII mapping (no special graphics, no UTF-8
sequences, etc). I really like a super dumb terminal, where taking
simple "ASCII screenshots" (and pasting them into plaintext emails!)
is *trivial*.

... Looking at your "Expected Behavior" table, there is only one line
left with "poor man's ASCII" -- namely, TTY_TERM. Unfortunately,
TTY_TERM breaks my BackSpace / Delete settings :(

* In summary, I'd prefer if (a) VT100 stayed as-is (using "poor man's
ASCII", as seen in ArmVirtPkg), and (b) if OVMF used *that* VT100,
rather than PC_ANSI, by default.

Thanks!
Laszlo


Re: Redfish Host Interface

Russell Peterson
 

Thanks so much, Abner. This is all very helpful to me.

Regards,

Russell

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of Abner Chang via groups.io
Sent: Monday, March 15, 2021 9:10 PM
To: Russell Peterson <russell@nvidia.com>; discuss@edk2.groups.io; Laszlo Ersek <lersek@redhat.com>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>
Subject: Re: [edk2-discuss] Redfish Host Interface



-----Original Message-----
From: Russell Peterson [mailto:russell@nvidia.com]
Sent: Monday, March 15, 2021 7:42 PM
To: discuss@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist)
<abner.chang@hpe.com>; Laszlo Ersek <lersek@redhat.com>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>
Subject: RE: [edk2-discuss] Redfish Host Interface

Thanks for the response Laszlo and Abner. Very much appreciated.

Do I understand correctly that if I have a BMC with the proper schema
for my system UEFI, when the system boots it (UEFI) will fetch data
from the BMC via the Redfish host interface and that data would in effect, "take priority"
over the data stored in the UPVS? I mean not only for boot order but
also any other configuration data?
Not quite sure what is UPVS stands for.
You are right, not only the boot order but also other properties defined in schema. Map HII to Redfish property using EFI keyword is the phase 2 plan for edk2.


I assume UEFI has some type of discovery mechanism and authentication
mechanism to find and use the BMC Redfish data. From what I have read
this seems to be the case.
Yes, the discovery mechanism for Redfish service is over Redfish HI or SSDP. Platform library has to provide the library to edk2 Redfish core code to build up SMBIOS type 42 table, and the library for acquiring Redfish credential which is OEM implementation-specific.
You can refer to below link for detail information, https://github.com/tianocore/edk2-staging/tree/UEFI_Redfish, few diagrams and descriptions may out of date but not too far from the implementation on edk2.

We still in the progress of contributing edk2 Redfish foundation code. It is almost there, we can finish this phase 1 work once UEFI spec 2.9 is released because there are some spec updates for EFI Redfish Discover Protocol.
After edk2 Redfish foundation, we will start to contribute edk2 Redfish client code. That is what I mentioned to map HII (BIOS configuration) to Redfish property for the remote management.

Thanks
Abner





--Russ

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
Abner Chang via groups.io
Sent: Sunday, March 14, 2021 9:24 PM
To: Laszlo Ersek <lersek@redhat.com>; Russell Peterson
<russell@nvidia.com>
Cc: discuss@edk2.groups.io; Wang, Nickle (HPS SW)
<nickle.wang@hpe.com>
Subject: Re: [edk2-discuss] Redfish Host Interface



-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Saturday, March 13, 2021 6:11 AM
To: russell@nvidia.com
Cc: discuss@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist)
<abner.chang@hpe.com>; Wang, Nickle (HPS SW)
<nickle.wang@hpe.com>
Subject: Re: [edk2-discuss] Redfish Host Interface

Hi Russell,

On 03/11/21 22:02, Russell Peterson wrote:
Hello,

I am interested in a UEFI Redfish Host Interface that does not
require an
OS.
Redfish Host Interface is built for OS to communicate with Redfish
service on either BMC or out-of-band service (I don’t see this use
case yet). UEFI use case is similar to how OS communicates with
Redfish service through Redfish HI.

That is, the Redfish service would be provided by UEFI itself (via
an IP interface)... or perhaps an EFI application.
This is interesting , however you will need edk2 web server and the
implementation of Redfish service on edk2 as well. Not sure how this
use case works for OS <-> in-band Redfish service on edk2. Or you just
would like to build up an stand along Redfish service on edk2 and
other Redfish clients can connect to it?

For example, if a BMC wanted to access EFI variables directly from
UEFI via
an IP interface with no functional OS present.
Redfish HI just provides the Redfish information to Redfish clients,
the majority users are OS and firmware. Client sends the request to
service for the property through HTTP. Rare use cases that service
asks client for something on its own initiative, Redfish event service
is one of the service- to-client action however client has to registers even first.
So BMC accesses to EFI variable through Redfish HI seems to me not an
use case of generic Redfish services.


Is this crazy for me to think this? Already exists? Under
development? An
idea... but I would need to do some work?

At least the following TianoCore bugzilla <INVALID URI REMOVED
re.org_&d=DwIGaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=_SN6FZBN4Vgi4Ulk
skz6qU3NYR
O03nHp9P7Z5q59A3E&m=cleBHtjxzgg6TX8uY7CL9wbNT3KZExKhTcVbhTAay
M4&s=Bj48
Z9S2NrOzp1mB8LD4F83t22B1dBuQqeIcseTXQYQ&e= > tickets relate to
RedFish, one way or
another:
Laszlo, thanks for providing below information.
Abner

2337
2906
2910
2911
2912
2913
2918
2919
2930
2931
2932
2933
2934
2936
3102
3173
3174

At least the following USWG Mantis tickets relate to RedFish:

0001221
0001834
0001853
0001920
0001925
0001935
0001941
0002000
0002009
0002172

In UEFI v2.8B, there is a chapter called "31 - EFI Redfish Service Support".

A top-level RedfishPkg exists in edk2, and it seems like EmulatorPkg
has received some enablement. Not sure about the state in
edk2-platforms (or in other out-of-tree platforms).

Adding Abner and Nickle to the CC list (per "Maintainers.txt");
hopefully they can summarize the status for you.

Thanks
Laszlo





Re: Redfish Host Interface

Abner Chang
 

-----Original Message-----
From: Russell Peterson [mailto:russell@nvidia.com]
Sent: Monday, March 15, 2021 7:42 PM
To: discuss@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist)
<abner.chang@hpe.com>; Laszlo Ersek <lersek@redhat.com>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>
Subject: RE: [edk2-discuss] Redfish Host Interface

Thanks for the response Laszlo and Abner. Very much appreciated.

Do I understand correctly that if I have a BMC with the proper schema for my
system UEFI, when the system boots it (UEFI) will fetch data from the BMC
via the Redfish host interface and that data would in effect, "take priority"
over the data stored in the UPVS? I mean not only for boot order but also
any other configuration data?
Not quite sure what is UPVS stands for.
You are right, not only the boot order but also other properties defined in schema. Map HII to Redfish property using EFI keyword is the phase 2 plan for edk2.


I assume UEFI has some type of discovery mechanism and authentication
mechanism to find and use the BMC Redfish data. From what I have read
this seems to be the case.
Yes, the discovery mechanism for Redfish service is over Redfish HI or SSDP. Platform library has to provide the library to edk2 Redfish core code to build up SMBIOS type 42 table, and the library for acquiring Redfish credential which is OEM implementation-specific.
You can refer to below link for detail information,
https://github.com/tianocore/edk2-staging/tree/UEFI_Redfish, few diagrams and descriptions may out of date but not too far from the implementation on edk2.

We still in the progress of contributing edk2 Redfish foundation code. It is almost there, we can finish this phase 1 work once UEFI spec 2.9 is released because there are some spec updates for EFI Redfish Discover Protocol.
After edk2 Redfish foundation, we will start to contribute edk2 Redfish client code. That is what I mentioned to map HII (BIOS configuration) to Redfish property for the remote management.

Thanks
Abner





--Russ

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of
Abner Chang via groups.io
Sent: Sunday, March 14, 2021 9:24 PM
To: Laszlo Ersek <lersek@redhat.com>; Russell Peterson
<russell@nvidia.com>
Cc: discuss@edk2.groups.io; Wang, Nickle (HPS SW)
<nickle.wang@hpe.com>
Subject: Re: [edk2-discuss] Redfish Host Interface



-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com]
Sent: Saturday, March 13, 2021 6:11 AM
To: russell@nvidia.com
Cc: discuss@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist)
<abner.chang@hpe.com>; Wang, Nickle (HPS SW)
<nickle.wang@hpe.com>
Subject: Re: [edk2-discuss] Redfish Host Interface

Hi Russell,

On 03/11/21 22:02, Russell Peterson wrote:
Hello,

I am interested in a UEFI Redfish Host Interface that does not require an
OS.
Redfish Host Interface is built for OS to communicate with Redfish service on
either BMC or out-of-band service (I don’t see this use case yet). UEFI use
case is similar to how OS communicates with Redfish service through Redfish
HI.

That is, the Redfish service would be provided by UEFI itself (via an
IP interface)... or perhaps an EFI application.
This is interesting , however you will need edk2 web server and the
implementation of Redfish service on edk2 as well. Not sure how this use
case works for OS <-> in-band Redfish service on edk2. Or you just would like
to build up an stand along Redfish service on edk2 and other Redfish clients
can connect to it?

For example, if a BMC wanted to access EFI variables directly from
UEFI via
an IP interface with no functional OS present.
Redfish HI just provides the Redfish information to Redfish clients, the
majority users are OS and firmware. Client sends the request to service for
the property through HTTP. Rare use cases that service asks client for
something on its own initiative, Redfish event service is one of the service-
to-client action however client has to registers even first.
So BMC accesses to EFI variable through Redfish HI seems to me not an use
case of generic Redfish services.


Is this crazy for me to think this? Already exists? Under
development? An
idea... but I would need to do some work?

At least the following TianoCore bugzilla
<INVALID URI REMOVED
re.org_&d=DwIGaQ&c=C5b8zRQO1miGmBeVZ2LFWg&r=_SN6FZBN4Vgi4Ulk
skz6qU3NYR
O03nHp9P7Z5q59A3E&m=cleBHtjxzgg6TX8uY7CL9wbNT3KZExKhTcVbhTAay
M4&s=Bj48
Z9S2NrOzp1mB8LD4F83t22B1dBuQqeIcseTXQYQ&e= > tickets relate to
RedFish, one way or
another:
Laszlo, thanks for providing below information.
Abner

2337
2906
2910
2911
2912
2913
2918
2919
2930
2931
2932
2933
2934
2936
3102
3173
3174

At least the following USWG Mantis tickets relate to RedFish:

0001221
0001834
0001853
0001920
0001925
0001935
0001941
0002000
0002009
0002172

In UEFI v2.8B, there is a chapter called "31 - EFI Redfish Service Support".

A top-level RedfishPkg exists in edk2, and it seems like EmulatorPkg
has received some enablement. Not sure about the state in
edk2-platforms (or in other out-of-tree platforms).

Adding Abner and Nickle to the CC list (per "Maintainers.txt");
hopefully they can summarize the status for you.

Thanks
Laszlo





回复: [edk2-discuss] Non-reproducible binaries generated by GenFw

gaoliming
 

Andrew:

-----邮件原件-----
发件人: discuss@edk2.groups.io <discuss@edk2.groups.io> 代表 Andrew
Fish via groups.io
发送时间: 2021年3月16日 2:56
收件人: discuss <discuss@edk2.groups.io>; ross@burtonini.com
抄送: Feng, Bob C <bob.c.feng@intel.com>; Liming Gao (Byosoft address)
<gaoliming@byosoft.com.cn>; yuwei.chen@intel.com
主题: Re: [edk2-discuss] Non-reproducible binaries generated by GenFw


On Mar 10, 2021, at 11:36 AM, Ross Burton <ross@burtonini.com> wrote:

Hi,

As per https://bugzilla.tianocore.org/show_bug.cgi?id=3256, GenFw
writes non-reproducible binaries by embedding a build path. In fact in
a build of ovmf with embedded shell, this one path is the sole source
of non-determinism.

WriteDebug64() is always called in GenFw on output and that embeds
into the NB10 entry the input filename. As build paths change this is
a source of non-determinism. There already exists a --zero option to
wipe out debug paths but this is in release builds so I'm not sure
what the best solution is. I can see several options:

1) Release builds should not call WriteDebug64() at all
2) --zero should wipe the NB10 entry, and release builds should pass that
3) GenFw should support path remapping (like gcc's -ffile-prefix-map)
to turn build paths into something consistent.

Any suggestions?
This is kind of a tangent, but for clang we solved a similar problem a different
way.
1) in the build_rule.txt file we added a cd $(WORKSPACE); on the compile line
so __FILE__ is always relative to the $(WORKSPACE);
[Liming] CLANG compiler supports __FILE_NAME__ macro for the file name only. MdePkg DebugLib.h has applied this definition.

#if defined(__clang__) && defined(__FILE_NAME__)
#define _ASSERT(Expression) DebugAssert (__FILE_NAME__, __LINE__, #Expression)
#else
#define _ASSERT(Expression) DebugAssert (__FILE__, __LINE__, #Expression)
#endif

Thanks
Liming
2) We added a post build step to create a Symbols directory in the Build
output.
a) The debug directory entry paths are relative to that symbols directory.
They generally look like IA32/PeiCore or X64/DxeCore etc.
b) In the old days we would launch the debugger from the Symbols
directory so relative files got searched relative to the symbols directory.
c) Now we use the UUID we have in the
EFI_IMAGE_DEBUG_CODEVIEW_RSDS_ENTRY[1]PE/COFF debug entry. On a
Mac these mach-O UUIDs are automatically indexed by Spotlight so the
debugger can just find them. It is also possible to write a tool to lookup these
UUIDs that makes them available to the debugger.

I’m not sure it is possible to do something like this with all the toolchains, but
I wanted to at least let people know we figured out a way with clang.

[1]
https://github.com/tianocore/edk2/blob/master/MdePkg/Include/IndustrySt
andard/PeImage.h#L659

Thanks,

Andrew Fish

Cheers,
Ross








回复: [edk2-discuss] 回复: Non-reproducible binaries generated by GenFw

gaoliming
 

Ross:
This solution is for the reproduced binary images. If your platform has this request, you can update the platform DSC [BuildOptions] to include the option RELEASE_*_*_GENFW_FLAGS=-z.

Thanks
Liming

-----邮件原件-----
发件人: Ross Burton <ross@burtonini.com>
发送时间: 2021年3月16日 1:40
收件人: Laszlo Ersek <lersek@redhat.com>
抄送: discuss@edk2.groups.io; gaoliming <gaoliming@byosoft.com.cn>;
bob.c.feng@intel.com; yuwei.chen@intel.com
主题: Re: [edk2-discuss] 回复: Non-reproducible binaries generated by
GenFw

On Fri, 12 Mar 2021 at 18:37, Laszlo Ersek <lersek@redhat.com> wrote:
I'm failing at making this actually stick in the makefiles though...
Can you provide your platform DSC file?

What is your exact "build" command line? (I'm looking for the "-a",
"-b", "-t" options.)
Essentially, OvmfPkg/build.sh -a X86 -b RELEASE -t GCC5. It appears I
should be editing OvmfPkg/OvmfX86.dsc instead of ShellPkg.dsc.

Is it agreed that this should be fixed? Is putting
RELEASE_*_*_GENFW_FLAGS=-z all over the repository the right solution,
or is there a better solution that can be done once centrally?

Ross


Re: Non-reproducible binaries generated by GenFw

Andrew Fish
 

On Mar 10, 2021, at 11:36 AM, Ross Burton <ross@burtonini.com> wrote:

Hi,

As per https://bugzilla.tianocore.org/show_bug.cgi?id=3256, GenFw
writes non-reproducible binaries by embedding a build path. In fact in
a build of ovmf with embedded shell, this one path is the sole source
of non-determinism.

WriteDebug64() is always called in GenFw on output and that embeds
into the NB10 entry the input filename. As build paths change this is
a source of non-determinism. There already exists a --zero option to
wipe out debug paths but this is in release builds so I'm not sure
what the best solution is. I can see several options:

1) Release builds should not call WriteDebug64() at all
2) --zero should wipe the NB10 entry, and release builds should pass that
3) GenFw should support path remapping (like gcc's -ffile-prefix-map)
to turn build paths into something consistent.

Any suggestions?
This is kind of a tangent, but for clang we solved a similar problem a different way.
1) in the build_rule.txt file we added a cd $(WORKSPACE); on the compile line so __FILE__ is always relative to the $(WORKSPACE);
2) We added a post build step to create a Symbols directory in the Build output.
a) The debug directory entry paths are relative to that symbols directory. They generally look like IA32/PeiCore or X64/DxeCore etc.
b) In the old days we would launch the debugger from the Symbols directory so relative files got searched relative to the symbols directory.
c) Now we use the UUID we have in the EFI_IMAGE_DEBUG_CODEVIEW_RSDS_ENTRY[1]PE/COFF debug entry. On a Mac these mach-O UUIDs are automatically indexed by Spotlight so the debugger can just find them. It is also possible to write a tool to lookup these UUIDs that makes them available to the debugger.

I’m not sure it is possible to do something like this with all the toolchains, but I wanted to at least let people know we figured out a way with clang.

[1] https://github.com/tianocore/edk2/blob/master/MdePkg/Include/IndustryStandard/PeImage.h#L659

Thanks,

Andrew Fish

Cheers,
Ross





Re: 回复: Non-reproducible binaries generated by GenFw

Ross Burton
 

On Fri, 12 Mar 2021 at 18:37, Laszlo Ersek <lersek@redhat.com> wrote:
I'm failing at making this actually stick in the makefiles though...
Can you provide your platform DSC file?

What is your exact "build" command line? (I'm looking for the "-a",
"-b", "-t" options.)
Essentially, OvmfPkg/build.sh -a X86 -b RELEASE -t GCC5. It appears I
should be editing OvmfPkg/OvmfX86.dsc instead of ShellPkg.dsc.

Is it agreed that this should be fixed? Is putting
RELEASE_*_*_GENFW_FLAGS=-z all over the repository the right solution,
or is there a better solution that can be done once centrally?

Ross

201 - 220 of 834