Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.
2) After this change, how to append new link option in platform DSC? Use style -Wl, ?
3) I see GCC5 uses gcc-ar as its SLINK, and GCC49 uses ar as its SLINK. Is gcc-ar required only by LTO?
4) Before GCC49 optimization, GCC49 means GCC49 or later, GCC5 can work with GCC49 tool chain configuration. But now, I configure gcc to point to GCC5, and build OVMF with GCC49 tool chain, it reports GenFw failure. I expect GCC5 work with GCC49 and GCC5 tool chain both. GCC49 for no lto, GCC5 for lto. I know Steven has provided the patch to fix this GenFw issue.

GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid

Thanks
Liming

On 29 July 2016 at 06:47, Gao, Liming <liming.gao@...> wrote:

Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.

Yes, I tested all of them.

2) After this change, how to append new link option in platform DSC? Use style -Wl, ?

It depends. Some options (like -z) don't need it, but others do.

3) I see GCC5 uses gcc-ar as its SLINK, and GCC49 uses ar as its SLINK. Is gcc-ar required only by LTO?

Yes

4) Before GCC49 optimization, GCC49 means GCC49 or later, GCC5 can work with GCC49 tool chain configuration. But now, I configure gcc to point to GCC5, and build OVMF with GCC49 tool chain, it reports GenFw failure. I expect GCC5 work with GCC49 and GCC5 tool chain both. GCC49 for no lto, GCC5 for lto. I know Steven has provided the patch to fix this GenFw issue.

GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid

Which GCC version are you using?

-----Original Message-----
From: Ard Biesheuvel [mailto:ard.biesheuvel@...]
Sent: Wednesday, July 27, 2016 7:14 PM
To: edk2-devel@...; lersek@...; Gao, Liming
<liming.gao@...>; Shi, Steven <steven.shi@...>; Zhu,
Yonghong <yonghong.zhu@...>; Justen, Jordan L
<jordan.l.justen@...>
Cc: leif.lindholm@...; Ard Biesheuvel <ard.biesheuvel@...>
Subject: [PATCH v4 0/7] BaseTools: add support for GCC5 in LTO mode

This v4 to introduce GCC5 is now a 7 piece series, including some
preparatory cleanup patches that allow all GCC4x and CLANG35 toolchains
to switch to using 'gcc' as the linker. This allows us to get rid of
the wrapper script to marshall ld arguments in order to make them
understandable by gcc, which is fragile and likely to cause problems in
the future.

Since there appears to be a natural split between the 'legacy' GCC
toolchains UNIXGCC, ELFGCC, and CYGGCC[xASL], both in term of supported
architectures [IA32, X64, IPF] vs [IA32, X64, ARM, AARCH64], and in
terms of maintenance, these toolchains are not moved to using 'gcc' as
the linker, and instead, a new BUILDRULEFAMILY is introduced called GCCLD
that will retain the old behavior.

The result is that GCC5 can align much more closely with its predecessors,
making the expected maintenance burden of supporting GCC back to v4.4
much lower.

Changes since v3:
- like Steven does in his GCC5LTO patch, add -fno-builtin to IA32 and X64
CC_FLAGS; this addresses a build issue reported by Liming
- add -Os the the linker flags as well, for AARCH64 this does not seem to
make
a difference, but it is arguably correct since the LTO processing at link
time involves code generation as well
- add Laszlo's ack to #2
- new patch #6 to omit the autogenerated build-id from the PE/COFF binary

Changes since v2:
- add license headers to LTO glue files for ARM and AARCH64 (#5)
- get rid of lto-ld-wrapper script

Ard Biesheuvel (7):
BaseTools CLANG35: drop problematic use-movt and save-temps options
ArmVirtPkg/ArmVirtPrePiUniCoreRelocatable: ignore .hash and .note
sections
BaseTools UNIXGCC ELFGCC CYGGCC: clone GCC build rule family into
GCCLD
BaseTools GCC: use 'gcc' as the linker command for GCC44 and later
ArmPkg: add prebuilt glue binaries for GCC5 LTO support
BaseTools GCC: drop GNU notes section from EFI image
BaseTools GCC: add support for GCC v5.x in LTO mode

ArmPkg/GccLto/liblto-aarch64.a | Bin 0 -> 1016 bytes
ArmPkg/GccLto/liblto-aarch64.s | 27 ++
ArmPkg/GccLto/liblto-arm.a | Bin 0 -> 2096 bytes
ArmPkg/GccLto/liblto-arm.s | 61 ++++
ArmVirtPkg/PrePi/ArmVirtPrePiUniCoreRelocatable.inf | 2 +-
ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds | 3 +
BaseTools/Conf/build_rule.template | 31 +-
BaseTools/Conf/tools_def.template | 344 ++++++++++++++------
BaseTools/Scripts/GccBase.lds | 6 +
EmulatorPkg/Unix/Host/Host.inf | 6 +-
10 files changed, 372 insertions(+), 108 deletions(-)
create mode 100644 ArmPkg/GccLto/liblto-aarch64.a
create mode 100644 ArmPkg/GccLto/liblto-aarch64.s
create mode 100644 ArmPkg/GccLto/liblto-arm.a
create mode 100644 ArmPkg/GccLto/liblto-arm.s

--
2.7.4

On 29 July 2016 at 08:09, Ard Biesheuvel <ard.biesheuvel@...> wrote:

On 29 July 2016 at 06:47, Gao, Liming <liming.gao@...> wrote:

Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.

Yes, I tested all of them.

2) After this change, how to append new link option in platform DSC? Use style -Wl, ?

It depends. Some options (like -z) don't need it, but others do.

3) I see GCC5 uses gcc-ar as its SLINK, and GCC49 uses ar as its SLINK. Is gcc-ar required only by LTO?

Yes

4) Before GCC49 optimization, GCC49 means GCC49 or later, GCC5 can work with GCC49 tool chain configuration. But now, I configure gcc to point to GCC5, and build OVMF with GCC49 tool chain, it reports GenFw failure. I expect GCC5 work with GCC49 and GCC5 tool chain both. GCC49 for no lto, GCC5 for lto. I know Steven has provided the patch to fix this GenFw issue.

GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid

Which GCC version are you using?

I cannot reproduce this with gcc version 5.4.0 20160609 (Ubuntu
5.4.0-6ubuntu1~16.04.1)

In any case, I think we should merge Steven's patch that adds handling
to the relocation types to GenFw. The issue is only that having a GOT
does not make a lot of sense for UEFI executables, since it forces a
symbol reference to be absolute, which uses more space in the code,
but also in the .reloc section. The visibility pragma I introduced for
GCC4x was intended to prevent GOT based relocations from being
emitted.

Ard:
My gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04). I will try GCC54.

Besides, for new GCC5 tool chain, could you add one brief introduction in tools_def.txt like GCC49? And, highlight it enable LTO by default.

# GCC49 -Linux,Windows- Requires:
# GCC 4.9 targeting x86_64-linux-gnu, aarch64-linux-gnu, or arm-linux-gnueabi
# Optional:
# Required to build platforms or ACPI tables:
# Intel(r) ACPI Compiler from
# https://acpica.org/downloads


Thanks
Liming
From: Ard Biesheuvel [mailto:ard.biesheuvel@...]
Sent: Friday, July 29, 2016 2:41 PM
To: Gao, Liming <liming.gao@...>
Cc: edk2-devel@...; lersek@...; Shi, Steven <steven.shi@...>; Zhu, Yonghong <yonghong.zhu@...>; Justen, Jordan L <jordan.l.justen@...>; leif.lindholm@...
Subject: Re: [PATCH v4 0/7] BaseTools: add support for GCC5 in LTO mode

On 29 July 2016 at 08:09, Ard Biesheuvel wrote:

On 29 July 2016 at 06:47, Gao, Liming wrote:

Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.

Yes, I tested all of them.

2) After this change, how to append new link option in platform DSC? Use style -Wl, ?

It depends. Some options (like -z) don't need it, but others do.

3) I see GCC5 uses gcc-ar as its SLINK, and GCC49 uses ar as its SLINK. Is gcc-ar required only by LTO?

Yes

4) Before GCC49 optimization, GCC49 means GCC49 or later, GCC5 can work with GCC49 tool chain configuration. But now, I configure gcc to point to GCC5, and build OVMF with GCC49 tool chain, it reports GenFw failure. I expect GCC5 work with GCC49 and GCC5 tool chain both. GCC49 for no lto, GCC5 for lto. I know Steven has provided the patch to fix this GenFw issue.

GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid

Which GCC version are you using?

I cannot reproduce this with gcc version 5.4.0 20160609 (Ubuntu
5.4.0-6ubuntu1~16.04.1)

In any case, I think we should merge Steven's patch that adds handling
to the relocation types to GenFw. The issue is only that having a GOT
does not make a lot of sense for UEFI executables, since it forces a
symbol reference to be absolute, which uses more space in the code,
but also in the .reloc section. The visibility pragma I introduced for
GCC4x was intended to prevent GOT based relocations from being
emitted.

On 29 July 2016 at 17:18, Gao, Liming <liming.gao@...> wrote:

Ard:

My gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04). I will try
GCC54.

Could you check if the issue also occurs with the 'old' GCC49, i.e.,
the version that uses LD as the linker?

Besides, for new GCC5 tool chain, could you add one brief introduction in
tools_def.txt like GCC49? And, highlight it enable LTO by default.



# GCC49 -Linux,Windows- Requires:

# GCC 4.9 targeting x86_64-linux-gnu,
aarch64-linux-gnu, or arm-linux-gnueabi

# Optional:

# Required to build platforms or ACPI tables:

# Intel(r) ACPI Compiler from

# https://acpica.org/downloads

OK

--
Ard.

On 2016-07-28 23:09:15, Ard Biesheuvel wrote:

On 29 July 2016 at 06:47, Gao, Liming <liming.gao@...> wrote:

Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.

Yes, I tested all of them.

I tested GCC44 (X64) on an old live-cd where GCC 4.4 was the supported
version. OVMF built and booted UEFI Linux. Therefore, I don't think we
are likely to have a major issue with GCC44-GCC49.

I also tested GCC49 and GCC5 with X64 on a system with GCC 5.4. It
built and booted UEFI Linux.

Can you add something like 'added GCC5 toolchain' into the subject for
patch 7? I really think that a patch that adds a new toolchain should
make it immediately obvious in the subject line.

3, 4 & 7 Reviewed-by: Jordan Justen <jordan.l.justen@...>

The others, Acked-by: Jordan Justen <jordan.l.justen@...>

2) After this change, how to append new link option in platform DSC? Use style -Wl, ?

It depends. Some options (like -z) don't need it, but others do.

3) I see GCC5 uses gcc-ar as its SLINK, and GCC49 uses ar as its SLINK. Is gcc-ar required only by LTO?

Yes

4) Before GCC49 optimization, GCC49 means GCC49 or later, GCC5 can work with GCC49 tool chain configuration. But now, I configure gcc to point to GCC5, and build OVMF with GCC49 tool chain, it reports GenFw failure. I expect GCC5 work with GCC49 and GCC5 tool chain both. GCC49 for no lto, GCC5 for lto. I know Steven has provided the patch to fix this GenFw issue.

GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid
/home/hwu/work/lgao4/AllPkg/Build/Ovmf3264/DEBUG_GCC49/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll unsupported ELF EM_X86_64 relocation 0x9.
GenFw: ERROR 3000: Invalid

Which GCC version are you using?

-----Original Message-----
From: Ard Biesheuvel [mailto:ard.biesheuvel@...]
Sent: Wednesday, July 27, 2016 7:14 PM
To: edk2-devel@...; lersek@...; Gao, Liming
<liming.gao@...>; Shi, Steven <steven.shi@...>; Zhu,
Yonghong <yonghong.zhu@...>; Justen, Jordan L
<jordan.l.justen@...>
Cc: leif.lindholm@...; Ard Biesheuvel <ard.biesheuvel@...>
Subject: [PATCH v4 0/7] BaseTools: add support for GCC5 in LTO mode

This v4 to introduce GCC5 is now a 7 piece series, including some
preparatory cleanup patches that allow all GCC4x and CLANG35 toolchains
to switch to using 'gcc' as the linker. This allows us to get rid of
the wrapper script to marshall ld arguments in order to make them
understandable by gcc, which is fragile and likely to cause problems in
the future.

Since there appears to be a natural split between the 'legacy' GCC
toolchains UNIXGCC, ELFGCC, and CYGGCC[xASL], both in term of supported
architectures [IA32, X64, IPF] vs [IA32, X64, ARM, AARCH64], and in
terms of maintenance, these toolchains are not moved to using 'gcc' as
the linker, and instead, a new BUILDRULEFAMILY is introduced called GCCLD
that will retain the old behavior.

The result is that GCC5 can align much more closely with its predecessors,
making the expected maintenance burden of supporting GCC back to v4.4
much lower.

Changes since v3:
- like Steven does in his GCC5LTO patch, add -fno-builtin to IA32 and X64
CC_FLAGS; this addresses a build issue reported by Liming
- add -Os the the linker flags as well, for AARCH64 this does not seem to
make
a difference, but it is arguably correct since the LTO processing at link
time involves code generation as well
- add Laszlo's ack to #2
- new patch #6 to omit the autogenerated build-id from the PE/COFF binary

Changes since v2:
- add license headers to LTO glue files for ARM and AARCH64 (#5)
- get rid of lto-ld-wrapper script

Ard Biesheuvel (7):
BaseTools CLANG35: drop problematic use-movt and save-temps options
ArmVirtPkg/ArmVirtPrePiUniCoreRelocatable: ignore .hash and .note
sections
BaseTools UNIXGCC ELFGCC CYGGCC: clone GCC build rule family into
GCCLD
BaseTools GCC: use 'gcc' as the linker command for GCC44 and later
ArmPkg: add prebuilt glue binaries for GCC5 LTO support
BaseTools GCC: drop GNU notes section from EFI image
BaseTools GCC: add support for GCC v5.x in LTO mode

ArmPkg/GccLto/liblto-aarch64.a | Bin 0 -> 1016 bytes
ArmPkg/GccLto/liblto-aarch64.s | 27 ++
ArmPkg/GccLto/liblto-arm.a | Bin 0 -> 2096 bytes
ArmPkg/GccLto/liblto-arm.s | 61 ++++
ArmVirtPkg/PrePi/ArmVirtPrePiUniCoreRelocatable.inf | 2 +-
ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds | 3 +
BaseTools/Conf/build_rule.template | 31 +-
BaseTools/Conf/tools_def.template | 344 ++++++++++++++------
BaseTools/Scripts/GccBase.lds | 6 +
EmulatorPkg/Unix/Host/Host.inf | 6 +-
10 files changed, 372 insertions(+), 108 deletions(-)
create mode 100644 ArmPkg/GccLto/liblto-aarch64.a
create mode 100644 ArmPkg/GccLto/liblto-aarch64.s
create mode 100644 ArmPkg/GccLto/liblto-arm.a
create mode 100644 ArmPkg/GccLto/liblto-arm.s

--
2.7.4

_______________________________________________
edk2-devel mailing list
edk2-devel@...
https://lists.01.org/mailman/listinfo/edk2-devel

On 29 July 2016 at 22:33, Jordan Justen <jordan.l.justen@...> wrote:

On 2016-07-28 23:09:15, Ard Biesheuvel wrote:

On 29 July 2016 at 06:47, Gao, Liming <liming.gao@...> wrote:

Ard:
Thanks for your update. I have some comments for them.
1) It uses GCC as Link for GCC44-GCC49. Have you done verification on them? I verify GCC49 in OVMFIa32X64 platform. It works.

Yes, I tested all of them.

I tested GCC44 (X64) on an old live-cd where GCC 4.4 was the supported
version. OVMF built and booted UEFI Linux. Therefore, I don't think we
are likely to have a major issue with GCC44-GCC49.

I also tested GCC49 and GCC5 with X64 on a system with GCC 5.4. It
built and booted UEFI Linux.

OK, so you are not seeing the issue Liming reported when using the
GCC49 toolchain tag with GCC 5.4?

In that case, I am inclined to diagnose Liming's issue as a GCC 5.x
regression that was fixed in 5.4. Adding the GOT handling to GenFw
would still be possible, of course, but GOT based symbol references
are sub-optimal so we should try to avoid them imo. If GCC 4.x does
not emit any such references with protected visibility enabled, and
nor does GCC 5.4, I think it it reasonable to require that GCC 5.3
users simply use the GCC5 profile (with LTO), and switch to either 4.9
or 5.4 if they want to use the GCC49 profile instead.

Can you add something like 'added GCC5 toolchain' into the subject for
patch 7? I really think that a patch that adds a new toolchain should
make it immediately obvious in the subject line.

Of course, that makes sense.

3, 4 & 7 Reviewed-by: Jordan Justen <jordan.l.justen@...>

The others, Acked-by: Jordan Justen <jordan.l.justen@...>

Thanks,
Ard.

On 29 July 2016 at 17:18, Gao, Liming <liming.gao@...> wrote:

Ard:

My gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04). I will try
GCC54.

I cannot reproduce this with 5.3.1-14ubuntu2.1 either.

On 30 July 2016 at 16:34, Ard Biesheuvel <ard.biesheuvel@...> wrote:

On 29 July 2016 at 17:18, Gao, Liming <liming.gao@...> wrote:

Ard:

My gcc version 5.3.0 20151204 (Ubuntu 5.3.0-3ubuntu1~14.04). I will try
GCC54.

I cannot reproduce this with 5.3.1-14ubuntu2.1 either.

@Liming: is the issue still reproducible with this change?

"""
diff --git a/MdePkg/Include/X64/ProcessorBind.h
b/MdePkg/Include/X64/ProcessorBind.h
index a4aad3e..73cf799 100644
--- a/MdePkg/Include/X64/ProcessorBind.h
+++ b/MdePkg/Include/X64/ProcessorBind.h
@@ -34,7 +34,7 @@
// symbols directly using relative references rather than via the GOT, which
// contains absolute symbol addresses that are subject to runtime relocation.
//
-#pragma GCC visibility push (protected)
+#pragma GCC visibility push (hidden)
#endif

#if defined(__INTEL_COMPILER)
"""

Thanks,
Ard.

Ard:
My GNU ld (GNU Binutils for Ubuntu) 2.24. Which version you use?

1. #pragma GCC visibility push (hidden) , GCC5 with GCC49 tool chain pass. GCC5 with GCC5 tool chain failure. Here is failure message.
GenFw: Elf64Convert.c:424: ScanSections64: Assertion `((BOOLEAN)(0==1))' failed.
GenFw: ERROR 3000: Invalid
Did not find any '.text' section.
Aborted (core dumped)

2. #pragma GCC visibility push (protected), GCC5 with GCC49 tool chain failure, GCC5 with GCC5 tool chain pass. Failure message is below.

Thanks
Liming

On 1 August 2016 at 04:26, Gao, Liming <liming.gao@...> wrote:

Ard:
My GNU ld (GNU Binutils for Ubuntu) 2.24. Which version you use?

1. #pragma GCC visibility push (hidden) , GCC5 with GCC49 tool chain pass. GCC5 with GCC5 tool chain failure. Here is failure message.
GenFw: Elf64Convert.c:424: ScanSections64: Assertion `((BOOLEAN)(0==1))' failed.
GenFw: ERROR 3000: Invalid
Did not find any '.text' section.
Aborted (core dumped)

2. #pragma GCC visibility push (protected), GCC5 with GCC49 tool chain failure, GCC5 with GCC5 tool chain pass. Failure message is below.

Thanks for checking. The problem with GCC5 is that the fact that
'_ModuleEntryPoint' is also hidden, which confuses the LTO code and
makes it eliminate all input objects. I think this is a bug in LTO,
since the entry point is passed explicitly to the linker using the -e
option. But we still need to work around it.

Since the current issue (#2) is a problem with GCC49, I will propose a
separate patch to fix it by changing the 'protected' to 'hidden'. I
will then add a patch to my GCC5 series as well to work around the LTO
problem.

Thanks,
Ard.