Date   

Re: [PATCH] .azurepipelines: Enable CI for OvmfPkg and EmulatorPkg

Laszlo Ersek
 

On 04/06/20 12:11, Ard Biesheuvel wrote:
On 4/5/20 8:39 AM, Sean via groups.io wrote:
Platform and Core CI patches are ready for review.  I have 14 commits
here.
https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages


This adds support for "Platform CI" for ArmVirtPkg, OvmfPkg, and
EmulatorPkg.
Each readme has live status and links to the builds as well as details
of how to build and run the same way the CI server will.
ArmVirt:
https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/ArmVirtPkg/README-pytools.md

Emulator:
https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/EmulatorPkg/README-pytools.md

OvmfPkg:
https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/OvmfPkg/README-pytools.md


It also adds ArmVirtPkg, OvmfPkg, and EmulatorPkg to Core CI just for
the code evaluation tests (not compiling).  Details of those tests are
here:
https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages/.pytool


ArmVirtPkg, OvmfPkg, and EmulatorPkg maintainers please review and let
me know what the next step for you package is.  This is ready to
commit from my perspective and has already caught a few issues in the
last couple of weeks.
Thanks Sean. This is all looking really good. I think this stuff is
ready to be sent to the mailing list for wider review.

Finally there are a few pending bugs that should get fixed and at
least a couple of gaps I have identified. These are not fixed in the
above 14 commits.

 1. OvmfPkg
     1. https://bugzilla.tianocore.org/show_bug.cgi?id=2662 -- Blocking
        Core CI
     2. https://bugzilla.tianocore.org/show_bug.cgi?id=2661 -- Blocking
Thanks Sean!

I've posted a patch for 2662, and commented on 2661.

        Platform CI
     3. Spell check in audit mode
 2. EmulatorPkg
     1. https://bugzilla.tianocore.org/show_bug.cgi?id=2663 -- Ignore
        added to Core CI
     2. https://bugzilla.tianocore.org/show_bug.cgi?id=2639 -- Feature
        in Platform CI turned off
     3. https://bugzilla.tianocore.org/show_bug.cgi?id=2638 -- Feature
        in Platform CI turned off
     4. Spell check in audit mode
 3. ArmVirtPkg
     1. No builds on Windows or with VS toolchain -- Feature in Platform
        CI turned off
For the ArmVirtPkg "README-pytools.md" file, I have a superficial comment:

The "E1000_ENABLE" reference may not be the best example (I understand
it's an example) with ArmVirtPkg, as E1000_ENABLE isn't actually used in
the ArmVirtPkg DSC/FDF files.

Thanks
Laszlo


Leif has looked into this in the past. As I understand it, it is simply
a lack of .asm versions of the various assembler source files in ArmLib,
ArmMmuLib and the startup code in ArmPlatformPkg.



Re: [PATCH v2 23/28] NXP/LS1043aRdbPkg/ArmPlatformLib: Use Allocate pool

Leif Lindholm
 

+Ard

On Mon, Apr 06, 2020 at 15:26:45 +0000, Pankaj Bansal (OSS) wrote:
-----Original Message-----
From: Leif Lindholm <leif@...>
Sent: Wednesday, April 1, 2020 11:34 PM
To: Pankaj Bansal (OSS) <pankaj.bansal@...>
Cc: Meenakshi Aggarwal <meenakshi.aggarwal@...>; Michael D Kinney
<michael.d.kinney@...>; devel@edk2.groups.io; Varun Sethi
<V.Sethi@...>; Samer El-Haj-Mahmoud <Samer.El-Haj-
Mahmoud@...>; Jon Nettleton <jon@...>
Subject: Re: [PATCH v2 23/28] NXP/LS1043aRdbPkg/ArmPlatformLib: Use
Allocate pool

On Fri, Mar 20, 2020 at 20:05:38 +0530, Pankaj Bansal wrote:
From: Pankaj Bansal <pankaj.bansal@...>

Allocate Pages may allocate more memory than required for
VirtualMemoryTable.
There is no special requirement that VirtualMemoryTable size should be
page size aligned.

Therefore, replace AllocatePages with AllocatePool.

Signed-off-by: Pankaj Bansal <pankaj.bansal@...>
I don't object to this as such (although one comment), but what is the
purpose of this change?

My comment is that most other platforms use AllocatePages for this. So
this is diverging from the norm.
I referred ArmVirtPkg/Library/QemuVirtMemInfoLib/QemuVirtMemInfoLib.c
Sure. This one was converted to AllocatePool because the QEMU virt
machine is very simple (because it does not emulate much real
hardware) and the port rarely changes.

Ard's what's your opinion - do you think this worth it even for a
platform that has
#define MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS 25
?

Clearly there is still wasteage going on, but it's 16% less bad.

Secondly, while I don't necessarily
*like* the current design (copied across most ARM platforms), it's
somewhat mitigated by the AllocatePages giving you a minimum of 128
entries before you start overwriting things. I don't know what you'll
overwrite if you do go too far, but you will overwrite it *before* the
ASSERT ever gets evaluated.
We can improve this by evaluating ASSERT after each entry like this :
VirtualMemoryTable[Index].PhysicalBase = 0xXXXXXXXX;
VirtualMemoryTable[Index].VirtualBase = 0xXXXXXXXX;
VirtualMemoryTable[Index].Length = 0xXXXXXXXX;
VirtualMemoryTable[Index++].Attributes = 0xXXXXXXXX;

ASSERT (Index < MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS);
Whilst functionally preferable, I think that would make for very
tedious reading. I'll let Ard call this one.

/
Leif

/
Leif

---
.../LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLib.inf | 1 +
.../LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLibMem.c | 5 +++--
2 files changed, 4 insertions(+), 2 deletions(-)

diff --git
a/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLib.inf
b/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLib.inf
index 1faf99b99c54..c64032f32772 100644
---
a/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLib.inf
+++
b/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLib.inf
@@ -25,6 +25,7 @@ [Packages]

[LibraryClasses]
ArmLib
+ DebugLib
SocLib

[Sources.common]
diff --git
a/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLibMem.
c
b/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLibMem.
c
index f5fa308551aa..f8dd642e3cff 100644
---
a/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLibMem.
c
+++
b/Platform/NXP/LS1043aRdbPkg/Library/ArmPlatformLib/ArmPlatformLibMem.
c
@@ -43,10 +43,11 @@ ArmPlatformGetVirtualMemoryMap (

ASSERT (VirtualMemoryMap != NULL);

- VirtualMemoryTable =
(ARM_MEMORY_REGION_DESCRIPTOR*)AllocatePages (
- EFI_SIZE_TO_PAGES (sizeof (ARM_MEMORY_REGION_DESCRIPTOR) *
MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS));
+ VirtualMemoryTable = AllocatePool (sizeof
(ARM_MEMORY_REGION_DESCRIPTOR) *
+ MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS);

if (VirtualMemoryTable == NULL) {
+ DEBUG ((DEBUG_ERROR, "%a: Error: Failed AllocatePool()\n",
__FUNCTION__));
return;
}

--
2.17.1


Re: [PATCH v2 21/28] Slicon/NXP: Add PlatformPei Lib

Leif Lindholm
 

OK, taking another look at this patch, this simply needs to be
deleted. Here is the sum total relevant difference compared to the
ArmPlatformPkg one.

DEBUG ((DEBUG_INIT, "Edk2 version is %a\n", XPRINT (WORKSPACE_GIT_VERSION)));
DEBUG ((DEBUG_INIT, "Edk2 platforms version is %a\n", XPRINT (PACKAGES_PATH_GIT_VERSION)));

If all you want to do is to print that sort of thing, please don't
fork a core library to do so.

First of all, please do like most other platforms and override
gEfiMdeModulePkgTokenSpaceGuid.PcdFirmwareVersionString
if -D FIRMWARE_VER is specified on your build command line.

You can then extract current top commits of your respective
repositories and not worry about getting this.
I would suggest iterating across all locations in PACKAGES_PATH and
then doing something similar to
https://git.linaro.org/uefi/uefi-tools.git/tree/edk2-build.sh#n400
appending together.

If something like this should be integrated into the build system
(which might not be a bad idea), then it needs to be so properly,
rather than shoehorned in for each platform.
(In the past, this was difficult because we supported both git and
svn, but I would say we have given up pretending that is possible.)

For now, you could add the printout in a standalone Depex TRUE PEIM
added to your [FV.FVMAIN_COMPACT].

/
Leif

On Mon, Apr 06, 2020 at 14:53:02 +0000, Pankaj Bansal (OSS) wrote:


-----Original Message-----
From: Leif Lindholm <leif@...>
Sent: Wednesday, April 1, 2020 8:23 PM
To: Pankaj Bansal (OSS) <pankaj.bansal@...>
Cc: Meenakshi Aggarwal <meenakshi.aggarwal@...>; Michael D Kinney
<michael.d.kinney@...>; devel@edk2.groups.io; Varun Sethi
<V.Sethi@...>; Samer El-Haj-Mahmoud <Samer.El-Haj-
Mahmoud@...>; Jon Nettleton <jon@...>
Subject: Re: [PATCH v2 21/28] Slicon/NXP: Add PlatformPei Lib

On Fri, Mar 20, 2020 at 20:05:36 +0530, Pankaj Bansal wrote:
From: Pankaj Bansal <pankaj.bansal@...>

PlatformPeiLib is going to be linked to Platform PEIM.

Signed-off-by: Pankaj Bansal <pankaj.bansal@...>
---
.../Library/PlatformPeiLib/PlatformPeiLib.c | 30 ++++++++++++++
.../Library/PlatformPeiLib/PlatformPeiLib.inf | 41 +++++++++++++++++++
Silicon/NXP/NxpQoriqLs.dsc.inc | 3 +-
3 files changed, 73 insertions(+), 1 deletion(-)
create mode 100644 Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.c
create mode 100644 Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.inf

diff --git a/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.c
b/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.c
new file mode 100644
index 000000000000..f64e564469f8
--- /dev/null
+++ b/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.c
@@ -0,0 +1,30 @@
+/** @file
+*
+* Copyright (c) 2011-2014, ARM Limited. All rights reserved.
+* Copyright 2020 NXP
+*
+* SPDX-License-Identifier: BSD-2-Clause-Patent
+*
+**/
+
+#include <PiPei.h>
+
+#include <Library/HobLib.h>
+#include <Library/DebugLib.h>
+#include <Library/PcdLib.h>
Although this is based on an existing library, please sort includes
alphabetically here.

+
+#define XPRINT(x) PRINT(x)
+#define PRINT(x) #x
This isn't a PRINT operation, this is a Stringize operation.
OK, I can rename these to
#define PRINTSTR(x) STR(x)
#define STR(x) #x


+
+EFI_STATUS
+EFIAPI
+PlatformPeim (
+ VOID
+ )
+{
+ BuildFvHob (PcdGet64 (PcdFvBaseAddress), PcdGet32 (PcdFvSize));
+ DEBUG ((DEBUG_INIT, "Edk2 version is %a\n", XPRINT
(WORKSPACE_GIT_VERSION)));
+ DEBUG ((DEBUG_INIT, "Edk2 platforms version is %a\n", XPRINT
(PACKAGES_PATH_GIT_VERSION)));

The only benefit I can see from the macro as opposed to using '#'
directly is that it permits wrapping of too long lines, so please do
that.
OK.


+
+ return EFI_SUCCESS;
+}
diff --git a/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.inf
b/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.inf
new file mode 100644
index 000000000000..fb42693daa20
--- /dev/null
+++ b/Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.inf
@@ -0,0 +1,41 @@
+#/** @file
+#
+# Copyright (c) 2011-2012, ARM Limited. All rights reserved.
+# Copyright 2020 NXP
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+#**/
+
+[Defines]
+ INF_VERSION = 0x00010005
Update version.

+ BASE_NAME = ArmPlatformPeiLib
+ FILE_GUID = 49d37060-70b5-11e0-aa2d-0002a5d5c51b
Unless this is another magic GUID filename (like ACPI), please
generate a new GUID.
Since this Library is replacement implementation of
ArmPlatformPkg/PlatformPei/PlatformPeiLib.inf,
Couldn't we use the same GUID ?


+ MODULE_TYPE = PEIM
+ VERSION_STRING = 1.0
+ LIBRARY_CLASS = PlatformPeiLib
+
+[BuildOptions]
+ GCC:*_*_*_CC_FLAGS = -
DWORKSPACE_GIT_VERSION="$(WORKSPACE_GIT_VERSION)"
+ GCC:*_*_*_CC_FLAGS = -
DPACKAGES_PATH_GIT_VERSION="$(PACKAGES_PATH_GIT_VERSION)"

Does this not require special magic build command line options to do
anything useful? This needs documenting.
Actually I had submitted a patch is BaseTools for this:
https://edk2.groups.io/g/devel/message/53146

This patch makes use of that BaseTools patch.
But the BaseTools patch was not accepted because that is Linux specific.
Still these changes don't cause any negative affect.
Without BaseTools patch, empty string would be printed.


/
Leif

+
+[Sources]
+ PlatformPeiLib.c
+
+[Packages]
+ ArmPkg/ArmPkg.dec
+ MdeModulePkg/MdeModulePkg.dec
+ MdePkg/MdePkg.dec
+ Silicon/NXP/NxpQoriqLs.dec
+
+[LibraryClasses]
+ DebugLib
+ HobLib
+ PcdLib
+
+[FixedPcd]
+ gArmTokenSpaceGuid.PcdFvBaseAddress
+ gArmTokenSpaceGuid.PcdFvSize
+
+[depex]
+ TRUE
diff --git a/Silicon/NXP/NxpQoriqLs.dsc.inc b/Silicon/NXP/NxpQoriqLs.dsc.inc
index 234a5e2707cd..5f77f47f0399 100644
--- a/Silicon/NXP/NxpQoriqLs.dsc.inc
+++ b/Silicon/NXP/NxpQoriqLs.dsc.inc
@@ -101,6 +101,8 @@ [LibraryClasses.common]
PciExpressLib|MdePkg/Library/BasePciExpressLib/BasePciExpressLib.inf
PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf

+ PlatformPeiLib|Silicon/NXP/Library/PlatformPeiLib/PlatformPeiLib.inf
+
[LibraryClasses.common.SEC]
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
UefiDecompressLib|MdePkg/Library/BaseUefiDecompressLib/BaseUefiDecomp
ressLib.inf
@@ -111,7 +113,6 @@ [LibraryClasses.common.SEC]
PrePiHobListPointerLib|ArmPlatformPkg/Library/PrePiHobListPointerLib/PrePiH
obListPointerLib.inf
MemoryAllocationLib|EmbeddedPkg/Library/PrePiMemoryAllocationLib/PrePiM
emoryAllocationLib.inf
PerformanceLib|MdeModulePkg/Library/PeiPerformanceLib/PeiPerformanceLib
.inf
- PlatformPeiLib|ArmPlatformPkg/PlatformPei/PlatformPeiLib.inf
MemoryInitPeiLib|Silicon/NXP/Library/MemoryInitPei/MemoryInitPeiLib.inf

# 1/123 faster than Stm or Vstm version
--
2.17.1


Re: [PATCH] OvmfPkg: supply missing lib class declarations in the DEC file

Philippe Mathieu-Daudé <philmd@...>
 

On 4/7/20 12:05 PM, Laszlo Ersek wrote:
List the header files in the OvmfPkg DEC file for the following lib
classes:
- MemEncryptSevLib (one instance: BaseMemEncryptSevLib)
- PlatformFvbLib (two instances: EmuVariableFvbLib, PlatformFvbLibNull)
- VirtioLib (one instance: VirtioLib)
- VirtioMmioDeviceLib (one instance: VirtioMmioDeviceLib)
Cc: Ard Biesheuvel <ard.biesheuvel@...>
Cc: Jordan Justen <jordan.l.justen@...>
Cc: Philippe Mathieu-Daudé <philmd@...>
Cc: Sean Brogan <sean.brogan@...>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2662
Signed-off-by: Laszlo Ersek <lersek@...>
Reviewed-by: Philippe Mathieu-Daude <philmd@...>

---
Notes:
Repo: https://pagure.io/lersek/edk2.git
Branch: lib_classes_bz_2662
OvmfPkg/OvmfPkg.dec | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index eae4d5e7ab42..28030391cff2 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -22,6 +22,10 @@ [LibraryClasses]
#
LoadLinuxLib|Include/Library/LoadLinuxLib.h
+ ## @libraryclass Declares helper functions for Secure Encrypted
+ # Virtualization (SEV) guests.
+ MemEncryptSevLib|Include/Library/MemEncryptSevLib.h
+
## @libraryclass Save and restore variables using a file
#
NvVarsFileLib|Include/Library/NvVarsFileLib.h
@@ -45,6 +49,9 @@ [LibraryClasses]
# return codes, to the UEFI console.
PlatformBmPrintScLib|Include/Library/PlatformBmPrintScLib.h
+ ## @libraryclass Customize FVB2 protocol member functions for a platform.
+ PlatformFvbLib|Include/Library/PlatformFvbLib.h
+
## @libraryclass Access QEMU's firmware configuration interface
#
QemuFwCfgLib|Include/Library/QemuFwCfgLib.h
@@ -67,6 +74,13 @@ [LibraryClasses]
#
SerializeVariablesLib|Include/Library/SerializeVariablesLib.h
+ ## @libraryclass Declares utility functions for virtio device drivers.
+ VirtioLib|Include/Library/VirtioLib.h
+
+ ## @libraryclass Install Virtio Device Protocol instances on virtio-mmio
+ # transports.
+ VirtioMmioDeviceLib|Include/Library/VirtioMmioDeviceLib.h
+
## @libraryclass Invoke Xen hypercalls
#
XenHypercallLib|Include/Library/XenHypercallLib.h


Re: [PATCH] OvmfPkg: supply missing lib class declarations in the DEC file

Ard Biesheuvel
 

On 4/7/20 12:05 PM, Laszlo Ersek wrote:
List the header files in the OvmfPkg DEC file for the following lib
classes:
- MemEncryptSevLib (one instance: BaseMemEncryptSevLib)
- PlatformFvbLib (two instances: EmuVariableFvbLib, PlatformFvbLibNull)
- VirtioLib (one instance: VirtioLib)
- VirtioMmioDeviceLib (one instance: VirtioMmioDeviceLib)
Cc: Ard Biesheuvel <ard.biesheuvel@...>
Cc: Jordan Justen <jordan.l.justen@...>
Cc: Philippe Mathieu-Daudé <philmd@...>
Cc: Sean Brogan <sean.brogan@...>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2662
Signed-off-by: Laszlo Ersek <lersek@...>
Reviewed-by: Ard Biesheuvel <ard.biesheuvel@...>

---
Notes:
Repo: https://pagure.io/lersek/edk2.git
Branch: lib_classes_bz_2662
OvmfPkg/OvmfPkg.dec | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index eae4d5e7ab42..28030391cff2 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -22,6 +22,10 @@ [LibraryClasses]
#
LoadLinuxLib|Include/Library/LoadLinuxLib.h
+ ## @libraryclass Declares helper functions for Secure Encrypted
+ # Virtualization (SEV) guests.
+ MemEncryptSevLib|Include/Library/MemEncryptSevLib.h
+
## @libraryclass Save and restore variables using a file
#
NvVarsFileLib|Include/Library/NvVarsFileLib.h
@@ -45,6 +49,9 @@ [LibraryClasses]
# return codes, to the UEFI console.
PlatformBmPrintScLib|Include/Library/PlatformBmPrintScLib.h
+ ## @libraryclass Customize FVB2 protocol member functions for a platform.
+ PlatformFvbLib|Include/Library/PlatformFvbLib.h
+
## @libraryclass Access QEMU's firmware configuration interface
#
QemuFwCfgLib|Include/Library/QemuFwCfgLib.h
@@ -67,6 +74,13 @@ [LibraryClasses]
#
SerializeVariablesLib|Include/Library/SerializeVariablesLib.h
+ ## @libraryclass Declares utility functions for virtio device drivers.
+ VirtioLib|Include/Library/VirtioLib.h
+
+ ## @libraryclass Install Virtio Device Protocol instances on virtio-mmio
+ # transports.
+ VirtioMmioDeviceLib|Include/Library/VirtioMmioDeviceLib.h
+
## @libraryclass Invoke Xen hypercalls
#
XenHypercallLib|Include/Library/XenHypercallLib.h


[PATCH] OvmfPkg: supply missing lib class declarations in the DEC file

Laszlo Ersek
 

List the header files in the OvmfPkg DEC file for the following lib
classes:

- MemEncryptSevLib (one instance: BaseMemEncryptSevLib)

- PlatformFvbLib (two instances: EmuVariableFvbLib, PlatformFvbLibNull)

- VirtioLib (one instance: VirtioLib)

- VirtioMmioDeviceLib (one instance: VirtioMmioDeviceLib)

Cc: Ard Biesheuvel <ard.biesheuvel@...>
Cc: Jordan Justen <jordan.l.justen@...>
Cc: Philippe Mathieu-Daudé <philmd@...>
Cc: Sean Brogan <sean.brogan@...>
Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2662
Signed-off-by: Laszlo Ersek <lersek@...>
---

Notes:
Repo: https://pagure.io/lersek/edk2.git
Branch: lib_classes_bz_2662

OvmfPkg/OvmfPkg.dec | 14 ++++++++++++++
1 file changed, 14 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index eae4d5e7ab42..28030391cff2 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -22,6 +22,10 @@ [LibraryClasses]
#
LoadLinuxLib|Include/Library/LoadLinuxLib.h

+ ## @libraryclass Declares helper functions for Secure Encrypted
+ # Virtualization (SEV) guests.
+ MemEncryptSevLib|Include/Library/MemEncryptSevLib.h
+
## @libraryclass Save and restore variables using a file
#
NvVarsFileLib|Include/Library/NvVarsFileLib.h
@@ -45,6 +49,9 @@ [LibraryClasses]
# return codes, to the UEFI console.
PlatformBmPrintScLib|Include/Library/PlatformBmPrintScLib.h

+ ## @libraryclass Customize FVB2 protocol member functions for a platform.
+ PlatformFvbLib|Include/Library/PlatformFvbLib.h
+
## @libraryclass Access QEMU's firmware configuration interface
#
QemuFwCfgLib|Include/Library/QemuFwCfgLib.h
@@ -67,6 +74,13 @@ [LibraryClasses]
#
SerializeVariablesLib|Include/Library/SerializeVariablesLib.h

+ ## @libraryclass Declares utility functions for virtio device drivers.
+ VirtioLib|Include/Library/VirtioLib.h
+
+ ## @libraryclass Install Virtio Device Protocol instances on virtio-mmio
+ # transports.
+ VirtioMmioDeviceLib|Include/Library/VirtioMmioDeviceLib.h
+
## @libraryclass Invoke Xen hypercalls
#
XenHypercallLib|Include/Library/XenHypercallLib.h
--
2.19.1.3.g30247aa5d201


[PATCH] EmulatorPkg: Add MagicPageLib header file declaration.

Guomin Jiang
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2663

Add the public header file declaration.

Cc: Jordan Justen <jordan.l.justen@...>
Cc: Andrew Fish <afish@...>
Cc: Ray Ni <ray.ni@...>
Signed-off-by: Guomin Jiang <guomin.jiang@...>
---
EmulatorPkg/EmulatorPkg.dec | 1 +
1 file changed, 1 insertion(+)

diff --git a/EmulatorPkg/EmulatorPkg.dec b/EmulatorPkg/EmulatorPkg.dec
index 99250d9fe5..27d8b2be4e 100644
--- a/EmulatorPkg/EmulatorPkg.dec
+++ b/EmulatorPkg/EmulatorPkg.dec
@@ -27,6 +27,7 @@
KeyMap|Include/Library/KeyMapLib.h=0D
PpiListLib|Include/Library/PpiListLib.h=0D
SmbiosLib|Include/Library/SmbiosLib.h=0D
+ EmuMagicPageLib|Include/Library/EmuMagicPageLib.h=0D
=0D
[Protocols]=0D
gEmuThunkProtocolGuid =3D { 0x5CF32E0B, 0x8EDF, 0x2E44, { 0x9C,=
0xDA, 0x93, 0x20, 0x5E, 0x99, 0xEC, 0x1C } }=0D
--=20
2.25.1.windows.1


Re: [PATCH 0/4] remove generation of EFI properties table

Ard Biesheuvel
 

On 4/7/20 9:27 AM, Wang, Jian J via groups.io wrote:
Ard,
My apologies. I was indeed working desperately to catch some deadlines recently.
I agree to remove the properties table.
For the whole series,
Reviewed-by: Jian J Wang <jian.j.wang@...>
Thanks all.

Merged into edk2/master.


Re: [EXT] Re: [PATCH 1/1] EmbeddedPkg/MmcDxe: Added MaxBlock Transfer Limit 65535 in R/W.

Loh, Tien Hock
 

Hi Leif, Gaurav,

The changes look good to me, but I haven't tested it on Intel's SoCFPGA platform.
I will need some time to test it as I'm working on some other tasks, maybe in a week or so.

Thanks

-----Original Message-----
From: Gaurav Jain <gaurav.jain@...>
Sent: Tuesday, April 7, 2020 3:02 PM
To: Ard Biesheuvel <ard.biesheuvel@...>; Leif Lindholm
<leif@...>
Cc: devel@edk2.groups.io; Pankaj Bansal <pankaj.bansal@...>; Haojian
Zhuang <haojian.zhuang@...>; Loh, Tien Hock
<tien.hock.loh@...>
Subject: RE: [EXT] Re: [PATCH 1/1] EmbeddedPkg/MmcDxe: Added MaxBlock
Transfer Limit 65535 in R/W.



-----Original Message-----
From: Ard Biesheuvel <ard.biesheuvel@...>
Sent: Monday, April 6, 2020 7:42 PM
To: Leif Lindholm <leif@...>; Gaurav Jain
<gaurav.jain@...>
Cc: devel@edk2.groups.io; Pankaj Bansal <pankaj.bansal@...>;
Haojian Zhuang <haojian.zhuang@...>; Loh, Tien Hock
<tien.hock.loh@...>
Subject: [EXT] Re: [PATCH 1/1] EmbeddedPkg/MmcDxe: Added MaxBlock
Transfer Limit 65535 in R/W.

Caution: EXT Email

On 4/6/20 4:08 PM, Leif Lindholm wrote:
Hi Gaurav,

Haojian, Tien Hock - can you help review/test this change?

Best Regards,

Leif

On Fri, Apr 03, 2020 at 14:54:07 +0530, Gaurav Jain wrote:
Moved BlockCount calculation below BufferSize Validation checks.
First Ensure Buffersize is Not Zero and multiple of Media BlockSize.
then calculate BlockCount and perform Block checks.

Corrected BlockCount calculation, as BufferSize is multiple of
BlockSize, So adding (BlockSize-1) bytes to BufferSize and then
divide by BlockSize will have no impact on BlockCount.

Reading Large Images from MMC causes errors.
As per SD Host Controller Spec version 4.20, Restriction of 16-bit
Block Count transfer is 65535.
Max block transfer limit in single cmd is 65535 blocks.
Added Max Block check that can be processed is 0xFFFF.
then Update BlockCount on the basis of MaxBlock.

Signed-off-by: Gaurav Jain <gaurav.jain@...>

Hello Gaurav,

Could you please elaborate on the underlying need for this change? If
you are considering using this driver for future NXP platforms, I
should point out that this legacy driver is only kept around for
existing users, and new users should use the driver stack in
MdeModulePkg, which is based on the UEFI spec.

--
Ard.
Hello Ard

This change is for existing Platforms as well, that are using EmbeddedPkg
driver.
I can see Max Block Transfer Limit in MdeModulePkg also.
This Limit is not defined in EmbeddedPkg, which is causing errors on NXP
existing platform While reading Large images from MMC.
Block transfer limit is defined in SD spec.

Regards
Gaurav Jain



---
EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c | 38
++++++++++++++++++++-----------
1 file changed, 25 insertions(+), 13 deletions(-)

diff --git a/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
b/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
index 17c20c0159ba..b508c466d9c5 100644
--- a/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
+++ b/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
@@ -242,6 +242,8 @@ MmcIoBlocks (
UINTN BytesRemainingToBeTransfered;
UINTN BlockCount;
UINTN ConsumeSize;
+ UINT32 MaxBlock;
+ UINTN RemainingBlock;

BlockCount = 1;
MmcHostInstance = MMC_HOST_INSTANCE_FROM_BLOCK_IO_THIS
(This); @@
-262,19 +264,6 @@ MmcIoBlocks (
return EFI_NO_MEDIA;
}

- if (MMC_HOST_HAS_ISMULTIBLOCK(MmcHost) && MmcHost-
IsMultiBlock(MmcHost)) {
- BlockCount = (BufferSize + This->Media->BlockSize - 1) / This->Media-
BlockSize;
- }
-
- // All blocks must be within the device
- if ((Lba + (BufferSize / This->Media->BlockSize)) >
(This->Media-
LastBlock + 1)) {
- return EFI_INVALID_PARAMETER;
- }
-
- if ((Transfer == MMC_IOBLOCKS_WRITE) && (This->Media->ReadOnly
== TRUE)) {
- return EFI_WRITE_PROTECTED;
- }
-
// Reading 0 Byte is valid
if (BufferSize == 0) {
return EFI_SUCCESS;
@@ -285,14 +274,36 @@ MmcIoBlocks (
return EFI_BAD_BUFFER_SIZE;
}

+ if (MMC_HOST_HAS_ISMULTIBLOCK(MmcHost) && MmcHost-
IsMultiBlock(MmcHost)) {
+ BlockCount = BufferSize / This->Media->BlockSize; }
+
+ // All blocks must be within the device if ((Lba + (BufferSize
+ /
+ This->Media->BlockSize)) > (This->Media->LastBlock + 1)) {
+ return EFI_INVALID_PARAMETER;
+ }
+
+ if ((Transfer == MMC_IOBLOCKS_WRITE) && (This->Media->ReadOnly
== TRUE)) {
+ return EFI_WRITE_PROTECTED;
+ }
+
// Check the alignment
if ((This->Media->IoAlign > 2) && (((UINTN)Buffer &
(This->Media-
IoAlign - 1)) != 0)) {
return EFI_INVALID_PARAMETER;
}

+ // Max block number in single cmd is 65535 blocks.
+ MaxBlock = 0xFFFF;
+ RemainingBlock = BlockCount;
BytesRemainingToBeTransfered = BufferSize;
while (BytesRemainingToBeTransfered > 0) {

+ if (RemainingBlock <= MaxBlock) {
+ BlockCount = RemainingBlock;
+ } else {
+ BlockCount = MaxBlock;
+ }
+
// Check if the Card is in Ready status
CmdArg = MmcHostInstance->CardInfo.RCA << 16;
Response[0] = 0;
@@ -338,6 +349,7 @@ MmcIoBlocks (
DEBUG ((EFI_D_ERROR, "%a(): Failed to transfer block and
Status:%r\n", __func__, Status));
}

+ RemainingBlock -= BlockCount;
BytesRemainingToBeTransfered -= ConsumeSize;
if (BytesRemainingToBeTransfered > 0) {
Lba += BlockCount;
--
2.7.4


Re: [PATCH] .azurepiplines/pr-gate-steps.yml: Update python to 3.8.x for ci build

Liming Gao
 

Reviewed-by: Liming Gao <liming.gao@...>

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Zhang, Shenglei
Sent: Tuesday, April 7, 2020 2:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Bret Barkelew <Bret.Barkelew@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <liming.gao@...>
Subject: [edk2-devel] [PATCH] .azurepiplines/pr-gate-steps.yml: Update python to 3.8.x for ci build

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2617
Update edk2 build and test ci to use Python 3.8.x

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
.azurepipelines/templates/pr-gate-steps.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index a969661dea15..8c4ad4e2efe6 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -20,7 +20,7 @@ steps:

- task: UsePythonVersion@0
inputs:
- versionSpec: '3.7.x'
+ versionSpec: '3.8.x'
architecture: 'x64'

- script: pip install -r pip-requirements.txt --upgrade
--
2.18.0.windows.1



Re: [PATCH] pip-requirements.txt: Update extensions min version to 0.13.3

Liming Gao
 

Reviewed-by: Liming Gao <liming.gao@...>

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Zhang, Shenglei
Sent: Tuesday, April 7, 2020 2:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Bret Barkelew <Bret.Barkelew@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <liming.gao@...>
Subject: [edk2-devel] [PATCH] pip-requirements.txt: Update extensions min version to 0.13.3

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2616
Pytool extensions are locked on 0.12.x but extensions has
moved to 0.13.x. So update the pip-requirements.txt.

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
pip-requirements.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pip-requirements.txt b/pip-requirements.txt
index 6a41a95ec594..574dac43b1a6 100644
--- a/pip-requirements.txt
+++ b/pip-requirements.txt
@@ -9,8 +9,8 @@
# https://pypi.org/project/pip/
# https://pip.pypa.io/en/stable/user_guide/#requirements-files
# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
-#
+# https://www.python.org/dev/peps/pep-0440/#version-specifiers
##

edk2-pytool-library==0.10.*
-edk2-pytool-extensions==0.12.*
+edk2-pytool-extensions~=0.13.3
--
2.18.0.windows.1



Re: [PATCH 0/4] remove generation of EFI properties table

Wang, Jian J
 

Ard,

My apologies. I was indeed working desperately to catch some deadlines recently.
I agree to remove the properties table.

For the whole series,

Reviewed-by: Jian J Wang <jian.j.wang@...>

Regards,
Jian

-----Original Message-----
From: Ard Biesheuvel <ard.biesheuvel@...>
Sent: Monday, April 06, 2020 7:42 PM
To: Bi, Dandan <dandan.bi@...>; Wang, Jian J <jian.j.wang@...>;
Wu, Hao A <hao.a.wu@...>
Cc: devel@edk2.groups.io; Laszlo Ersek <lersek@...>; Leif Lindholm
<leif@...>; Kinney, Michael D <michael.d.kinney@...>; Ni, Ray
<ray.ni@...>; Yao, Jiewen <jiewen.yao@...>; Bret Barkelew
<Bret.Barkelew@...>
Subject: Re: [edk2-devel] [PATCH 0/4] remove generation of EFI properties table

On Fri, 3 Apr 2020 at 04:22, Bi, Dandan <dandan.bi@...> wrote:

For the functionality, it is the same with before for platforms which set
PcdPropertiesTableEnable to false by default.
Reviewed-by: Dandan Bi <dandan.bi@...> for patch [PATCH 2/4]
[PATCH 3/4] [PATCH 4/4].


If anyone still has the use case of enabling PcdPropertiesTableEnable, please
comment.
Thank you Dandan.

Jian, Hao, do you have any comments on this series? If you are too
busy to have a closer look immediately, could you please indicate so
instead of not responding at all? Thanks.


Re: [EXT] Re: [PATCH 1/1] EmbeddedPkg/MmcDxe: Added MaxBlock Transfer Limit 65535 in R/W.

Gaurav Jain
 

-----Original Message-----
From: Ard Biesheuvel <ard.biesheuvel@...>
Sent: Monday, April 6, 2020 7:42 PM
To: Leif Lindholm <leif@...>; Gaurav Jain <gaurav.jain@...>
Cc: devel@edk2.groups.io; Pankaj Bansal <pankaj.bansal@...>;
Haojian Zhuang <haojian.zhuang@...>; Loh, Tien Hock
<tien.hock.loh@...>
Subject: [EXT] Re: [PATCH 1/1] EmbeddedPkg/MmcDxe: Added MaxBlock
Transfer Limit 65535 in R/W.

Caution: EXT Email

On 4/6/20 4:08 PM, Leif Lindholm wrote:
Hi Gaurav,

Haojian, Tien Hock - can you help review/test this change?

Best Regards,

Leif

On Fri, Apr 03, 2020 at 14:54:07 +0530, Gaurav Jain wrote:
Moved BlockCount calculation below BufferSize Validation checks.
First Ensure Buffersize is Not Zero and multiple of Media BlockSize.
then calculate BlockCount and perform Block checks.

Corrected BlockCount calculation, as BufferSize is multiple of
BlockSize, So adding (BlockSize-1) bytes to BufferSize and then
divide by BlockSize will have no impact on BlockCount.

Reading Large Images from MMC causes errors.
As per SD Host Controller Spec version 4.20, Restriction of 16-bit
Block Count transfer is 65535.
Max block transfer limit in single cmd is 65535 blocks.
Added Max Block check that can be processed is 0xFFFF.
then Update BlockCount on the basis of MaxBlock.

Signed-off-by: Gaurav Jain <gaurav.jain@...>

Hello Gaurav,

Could you please elaborate on the underlying need for this change? If you
are considering using this driver for future NXP platforms, I should point out
that this legacy driver is only kept around for existing users, and new users
should use the driver stack in MdeModulePkg, which is based on the UEFI
spec.

--
Ard.
Hello Ard

This change is for existing Platforms as well, that are using EmbeddedPkg driver.
I can see Max Block Transfer Limit in MdeModulePkg also.
This Limit is not defined in EmbeddedPkg, which is causing errors on NXP existing platform While reading Large images from MMC.
Block transfer limit is defined in SD spec.

Regards
Gaurav Jain



---
EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c | 38
++++++++++++++++++++-----------
1 file changed, 25 insertions(+), 13 deletions(-)

diff --git a/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
b/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
index 17c20c0159ba..b508c466d9c5 100644
--- a/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
+++ b/EmbeddedPkg/Universal/MmcDxe/MmcBlockIo.c
@@ -242,6 +242,8 @@ MmcIoBlocks (
UINTN BytesRemainingToBeTransfered;
UINTN BlockCount;
UINTN ConsumeSize;
+ UINT32 MaxBlock;
+ UINTN RemainingBlock;

BlockCount = 1;
MmcHostInstance = MMC_HOST_INSTANCE_FROM_BLOCK_IO_THIS
(This); @@
-262,19 +264,6 @@ MmcIoBlocks (
return EFI_NO_MEDIA;
}

- if (MMC_HOST_HAS_ISMULTIBLOCK(MmcHost) && MmcHost-
IsMultiBlock(MmcHost)) {
- BlockCount = (BufferSize + This->Media->BlockSize - 1) / This->Media-
BlockSize;
- }
-
- // All blocks must be within the device
- if ((Lba + (BufferSize / This->Media->BlockSize)) > (This->Media-
LastBlock + 1)) {
- return EFI_INVALID_PARAMETER;
- }
-
- if ((Transfer == MMC_IOBLOCKS_WRITE) && (This->Media->ReadOnly
== TRUE)) {
- return EFI_WRITE_PROTECTED;
- }
-
// Reading 0 Byte is valid
if (BufferSize == 0) {
return EFI_SUCCESS;
@@ -285,14 +274,36 @@ MmcIoBlocks (
return EFI_BAD_BUFFER_SIZE;
}

+ if (MMC_HOST_HAS_ISMULTIBLOCK(MmcHost) && MmcHost-
IsMultiBlock(MmcHost)) {
+ BlockCount = BufferSize / This->Media->BlockSize; }
+
+ // All blocks must be within the device if ((Lba + (BufferSize /
+ This->Media->BlockSize)) > (This->Media->LastBlock + 1)) {
+ return EFI_INVALID_PARAMETER;
+ }
+
+ if ((Transfer == MMC_IOBLOCKS_WRITE) && (This->Media->ReadOnly
== TRUE)) {
+ return EFI_WRITE_PROTECTED;
+ }
+
// Check the alignment
if ((This->Media->IoAlign > 2) && (((UINTN)Buffer & (This->Media-
IoAlign - 1)) != 0)) {
return EFI_INVALID_PARAMETER;
}

+ // Max block number in single cmd is 65535 blocks.
+ MaxBlock = 0xFFFF;
+ RemainingBlock = BlockCount;
BytesRemainingToBeTransfered = BufferSize;
while (BytesRemainingToBeTransfered > 0) {

+ if (RemainingBlock <= MaxBlock) {
+ BlockCount = RemainingBlock;
+ } else {
+ BlockCount = MaxBlock;
+ }
+
// Check if the Card is in Ready status
CmdArg = MmcHostInstance->CardInfo.RCA << 16;
Response[0] = 0;
@@ -338,6 +349,7 @@ MmcIoBlocks (
DEBUG ((EFI_D_ERROR, "%a(): Failed to transfer block and
Status:%r\n", __func__, Status));
}

+ RemainingBlock -= BlockCount;
BytesRemainingToBeTransfered -= ConsumeSize;
if (BytesRemainingToBeTransfered > 0) {
Lba += BlockCount;
--
2.7.4


[PATCH] BaseTools/WindowsVsToolChain.py: Update toolchain plugin

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2659
Allow WindowsVsToolChain Plugin to add libraries and headers
of user defined ARCH for VS2017 and VS2019.

Cc: Bob Feng <bob.c.feng@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
.../WindowsVsToolChain/WindowsVsToolChain.py | 73 ++++++++++++++++---
1 file changed, 62 insertions(+), 11 deletions(-)

diff --git a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
index c9279e1c75b5..0fba2c1b5325 100644
--- a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
+++ b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
@@ -13,6 +13,7 @@ import edk2toollib.windows.locate_tools as locate_tools
from edk2toollib.windows.locate_tools import FindWithVsWhere
from edk2toolext.environment import shell_environment
from edk2toolext.environment import version_aggregator
+from edk2toollib.utility_functions import GetHostInfo


class WindowsVsToolChain(IUefiBuildPlugin):
@@ -26,14 +27,41 @@ class WindowsVsToolChain(IUefiBuildPlugin):
"UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath", "WindowsSdkDir", "WindowsSdkVerBinPath",
"WindowsSDKVersion", "VCToolsInstallDir", "Path"]

-#
+ #
# VS2017 - Follow VS2017 where there is potential for many versions of the tools.
# If a specific version is required then the user must set both env variables:
# VS150INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
# VS150TOOLVER: version number for the VC compiler tools
# VS2017_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ # VS2017_HOST: set the host architecture to use for host tools, and host libs, etc
if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":

+ # check to see if host is configured
+ # HostType for VS2017 should be (defined in tools_def):
+ # x86 == 32bit Intel
+ # x64 == 64bit Intel
+ # arm == 32bit Arm
+ # arm64 == 64bit Arm
+ #
+ HostType = shell_environment.GetEnvironment().get_shell_var("VS2017_HOST")
+ if HostType is not None:
+ HostType = HostType.lower()
+ self.Logger.info(
+ f"HOST TYPE defined by environment. Host Type is {HostType}")
+ else:
+ HostInfo = GetHostInfo()
+ if HostInfo.arch == "x86":
+ if HostInfo.bit == "32":
+ HostType = "x86"
+ elif HostInfo.bit == "64":
+ HostType = "x64"
+ else:
+ raise NotImplementedError()
+
+ # VS2017_HOST options are not exactly the same as QueryVcVariables. This translates.
+ VC_HOST_ARCH_TRANSLATOR = {
+ "x86": "x86", "x64": "AMD64", "arm": "not supported", "arm64": "not supported"}
+
# check to see if full path already configured
if shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX") != None:
self.Logger.info("VS2017_PREFIX is already set.")
@@ -58,16 +86,14 @@ class WindowsVsToolChain(IUefiBuildPlugin):
"Tools", "MSVC", vc_ver)
prefix = prefix + os.path.sep
shell_environment.GetEnvironment().set_shell_var("VS2017_PREFIX", prefix)
+ shell_environment.GetEnvironment().set_shell_var("VS2017_HOST", HostType)

shell_env = shell_environment.GetEnvironment()
# Use the tools lib to determine the correct values for the vars that interest us.
vs_vars = locate_tools.QueryVcVariables(
- interesting_keys, "amd64", vs_version="vs2017")
+ interesting_keys, VC_HOST_ARCH_TRANSLATOR[HostType], vs_version="vs2017")
for (k, v) in vs_vars.items():
- if k.upper() == "PATH":
- shell_env.insert_path(v)
- else:
- shell_env.set_shell_var(k, v)
+ shell_env.set_shell_var(k, v)

# now confirm it exists
if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX")):
@@ -80,8 +106,35 @@ class WindowsVsToolChain(IUefiBuildPlugin):
# VS160INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
# VS160TOOLVER: version number for the VC compiler tools
# VS2019_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ # VS2017_HOST: set the host architecture to use for host tools, and host libs, etc
elif thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2019":

+ # check to see if host is configured
+ # HostType for VS2019 should be (defined in tools_def):
+ # x86 == 32bit Intel
+ # x64 == 64bit Intel
+ # arm == 32bit Arm
+ # arm64 == 64bit Arm
+ #
+ HostType = shell_environment.GetEnvironment().get_shell_var("VS2019_HOST")
+ if HostType is not None:
+ HostType = HostType.lower()
+ self.Logger.info(
+ f"HOST TYPE defined by environment. Host Type is {HostType}")
+ else:
+ HostInfo = GetHostInfo()
+ if HostInfo.arch == "x86":
+ if HostInfo.bit == "32":
+ HostType = "x86"
+ elif HostInfo.bit == "64":
+ HostType = "x64"
+ else:
+ raise NotImplementedError()
+
+ # VS2019_HOST options are not exactly the same as QueryVcVariables. This translates.
+ VC_HOST_ARCH_TRANSLATOR = {
+ "x86": "x86", "x64": "AMD64", "arm": "not supported", "arm64": "not supported"}
+
# check to see if full path already configured
if shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX") != None:
self.Logger.info("VS2019_PREFIX is already set.")
@@ -106,16 +159,14 @@ class WindowsVsToolChain(IUefiBuildPlugin):
"Tools", "MSVC", vc_ver)
prefix = prefix + os.path.sep
shell_environment.GetEnvironment().set_shell_var("VS2019_PREFIX", prefix)
+ shell_environment.GetEnvironment().set_shell_var("VS2019_HOST", HostType)

shell_env = shell_environment.GetEnvironment()
# Use the tools lib to determine the correct values for the vars that interest us.
vs_vars = locate_tools.QueryVcVariables(
- interesting_keys, "amd64", vs_version="vs2019")
+ interesting_keys, VC_HOST_ARCH_TRANSLATOR[HostType], vs_version="vs2019")
for (k, v) in vs_vars.items():
- if k.upper() == "PATH":
- shell_env.insert_path(v)
- else:
- shell_env.set_shell_var(k, v)
+ shell_env.set_shell_var(k, v)

# now confirm it exists
if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX")):
--
2.18.0.windows.1


[PATCH] .azurepiplines/pr-gate-steps.yml: Update python to 3.8.x for ci build

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2617
Update edk2 build and test ci to use Python 3.8.x

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
.azurepipelines/templates/pr-gate-steps.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index a969661dea15..8c4ad4e2efe6 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -20,7 +20,7 @@ steps:

- task: UsePythonVersion@0
inputs:
- versionSpec: '3.7.x'
+ versionSpec: '3.8.x'
architecture: 'x64'

- script: pip install -r pip-requirements.txt --upgrade
--
2.18.0.windows.1


[PATCH] pip-requirements.txt: Update extensions min version to 0.13.3

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2616
Pytool extensions are locked on 0.12.x but extensions has
moved to 0.13.x. So update the pip-requirements.txt.

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
pip-requirements.txt | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pip-requirements.txt b/pip-requirements.txt
index 6a41a95ec594..574dac43b1a6 100644
--- a/pip-requirements.txt
+++ b/pip-requirements.txt
@@ -9,8 +9,8 @@
# https://pypi.org/project/pip/
# https://pip.pypa.io/en/stable/user_guide/#requirements-files
# https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format
-#
+# https://www.python.org/dev/peps/pep-0440/#version-specifiers
##

edk2-pytool-library==0.10.*
-edk2-pytool-extensions==0.12.*
+edk2-pytool-extensions~=0.13.3
--
2.18.0.windows.1


Re: [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

Sean
 

Bob and Ray,

My point isn't that there is anything wrong with script.  This script is designed to work with edk2-platforms repo or other repos of similar layout.   This is a "platform" thing that is needed on some platforms (not edk2 core platforms).  To me this means it does't belong in edk2 repo. 

Anytime new functionality is proposed I think it is important to evaluate the stakeholders of the given repository that is targeted and make sure the functionality is aligned.  In my opinion when edk2 get too prescriptive about how to do things outside of edk2 or tries to prematurely provide functionality for all consumers to use then we often see this overhead added to the tree and low adoption rate.   

I would propose to add this to edk2-platforms since that is where this is needed and build up a nice "platform" design pattern.   If it provides great value then talk about moving up in the repository stack.  

Finally, I know this is a trivial and small script.  The overhead is minimal but i see this as a barometer for how the community aligns to support all isn't consumers.  Many of us have to maintain forks with significant changes because edk2 carries too many prescriptive ideas.  I want to push Tianocore to have better alignment with different repositories, their functionality and focus.  I want to see downstream consumers spend less time maintaining their fork and shipping code more closely aligned with the open source edk2.  

thanks
Sean


Re: [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

Bob Feng
 

Hi Sean,

 

That the Basetools works correctly depends on the PACAKGES_PATH is set correctly. I think this patch provides common function and provides assistance for the Basetools.

 

Thanks,

Bob

 

From: Ni, Ray
Sent: Tuesday, April 7, 2020 10:16 AM
To: devel@edk2.groups.io; sean.brogan@...; Luo, Heng <heng.luo@...>
Cc: Feng, Bob C <bob.c.feng@...>
Subject: RE: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

 

Sean,

PyTools is good. This patch is to support those platforms that haven’t adopted PyTools for their platform build.

 

Thanks,

Ray

 

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean via groups.io
Sent: Thursday, April 2, 2020 11:59 PM
To: Luo, Heng <heng.luo@...>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

 

I am not a fan of this.  The community and design meetings have had a few discussions about tools (PyTools has been presented twice) and a few brief discussions about common patterns to build platforms but I don't think there is real alignment.  Each "platform" has its own way of doing things. 

As for this patch, I don't really want to see a bunch of scripts added to edk2 basetools that are not aligned with a community agreed direction (or any clear direction).  It has also been brought up that basetools as a python package/project is a challenge to work with and has some fundamental problems.  Adding more in a similar design pattern is not how we start fixing it.   As an example for edk2-pytools, an RFC was offered and even then it was added as its own repositories to avoid more directly to the edk2 repo.   

@Mike K - Any progress on getting a tools subtream setup?  Any governance ideas to help align these efforts.  

Thanks
Sean


Re: [PATCH] EmulatorPkg/WinHost: Add link flags for VS2019 tool chains.

Guomin Jiang
 

Hi Justen, Andrew, Ray,

Could you help review the changes.

Best Regards
Guomin

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Jiang,
Guomin
Sent: Friday, March 20, 2020 1:26 PM
To: devel@edk2.groups.io
Cc: Justen, Jordan L <jordan.l.justen@...>; Andrew Fish
<afish@...>; Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH] EmulatorPkg/WinHost: Add link flags for
VS2019 tool chains.

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2604

The link flags for VS2019 is absent and result the build fail.

Cc: Jordan Justen <jordan.l.justen@...>
Cc: Andrew Fish <afish@...>
Cc: Ray Ni <ray.ni@...>
Signed-off-by: Guomin Jiang <guomin.jiang@...>
---
EmulatorPkg/Win/Host/WinHost.inf | 2 ++
1 file changed, 2 insertions(+)

diff --git a/EmulatorPkg/Win/Host/WinHost.inf
b/EmulatorPkg/Win/Host/WinHost.inf
index e0b3ecb15b..44e938761d 100644
--- a/EmulatorPkg/Win/Host/WinHost.inf
+++ b/EmulatorPkg/Win/Host/WinHost.inf
@@ -87,12 +87,14 @@
MSFT:*_VS2015_IA32_DLINK_FLAGS = /LIBPATH:"%VS2015_PREFIX%Lib"
/LIBPATH:"%VS2015_PREFIX%VC\Lib"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x86"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:I386 /LTCG Kernel32.lib MSVCRTD.lib Gdi32.lib
User32.lib Winmm.lib Advapi32.lib vcruntimed.lib ucrtd.lib
MSFT:*_VS2015x86_IA32_DLINK_FLAGS = /LIBPATH:"%VS2015_PREFIX%Lib"
/LIBPATH:"%VS2015_PREFIX%VC\Lib"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x86"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:I386 /LTCG Kernel32.lib MSVCRTD.lib Gdi32.lib
User32.lib Winmm.lib Advapi32.lib vcruntimed.lib ucrtd.lib
MSFT:*_VS2017_IA32_DLINK_FLAGS =
/LIBPATH:"%VCToolsInstallDir%lib\x86"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x86"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:I386 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib+
MSFT:*_VS2019_IA32_DLINK_FLAGS =
/LIBPATH:"%VCToolsInstallDir%lib\x86"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x86"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:I386 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib
MSFT:*_*_IA32_ASM_FLAGS == /nologo /W3 /WX /c /coff /Cx /Zd /W0
/Zi MSFT:*_*_IA32_ASMLINK_FLAGS == /link /nologo /tiny
MSFT:*_VS2015_X64_DLINK_FLAGS =
/LIBPATH:"%VS2015_PREFIX%VC\Lib\AMD64"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:AMD64 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib
MSFT:*_VS2015x86_X64_DLINK_FLAGS =
/LIBPATH:"%VS2015_PREFIX%VC\Lib\AMD64"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:AMD64 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib
MSFT:*_VS2017_X64_DLINK_FLAGS =
/LIBPATH:"%VCToolsInstallDir%lib\x64"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:AMD64 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib+
MSFT:*_VS2019_X64_DLINK_FLAGS =
/LIBPATH:"%VCToolsInstallDir%lib\x64"
/LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64"
/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64"
/NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP
/OPT:REF /DEBUG /MACHINE:AMD64 /LTCG Kernel32.lib MSVCRTD.lib
vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib
MSFT:*_*_X64_ASM_FLAGS == /nologo /W3 /WX /c /Cx /Zd /W0 /Zi
MSFT:*_*_X64_ASMLINK_FLAGS == /link /nologo --
2.25.1.windows.1


-=-=-=-=-=-=
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#56049): https://edk2.groups.io/g/devel/message/56049
Mute This Topic: https://groups.io/mt/72093907/4399222
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub
[guomin.jiang@...] -=-=-=-=-=-=


Re: [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

Ni, Ray
 

Sean,

PyTools is good. This patch is to support those platforms that haven’t adopted PyTools for their platform build.

 

Thanks,

Ray

 

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean via groups.io
Sent: Thursday, April 2, 2020 11:59 PM
To: Luo, Heng <heng.luo@...>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [PATCH] BaseTools/Scripts: Add scripts to set PACKAGES_PATH environment

 

I am not a fan of this.  The community and design meetings have had a few discussions about tools (PyTools has been presented twice) and a few brief discussions about common patterns to build platforms but I don't think there is real alignment.  Each "platform" has its own way of doing things. 

As for this patch, I don't really want to see a bunch of scripts added to edk2 basetools that are not aligned with a community agreed direction (or any clear direction).  It has also been brought up that basetools as a python package/project is a challenge to work with and has some fundamental problems.  Adding more in a similar design pattern is not how we start fixing it.   As an example for edk2-pytools, an RFC was offered and even then it was added as its own repositories to avoid more directly to the edk2 repo.   

@Mike K - Any progress on getting a tools subtream setup?  Any governance ideas to help align these efforts.  

Thanks
Sean