|
[PATCH v9 03/12] EmulatorPkg: CI: use Python version from defaults template
Oliver Steffen
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows CI jobs. Previous changes to the CI job templates make it necessary to specify a version number, if Python shall be pulled at CI runtime. The Linux CI jobs will use Python from the container image, so these changes are for Windows only. Signed-off-by: Oliver Steffen <osteffen@...> --- EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml index e7ead06ae266..09960e7c7a6e 100644 --- a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml @@ -16,6 +16,9 @@ pr: - master - stable/* +variables: + - template: ../../../.azurepipelines/templates/defaults.yml + jobs: - job: Platform_CI variables: @@ -128,3 +131,4 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) + usePythonVersion: ${{ variables.default_python_version }} -- 2.38.1 |
|
|
|
[PATCH v9 02/12] OvmfPkg: CI: use Python version from defaults template
Oliver Steffen
Use the default Python version from the defaults template
(.azurepipelines/templates/defaults.yml) in the Windows CI jobs. Previous changes to the CI job templates make it necessary to specify a version number, if Python shall be pulled at CI runtime. The Linux CI jobs will use Python from the container image, so these changes are for Windows only. Signed-off-by: Oliver Steffen <osteffen@...> --- OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml index 7d6344d6383d..7e63f419b26b 100644 --- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml +++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml @@ -14,6 +14,10 @@ trigger: pr: - master - stable/* + +variables: + - template: ../../../.azurepipelines/templates/defaults.yml + jobs: - job: Platform_CI variables: @@ -133,6 +137,7 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) + usePythonVersion: ${{ variables.default_python_version }} extra_install_step: - powershell: choco install qemu --version=2021.5.5; Write-Host "##vso[task.prependpath]c:\Program Files\qemu" displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI -- 2.38.1 |
|
|
|
[PATCH v9 01/12] CI: make Python version configurable
Oliver Steffen
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use. The default value is '', in which case Python will not be downloaded at runtime and the one provided by the VM/container image will be used. Additionally, add a template .azurepipelines/templates/defaults.yml, from which the default Pyhton version string can be obtained. Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/Windows-VS2019.yml | 4 ++++ .azurepipelines/templates/defaults.yml | 3 +++ .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++- .azurepipelines/templates/pr-gate-build-job.yml | 2 ++ .azurepipelines/templates/pr-gate-steps.yml | 6 ++++-- 5 files changed, 18 insertions(+), 3 deletions(-) create mode 100644 .azurepipelines/templates/defaults.yml diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml index e4bd4b1d2283..89bccde82575 100644 --- a/.azurepipelines/Windows-VS2019.yml +++ b/.azurepipelines/Windows-VS2019.yml @@ -12,9 +12,13 @@ pr: - master - stable/* +variables: + - template: templates/defaults.yml + jobs: - template: templates/pr-gate-build-job.yml parameters: tool_chain_tag: 'VS2019' vm_image: 'windows-2019' arch_list: "IA32,X64" + usePythonVersion: ${{ variables.default_python_version }} diff --git a/.azurepipelines/templates/defaults.yml b/.azurepipelines/templates/defaults.yml new file mode 100644 index 000000000000..314d03e33af8 --- /dev/null +++ b/.azurepipelines/templates/defaults.yml @@ -0,0 +1,3 @@ + +variables: + default_python_version: ">=3.10.6" diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml index 40a31a509fc5..8803d80cf51c 100644 --- a/.azurepipelines/templates/platform-build-run-steps.yml +++ b/.azurepipelines/templates/platform-build-run-steps.yml @@ -34,6 +34,9 @@ parameters: - name: extra_install_step type: stepList default: [] +- name: usePythonVersion + type: string + default: '' steps: - checkout: self @@ -42,8 +45,9 @@ steps: - task: UsePythonVersion@0 inputs: - versionSpec: ">=3.10.6" + versionSpec: ${{ parameters.usePythonVersion }} architecture: "x64" + condition: ne('${{ parameters.usePythonVersion }}', '') - script: pip install -r pip-requirements.txt --upgrade displayName: 'Install/Upgrade pip modules' diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml index 7f88b41dc8d3..f0ef1142da49 100644 --- a/.azurepipelines/templates/pr-gate-build-job.yml +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -12,6 +12,7 @@ parameters: tool_chain_tag: '' vm_image: '' arch_list: '' + usePythonVersion: '' # Build step jobs: @@ -77,3 +78,4 @@ jobs: build_pkgs: $(Build.Pkgs) build_targets: $(Build.Targets) build_archs: ${{ parameters.arch_list }} + usePythonVersion: ${{ parameters.usePythonVersion }} diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml index cb431e53fcd1..27a44c06011c 100644 --- a/.azurepipelines/templates/pr-gate-steps.yml +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -12,6 +12,7 @@ parameters: build_pkgs: '' build_targets: '' build_archs: '' + usePythonVersion: '' steps: - checkout: self @@ -20,8 +21,9 @@ steps: - task: UsePythonVersion@0 inputs: - versionSpec: '>=3.10.6' - architecture: 'x64' + versionSpec: ${{ parameters.usePythonVersion }} + architecture: "x64" + condition: ne('${{ parameters.usePythonVersion }}', '') - script: pip install -r pip-requirements.txt --upgrade displayName: 'Install/Upgrade pip modules' -- 2.38.1 |
|
|
|
Re: [PATCH v8 12/12] CI: use latest vm_image (Linux only)
Oliver Steffen
On Wed, Nov 30, 2022 at 9:39 PM Kinney, Michael D
<michael.d.kinney@...> wrote: I'll set it to ubuntu-22.04. Although the actual VM does not do much anymore except spawning a container, which I would assume to "always" work. But better be safe than sorry. -Oliver
|
|
|
|
Re: [PATCH v8 10/12] CI: Don't install cspell
Oliver Steffen
On Wed, Nov 30, 2022 at 9:45 PM Chris Fernald <chris.fernald@...> wrote:
No worries. Sorry for misunderstanding. I'll drop this patch for now. Don't have a simple and good way to skip this on Linux only. I also don't think this is too critical now, the install should not do much if cspell is already there. We can clean this up later, and track this in an issue / BZ. - Oliver On 11/30/2022 12:01 PM, Oliver Steffen wrote:cspell is already included in the container image. |
|
|
|
[Patch 1/1] UnitTestFrameworkPkg: Library classes private to public
Michael D Kinney
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4178
* Change GoogleTestLib class from private to public. * Change UnitTestPersistenceLib class from private to public. Cc: Michael Kubacki <mikuback@...> Cc: Sean Brogan <sean.brogan@...> Cc: Ray Ni <ray.ni@...> Cc: Zhiguang Liu <zhiguang.liu@...> Signed-off-by: Michael D Kinney <michael.d.kinney@...> --- .../Library/UnitTestPersistenceLib.h | 0 UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec | 13 +++++++------ 2 files changed, 7 insertions(+), 6 deletions(-) rename UnitTestFrameworkPkg/{PrivateInclude => Include}/Library/UnitTestPersistenceLib.h (100%) diff --git a/UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h b/UnitTestFrameworkPkg/Include/Library/UnitTestPersistenceLib.h similarity index 100% rename from UnitTestFrameworkPkg/PrivateInclude/Library/UnitTestPersistenceLib.h rename to UnitTestFrameworkPkg/Include/Library/UnitTestPersistenceLib.h diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec index ed12f32009d8..14e387d63a0f 100644 --- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec +++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec @@ -26,11 +26,16 @@ [Includes.Common.Private] Library/CmockaLib/cmocka/include/cmockery Library/GoogleTestLib/googletest/googletest -[LibraryClasses.Common.Private] +[LibraryClasses] ## @libraryclass Allows save and restore unit test internal state # - UnitTestPersistenceLib|PrivateInclude/Library/UnitTestPersistenceLib.h + UnitTestPersistenceLib|Include/Library/UnitTestPersistenceLib.h + ## @libraryclass GoogleTest infrastructure + # + GoogleTestLib|Include/Library/GoogleTestLib.h + +[LibraryClasses.Common.Private] ## @libraryclass Provides a unit test result report # UnitTestResultReportLib|PrivateInclude/Library/UnitTestResultReportLib.h @@ -39,10 +44,6 @@ [LibraryClasses.Common.Private] # UnitTestBootLib|PrivateInclude/Library/UnitTestBootLib.h - ## @libraryclass GoogleTest infrastructure - # - GoogleTestLib|Include/Library/GoogleTestLib.h - [Guids] gUnitTestFrameworkPkgTokenSpaceGuid = { 0x833d3aba, 0x39b4, 0x43a2, { 0xb9, 0x30, 0x7a, 0x34, 0x53, 0x39, 0x31, 0xb3 } } -- 2.37.1.windows.1 |
|
|
|
Re: [PATCH] UnitTestFrameworkPkg/UnitTestLib: Print expected Status on ASSERT fail
Michael D Kinney
Hi Jeshuas,
toggle quoted message
Show quoted text
This is a good idea to show the expected value. %p was used on purpose because unit tests can be built for 32-bit or 64-bit and the EFI_STATUS is same as RETURN_STATUS which is same as UINTN. UINTN is 32-bits for 32-bit unit test apps and 64-bit for 64-bit unit test apps. %p prints a pointer sized value, which happens to match the UINTN for support CPU archs. A couple options to consider: 1) Keep using %p instead of %llx. 2) Use UINT64 local variables to hold Status and Expected values and use %llx. 3) Use the MdePkg PrintLib to convert Status and Expected to string names and update message to show the name of the status value instead of the hex value. 4) Don't add a dependency on PrintLib and instead convert to string names in this same C file. Best regards, Mike -----Original Message----- |
|
|
|
Re: [PATCH v8 10/12] CI: Don't install cspell
Chris Fernald
I apologize as I think I mislead you on this. I think we can only remove this operation for the builds using the container. I believe Windows builds will still require this step for the time being. The Windows pipeline on your PR is currently failing to find cspell. Sorry for not being clear on this originally.
toggle quoted message
Show quoted text
On 11/30/2022 12:01 PM, Oliver Steffen wrote:
cspell is already included in the container image. |
|
|
|
[PATCH] UnitTestFrameworkPkg/UnitTestLib: Print expected Status on ASSERT fail
Jeshua Smith
Update the UnitTestAssertStatusEqual error message to print out the
expected value in addition to the seen value. Signed-off-by: Jeshua Smith <jeshuas@...> --- UnitTestFrameworkPkg/Library/UnitTestLib/AssertCmocka.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/UnitTestFrameworkPkg/Library/UnitTestLib/AssertCmocka.c b/UnitTestFrameworkPkg/Library/UnitTestLib/AssertCmocka.c index dc05bbd438..322daf318a 100644 --- a/UnitTestFrameworkPkg/Library/UnitTestLib/AssertCmocka.c +++ b/UnitTestFrameworkPkg/Library/UnitTestLib/AssertCmocka.c @@ -290,7 +290,7 @@ UnitTestAssertStatusEqual ( { CHAR8 TempStr[MAX_STRING_SIZE]; - snprintf (TempStr, sizeof (TempStr), "UT_ASSERT_STATUS_EQUAL(%s:%p)", Description, (VOID *)Status); + snprintf (TempStr, sizeof (TempStr), "UT_ASSERT_STATUS_EQUAL(%s:0x%llx expected:0x%llx)", Description, Status, Expected); _assert_true ((Status == Expected), TempStr, FileName, (INT32)LineNumber); return (Status == Expected); -- 2.25.1 |
|
|
|
Re: [PATCH v8 12/12] CI: use latest vm_image (Linux only)
Michael D Kinney
I know we have run into issues before when using ubuntu-latest when
toggle quoted message
Show quoted text
a change was made to the latest image that broke CI. I would prefer to always set this to a specific version, so we can choose to move to a newer one with a PR that can run through CI. I am ok with selecting the highest version available that matches current ubuntu-latest version number. Thanks, Mike -----Original Message----- |
|
|
|
[PATCH v8 10/12] CI: Don't install cspell
Oliver Steffen
cspell is already included in the container image.
Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/templates/spell-check-prereq-steps.yml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml index 226909d54c4e..66354ad78b5c 100644 --- a/.azurepipelines/templates/spell-check-prereq-steps.yml +++ b/.azurepipelines/templates/spell-check-prereq-steps.yml @@ -16,7 +16,3 @@ steps: versionSpec: '14.x' #checkLatest: false # Optional condition: and(gt(variables.pkg_count, 0), succeeded()) - -- script: npm install -g cspell@5.20.0 - displayName: 'Install cspell npm' - condition: and(gt(variables.pkg_count, 0), succeeded()) -- 2.38.1 |
|
|
|
[PATCH v8 11/12] .pytool: CISettings.py: don't add scopes for GCC
Oliver Steffen
All ext_dep.yml files for gcc have been removed and gcc
is expected to be installed on the system (GCC5_*_PREFIX may indicate the location). No need to adjust the toolchain scopes for Linux builds anymore. Signed-off-by: Oliver Steffen <osteffen@...> --- .pytool/CISettings.py | 9 --------- 1 file changed, 9 deletions(-) diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py index 76ac2b09dba6..d87c8e838e61 100644 --- a/.pytool/CISettings.py +++ b/.pytool/CISettings.py @@ -169,15 +169,6 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag else: logging.warning("Falling back to using in-tree BaseTools") - if is_linux and self.ActualToolChainTag.upper().startswith("GCC"): - if "AARCH64" in self.ActualArchitectures: - scopes += ("gcc_aarch64_linux",) - if "ARM" in self.ActualArchitectures: - scopes += ("gcc_arm_linux",) - if "RISCV64" in self.ActualArchitectures: - scopes += ("gcc_riscv64_unknown",) - if "LOONGARCH64" in self.ActualArchitectures: - scopes += ("gcc_loongarch64_unknown_linux",) self.ActualScopes = scopes return self.ActualScopes -- 2.38.1 |
|
|
|
[PATCH v8 12/12] CI: use latest vm_image (Linux only)
Oliver Steffen
Switch over to `ubuntu-latest` as the vm_image for Linux
CI jobs (EmulatorPkg, ArmVirtPkg, OvmfPkg). The previously used ubuntu-18.04 will not be available after Dec 1st 2022. Signed-off-by: Oliver Steffen <osteffen@...> --- ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +- EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +- OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index 6d75502b5628..f7844229e60c 100644 --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -19,7 +19,7 @@ jobs: - job: Platform_CI variables: package: 'ArmVirtPkg' - vm_image: 'ubuntu-18.04' + vm_image: 'ubuntu-latest' should_run: true run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE" diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index e58bb5b55245..3124de6634b2 100644 --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -19,7 +19,7 @@ jobs: - job: Platform_CI variables: package: 'EmulatorPkg' - vm_image: 'ubuntu-18.04' + vm_image: 'ubuntu-latest' should_run: false run_flags: "MAKE_STARTUP_NSH=TRUE" diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index 984782aa7d19..831b0eb254b4 100644 --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -19,7 +19,7 @@ jobs: - job: Platform_CI variables: package: 'OvmfPkg' - vm_image: 'ubuntu-18.04' + vm_image: 'ubuntu-latest' should_run: true run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE" -- 2.38.1 |
|
|
|
[PATCH v8 09/12] EmulatorPkg: CI: Use Fedora 35 container (Linux only)
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are here: https://github.com/tianocore/containers The version numbers of gcc, iasl, and nasm are pinned to avoid unintended upgrades during image rebuild. Use Python from the container image, do not download at runtime. Signed-off-by: Oliver Steffen <osteffen@...> --- EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index 416c15e70840..e58bb5b55245 100644 --- a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -75,6 +75,8 @@ jobs: pool: vmImage: $(vm_image) + container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e' + steps: - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml parameters: @@ -85,3 +87,4 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) + usePythonVersion: '' # use Python from the container image -- 2.38.1 |
|
|
|
[PATCH v8 06/12] BaseTools: CI: Use Fedora 35 container (Linux only)
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are here: https://github.com/tianocore/containers The version numbers of gcc, iasl, and nasm are pinned to avoid unintended upgrades during image rebuild. Do not run apt-get in CI jobs to install qemu and gcc dependencies. Assume the container image provides these. Use Python from the container image, do not download at runtime. Remove BaseTools/Bin/gcc_*_ext_dep.yaml to stop downloading gcc from external locations; use the gcc provided by the container image instead. The container image sets the variable GCC5_*_PREFIX accordingly. Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/Ubuntu-GCC5.yml | 2 ++ .../templates/basetools-build-steps.yml | 9 -------- BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 ------------------ BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 ------------------ ...gcc_loongarch64_unknown_linux_ext_dep.yaml | 22 ------------------- .../Bin/gcc_riscv64_unknown_ext_dep.yaml | 22 ------------------- 6 files changed, 2 insertions(+), 95 deletions(-) delete mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml delete mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml delete mode 100644 BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml delete mode 100644 BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml index 1acd8d2a46a7..0dd9b74de57a 100644 --- a/.azurepipelines/Ubuntu-GCC5.yml +++ b/.azurepipelines/Ubuntu-GCC5.yml @@ -18,5 +18,7 @@ jobs: parameters: tool_chain_tag: 'GCC5' vm_image: 'ubuntu-latest' + container: 'ghcr.io/tianocore/containers/fedora-35-build:2113a0e' + usePythonVersion: '' # use Python from the container image arch_list: "IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64" diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml index d8c108c6e212..a72758bc3395 100644 --- a/.azurepipelines/templates/basetools-build-steps.yml +++ b/.azurepipelines/templates/basetools-build-steps.yml @@ -10,15 +10,6 @@ parameters: tool_chain_tag: '' steps: -- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}: - - bash: sudo apt-get update - displayName: Update apt - condition: and(gt(variables.pkg_count, 0), succeeded()) - - - bash: sudo apt-get install gcc g++ make uuid-dev - displayName: Install required tools - condition: and(gt(variables.pkg_count, 0), succeeded()) - - task: CmdLine@1 displayName: Build Base Tools from source inputs: diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml deleted file mode 100644 index ff8a9e868100..000000000000 --- a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml +++ /dev/null @@ -1,21 +0,0 @@ -## @file -# Download GCC AARCH64 compiler from Arm's release site -# Set shell variable GCC5_AARCH64_INSTALL to this folder -# -# This is only downloaded when a build activates scope gcc_aarch64_linux -# -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: BSD-2-Clause-Patent -## -{ - "scope": "gcc_aarch64_linux", - "type": "web", - "name": "gcc_aarch64_linux", - "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu.tar.xz", - "version": "11.2-2022.02", - "sha256": "52dbac3eb71dbe0916f60a8c5ab9b7dc9b66b3ce513047baa09fae56234e53f3", - "compression_type": "tar", - "internal_path": "/gcc-arm-11.2-2022.02-x86_64-aarch64-none-linux-gnu/", - "flags": ["set_shell_var", ], - "var_name": "GCC5_AARCH64_INSTALL" -} diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml deleted file mode 100644 index 151cbfa4b532..000000000000 --- a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml +++ /dev/null @@ -1,21 +0,0 @@ -## @file -# Download GCC ARM compiler from Arm's release site -# Set shell variable GCC5_ARM_INSTALL to this folder -# -# This is only downloaded when a build activates scope gcc_arm_linux -# -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: BSD-2-Clause-Patent -## -{ - "scope": "gcc_arm_linux", - "type": "web", - "name": "gcc_arm_linux", - "source": "https://developer.arm.com/-/media/Files/downloads/gnu/11.2-2022.02/binrel/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf.tar.xz", - "version": "11.2-2022.02", - "sha256": "c254f7199261fe76c32ef42187502839bda7efad0a66646cf739d074eff45fad", - "compression_type": "tar", - "internal_path": "/gcc-arm-11.2-2022.02-x86_64-arm-none-linux-gnueabihf/", - "flags": ["set_shell_var", ], - "var_name": "GCC5_ARM_INSTALL" -} diff --git a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml b/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml deleted file mode 100644 index fe9a2448531a..000000000000 --- a/BaseTools/Bin/gcc_loongarch64_unknown_linux_ext_dep.yaml +++ /dev/null @@ -1,22 +0,0 @@ -## @file -# Download GCC LoongArch64 compiler from LoongArch GitHub release site -# Set shell variable GCC5_LOONGARCH64_INSTALL to this folder -# -# This is only downloaded when a build activates scope gcc_loongarch64_unknown_linux -# -# Copyright (c) Microsoft Corporation. -# Copyright (c) 2022, Loongson Technology Corporation Limited. All rights reserved.<BR> -# SPDX-License-Identifier: BSD-2-Clause-Patent -## -{ - "scope": "gcc_loongarch64_unknown_linux", - "type": "web", - "name": "gcc_loongarch64_unknown_linux", - "source":"https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-gcc-full.tar.xz", - "version": "13.0.0", - "sha256":"27a43c5bb127794f091d0e75da0003c4d0eec28a958d8f2cc7cd290a6e6133ab", - "compression_type": "tar", - "internal_path": "/cross-tools/", - "flags": ["set_shell_var", ], - "var_name": "GCC5_LOONGARCH64_INSTALL" -} diff --git a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml b/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml deleted file mode 100644 index 8abbcd7ba040..000000000000 --- a/BaseTools/Bin/gcc_riscv64_unknown_ext_dep.yaml +++ /dev/null @@ -1,22 +0,0 @@ -## @file -# Download GCC RISCV64 compiler from RISC-V Organization release site -# Set shell variable GCC5_RISCV64_INSTALL to this folder -# -# This is only downloaded when a build activates scope gcc_riscv64_unknown -# -# Copyright (c) Microsoft Corporation. -# Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rights reserved.<BR> -# SPDX-License-Identifier: BSD-2-Clause-Patent -## -{ - "scope": "gcc_riscv64_unknown", - "type": "web", - "name": "gcc_riscv64_unknown", - "source": "https://raw.githubusercontent.com/riscv/riscv-uefi-edk2-docs/master/gcc-riscv-edk2-ci-toolchain/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu.tar.xz", - "version": "9.2.0", - "compression_type": "tar", - "sha256": "28373643b69f0ce008273c3dc63f172aa1121952f1b9ae94d7485ac94af7f344", - "internal_path": "/gcc-riscv-9.2.0-2020.04-x86_64_riscv64-unknown-gnu", - "flags": ["set_shell_var", ], - "var_name": "GCC5_RISCV64_INSTALL" -} -- 2.38.1 |
|
|
|
[PATCH v8 08/12] OvmfPkg: CI: Use Fedora 35 container (Linux only)
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are here: https://github.com/tianocore/containers The version numbers of gcc, iasl, and nasm are pinned to avoid unintended upgrades during image rebuild. Do not run apt-get in CI jobs to install qemu and gcc dependencies. Assume the container image provides these. Use Python from the container image, do not download at runtime. Signed-off-by: Oliver Steffen <osteffen@...> --- OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index 7160d95f7e04..984782aa7d19 100644 --- a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -177,6 +177,8 @@ jobs: pool: vmImage: $(vm_image) + container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e' + steps: - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml parameters: @@ -187,7 +189,4 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) - extra_install_step: - - bash: sudo apt-get install qemu - displayName: Install qemu - condition: and(gt(variables.pkg_count, 0), succeeded()) + usePythonVersion: '' # use Python from the container image -- 2.38.1 |
|
|
|
[PATCH v8 04/12] CI: add ~/.local/bin to PATH (Linux only)
Oliver Steffen
Fix pip install under certain conditions.
Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++ .azurepipelines/templates/pr-gate-steps.yml | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml index 8803d80cf51c..087f460d7fec 100644 --- a/.azurepipelines/templates/platform-build-run-steps.yml +++ b/.azurepipelines/templates/platform-build-run-steps.yml @@ -39,6 +39,12 @@ parameters: default: '' steps: +- bash: | + echo "##vso[task.prependpath]${HOME}/.local/bin" + echo "new PATH=${PATH}" + displayName: Set PATH + condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5') + - checkout: self clean: true fetchDepth: 1 diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml index 27a44c06011c..99d52cfa0605 100644 --- a/.azurepipelines/templates/pr-gate-steps.yml +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -15,6 +15,12 @@ parameters: usePythonVersion: '' steps: +- bash: | + echo "##vso[task.prependpath]${HOME}/.local/bin" + echo "new PATH=${PATH}" + displayName: Set PATH + condition: eq('${{ parameters.tool_chain_tag }}', 'GCC5') + - checkout: self clean: true fetchDepth: 1 -- 2.38.1 |
|
|
|
[PATCH v8 07/12] ArmVirtPkg: CI: Use Fedora 35 container (Linux only)
Oliver Steffen
Run all Linux based jobs in a container, using a custom Fedora 35 image
(gcc 11). The image is hosted on ghcr.io and the Dockerfiles are here: https://github.com/tianocore/containers The version numbers of gcc, iasl, and nasm are pinned to avoid unintended upgrades during image rebuild. Do not run apt-get in CI jobs to install qemu and gcc dependencies. Assume the container image provides these. Use Python from the container image, do not download at runtime. Signed-off-by: Oliver Steffen <osteffen@...> --- ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml index b07e3199f143..6d75502b5628 100644 --- a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml @@ -75,6 +75,8 @@ jobs: pool: vmImage: $(vm_image) + container: 'ghcr.io/tianocore/containers/fedora-35-test:2113a0e' + steps: - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml parameters: @@ -85,7 +87,4 @@ jobs: build_file: $(Build.File) build_flags: $(Build.Flags) run_flags: $(Run.Flags) - extra_install_step: - - bash: sudo apt-get install qemu - displayName: Install qemu - condition: and(gt(variables.pkg_count, 0), succeeded()) + usePythonVersion: '' # use Python from the container image -- 2.38.1 |
|
|
|
[PATCH v8 01/12] CI: make Python version configurable
Oliver Steffen
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use. The default value is '', in which case Python will not be downloaded at runtime and the one provided by the VM/container image will be used. Additionally, add a template .azurepipelines/templates/defaults.yml, from which the default Pyhton version string can be obtained. Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/Windows-VS2019.yml | 4 ++++ .azurepipelines/templates/defaults.yml | 3 +++ .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++- .azurepipelines/templates/pr-gate-build-job.yml | 2 ++ .azurepipelines/templates/pr-gate-steps.yml | 6 ++++-- 5 files changed, 18 insertions(+), 3 deletions(-) create mode 100644 .azurepipelines/templates/defaults.yml diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml index e4bd4b1d2283..89bccde82575 100644 --- a/.azurepipelines/Windows-VS2019.yml +++ b/.azurepipelines/Windows-VS2019.yml @@ -12,9 +12,13 @@ pr: - master - stable/* +variables: + - template: templates/defaults.yml + jobs: - template: templates/pr-gate-build-job.yml parameters: tool_chain_tag: 'VS2019' vm_image: 'windows-2019' arch_list: "IA32,X64" + usePythonVersion: ${{ variables.default_python_version }} diff --git a/.azurepipelines/templates/defaults.yml b/.azurepipelines/templates/defaults.yml new file mode 100644 index 000000000000..314d03e33af8 --- /dev/null +++ b/.azurepipelines/templates/defaults.yml @@ -0,0 +1,3 @@ + +variables: + default_python_version: ">=3.10.6" diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml index 40a31a509fc5..8803d80cf51c 100644 --- a/.azurepipelines/templates/platform-build-run-steps.yml +++ b/.azurepipelines/templates/platform-build-run-steps.yml @@ -34,6 +34,9 @@ parameters: - name: extra_install_step type: stepList default: [] +- name: usePythonVersion + type: string + default: '' steps: - checkout: self @@ -42,8 +45,9 @@ steps: - task: UsePythonVersion@0 inputs: - versionSpec: ">=3.10.6" + versionSpec: ${{ parameters.usePythonVersion }} architecture: "x64" + condition: ne('${{ parameters.usePythonVersion }}', '') - script: pip install -r pip-requirements.txt --upgrade displayName: 'Install/Upgrade pip modules' diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml index 7f88b41dc8d3..f0ef1142da49 100644 --- a/.azurepipelines/templates/pr-gate-build-job.yml +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -12,6 +12,7 @@ parameters: tool_chain_tag: '' vm_image: '' arch_list: '' + usePythonVersion: '' # Build step jobs: @@ -77,3 +78,4 @@ jobs: build_pkgs: $(Build.Pkgs) build_targets: $(Build.Targets) build_archs: ${{ parameters.arch_list }} + usePythonVersion: ${{ parameters.usePythonVersion }} diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml index cb431e53fcd1..27a44c06011c 100644 --- a/.azurepipelines/templates/pr-gate-steps.yml +++ b/.azurepipelines/templates/pr-gate-steps.yml @@ -12,6 +12,7 @@ parameters: build_pkgs: '' build_targets: '' build_archs: '' + usePythonVersion: '' steps: - checkout: self @@ -20,8 +21,9 @@ steps: - task: UsePythonVersion@0 inputs: - versionSpec: '>=3.10.6' - architecture: 'x64' + versionSpec: ${{ parameters.usePythonVersion }} + architecture: "x64" + condition: ne('${{ parameters.usePythonVersion }}', '') - script: pip install -r pip-requirements.txt --upgrade displayName: 'Install/Upgrade pip modules' -- 2.38.1 |
|
|
|
[PATCH v8 05/12] CI: Allow running in a container.
Oliver Steffen
Add a parameter of the pr-gate-build-job template to specify a
container image URL. If the value is not '' (default), then the jobs will be run inside a container based on that image. Signed-off-by: Oliver Steffen <osteffen@...> --- .azurepipelines/templates/pr-gate-build-job.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml index f0ef1142da49..5ffcaca7b591 100644 --- a/.azurepipelines/templates/pr-gate-build-job.yml +++ b/.azurepipelines/templates/pr-gate-build-job.yml @@ -12,6 +12,7 @@ parameters: tool_chain_tag: '' vm_image: '' arch_list: '' + container: '' usePythonVersion: '' # Build step @@ -71,6 +72,9 @@ jobs: pool: vmImage: ${{ parameters.vm_image }} + ${{ if not(eq(parameters.container, '')) }}: + container: ${{ parameters.container }} + steps: - template: pr-gate-steps.yml parameters: -- 2.38.1 |
|
|