devel@edk2.groups.io | [PATCH v7 1/6] CI: make Python version configurable

Home Messages Hashtags Subgroups Calendar

Date Date 1 - 4 of 4

[PATCH v7 1/6] CI: make Python version configurable

Oliver Steffen

Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is ">=3.10.6".
If '' is specified, Python will not be downloaded at runtime and
the one provided by the VM/container image will be used.

Signed-off-by: Oliver Steffen <osteffen@...>
---
 .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++-
 .azurepipelines/templates/pr-gate-build-job.yml        | 2 ++
 .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++--
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..6c9189a20fde 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
 - name: extra_install_step
   type: stepList
   default: []
+- name: usePythonVersion
+  type: string
+  default: ">=3.10.6"

 steps:
 - checkout: self
@@ -42,8 +45,9 @@ steps:

 - task: UsePythonVersion@0
   inputs:
-    versionSpec: ">=3.10.6"
+    versionSpec: ${{ parameters.usePythonVersion }}
     architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

 - script: pip install -r pip-requirements.txt --upgrade
   displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8d3..d0a6ccd10700 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
   tool_chain_tag: ''
   vm_image: ''
   arch_list: ''
+  usePythonVersion: '>=3.10.6'

 # Build step
 jobs:
@@ -77,3 +78,4 @@ jobs:
       build_pkgs: $(Build.Pkgs)
       build_targets: $(Build.Targets)
       build_archs: ${{ parameters.arch_list }}
+      usePythonVersion: ${{ parameters.usePythonVersion }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fcd1..bf6574b33692 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
   build_pkgs: ''
   build_targets: ''
   build_archs: ''
+  usePythonVersion: '>=3.10.6'

 steps:
 - checkout: self
@@ -20,8 +21,9 @@ steps:

 - task: UsePythonVersion@0
   inputs:
-    versionSpec: '>=3.10.6'
-    architecture: 'x64'
+    versionSpec: ${{ parameters.usePythonVersion }}
+    architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

 - script: pip install -r pip-requirements.txt --upgrade
   displayName: 'Install/Upgrade pip modules'
-- 
2.38.1

Michael Kubacki

#96735

This distributes the Python version (as a default value) across several files - ">=3.10.6". For maintainability, we'd like to manage versioning more centrally to reduce likelihood of misses on updates.

You could potentially adjust it such that the default does not specify a version, put the Python version in a template and call that when needed, etc.

Thanks,
Michael

Show quoted text

On 11/29/2022 2:26 PM, Oliver Steffen wrote:
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is ">=3.10.6".
If '' is specified, Python will not be downloaded at runtime and
the one provided by the VM/container image will be used.
Signed-off-by: Oliver Steffen <osteffen@...>
---
  .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++-
  .azurepipelines/templates/pr-gate-build-job.yml        | 2 ++
  .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++--
  3 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..6c9189a20fde 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
  - name: extra_install_step
    type: stepList
    default: []
+- name: usePythonVersion
+  type: string
+  default: ">=3.10.6"
steps:
  - checkout: self
@@ -42,8 +45,9 @@ steps:
- task: UsePythonVersion@0
    inputs:
-    versionSpec: ">=3.10.6"
+    versionSpec: ${{ parameters.usePythonVersion }}
      architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
    displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8d3..d0a6ccd10700 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
    tool_chain_tag: ''
    vm_image: ''
    arch_list: ''
+  usePythonVersion: '>=3.10.6'
# Build step
  jobs:
@@ -77,3 +78,4 @@ jobs:
        build_pkgs: $(Build.Pkgs)
        build_targets: $(Build.Targets)
        build_archs: ${{ parameters.arch_list }}
+      usePythonVersion: ${{ parameters.usePythonVersion }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fcd1..bf6574b33692 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
    build_pkgs: ''
    build_targets: ''
    build_archs: ''
+  usePythonVersion: '>=3.10.6'
steps:
  - checkout: self
@@ -20,8 +21,9 @@ steps:
- task: UsePythonVersion@0
    inputs:
-    versionSpec: '>=3.10.6'
-    architecture: 'x64'
+    versionSpec: ${{ parameters.usePythonVersion }}
+    architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')
- script: pip install -r pip-requirements.txt --upgrade
    displayName: 'Install/Upgrade pip modules'

Oliver Steffen

#96741

Hi Michael

On Wed, Nov 30, 2022 at 3:13 PM Michael Kubacki
<mikuback@...> wrote:

This distributes the Python version (as a default value) across several
files - ">=3.10.6". For maintainability, we'd like to manage versioning
more centrally to reduce likelihood of misses on updates.

You could potentially adjust it such that the default does not specify a
version, put the Python version in a template and call that when needed,
etc.Yes, it sounds like a good idea.

Can we work on this in a separate patch set?
I am experimenting with this now, but I fear this might delay
these patches too much and go for a minimal version for now.
Maybe we have a chance to merge something before the CI stops
working tomorrow.

Thanks,
  Oliver

Thanks,
Michael

On 11/29/2022 2:26 PM, Oliver Steffen wrote:
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is ">=3.10.6".
If '' is specified, Python will not be downloaded at runtime and
the one provided by the VM/container image will be used.

Signed-off-by: Oliver Steffen <osteffen@...>
---
  .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++-
  .azurepipelines/templates/pr-gate-build-job.yml        | 2 ++
  .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++--
  3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..6c9189a20fde 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
  - name: extra_install_step
    type: stepList
    default: []
+- name: usePythonVersion
+  type: string
+  default: ">=3.10.6"

  steps:
  - checkout: self
@@ -42,8 +45,9 @@ steps:

  - task: UsePythonVersion@0
    inputs:
-    versionSpec: ">=3.10.6"
+    versionSpec: ${{ parameters.usePythonVersion }}
      architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

  - script: pip install -r pip-requirements.txt --upgrade
    displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8d3..d0a6ccd10700 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
    tool_chain_tag: ''
    vm_image: ''
    arch_list: ''
+  usePythonVersion: '>=3.10.6'

  # Build step
  jobs:
@@ -77,3 +78,4 @@ jobs:
        build_pkgs: $(Build.Pkgs)
        build_targets: $(Build.Targets)
        build_archs: ${{ parameters.arch_list }}
+      usePythonVersion: ${{ parameters.usePythonVersion }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fcd1..bf6574b33692 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
    build_pkgs: ''
    build_targets: ''
    build_archs: ''
+  usePythonVersion: '>=3.10.6'

  steps:
  - checkout: self
@@ -20,8 +21,9 @@ steps:

  - task: UsePythonVersion@0
    inputs:
-    versionSpec: '>=3.10.6'
-    architecture: 'x64'
+    versionSpec: ${{ parameters.usePythonVersion }}
+    architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

  - script: pip install -r pip-requirements.txt --upgrade
    displayName: 'Install/Upgrade pip modules'

Michael Kubacki

#96743

I'm fine for it to come in a separate patch if necessary. If you end up deferring it, maybe file a brief BZ to track it.

Thanks,
Michael

Show quoted text

On 11/30/2022 12:52 PM, Oliver Steffen wrote:
Hi Michael
On Wed, Nov 30, 2022 at 3:13 PM Michael Kubacki
<mikuback@...> wrote:

This distributes the Python version (as a default value) across several
files - ">=3.10.6". For maintainability, we'd like to manage versioning
more centrally to reduce likelihood of misses on updates.

You could potentially adjust it such that the default does not specify a
version, put the Python version in a template and call that when needed,
etc.Yes, it sounds like a good idea.
Can we work on this in a separate patch set?
I am experimenting with this now, but I fear this might delay
these patches too much and go for a minimal version for now.
Maybe we have a chance to merge something before the CI stops
working tomorrow.
Thanks,
   Oliver

Thanks,
Michael

On 11/29/2022 2:26 PM, Oliver Steffen wrote:
Add a new parameter "usePythonVersion" to the CI job templates.
This makes it possible to specify the version of Python to use.
The default value is ">=3.10.6".
If '' is specified, Python will not be downloaded at runtime and
the one provided by the VM/container image will be used.

Signed-off-by: Oliver Steffen <osteffen@...>
---
   .azurepipelines/templates/platform-build-run-steps.yml | 6 +++++-
   .azurepipelines/templates/pr-gate-build-job.yml        | 2 ++
   .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++--
   3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509fc5..6c9189a20fde 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -34,6 +34,9 @@ parameters:
   - name: extra_install_step
     type: stepList
     default: []
+- name: usePythonVersion
+  type: string
+  default: ">=3.10.6"

   steps:
   - checkout: self
@@ -42,8 +45,9 @@ steps:

   - task: UsePythonVersion@0
     inputs:
-    versionSpec: ">=3.10.6"
+    versionSpec: ${{ parameters.usePythonVersion }}
       architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

   - script: pip install -r pip-requirements.txt --upgrade
     displayName: 'Install/Upgrade pip modules'
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml
b/.azurepipelines/templates/pr-gate-build-job.yml
index 7f88b41dc8d3..d0a6ccd10700 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -12,6 +12,7 @@ parameters:
     tool_chain_tag: ''
     vm_image: ''
     arch_list: ''
+  usePythonVersion: '>=3.10.6'

   # Build step
   jobs:
@@ -77,3 +78,4 @@ jobs:
         build_pkgs: $(Build.Pkgs)
         build_targets: $(Build.Targets)
         build_archs: ${{ parameters.arch_list }}
+      usePythonVersion: ${{ parameters.usePythonVersion }}
diff --git a/.azurepipelines/templates/pr-gate-steps.yml
b/.azurepipelines/templates/pr-gate-steps.yml
index cb431e53fcd1..bf6574b33692 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -12,6 +12,7 @@ parameters:
     build_pkgs: ''
     build_targets: ''
     build_archs: ''
+  usePythonVersion: '>=3.10.6'

   steps:
   - checkout: self
@@ -20,8 +21,9 @@ steps:

   - task: UsePythonVersion@0
     inputs:
-    versionSpec: '>=3.10.6'
-    architecture: 'x64'
+    versionSpec: ${{ parameters.usePythonVersion }}
+    architecture: "x64"
+  condition: ne('${{ parameters.usePythonVersion }}', '')

   - script: pip install -r pip-requirements.txt --upgrade
     displayName: 'Install/Upgrade pip modules'

1 - 4 of 4

Previous Topic Next Topic

Home Hashtags Subgroups Calendar About Features Pricing Updates Terms Help Twitter