[PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest


duntan
 

Hi all,
Could you please help to review this patch? Thanks a lot!

Thanks,
Dun

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki <mikuback@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.

In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.

Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''

- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'

# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"

# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target
+ origin/$(System.PullRequest.targetBranch) --output-count-format-string
+ "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')

# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c
+${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{
+parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))

# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1

--
2.31.1.windows.1


duntan
 

Hi all,
Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.

Thanks,
Dun

-----Original Message-----
From: Tan, Dun
Sent: Monday, November 28, 2022 5:34 PM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki <mikuback@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Hi all,
Could you please help to review this patch? Thanks a lot!

Thanks,
Dun

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki <mikuback@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.

In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.

Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''

- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'

# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"

# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target
+ origin/$(System.PullRequest.targetBranch) --output-count-format-string
+ "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')

# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c
+${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{
+parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))

# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1

--
2.31.1.windows.1


Michael Kubacki
 

Sorry for the delay Dun. Can you please share an edk2 pull request with this change?

Thanks,
Michael

On 12/4/2022 10:57 PM, duntan wrote:
Hi all,
Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.
Thanks,
Dun
-----Original Message-----
From: Tan, Dun
Sent: Monday, November 28, 2022 5:34 PM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki <mikuback@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest
Hi all,
Could you please help to review this patch? Thanks a lot!
Thanks,
Dun
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki <mikuback@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest
Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.
In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.
Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''
- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'
# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"
# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target
+ origin/$(System.PullRequest.targetBranch) --output-count-format-string
+ "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')
# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c
+${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{
+parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))
# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1
--
2.31.1.windows.1


duntan
 

Hi Michael,
Thanks for the reply! In the following PR, I added an unit test list in the new OvmfPkg platform CI JOB. In PlatformCI_OvmfPkg_Ubuntu_GCC5_PR and PlatformCI_OvmfPkg_Windows_VS2019_PR, the CI for specific unit test list was triggered.
https://github.com/tianocore/edk2/pull/3651

Thanks,
Dun

-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Tuesday, December 6, 2022 9:24 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Sorry for the delay Dun. Can you please share an edk2 pull request with this change?

Thanks,
Michael

On 12/4/2022 10:57 PM, duntan wrote:
Hi all,
Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.

Thanks,
Dun
-----Original Message-----
From: Tan, Dun
Sent: Monday, November 28, 2022 5:34 PM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
PlatformCI template for Shell UnitTest

Hi all,
Could you please help to review this patch? Thanks a lot!

Thanks,
Dun

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI
template for Shell UnitTest

Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.

In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.

Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''

- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'

# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"

# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target
+ origin/$(System.PullRequest.targetBranch)
+ --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')

# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c
+${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))

# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1

--
2.31.1.windows.1










Michael Kubacki
 

Hi Dun,

Sean Brogan and I have some feedback we'll send soon after we sync on it. If anything is still open as of the upcoming TianoCore tools & CI meeting and you're able to attend, we can talk there as well.

Thanks,
Michael

On 12/5/2022 11:46 PM, duntan wrote:
Hi Michael,
Thanks for the reply! In the following PR, I added an unit test list in the new OvmfPkg platform CI JOB. In PlatformCI_OvmfPkg_Ubuntu_GCC5_PR and PlatformCI_OvmfPkg_Windows_VS2019_PR, the CI for specific unit test list was triggered.
https://github.com/tianocore/edk2/pull/3651
Thanks,
Dun
-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Tuesday, December 6, 2022 9:24 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest
Sorry for the delay Dun. Can you please share an edk2 pull request with this change?
Thanks,
Michael
On 12/4/2022 10:57 PM, duntan wrote:
Hi all,
Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.

Thanks,
Dun
-----Original Message-----
From: Tan, Dun
Sent: Monday, November 28, 2022 5:34 PM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
PlatformCI template for Shell UnitTest

Hi all,
Could you please help to review this patch? Thanks a lot!

Thanks,
Dun

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI
template for Shell UnitTest

Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.

In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.

Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''
- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'
# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"
# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target
+ origin/$(System.PullRequest.targetBranch)
+ --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')
# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build) -c
+${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))
# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1
--
2.31.1.windows.1










duntan
 

Thank you and Sean a lot! I'll attend the CI meeting if there is any open.

Thanks,
Dun

-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Thursday, December 8, 2022 10:57 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Hi Dun,

Sean Brogan and I have some feedback we'll send soon after we sync on it. If anything is still open as of the upcoming TianoCore tools & CI meeting and you're able to attend, we can talk there as well.

Thanks,
Michael

On 12/5/2022 11:46 PM, duntan wrote:
Hi Michael,
Thanks for the reply! In the following PR, I added an unit test list in the new OvmfPkg platform CI JOB. In PlatformCI_OvmfPkg_Ubuntu_GCC5_PR and PlatformCI_OvmfPkg_Windows_VS2019_PR, the CI for specific unit test list was triggered.
https://github.com/tianocore/edk2/pull/3651

Thanks,
Dun

-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Tuesday, December 6, 2022 9:24 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
PlatformCI template for Shell UnitTest

Sorry for the delay Dun. Can you please share an edk2 pull request with this change?

Thanks,
Michael

On 12/4/2022 10:57 PM, duntan wrote:
Hi all,
Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.

Thanks,
Dun
-----Original Message-----
From: Tan, Dun
Sent: Monday, November 28, 2022 5:34 PM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
PlatformCI template for Shell UnitTest

Hi all,
Could you please help to review this patch? Thanks a lot!

Thanks,
Dun

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
Sent: Tuesday, November 22, 2022 7:48 PM
To: devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
<mikuback@...>; Kinney, Michael D
<michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
Ni, Ray <ray.ni@...>
Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI
template for Shell UnitTest

Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.

In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.

Signed-off-by: Dun Tan <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>
Cc: Michael Kubacki <mikuback@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <gaoliming@...>
Cc: Ray Ni <ray.ni@...>
---
.azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
b/.azurepipelines/templates/platform-build-run-steps.yml
index 40a31a509f..51503287c4 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -30,6 +30,9 @@ parameters:
- name: run_flags
type: string
default: ''
+- name: unit_test_list
+ type: string
+ default: ''

- name: extra_install_step
type: stepList
@@ -49,7 +52,9 @@ steps:
displayName: 'Install/Upgrade pip modules'

# Set default
-- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+- bash: |
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"

# Fetch the target branch so that pr_eval can diff them.
# Seems like azure pipelines/github changed checkout process in nov 2020.
@@ -62,7 +67,7 @@ steps:
displayName: Check if ${{ parameters.build_pkg }} need testing
inputs:
filename: stuart_pr_eval
- arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ arguments: -c ${{ parameters.build_file }} -t ${{
+ parameters.build_target}} -a ${{ parameters.build_arch}}
+ --pr-target
+ origin/$(System.PullRequest.targetBranch)
+ --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ --output-csv-format-string "##vso[task.setvariable
+ variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
condition: eq(variables['Build.Reason'], 'PullRequest')

# Setup repo
@@ -97,14 +102,22 @@ steps:
inputs:
filename: stuart_build
arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
${{ parameters.build_arch}} ${{ parameters.build_flags}}
- condition: and(gt(variables.pkg_count, 0), succeeded())
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+ eq(variables.unit_test_list, ''))
+
+# Build specific pkg for UnitTest
+- task: CmdLine@1
+ displayName: Build UnitTest
+ inputs:
+ filename: stuart_build
+ arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build)
+-c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()),
+not(eq(variables.unit_test_list, '')))

# Run
- task: CmdLine@1
displayName: Run to shell
inputs:
filename: stuart_build
- arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
+ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
+ ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
+ parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
timeoutInMinutes: 1

--
2.31.1.windows.1













Sean Brogan <sean.brogan@...>
 

Dun,

I created this discussion on github hoping that others might have ideas/feedback.  

Lets use that to discuss and then on monday we can talk more if needed. 

Thanks
Sean


Starting a discussion here to see if more individuals in the community have opinions on how best to integrate target-based testing into edk2 CI. There is a current proposed patch on the mailing lis...
github.com


From: Tan, Dun <dun.tan@...>
Sent: Thursday, December 8, 2022 12:17 AM
To: Michael Kubacki <mikuback@...>; devel@edk2.groups.io <devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: [EXTERNAL] RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest
 
Thank you and Sean a lot! I'll attend the CI meeting if there is any open.

Thanks,
Dun

-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Thursday, December 8, 2022 10:57 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Hi Dun,

Sean Brogan and I have some feedback we'll send soon after we sync on it. If anything is still open as of the upcoming TianoCore tools & CI meeting and you're able to attend, we can talk there as well.

Thanks,
Michael

On 12/5/2022 11:46 PM, duntan wrote:
> Hi Michael,
> Thanks for the reply! In the following PR, I added an unit test list in the new OvmfPkg platform CI JOB. In  PlatformCI_OvmfPkg_Ubuntu_GCC5_PR and PlatformCI_OvmfPkg_Windows_VS2019_PR, the CI for specific unit test list was triggered.
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F3651&data=05%7C01%7Csean.brogan%40microsoft.com%7C884211b3375b4f0618de08dad8f4bdea%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638060842928005678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FPbU3fGHlHv5%2FlKn8tvUdEO6%2Ff2SjCAzc7u2KVRxyK0%3D&reserved=0
>
> Thanks,
> Dun
>
> -----Original Message-----
> From: Michael Kubacki <mikuback@...>
> Sent: Tuesday, December 6, 2022 9:24 AM
> To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
> Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D
> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
> Ni, Ray <ray.ni@...>
> Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
> PlatformCI template for Shell UnitTest
>
> Sorry for the delay Dun. Can you please share an edk2 pull request with this change?
>
> Thanks,
> Michael
>
> On 12/4/2022 10:57 PM, duntan wrote:
>> Hi all,
>> Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.
>>
>> Thanks,
>> Dun
>> -----Original Message-----
>> From: Tan, Dun
>> Sent: Monday, November 28, 2022 5:34 PM
>> To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
>> Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
>> <mikuback@...>; Kinney, Michael D
>> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
>> Ni, Ray <ray.ni@...>
>> Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
>> PlatformCI template for Shell UnitTest
>>
>> Hi all,
>> Could you please help to review this patch? Thanks a lot!
>>
>> Thanks,
>> Dun
>>
>> -----Original Message-----
>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
>> Sent: Tuesday, November 22, 2022 7:48 PM
>> To: devel@edk2.groups.io
>> Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
>> <mikuback@...>; Kinney, Michael D
>> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
>> Ni, Ray <ray.ni@...>
>> Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI
>> template for Shell UnitTest
>>
>> Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.
>>
>> In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
>> In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
>> In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.
>>
>> Signed-off-by: Dun Tan <dun.tan@...>
>> Cc: Sean Brogan <sean.brogan@...>
>> Cc: Michael Kubacki <mikuback@...>
>> Cc: Michael D Kinney <michael.d.kinney@...>
>> Cc: Liming Gao <gaoliming@...>
>> Cc: Ray Ni <ray.ni@...>
>> ---
>>    .azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
>>    1 file changed, 17 insertions(+), 4 deletions(-)
>>
>> diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
>> b/.azurepipelines/templates/platform-build-run-steps.yml
>> index 40a31a509f..51503287c4 100644
>> --- a/.azurepipelines/templates/platform-build-run-steps.yml
>> +++ b/.azurepipelines/templates/platform-build-run-steps.yml
>> @@ -30,6 +30,9 @@ parameters:
>>    - name: run_flags
>>      type: string
>>      default: ''
>> +- name: unit_test_list
>> +  type: string
>> +  default: ''
>>   
>>    - name: extra_install_step
>>      type: stepList
>> @@ -49,7 +52,9 @@ steps:
>>      displayName: 'Install/Upgrade pip modules'
>>   
>>    # Set default
>> -- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
>> +- bash: |
>> +    echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
>> +    echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"
>>   
>>    # Fetch the target branch so that pr_eval can diff them.
>>    # Seems like azure pipelines/github changed checkout process in nov 2020.
>> @@ -62,7 +67,7 @@ steps:
>>      displayName: Check if ${{ parameters.build_pkg }} need testing
>>      inputs:
>>        filename: stuart_pr_eval
>> -    arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
>> +    arguments: -c ${{ parameters.build_file }} -t ${{
>> + parameters.build_target}} -a ${{ parameters.build_arch}}
>> + --pr-target
>> + origin/$(System.PullRequest.targetBranch)
>> + --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
>> + --output-csv-format-string "##vso[task.setvariable
>> + variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
>>      condition: eq(variables['Build.Reason'], 'PullRequest')
>>   
>>     # Setup repo
>> @@ -97,14 +102,22 @@ steps:
>>      inputs:
>>        filename: stuart_build
>>        arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> ${{ parameters.build_arch}} ${{ parameters.build_flags}}
>> -  condition: and(gt(variables.pkg_count, 0), succeeded())
>> +  condition: and(and(gt(variables.pkg_count, 0), succeeded()),
>> + eq(variables.unit_test_list, ''))
>> +
>> +# Build specific pkg for UnitTest
>> +- task: CmdLine@1
>> +  displayName: Build UnitTest
>> +  inputs:
>> +    filename: stuart_build
>> +    arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build)
>> +-c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> +parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> +${{ parameters.build_arch}} ${{ parameters.build_flags}}
>> +  condition: and(and(gt(variables.pkg_count, 0), succeeded()),
>> +not(eq(variables.unit_test_list, '')))
>>   
>>    # Run
>>    - task: CmdLine@1
>>      displayName: Run to shell
>>      inputs:
>>        filename: stuart_build
>> -    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
>> +    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> + parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> + ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
>> + parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
>>      condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
>>      timeoutInMinutes: 1
>>   
>> --
>> 2.31.1.windows.1
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>


duntan
 

Hi Sean,

Thank you for creating this discussion. Looking forward to more feedback from community.

 

Thanks,

Dun

 

From: Sean Brogan <sean.brogan@...>
Sent: Friday, December 9, 2022 6:37 AM
To: Tan, Dun <dun.tan@...>; Michael Kubacki <mikuback@...>; devel@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

 

Dun,

 

I created this discussion on github hoping that others might have ideas/feedback.  

 

Lets use that to discuss and then on monday we can talk more if needed. 

 

Thanks

Sean

 

 

Starting a discussion here to see if more individuals in the community have opinions on how best to integrate target-based testing into edk2 CI. There is a current proposed patch on the mailing lis...

github.com

 


From: Tan, Dun <dun.tan@...>
Sent: Thursday, December 8, 2022 12:17 AM
To: Michael Kubacki <mikuback@...>; devel@edk2.groups.io <devel@edk2.groups.io>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: [EXTERNAL] RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

 

Thank you and Sean a lot! I'll attend the CI meeting if there is any open.

Thanks,
Dun

-----Original Message-----
From: Michael Kubacki <mikuback@...>
Sent: Thursday, December 8, 2022 10:57 AM
To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>; Gao, Liming <gaoliming@...>; Ni, Ray <ray.ni@...>
Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI template for Shell UnitTest

Hi Dun,

Sean Brogan and I have some feedback we'll send soon after we sync on it. If anything is still open as of the upcoming TianoCore tools & CI meeting and you're able to attend, we can talk there as well.

Thanks,
Michael

On 12/5/2022 11:46 PM, duntan wrote:
> Hi Michael,
> Thanks for the reply! In the following PR, I added an unit test list in the new OvmfPkg platform CI JOB. In  PlatformCI_OvmfPkg_Ubuntu_GCC5_PR and PlatformCI_OvmfPkg_Windows_VS2019_PR, the CI for specific unit test list was triggered.
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F3651&data=05%7C01%7Csean.brogan%40microsoft.com%7C884211b3375b4f0618de08dad8f4bdea%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638060842928005678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2FPbU3fGHlHv5%2FlKn8tvUdEO6%2Ff2SjCAzc7u2KVRxyK0%3D&reserved=0
>
> Thanks,
> Dun
>
> -----Original Message-----
> From: Michael Kubacki <mikuback@...>
> Sent: Tuesday, December 6, 2022 9:24 AM
> To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
> Cc: Sean Brogan <sean.brogan@...>; Kinney, Michael D
> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
> Ni, Ray <ray.ni@...>
> Subject: Re: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
> PlatformCI template for Shell UnitTest
>
> Sorry for the delay Dun. Can you please share an edk2 pull request with this change?
>
> Thanks,
> Michael
>
> On 12/4/2022 10:57 PM, duntan wrote:
>> Hi all,
>> Is there anything else I can do to speed up the review process for this patch set? Looking forward to your reply.
>>
>> Thanks,
>> Dun
>> -----Original Message-----
>> From: Tan, Dun
>> Sent: Monday, November 28, 2022 5:34 PM
>> To: devel@edk2.groups.io; Tan, Dun <dun.tan@...>
>> Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
>> <mikuback@...>; Kinney, Michael D
>> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
>> Ni, Ray <ray.ni@...>
>> Subject: RE: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand
>> PlatformCI template for Shell UnitTest
>>
>> Hi all,
>> Could you please help to review this patch? Thanks a lot!
>>
>> Thanks,
>> Dun
>>
>> -----Original Message-----
>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of duntan
>> Sent: Tuesday, November 22, 2022 7:48 PM
>> To: devel@edk2.groups.io
>> Cc: Sean Brogan <sean.brogan@...>; Michael Kubacki
>> <mikuback@...>; Kinney, Michael D
>> <michael.d.kinney@...>; Gao, Liming <gaoliming@...>;
>> Ni, Ray <ray.ni@...>
>> Subject: [edk2-devel] [PATCH 3/3] .azurepipelines: Expand PlatformCI
>> template for Shell UnitTest
>>
>> Expand PlatformCI build and run steps template for Shell UnitTest. Add a new parameter unit_test_list to support building and running specific Shell UnitTest modules.
>>
>> In stuart_pr_eval step, if the unit_test_list passed from platform yml file is not null, it will select some packages from the packages which contain the module in unit_test_list and set them into a new variable pkgs_to_build base on its evaluation rule.
>> In stuart_build step, if unit_test_list is not null, '${{ parameters.unit_test_list}} -p $(pkgs_to_build)' will be added into the arguments to build specific UnitTest modules in pkgs_to_build.
>> In 'Run to shell' step, if unit_test_list is not null, all the UnitTest modules built in stuart_build step will runs in shell.
>>
>> Signed-off-by: Dun Tan <dun.tan@...>
>> Cc: Sean Brogan <sean.brogan@...>
>> Cc: Michael Kubacki <mikuback@...>
>> Cc: Michael D Kinney <michael.d.kinney@...>
>> Cc: Liming Gao <gaoliming@...>
>> Cc: Ray Ni <ray.ni@...>
>> ---
>>    .azurepipelines/templates/platform-build-run-steps.yml | 21 +++++++++++++++++----
>>    1 file changed, 17 insertions(+), 4 deletions(-)
>>
>> diff --git a/.azurepipelines/templates/platform-build-run-steps.yml
>> b/.azurepipelines/templates/platform-build-run-steps.yml
>> index 40a31a509f..51503287c4 100644
>> --- a/.azurepipelines/templates/platform-build-run-steps.yml
>> +++ b/.azurepipelines/templates/platform-build-run-steps.yml
>> @@ -30,6 +30,9 @@ parameters:
>>    - name: run_flags
>>      type: string
>>      default: ''
>> +- name: unit_test_list
>> +  type: string
>> +  default: ''
>>   
>>    - name: extra_install_step
>>      type: stepList
>> @@ -49,7 +52,9 @@ steps:
>>      displayName: 'Install/Upgrade pip modules'
>>   
>>    # Set default
>> -- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
>> +- bash: |
>> +    echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
>> +    echo "##vso[task.setvariable variable=pkgs_to_build]${{ 'all' }}"
>>   
>>    # Fetch the target branch so that pr_eval can diff them.
>>    # Seems like azure pipelines/github changed checkout process in nov 2020.
>> @@ -62,7 +67,7 @@ steps:
>>      displayName: Check if ${{ parameters.build_pkg }} need testing
>>      inputs:
>>        filename: stuart_pr_eval
>> -    arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
>> +    arguments: -c ${{ parameters.build_file }} -t ${{
>> + parameters.build_target}} -a ${{ parameters.build_arch}}
>> + --pr-target
>> + origin/$(System.PullRequest.targetBranch)
>> + --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
>> + --output-csv-format-string "##vso[task.setvariable
>> + variable=pkgs_to_build]{pkgcsv}" ${{ parameters.unit_test_list}}
>>      condition: eq(variables['Build.Reason'], 'PullRequest')
>>   
>>     # Setup repo
>> @@ -97,14 +102,22 @@ steps:
>>      inputs:
>>        filename: stuart_build
>>        arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> ${{ parameters.build_arch}} ${{ parameters.build_flags}}
>> -  condition: and(gt(variables.pkg_count, 0), succeeded())
>> +  condition: and(and(gt(variables.pkg_count, 0), succeeded()),
>> + eq(variables.unit_test_list, ''))
>> +
>> +# Build specific pkg for UnitTest
>> +- task: CmdLine@1
>> +  displayName: Build UnitTest
>> +  inputs:
>> +    filename: stuart_build
>> +    arguments: ${{ parameters.unit_test_list}} -p $(pkgs_to_build)
>> +-c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> +parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> +${{ parameters.build_arch}} ${{ parameters.build_flags}}
>> +  condition: and(and(gt(variables.pkg_count, 0), succeeded()),
>> +not(eq(variables.unit_test_list, '')))
>>   
>>    # Run
>>    - task: CmdLine@1
>>      displayName: Run to shell
>>      inputs:
>>        filename: stuart_build
>> -    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
>> +    arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{
>> + parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a
>> + ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{
>> + parameters.run_flags }} --FlashOnly ${{ parameters.unit_test_list}}
>>      condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
>>      timeoutInMinutes: 1
>>   
>> --
>> 2.31.1.windows.1
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>