Date
1 - 7 of 7
[edk2-devel] [RFC] EDK II Continuous Integration Phase 1
Michael D Kinney
Hi Sean,
Which OS/Compiler configurations are currently enabled
for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler
configurations in Azure Pipelines. There are some tools
that need to be installed for each of these environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools
installed? Is it in the YML files or in the Python
scripts.
One critical task is to identify the tools and their
specific versions that the CI system is configured to use.
These configurations should be documented in a Wiki page
and updated as new tools are released and adopted by EDK II.
The inventory of tools used to validate a release should
Also be documented in a release notes for a stable tag.
Here are the YML files that install the additional tools
required to support EDK II builds. I need the source and
versions of these tools to be reviewed and approved.
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/WindowsPrerequisites.yml
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/UbuntuPrerequisites.yml
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/MacOsPrerequisites.yml
Thanks,
Mike
toggle quoted message
Show quoted text
Which OS/Compiler configurations are currently enabled
for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler
configurations in Azure Pipelines. There are some tools
that need to be installed for each of these environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools
installed? Is it in the YML files or in the Python
scripts.
One critical task is to identify the tools and their
specific versions that the CI system is configured to use.
These configurations should be documented in a Wiki page
and updated as new tools are released and adopted by EDK II.
The inventory of tools used to validate a release should
Also be documented in a release notes for a stable tag.
Here are the YML files that install the additional tools
required to support EDK II builds. I need the source and
versions of these tools to be reviewed and approved.
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/WindowsPrerequisites.yml
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/UbuntuPrerequisites.yml
https://github.com/mdkinney/edk2-ci/blob/master/AzurePipelines/MacOsPrerequisites.yml
Thanks,
Mike
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On
Behalf Of Sean via Groups.Io
Sent: Thursday, August 29, 2019 7:22 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; devel@edk2.groups.io
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: Re: [edk2-devel] [RFC] EDK II Continuous
Integration Phase 1
Mike, as you mentioned we have been working towards
enabling a practical and extensible CI for Edk2 using
Azure dev ops and the recently added edk2-pytool
infrastructure. We have been using similar CI for
Project Mu for the last few years.
Our approach is a little different in that we focus on
validating the whole code base rather than just the
incoming patch. We do this because we have found
unexpected consequences of patches and overall we want
all code to be compliant not just new additions. We
have found the time to test the whole tree is not much
longer than only the parts impacted by a code change
(except maybe when running the entire compile test on
every package). This obviously comes with an initial
tax of needing to get the codebase into compliant form.
Anyway we have prepared an RFC in addition to yours and
would like to see these two efforts merged together.
We are still working on making a few optimizations.
Currently if the full set of tests are run we take
about 20 minutes. This is because compiling
MdeModulePkg for debug, release, and host based tests
take a while. Most other packages are in the 10 minute
range. We do have easy ways to disable or limit
certain tests as well as expand the matrix to leverage
more cloud resources (more parallel builds).
Content is best viewed online with links to helpful
content but is also attached below:
https://github.com/spbrogan/edk2-staging/blob/edk2-
stuart-ci-latest/Readme-CI-RFC.md
# CI and PR Gates
## Background
Historically, while the TianoCore maintainers and
stewards have done a fantastic job of keeping
contribution policies consistent and contributions
clean and well-documented, there have been few
processes that ran to verify the sanity, cleanliness,
and efficacy of the codebase, and even fewer that
publicly published their results for the community at
large. This has caused inconsistancies and issues
within the codebase from time to time.
Adding continuous integration (and potentially PR
gates) to the checkin process ensures that simple
errors like these are caught and can be fixed on a
regular basis.
## Strategy
While a number of CI solutions exist, this proposal
will focus on the usage of Azure Dev Ops and Build
Pipelines. For demonstration, a sample [TianoCore
repo](https://github.com/spbrogan/edk2-staging.git)
(branch edk2-stuart-ci-latest) and [Dev Ops
Pipeline](https://dev.azure.com/tianocore/edk2-ci-
play/_build?definitionId=12) have been set up.
Furthermore, this proposal will leverage the TianoCore
python tools PIP modules:
[library](https://pypi.org/project/edk2-pytool-
library/) and
[extensions](https://pypi.org/project/edk2-pytool-
extensions/) (with repos located
[here](https://github.com/tianocore/edk2-pytool-
library) and [here](https://github.com/tianocore/edk2-
pytool-extensions)).
The primary execution flows can be found in the `azure-
pipelines-pr-gate.yml` and `azure-pipelines-pr-gate-
linux.yml` files. These YAML files are consumed by the
Azure Dev Ops Build Pipeline and dictate what server
resources should be used, how they should be
configured, and what processes should be run on them.
An overview of this schema can be found
[here](https://docs.microsoft.com/en-
us/azure/devops/pipelines/yaml-schema?view=azure-
devops&tabs=schema).
Inspection of these files reveals the EDKII Tools
commands that make up the primary processes for the CI
build: 'stuart_setup', 'stuart_update', and
'stuart_ci_build'. These commands come from the EDKII
Tools PIP modules and are configured as described
below. More documentation on the stuart tools can be
found [here](https://github.com/tianocore/edk2-pytool-
extensions/blob/master/docs/using.md) and
[here](https://github.com/tianocore/edk2-pytool-
extensions/blob/master/docs/features/feature_invocables
.md).
## Configuration
Configuration of the CI process consists of (in order
of precedence):
* command-line arguments passed in via the Pipeline
YAML
* a per-package configuration file (e.g. `<package-
name>.mu.yaml`) that is detected by the CI system in
EDKII Tools.
* a global configuration Python module (e.g.
`CISetting.py`) passed in via the command-line
The global configuration file is described in [this
readme](https://github.com/tianocore/edk2-pytool-
extensions/blob/master/docs/usability/using_settings_ma
nager.md) from the EDKII Tools documentation. This
configuration is written as a Python module so that
decisions can be made dynamically based on command line
parameters and codebase state.
The per-package configuration file can override most
settings in the global configuration file, but is not
dynamic. This file can be used to skip or customize
tests that may be incompatible with a specific package.
By default, the global configuration will try to run
all tests on all packages.
## CI Test Types
All CI tests are instances of EDKII Tools plugins.
Documentation on the plugin system can be found
[here](https://github.com/tianocore/edk2-pytool-
extensions/blob/master/docs/usability/using_plugin_mana
ger.md) and [here](https://github.com/tianocore/edk2-
pytool-
extensions/blob/master/docs/features/feature_plugin_man
ager.md). Upon invocation, each plugin will be passed
the path to the current package under test and a
dictionary containing its targeted configuration, as
assembled from the command line, per-package
configuration, and global configuration.
Note: CI plugins are considered unique from build
plugins and helper plugins, even though some CI plugins
may execute steps of a build.
In the example, these plugins live alongside the code
under test (in the `BaseTools` directory), but may be
moved to the 'edk2-test' repo if that location makes
more sense for the community.
### Module Inclusion Test - DscCompleteCheck
This test scans all available modules (via INF files)
and compares them to the package-level DSC file for the
package each module is contained within. The test
considers it an error if any module does not appear in
the `Components` section of at least one package-level
DSC (indicating that it would not be built if the
package were built).
### Code Compilation Test - CompilerPlugin
Once the Module Inclusion Test has verified that all
modules would be built if all package-level DSCs were
built, the Code Compilation Test simply runs through
and builds every package-level DSC on every toolchain
and for every architecture that is supported. Any
module that fails to build is considered an error.
### Host-Based UnitTests - HostUnitTestCompilerPlugin
and HostUnitTestDscCompleteCheck
The [Testing RFC doc](Readme-Testing-RFC.md) has much
more detail on this, but the basic idea is that host-
based unit tests can be compiled against individual
modules and libraries and run on the build agent (in
this case, the Dev Ops build server). The successful
and failing test case results are collected and
included in the final build report.
### GUID Uniqueness Test - GuidCheck
This test works on the collection of all packages
rather than an individual package. It looks at all
FILE_GUIDs and GUIDs declared in DEC files and ensures
that they are unique for the codebase. This prevents,
for example, accidental duplication of GUIDs when using
an existing INF as a template for a new module.
### Cross-Package Dependency Test - DependencyCheck
This test compares the list of all packages used in
INFs files for a given package against a list of
"allowed dependencies" in plugin configuration for that
package. Any module that depends on a disallowed
package will cause a test failure.
### Library Declaration Test - LibraryClassCheck
This test looks at all library header files found in a
package's `Include/Library` directory and ensures that
all files have a matching LibraryClass declaration in
the DEC file for the package. Any missing declarations
will cause a failure.
### Invalid Character Test - CharEncodingCheck
This test scans all files in a package to make sure
that there are no invalid Unicode characters that may
cause build errors in some character
sets/localizations.
## Next Steps
* Receive community feedback on RFC.
* Determine where this phase makes sense given existing
RFCs from other TianoCore contributors.
* Optimize testing beharior.
* Only run a subset of tests on PRs or individual
commits.
* Run full testing either once per day or once every
several commits.
* Add more tests/capabilities.
* Continue to improve results formatting.
* Continue to improve CI documentation.
* Much of this documentation effort is pending
community feedback on which parts are needed and what
phases are priorities.
Thanks
-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
Of Michael D Kinney via Groups.Io
Sent: Thursday, August 29, 2019 1:23 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Subject: [edk2-rfc] [RFC] EDK II Continuous Integration
Phase 1
Hello,
This is a proposal for a first step towards continuous
integration for all TianoCore repositories to help
improve to quality of commits and automate testing and
release processes for all EDK II packages and
platforms.
This is based on work from a number of EDK II community
members that have provide valuable input and
evaluations.
* Rebecca Cran <mailto:rebecca@...> Jenkins
evaluation
* Laszlo Ersek <mailto:lersek@...> GitLab
evaluation
* Philippe Mathieu-Daudé <mailto:philmd@...>
GitLab evaluation
* Sean Brogan <mailto:sean.brogan@...> Azure
Pipelines and HBFA
* Bret Barkelew <mailto:Bret.Barkelew@...>
Azure Pipelines and HBFA
* Jiewen Yao <mailto:jiewen.yao@...> HBFA
The following link is a link to an EDK II WIKI page
that contains a summary of the work to date. Please
provide feedback in the EDK II mailing lists. The WIKI
pages will be updated with input from the entire EDK II
community.
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Continuous-
Integration&data=02%7C01%7Csean.brogan%40microsoft.
com%7C6f67f169a6c746b4288608d72cbea7b6%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637027069686644659&sda
ta=GR9wN6gP3mJlCTopAaQ2rlzhby1nuF%2BwDVsfFIQAZjA%3D&
;reserved=0
Proposal
========
Phase 1 of adding continuous integration is limited to
the
edk2 repository. Additional repositories will be added
later.
The following changes are proposed:
* Remove EDK II Maintainers write access to edk2
repository.
Only EDK II Administrators will continue to have
write
access, and that should only be used to handle
extraordinary
events.
* EDK II Maintainers use a GitHub Pull Request instead
of push
to commit a patch series to the edk2 repository.
There are
no other changes to the development and review
process. The
patch series is prepared in an EDK II maintainer
branch with
all commit message requirements met on each patch in
the series.
* The edk2 repository only accepts Pull Requests from
members
of the EDK II Maintainers team. Pull Requests from
anyone else
are rejected.
* Run pre-commit checks using Azure Pipelines
* If all pre-commit checks pass, then the patch series
is auto
committed. The result of this commit must match the
contents
and commit history that would have occurred using the
previous
push operation.
* If any pre-commit checks fail, then notify the
submitter.
A typical reason for a failure would be a merge
conflict with
another pull request that was just processed.
* Limit pre-commit checks execution time to 10 minutes.
* Provide on-demand builds to EDK II Maintainers that
to allow
EDK II Maintainers to submit a branch through for the
same
set of pre-commit checks without submitting a pull
request.
## Pre-Commit Checks in Phase 1
* Run and pass PatchCheck.py with no errors
=====================================================
The following are some additional pre-commit check
ideas that could be quickly added once the initial
version using PatchCheck.py is fully functional.
Please provide feedback on the ones you like and
additional ones you think may improve the quality of
the commits to the edk2 repository.
## Proposed Pre-Commit Checks in Phase 2
* Verify Reviewed-by and Acked-by tags are present with
correct maintainer email addresses
* Verify no non-ASCII characters in modified files
* Verify no binary files in set of modified files
* Verify package dependency rules in modified files
## Proposed Pre-Commit Checks in Phase 3
* Run ECC on modified files
* Verify modified modules/libs build
* Run available host based tests (HBFA) against
modified
modules/libs
Best regards,
Mike
Sean
Currently it is building on Windows with VS2019. VS2017 would be trivial but not worth it in my perspective given how aligned the two are. If you really wanted to do a weekly or nightly build it could be added to that but I have been focused on a PR build.
I have a pipeline for GCC on linux. It doesn't' support host based unit tests but I am working to get the rest of the tests running.
I do not have a pipeline for Mac or LLVM/Clang. That would be next and I think your files below should help.
To date we use PYTOOLs features to install our extra tools. This makes it super easy and works for both local and server based builds. It gives strong versioning and management of that. It also tracks those versions and you can see them on every build in the Built Tool Report artifact. It can be downloaded here for html version. https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=803&view=artifacts. BUILD_TOOL_REPORT.html
Here it is pasted as plain text below. You can see the iasl and nasm versions here.
Key Value Type
TOOL_CHAIN_TAG VS2019 TOOL
VC Version 14.22.27905 TOOL
d:\a\1\s\Conf\build_rule.txt 1.04 INFO
d:\a\1\s\Conf\target.txt 1.03 INFO
d:\a\1\s\Conf\tools_def.txt 1.22 INFO
iasl 20190215.0.0 INFO
Mu-Basetools 2019.03.1 INFO
mu_nasm 2.14.02 INFO
Hope that helps.
I would be happy to queue up this topic for next design meeting and open it up to questions? I hope others would take a look prior to meeting.
Thanks
Sean
toggle quoted message
Show quoted text
I have a pipeline for GCC on linux. It doesn't' support host based unit tests but I am working to get the rest of the tests running.
I do not have a pipeline for Mac or LLVM/Clang. That would be next and I think your files below should help.
To date we use PYTOOLs features to install our extra tools. This makes it super easy and works for both local and server based builds. It gives strong versioning and management of that. It also tracks those versions and you can see them on every build in the Built Tool Report artifact. It can be downloaded here for html version. https://dev.azure.com/tianocore/edk2-ci-play/_build/results?buildId=803&view=artifacts. BUILD_TOOL_REPORT.html
Here it is pasted as plain text below. You can see the iasl and nasm versions here.
Key Value Type
TOOL_CHAIN_TAG VS2019 TOOL
VC Version 14.22.27905 TOOL
d:\a\1\s\Conf\build_rule.txt 1.04 INFO
d:\a\1\s\Conf\target.txt 1.03 INFO
d:\a\1\s\Conf\tools_def.txt 1.22 INFO
iasl 20190215.0.0 INFO
Mu-Basetools 2019.03.1 INFO
mu_nasm 2.14.02 INFO
Hope that helps.
I would be happy to queue up this topic for next design meeting and open it up to questions? I hope others would take a look prior to meeting.
Thanks
Sean
-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Thursday, September 19, 2019 2:56 PM
To: devel@edk2.groups.io; Sean Brogan <sean.brogan@...>; rfc@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1
Hi Sean,
Which OS/Compiler configurations are currently enabled for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler configurations in Azure Pipelines. There are some tools that need to be installed for each of these environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools installed? Is it in the YML files or in the Python scripts.
One critical task is to identify the tools and their specific versions that the CI system is configured to use.
These configurations should be documented in a Wiki page and updated as new tools are released and adopted by EDK II.
The inventory of tools used to validate a release should Also be documented in a release notes for a stable tag.
Here are the YML files that install the additional tools required to support EDK II builds. I need the source and versions of these tools to be reviewed and approved.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FWindowsPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=hM4AKoMutISI5Oc%2FeVMevx%2FVRUCjJHuhEwqom0R30Ak%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FUbuntuPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=KQ2zas2bJ%2FCjSRWGyMrxq5Rk4cW5lOgXQNR99QJbEKY%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FMacOsPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OzAmzYkRJ3rQOHEDgKTREz%2BNp7acOWUACh1s%2Fb4UPGk%3D&reserved=0
Thanks,
Mike
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Thursday, September 19, 2019 2:56 PM
To: devel@edk2.groups.io; Sean Brogan <sean.brogan@...>; rfc@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1
Hi Sean,
Which OS/Compiler configurations are currently enabled for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler configurations in Azure Pipelines. There are some tools that need to be installed for each of these environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools installed? Is it in the YML files or in the Python scripts.
One critical task is to identify the tools and their specific versions that the CI system is configured to use.
These configurations should be documented in a Wiki page and updated as new tools are released and adopted by EDK II.
The inventory of tools used to validate a release should Also be documented in a release notes for a stable tag.
Here are the YML files that install the additional tools required to support EDK II builds. I need the source and versions of these tools to be reviewed and approved.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FWindowsPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=hM4AKoMutISI5Oc%2FeVMevx%2FVRUCjJHuhEwqom0R30Ak%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FUbuntuPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=KQ2zas2bJ%2FCjSRWGyMrxq5Rk4cW5lOgXQNR99QJbEKY%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-ci%2Fblob%2Fmaster%2FAzurePipelines%2FMacOsPrerequisites.yml&;data=02%7C01%7Csean.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OzAmzYkRJ3rQOHEDgKTREz%2BNp7acOWUACh1s%2Fb4UPGk%3D&reserved=0
Thanks,
Mike
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
via Groups.Io
Sent: Thursday, August 29, 2019 7:22 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; devel@edk2.groups.io
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: Re: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1
Mike, as you mentioned we have been working towards enabling a
practical and extensible CI for Edk2 using Azure dev ops and the
recently added edk2-pytool infrastructure. We have been using similar
CI for Project Mu for the last few years.
Our approach is a little different in that we focus on validating the
whole code base rather than just the incoming patch. We do this
because we have found unexpected consequences of patches and overall
we want all code to be compliant not just new additions. We have
found the time to test the whole tree is not much longer than only the
parts impacted by a code change (except maybe when running the entire
compile test on every package). This obviously comes with an initial
tax of needing to get the codebase into compliant form.
Anyway we have prepared an RFC in addition to yours and would like to
see these two efforts merged together.
We are still working on making a few optimizations.
Currently if the full set of tests are run we take about 20 minutes.
This is because compiling MdeModulePkg for debug, release, and host
based tests take a while. Most other packages are in the 10 minute
range. We do have easy ways to disable or limit certain tests as well
as expand the matrix to leverage more cloud resources (more parallel
builds).
Content is best viewed online with links to helpful content but is
also attached below:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
ub.com%2Fspbrogan%2Fedk2-staging%2Fblob%2Fedk2-&data=02%7C01%7Csea
n.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=xOiPHH
VbMhFBsIpj%2F9mhOhrkhdsF2Gbehd6%2Frk0XYgM%3D&reserved=0
stuart-ci-latest/Readme-CI-RFC.md
# CI and PR Gates
## Background
Historically, while the TianoCore maintainers and stewards have done a
fantastic job of keeping contribution policies consistent and
contributions clean and well-documented, there have been few processes
that ran to verify the sanity, cleanliness, and efficacy of the
codebase, and even fewer that publicly published their results for the
community at large. This has caused inconsistancies and issues within
the codebase from time to time.
Adding continuous integration (and potentially PR
gates) to the checkin process ensures that simple errors like these
are caught and can be fixed on a regular basis.
## Strategy
While a number of CI solutions exist, this proposal will focus on the
usage of Azure Dev Ops and Build Pipelines. For demonstration, a
sample [TianoCore
repo](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
2Fgithub.com%2Fspbrogan%2Fedk2-staging.git&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=QmzQhemLUNB
7FGJwoGeeewExThrUjUArxnkmtro1b8A%3D&reserved=0)
(branch edk2-stuart-ci-latest) and [Dev Ops
Pipeline](https://nam06.safelinks.protection.outlook.com/?url=https%3A
%2F%2Fdev.azure.com%2Ftianocore%2Fedk2-ci-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=rS%2FrjIFkS
uEhI%2FDWyslf34i711%2FbY8Gw%2Byexq%2FwydjU%3D&reserved=0
play/_build?definitionId=12) have been set up.
Furthermore, this proposal will leverage the TianoCore python tools
PIP modules:
[library](https://nam06.safelinks.protection.outlook.com/?url=https%3A
%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.brogan
%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=jzM6MKZFFAEvyR
4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0
library/) and
[extensions](https://nam06.safelinks.protection.outlook.com/?url=https
%3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=jzM6MKZFFAE
vyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0
extensions/) (with repos located
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OwT%2BiNTN9Rv
r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0
library) and
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452466989&sdata=tInVwGwCXXGBgcmHQ%2B
0UwRaEbtZWRy8hTp5wpRtN0R0%3D&reserved=0
pytool-extensions)).
The primary execution flows can be found in the `azure-
pipelines-pr-gate.yml` and `azure-pipelines-pr-gate- linux.yml` files.
These YAML files are consumed by the Azure Dev Ops Build Pipeline and
dictate what server resources should be used, how they should be
configured, and what processes should be run on them.
An overview of this schema can be found
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fdocs.microsoft.com%2Fen-&data=02%7C01%7Csean.brogan%40microsoft
.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d7cd011
db47%7C1%7C0%7C637045269452466989&sdata=j8nyYG3VnizwKFZ8l1BWoLcPy%
2BuaSoVT0jN2Esi7wH8%3D&reserved=0
us/azure/devops/pipelines/yaml-schema?view=azure-
devops&tabs=schema).
Inspection of these files reveals the EDKII Tools commands that make
up the primary processes for the CI
build: 'stuart_setup', 'stuart_update', and 'stuart_ci_build'. These
commands come from the EDKII Tools PIP modules and are configured as
described below. More documentation on the stuart tools can be found
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OwT%2BiNTN9Rv
r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0
extensions/blob/master/docs/using.md) and
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OwT%2BiNTN9Rv
r14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0
extensions/blob/master/docs/features/feature_invocables
.md).
## Configuration
Configuration of the CI process consists of (in order of precedence):
* command-line arguments passed in via the Pipeline YAML
* a per-package configuration file (e.g. `<package-
name>.mu.yaml`) that is detected by the CI system in
EDKII Tools.
* a global configuration Python module (e.g.
`CISetting.py`) passed in via the command-line
The global configuration file is described in [this
readme](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.brog
an%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OwT%2BiNTN9R
vr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0
extensions/blob/master/docs/usability/using_settings_ma
nager.md) from the EDKII Tools documentation. This configuration is
written as a Python module so that decisions can be made dynamically
based on command line parameters and codebase state.
The per-package configuration file can override most settings in the
global configuration file, but is not dynamic. This file can be used
to skip or customize tests that may be incompatible with a specific
package.
By default, the global configuration will try to run all tests on all
packages.
## CI Test Types
All CI tests are instances of EDKII Tools plugins.
Documentation on the plugin system can be found
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452476994&sdata=1AM4geY%2BEFh
ekun7oQOYyVVt%2Bwn6CAF2hhtkS0sgwPU%3D&reserved=0
extensions/blob/master/docs/usability/using_plugin_mana
ger.md) and
[here](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452476994&sdata=3vyPMvrfG3V4doNM0%2B
cKTpDSHEEEvkfxKAtw3eiJWsY%3D&reserved=0
pytool-
extensions/blob/master/docs/features/feature_plugin_man
ager.md). Upon invocation, each plugin will be passed the path to the
current package under test and a dictionary containing its targeted
configuration, as assembled from the command line, per-package
configuration, and global configuration.
Note: CI plugins are considered unique from build plugins and helper
plugins, even though some CI plugins may execute steps of a build.
In the example, these plugins live alongside the code under test (in
the `BaseTools` directory), but may be moved to the 'edk2-test' repo
if that location makes more sense for the community.
### Module Inclusion Test - DscCompleteCheck
This test scans all available modules (via INF files) and compares
them to the package-level DSC file for the package each module is
contained within. The test considers it an error if any module does
not appear in the `Components` section of at least one package-level
DSC (indicating that it would not be built if the package were built).
### Code Compilation Test - CompilerPlugin
Once the Module Inclusion Test has verified that all modules would be
built if all package-level DSCs were built, the Code Compilation Test
simply runs through and builds every package-level DSC on every
toolchain and for every architecture that is supported. Any module
that fails to build is considered an error.
### Host-Based UnitTests - HostUnitTestCompilerPlugin and
HostUnitTestDscCompleteCheck
The [Testing RFC doc](Readme-Testing-RFC.md) has much more detail on
this, but the basic idea is that host- based unit tests can be
compiled against individual modules and libraries and run on the build
agent (in this case, the Dev Ops build server). The successful and
failing test case results are collected and included in the final
build report.
### GUID Uniqueness Test - GuidCheck
This test works on the collection of all packages rather than an
individual package. It looks at all FILE_GUIDs and GUIDs declared in
DEC files and ensures that they are unique for the codebase. This
prevents, for example, accidental duplication of GUIDs when using an
existing INF as a template for a new module.
### Cross-Package Dependency Test - DependencyCheck
This test compares the list of all packages used in INFs files for a
given package against a list of "allowed dependencies" in plugin
configuration for that package. Any module that depends on a
disallowed package will cause a test failure.
### Library Declaration Test - LibraryClassCheck
This test looks at all library header files found in a package's
`Include/Library` directory and ensures that all files have a matching
LibraryClass declaration in the DEC file for the package. Any missing
declarations will cause a failure.
### Invalid Character Test - CharEncodingCheck
This test scans all files in a package to make sure that there are no
invalid Unicode characters that may cause build errors in some
character sets/localizations.
## Next Steps
* Receive community feedback on RFC.
* Determine where this phase makes sense given existing RFCs from
other TianoCore contributors.
* Optimize testing beharior.
* Only run a subset of tests on PRs or individual commits.
* Run full testing either once per day or once every several
commits.
* Add more tests/capabilities.
* Continue to improve results formatting.
* Continue to improve CI documentation.
* Much of this documentation effort is pending community feedback on
which parts are needed and what phases are priorities.
Thanks
-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Michael D
Kinney via Groups.Io
Sent: Thursday, August 29, 2019 1:23 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Subject: [edk2-rfc] [RFC] EDK II Continuous Integration Phase 1
Hello,
This is a proposal for a first step towards continuous integration for
all TianoCore repositories to help improve to quality of commits and
automate testing and release processes for all EDK II packages and
platforms.
This is based on work from a number of EDK II community members that
have provide valuable input and evaluations.
* Rebecca Cran <mailto:rebecca@...> Jenkins evaluation
* Laszlo Ersek <mailto:lersek@...> GitLab evaluation
* Philippe Mathieu-Daudé <mailto:philmd@...> GitLab evaluation
* Sean Brogan <mailto:sean.brogan@...> Azure Pipelines and
HBFA
* Bret Barkelew <mailto:Bret.Barkelew@...>
Azure Pipelines and HBFA
* Jiewen Yao <mailto:jiewen.yao@...> HBFA
The following link is a link to an EDK II WIKI page that contains a
summary of the work to date. Please provide feedback in the EDK II
mailing lists. The WIKI pages will be updated with input from the
entire EDK II community.
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Continuous-
Integration&data=02%7C01%7Csean.brogan%40microsoft.
com%7C6f67f169a6c746b4288608d72cbea7b6%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637027069686644659&sda
ta=GR9wN6gP3mJlCTopAaQ2rlzhby1nuF%2BwDVsfFIQAZjA%3D&
;reserved=0
Proposal
========
Phase 1 of adding continuous integration is limited to the
edk2 repository. Additional repositories will be added later.
The following changes are proposed:
* Remove EDK II Maintainers write access to edk2 repository.
Only EDK II Administrators will continue to have write
access, and that should only be used to handle extraordinary
events.
* EDK II Maintainers use a GitHub Pull Request instead of push
to commit a patch series to the edk2 repository.
There are
no other changes to the development and review process. The
patch series is prepared in an EDK II maintainer branch with
all commit message requirements met on each patch in the series.
* The edk2 repository only accepts Pull Requests from members
of the EDK II Maintainers team. Pull Requests from anyone else
are rejected.
* Run pre-commit checks using Azure Pipelines
* If all pre-commit checks pass, then the patch series is auto
committed. The result of this commit must match the contents
and commit history that would have occurred using the previous
push operation.
* If any pre-commit checks fail, then notify the submitter.
A typical reason for a failure would be a merge conflict with
another pull request that was just processed.
* Limit pre-commit checks execution time to 10 minutes.
* Provide on-demand builds to EDK II Maintainers that to allow
EDK II Maintainers to submit a branch through for the same
set of pre-commit checks without submitting a pull request.
## Pre-Commit Checks in Phase 1
* Run and pass PatchCheck.py with no errors
=====================================================
The following are some additional pre-commit check
ideas that could be quickly added once the initial
version using PatchCheck.py is fully functional.
Please provide feedback on the ones you like and
additional ones you think may improve the quality of
the commits to the edk2 repository.
## Proposed Pre-Commit Checks in Phase 2
* Verify Reviewed-by and Acked-by tags are present with
correct maintainer email addresses
* Verify no non-ASCII characters in modified files
* Verify no binary files in set of modified files
* Verify package dependency rules in modified files
## Proposed Pre-Commit Checks in Phase 3
* Run ECC on modified files
* Verify modified modules/libs build
* Run available host based tests (HBFA) against
modified
modules/libs
Best regards,
Mike
Michael D Kinney
Hi Sean,
For host based tests, I agree that VS2017 or VS2019
would be a good choice. Pick the one with the best
coverage and easiest for developers to get feedback on
the test results and test coverage. That may be sufficient
for automated CI tests. Enabling other tool chains for
host based testing will be required to support developers
that implement unit tests and want to test them locally
before adding to the automated CI tests.
For build tests, I think we need VS2015, VS2017, GCC,
and XCODE5. We can add VS2019 if there is a request to make
that one of the fully validated tool chains for EDK II.
For pre-commit gates, we could choose to do build tests of
only the libs/modules touched by the patch series. Then
perform complete package/platform build tests in a
Daily/Weekly/Release scope.
We need to update the RFC with the specific tool chains and
tool versions for both Host Based tests and Code Compilation
Tests. They are not the same.
Thanks,
Mike
toggle quoted message
Show quoted text
For host based tests, I agree that VS2017 or VS2019
would be a good choice. Pick the one with the best
coverage and easiest for developers to get feedback on
the test results and test coverage. That may be sufficient
for automated CI tests. Enabling other tool chains for
host based testing will be required to support developers
that implement unit tests and want to test them locally
before adding to the automated CI tests.
For build tests, I think we need VS2015, VS2017, GCC,
and XCODE5. We can add VS2019 if there is a request to make
that one of the fully validated tool chains for EDK II.
For pre-commit gates, we could choose to do build tests of
only the libs/modules touched by the patch series. Then
perform complete package/platform build tests in a
Daily/Weekly/Release scope.
We need to update the RFC with the specific tool chains and
tool versions for both Host Based tests and Code Compilation
Tests. They are not the same.
Thanks,
Mike
-----Original Message-----
From: Sean Brogan <sean.brogan@...>
Sent: Friday, September 20, 2019 2:30 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous
Integration Phase 1
Currently it is building on Windows with VS2019.
VS2017 would be trivial but not worth it in my
perspective given how aligned the two are. If you
really wanted to do a weekly or nightly build it could
be added to that but I have been focused on a PR build.
I have a pipeline for GCC on linux. It doesn't'
support host based unit tests but I am working to get
the rest of the tests running.
I do not have a pipeline for Mac or LLVM/Clang. That
would be next and I think your files below should help.
To date we use PYTOOLs features to install our extra
tools. This makes it super easy and works for both
local and server based builds. It gives strong
versioning and management of that. It also tracks
those versions and you can see them on every build in
the Built Tool Report artifact. It can be downloaded
here for html version.
https://dev.azure.com/tianocore/edk2-ci-
play/_build/results?buildId=803&view=artifacts.
BUILD_TOOL_REPORT.html
Here it is pasted as plain text below. You can see the
iasl and nasm versions here.
Key Value Type
TOOL_CHAIN_TAG VS2019 TOOL
VC Version 14.22.27905 TOOL
d:\a\1\s\Conf\build_rule.txt 1.04 INFO
d:\a\1\s\Conf\target.txt 1.03 INFO
d:\a\1\s\Conf\tools_def.txt 1.22 INFO
iasl 20190215.0.0 INFO
Mu-Basetools 2019.03.1 INFO
mu_nasm 2.14.02 INFO
Hope that helps.
I would be happy to queue up this topic for next design
meeting and open it up to questions? I hope others
would take a look prior to meeting.
Thanks
Sean
-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Thursday, September 19, 2019 2:56 PM
To: devel@edk2.groups.io; Sean Brogan
<sean.brogan@...>; rfc@edk2.groups.io;
Kinney, Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous
Integration Phase 1
Hi Sean,
Which OS/Compiler configurations are currently enabled
for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler
configurations in Azure Pipelines. There are some
tools that need to be installed for each of these
environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools
installed? Is it in the YML files or in the Python
scripts.
One critical task is to identify the tools and their
specific versions that the CI system is configured to
use.
These configurations should be documented in a Wiki
page and updated as new tools are released and adopted
by EDK II.
The inventory of tools used to validate a release
should Also be documented in a release notes for a
stable tag.
Here are the YML files that install the additional
tools required to support EDK II builds. I need the
source and versions of these tools to be reviewed and
approved.
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FWindowsPrerequisi
tes.yml&data=02%7C01%7Csean.brogan%40microsoft.com%
7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=h
M4AKoMutISI5Oc%2FeVMevx%2FVRUCjJHuhEwqom0R30Ak%3D&r
eserved=0
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FUbuntuPrerequisit
es.yml&data=02%7C01%7Csean.brogan%40microsoft.com%7
C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=KQ
2zas2bJ%2FCjSRWGyMrxq5Rk4cW5lOgXQNR99QJbEKY%3D&rese
rved=0
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FMacOsPrerequisite
s.yml&data=02%7C01%7Csean.brogan%40microsoft.com%7C
9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OzA
mzYkRJ3rQOHEDgKTREz%2BNp7acOWUACh1s%2Fb4UPGk%3D&res
erved=0
Thanks,
Mike-----Original Message-----Behalf Of Sean
From: devel@edk2.groups.io <devel@edk2.groups.io> Onvia Groups.IoIntegration Phase 1
Sent: Thursday, August 29, 2019 7:22 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; devel@edk2.groups.io
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: Re: [edk2-devel] [RFC] EDK II Continuousenabling a
Mike, as you mentioned we have been working towardspractical and extensible CI for Edk2 using Azure devops and therecently added edk2-pytool infrastructure. We havebeen using similarCI for Project Mu for the last few years.on validating the
Our approach is a little different in that we focuswhole code base rather than just the incoming patch.We do thisbecause we have found unexpected consequences ofpatches and overallwe want all code to be compliant not just newadditions. We havefound the time to test the whole tree is not muchlonger than only theparts impacted by a code change (except maybe whenrunning the entirecompile test on every package). This obviously comeswith an initialtax of needing to get the codebase into compliantform.Anyway we have prepared an RFC in addition to yoursand would like tosee these two efforts merged together.about 20 minutes.
We are still working on making a few optimizations.
Currently if the full set of tests are run we takeThis is because compiling MdeModulePkg for debug,release, and hostbased tests take a while. Most other packages are inthe 10 minuterange. We do have easy ways to disable or limitcertain tests as wellas expand the matrix to leverage more cloud resources(more parallelbuilds).content but is
Content is best viewed online with links to helpfulalso attached below:https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fspbrogan%2Fedk2-staging%2Fblob%2Fedk2-&data=02%7C01%7Csea
n.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c
1e1b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&a
mp;sdata=xOiPHH
VbMhFBsIpj%2F9mhOhrkhdsF2Gbehd6%2Frk0XYgM%3D&reserv
ed=0stuart-ci-latest/Readme-CI-RFC.mdstewards have done a
# CI and PR Gates
## Background
Historically, while the TianoCore maintainers andfantastic job of keeping contribution policiesconsistent andcontributions clean and well-documented, there havebeen few processesthat ran to verify the sanity, cleanliness, andefficacy of thecodebase, and even fewer that publicly publishedtheir results for thecommunity at large. This has caused inconsistanciesand issues withinthe codebase from time to time.errors like these
Adding continuous integration (and potentially PR
gates) to the checkin process ensures that simpleare caught and can be fixed on a regular basis.will focus on the
## Strategy
While a number of CI solutions exist, this proposalusage of Azure Dev Ops and Build Pipelines. Fordemonstration, asample [TianoCorerepo](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Fspbrogan%2Fedk2-staging.git&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=QmzQhemLUNB7FGJwoGeeewExThrUjUArxnkmtro1b8A%3D&reserved=0)Pipeline](https://nam06.safelinks.protection.outlook.co
(branch edk2-stuart-ci-latest) and [Dev Ops
m/?url=https%3A%2F%2Fdev.azure.com%2Ftianocore%2Fedk2-ci-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=rS%2FrjIFkS
uEhI%2FDWyslf34i711%2FbY8Gw%2Byexq%2FwydjU%3D&reser
ved=0play/_build?definitionId=12) have been set up.TianoCore python tools
Furthermore, this proposal will leverage thePIP modules:[library](https://nam06.safelinks.protection.outlook.co
m/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.brogan
%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C7
2f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata
=jzM6MKZFFAEvyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0[extensions](https://nam06.safelinks.protection.outlook
library/) and
.com/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=jzM6MKZFFAEvyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
extensions/) (with repos located
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
library) and
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
f86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452466989&sdata=tInVw
GwCXXGBgcmHQ%2B0UwRaEbtZWRy8hTp5wpRtN0R0%3D&reserved=0`azure-
pytool-extensions)).
The primary execution flows can be found in thepipelines-pr-gate.yml` and `azure-pipelines-pr-gate-linux.yml` files.These YAML files are consumed by the Azure Dev OpsBuild Pipeline anddictate what server resources should be used, howthey should beconfigured, and what processes should be run on them.[here](https://nam06.safelinks.protection.outlook.com/?
An overview of this schema can be found
url=https%3A%2F%2Fdocs.microsoft.com%2Fen-&data=02%7C01%7Csean.brogan%40microsoft
.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
1af91ab2d7cd011
db47%7C1%7C0%7C637045269452466989&sdata=j8nyYG3Vniz
wKFZ8l1BWoLcPy%2BuaSoVT0jN2Esi7wH8%3D&reserved=0commands that make
us/azure/devops/pipelines/yaml-schema?view=azure-
devops&tabs=schema).
Inspection of these files reveals the EDKII Toolsup the primary processes for the CI'stuart_ci_build'. These
build: 'stuart_setup', 'stuart_update', andcommands come from the EDKII Tools PIP modules andare configured asdescribed below. More documentation on the stuarttools can be found
[here](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
extensions/blob/master/docs/using.md) and
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0extensions/blob/master/docs/features/feature_invocables.md).of precedence):
## Configuration
Configuration of the CI process consists of (in order* command-line arguments passed in via the PipelineYAML* a per-package configuration file (e.g. `<package-readme](https://nam06.safelinks.protection.outlook.com/
name>.mu.yaml`) that is detected by the CI system in
EDKII Tools.
* a global configuration Python module (e.g.
`CISetting.py`) passed in via the command-line
The global configuration file is described in [this
?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.brog
an%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7
C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sda
ta=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0extensions/blob/master/docs/usability/using_settings_manager.md) from the EDKII Tools documentation. Thisconfiguration iswritten as a Python module so that decisions can bemade dynamicallybased on command line parameters and codebase state.settings in the
The per-package configuration file can override mostglobal configuration file, but is not dynamic. Thisfile can be usedto skip or customize tests that may be incompatiblewith a specificpackage.all tests on all
By default, the global configuration will try to runpackages.[here](https://nam06.safelinks.protection.outlook.com/?
## CI Test Types
All CI tests are instances of EDKII Tools plugins.
Documentation on the plugin system can be found
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452476994&sdat
a=1AM4geY%2BEFhekun7oQOYyVVt%2Bwn6CAF2hhtkS0sgwPU%3D&reserved=0extensions/blob/master/docs/usability/using_plugin_manager.md) and[here](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
f86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452476994&sdata=3vyPM
vrfG3V4doNM0%2BcKTpDSHEEEvkfxKAtw3eiJWsY%3D&reserved=0extensions/blob/master/docs/features/feature_plugin_man
pytool-ager.md). Upon invocation, each plugin will be passedthe path to thecurrent package under test and a dictionarycontaining its targetedconfiguration, as assembled from the command line,per-packageconfiguration, and global configuration.plugins and helper
Note: CI plugins are considered unique from buildplugins, even though some CI plugins may executesteps of a build.under test (in
In the example, these plugins live alongside the codethe `BaseTools` directory), but may be moved to the'edk2-test' repoif that location makes more sense for the community.and compares
### Module Inclusion Test - DscCompleteCheck
This test scans all available modules (via INF files)them to the package-level DSC file for the packageeach module iscontained within. The test considers it an error ifany module doesnot appear in the `Components` section of at leastone package-levelDSC (indicating that it would not be built if thepackage were built).modules would be
### Code Compilation Test - CompilerPlugin
Once the Module Inclusion Test has verified that allbuilt if all package-level DSCs were built, the CodeCompilation Testsimply runs through and builds every package-levelDSC on everytoolchain and for every architecture that issupported. Any modulethat fails to build is considered an error.and
### Host-Based UnitTests - HostUnitTestCompilerPluginHostUnitTestDscCompleteCheckmore detail on
The [Testing RFC doc](Readme-Testing-RFC.md) has muchthis, but the basic idea is that host- based unittests can becompiled against individual modules and libraries andrun on the buildagent (in this case, the Dev Ops build server). Thesuccessful andfailing test case results are collected and includedin the finalbuild report.rather than an
### GUID Uniqueness Test - GuidCheck
This test works on the collection of all packagesindividual package. It looks at all FILE_GUIDs andGUIDs declared inDEC files and ensures that they are unique for thecodebase. Thisprevents, for example, accidental duplication ofGUIDs when using anexisting INF as a template for a new module.INFs files for a
### Cross-Package Dependency Test - DependencyCheck
This test compares the list of all packages used ingiven package against a list of "alloweddependencies" in pluginconfiguration for that package. Any module thatdepends on adisallowed package will cause a test failure.a package's
### Library Declaration Test - LibraryClassCheck
This test looks at all library header files found in`Include/Library` directory and ensures that allfiles have a matchingLibraryClass declaration in the DEC file for thepackage. Any missingdeclarations will cause a failure.that there are no
### Invalid Character Test - CharEncodingCheck
This test scans all files in a package to make sureinvalid Unicode characters that may cause builderrors in somecharacter sets/localizations.existing RFCs from
## Next Steps
* Receive community feedback on RFC.
* Determine where this phase makes sense givenother TianoCore contributors.commits.
* Optimize testing beharior.
* Only run a subset of tests on PRs or individual* Run full testing either once per day or onceevery severalcommits.community feedback on
* Add more tests/capabilities.
* Continue to improve results formatting.
* Continue to improve CI documentation.
* Much of this documentation effort is pendingwhich parts are needed and what phases arepriorities.Behalf Of Michael D
Thanks
-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> OnKinney via Groups.IoIntegration Phase 1
Sent: Thursday, August 29, 2019 1:23 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Subject: [edk2-rfc] [RFC] EDK II Continuouscontinuous integration for
Hello,
This is a proposal for a first step towardsall TianoCore repositories to help improve to qualityof commits andautomate testing and release processes for all EDK IIpackages andplatforms.community members that
This is based on work from a number of EDK IIhave provide valuable input and evaluations.evaluation
* Rebecca Cran <mailto:rebecca@...> Jenkins* Laszlo Ersek <mailto:lersek@...> GitLabevaluation* Philippe Mathieu-Daudé <mailto:philmd@...>GitLab evaluation* Sean Brogan <mailto:sean.brogan@...>Azure Pipelines andHBFAthat contains a
* Bret Barkelew <mailto:Bret.Barkelew@...>
Azure Pipelines and HBFA
* Jiewen Yao <mailto:jiewen.yao@...> HBFA
The following link is a link to an EDK II WIKI pagesummary of the work to date. Please provide feedbackin the EDK IImailing lists. The WIKI pages will be updated withinput from theentire EDK II community.https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Continuous-Integration&data=02%7C01%7Csean.brogan%40microsoft.
com%7C6f67f169a6c746b4288608d72cbea7b6%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637027069686644659&sda
ta=GR9wN6gP3mJlCTopAaQ2rlzhby1nuF%2BwDVsfFIQAZjA%3D&reserved=0to the
Proposal
========
Phase 1 of adding continuous integration is limitededk2 repository. Additional repositories will beadded later.repository.
The following changes are proposed:
* Remove EDK II Maintainers write access to edk2Only EDK II Administrators will continue to havewriteaccess, and that should only be used to handleextraordinaryevents.instead of push
* EDK II Maintainers use a GitHub Pull Requestto commit a patch series to the edk2 repository.process. The
There are
no other changes to the development and reviewpatch series is prepared in an EDK II maintainerbranch withall commit message requirements met on each patchin the series.* The edk2 repository only accepts Pull Requests frommembersof the EDK II Maintainers team. Pull Requests fromanyone elseare rejected.series is auto
* Run pre-commit checks using Azure Pipelines
* If all pre-commit checks pass, then the patchcommitted. The result of this commit must matchthe contentsand commit history that would have occurred usingthe previouspush operation.submitter.
* If any pre-commit checks fail, then notify theA typical reason for a failure would be a mergeconflict withanother pull request that was just processed.minutes.
* Limit pre-commit checks execution time to 10* Provide on-demand builds to EDK II Maintainers thatto allowEDK II Maintainers to submit a branch through forthe sameset of pre-commit checks without submitting a pullrequest.ideas that could be
## Pre-Commit Checks in Phase 1
* Run and pass PatchCheck.py with no errors
=====================================================
The following are some additional pre-commit checkquickly added once the initial version usingPatchCheck.py is fullyfunctional.additional ones you
Please provide feedback on the ones you like andthink may improve the quality of the commits to theedk2 repository.with
## Proposed Pre-Commit Checks in Phase 2
* Verify Reviewed-by and Acked-by tags are presentcorrect maintainer email addressesmodified
* Verify no non-ASCII characters in modified files
* Verify no binary files in set of modified files
* Verify package dependency rules in modified files
## Proposed Pre-Commit Checks in Phase 3
* Run ECC on modified files
* Verify modified modules/libs build
* Run available host based tests (HBFA) againstmodules/libs
Best regards,
Mike
Liming Gao
Mike:
toggle quoted message
Show quoted text
-----Original Message-----VS2019 tool chain has been added for IA32/X64/ARM/AARH64/EBC archs.
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D Kinney
Sent: Tuesday, September 24, 2019 1:44 AM
To: Sean Brogan <sean.brogan@...>; devel@edk2.groups.io; rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: Re: [edk2-devel] [RFC] EDK II Continuous Integration Phase 1
Hi Sean,
For host based tests, I agree that VS2017 or VS2019
would be a good choice. Pick the one with the best
coverage and easiest for developers to get feedback on
the test results and test coverage. That may be sufficient
for automated CI tests. Enabling other tool chains for
host based testing will be required to support developers
that implement unit tests and want to test them locally
before adding to the automated CI tests.
For build tests, I think we need VS2015, VS2017, GCC,
and XCODE5. We can add VS2019 if there is a request to make
that one of the fully validated tool chains for EDK II.
CLANG9 tool chain for IA32/X64 will be added. I request to add CLANG9 build test.
Thanks
Liming
For pre-commit gates, we could choose to do build tests of
only the libs/modules touched by the patch series. Then
perform complete package/platform build tests in a
Daily/Weekly/Release scope.
We need to update the RFC with the specific tool chains and
tool versions for both Host Based tests and Code Compilation
Tests. They are not the same.
Thanks,
Mike-----Original Message-----
From: Sean Brogan <sean.brogan@...>
Sent: Friday, September 20, 2019 2:30 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous
Integration Phase 1
Currently it is building on Windows with VS2019.
VS2017 would be trivial but not worth it in my
perspective given how aligned the two are. If you
really wanted to do a weekly or nightly build it could
be added to that but I have been focused on a PR build.
I have a pipeline for GCC on linux. It doesn't'
support host based unit tests but I am working to get
the rest of the tests running.
I do not have a pipeline for Mac or LLVM/Clang. That
would be next and I think your files below should help.
To date we use PYTOOLs features to install our extra
tools. This makes it super easy and works for both
local and server based builds. It gives strong
versioning and management of that. It also tracks
those versions and you can see them on every build in
the Built Tool Report artifact. It can be downloaded
here for html version.
https://dev.azure.com/tianocore/edk2-ci-
play/_build/results?buildId=803&view=artifacts.
BUILD_TOOL_REPORT.html
Here it is pasted as plain text below. You can see the
iasl and nasm versions here.
Key Value Type
TOOL_CHAIN_TAG VS2019 TOOL
VC Version 14.22.27905 TOOL
d:\a\1\s\Conf\build_rule.txt 1.04 INFO
d:\a\1\s\Conf\target.txt 1.03 INFO
d:\a\1\s\Conf\tools_def.txt 1.22 INFO
iasl 20190215.0.0 INFO
Mu-Basetools 2019.03.1 INFO
mu_nasm 2.14.02 INFO
Hope that helps.
I would be happy to queue up this topic for next design
meeting and open it up to questions? I hope others
would take a look prior to meeting.
Thanks
Sean
-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Thursday, September 19, 2019 2:56 PM
To: devel@edk2.groups.io; Sean Brogan
<sean.brogan@...>; rfc@edk2.groups.io;
Kinney, Michael D <michael.d.kinney@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: RE: [edk2-devel] [RFC] EDK II Continuous
Integration Phase 1
Hi Sean,
Which OS/Compiler configurations are currently enabled
for the Code Compilation Test?
I have been working on enabling multiple OS/Compiler
configurations in Azure Pipelines. There are some
tools that need to be installed for each of these
environments.
Examples include NASM, iASL, Python.
For the work you have done, how are these extra tools
installed? Is it in the YML files or in the Python
scripts.
One critical task is to identify the tools and their
specific versions that the CI system is configured to
use.
These configurations should be documented in a Wiki
page and updated as new tools are released and adopted
by EDK II.
The inventory of tools used to validate a release
should Also be documented in a release notes for a
stable tag.
Here are the YML files that install the additional
tools required to support EDK II builds. I need the
source and versions of these tools to be reviewed and
approved.
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FWindowsPrerequisi
tes.yml&data=02%7C01%7Csean.brogan%40microsoft.com%
7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=h
M4AKoMutISI5Oc%2FeVMevx%2FVRUCjJHuhEwqom0R30Ak%3D&r
eserved=0
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FUbuntuPrerequisit
es.yml&data=02%7C01%7Csean.brogan%40microsoft.com%7
C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=KQ
2zas2bJ%2FCjSRWGyMrxq5Rk4cW5lOgXQNR99QJbEKY%3D&rese
rved=0
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-
ci%2Fblob%2Fmaster%2FAzurePipelines%2FMacOsPrerequisite
s.yml&data=02%7C01%7Csean.brogan%40microsoft.com%7C
9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637045269452466989&sdata=OzA
mzYkRJ3rQOHEDgKTREz%2BNp7acOWUACh1s%2Fb4UPGk%3D&res
erved=0
Thanks,
Mike-----Original Message-----Behalf Of Sean
From: devel@edk2.groups.io <devel@edk2.groups.io> Onvia Groups.IoIntegration Phase 1
Sent: Thursday, August 29, 2019 7:22 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; devel@edk2.groups.io
Cc: Bret Barkelew <Bret.Barkelew@...>
Subject: Re: [edk2-devel] [RFC] EDK II Continuousenabling a
Mike, as you mentioned we have been working towardspractical and extensible CI for Edk2 using Azure devops and therecently added edk2-pytool infrastructure. We havebeen using similarCI for Project Mu for the last few years.on validating the
Our approach is a little different in that we focuswhole code base rather than just the incoming patch.We do thisbecause we have found unexpected consequences ofpatches and overallwe want all code to be compliant not just newadditions. We havefound the time to test the whole tree is not muchlonger than only theparts impacted by a code change (except maybe whenrunning the entirecompile test on every package). This obviously comeswith an initialtax of needing to get the codebase into compliantform.Anyway we have prepared an RFC in addition to yoursand would like tosee these two efforts merged together.about 20 minutes.
We are still working on making a few optimizations.
Currently if the full set of tests are run we takeThis is because compiling MdeModulePkg for debug,release, and hostbased tests take a while. Most other packages are inthe 10 minuterange. We do have easy ways to disable or limitcertain tests as wellas expand the matrix to leverage more cloud resources(more parallelbuilds).content but is
Content is best viewed online with links to helpfulalso attached below:https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fspbrogan%2Fedk2-staging%2Fblob%2Fedk2-&data=02%7C01%7Csea
n.brogan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c
1e1b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&a
mp;sdata=xOiPHH
VbMhFBsIpj%2F9mhOhrkhdsF2Gbehd6%2Frk0XYgM%3D&reserv
ed=0stuart-ci-latest/Readme-CI-RFC.mdstewards have done a
# CI and PR Gates
## Background
Historically, while the TianoCore maintainers andfantastic job of keeping contribution policiesconsistent andcontributions clean and well-documented, there havebeen few processesthat ran to verify the sanity, cleanliness, andefficacy of thecodebase, and even fewer that publicly publishedtheir results for thecommunity at large. This has caused inconsistanciesand issues withinthe codebase from time to time.errors like these
Adding continuous integration (and potentially PR
gates) to the checkin process ensures that simpleare caught and can be fixed on a regular basis.will focus on the
## Strategy
While a number of CI solutions exist, this proposalusage of Azure Dev Ops and Build Pipelines. Fordemonstration, asample [TianoCorerepo](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Fspbrogan%2Fedk2-staging.git&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=QmzQhemLUNB7FGJwoGeeewExThrUjUArxnkmtro1b8A%3D&reserved=0)Pipeline](https://nam06.safelinks.protection.outlook.co
(branch edk2-stuart-ci-latest) and [Dev Ops
m/?url=https%3A%2F%2Fdev.azure.com%2Ftianocore%2Fedk2-ci-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=rS%2FrjIFkS
uEhI%2FDWyslf34i711%2FbY8Gw%2Byexq%2FwydjU%3D&reser
ved=0play/_build?definitionId=12) have been set up.TianoCore python tools
Furthermore, this proposal will leverage thePIP modules:[library](https://nam06.safelinks.protection.outlook.co
m/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.brogan
%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C7
2f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdata
=jzM6MKZFFAEvyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0[extensions](https://nam06.safelinks.protection.outlook
library/) and
.com/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fedk2-pytool-&data=02%7C01%7Csean.bro
gan%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%
7C72f988bf86f14
1af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sd
ata=jzM6MKZFFAEvyR4h0%2Bkf5pUcBSsoVzkIHFFi1Bd6Il4%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
extensions/) (with repos located
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
library) and
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
f86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452466989&sdata=tInVw
GwCXXGBgcmHQ%2B0UwRaEbtZWRy8hTp5wpRtN0R0%3D&reserved=0`azure-
pytool-extensions)).
The primary execution flows can be found in thepipelines-pr-gate.yml` and `azure-pipelines-pr-gate-linux.yml` files.These YAML files are consumed by the Azure Dev OpsBuild Pipeline anddictate what server resources should be used, howthey should beconfigured, and what processes should be run on them.[here](https://nam06.safelinks.protection.outlook.com/?
An overview of this schema can be found
url=https%3A%2F%2Fdocs.microsoft.com%2Fen-&data=02%7C01%7Csean.brogan%40microsoft
.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988bf86f14
1af91ab2d7cd011
db47%7C1%7C0%7C637045269452466989&sdata=j8nyYG3Vniz
wKFZ8l1BWoLcPy%2BuaSoVT0jN2Esi7wH8%3D&reserved=0commands that make
us/azure/devops/pipelines/yaml-schema?view=azure-
devops&tabs=schema).
Inspection of these files reveals the EDKII Toolsup the primary processes for the CI'stuart_ci_build'. These
build: 'stuart_setup', 'stuart_update', andcommands come from the EDKII Tools PIP modules andare configured asdescribed below. More documentation on the stuarttools can be found
[here](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0[here](https://nam06.safelinks.protection.outlook.com/?
extensions/blob/master/docs/using.md) and
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sdat
a=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0extensions/blob/master/docs/features/feature_invocables.md).of precedence):
## Configuration
Configuration of the CI process consists of (in order* command-line arguments passed in via the PipelineYAML* a per-package configuration file (e.g. `<package-readme](https://nam06.safelinks.protection.outlook.com/
name>.mu.yaml`) that is detected by the CI system in
EDKII Tools.
* a global configuration Python module (e.g.
`CISetting.py`) passed in via the command-line
The global configuration file is described in [this
?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.brog
an%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7
C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637045269452466989&sda
ta=OwT%2BiNTN9Rvr14Oj67IdDlK8WGJClTIsmEzdyrBt2Ho%3D&reserved=0extensions/blob/master/docs/usability/using_settings_manager.md) from the EDKII Tools documentation. Thisconfiguration iswritten as a Python module so that decisions can bemade dynamicallybased on command line parameters and codebase state.settings in the
The per-package configuration file can override mostglobal configuration file, but is not dynamic. Thisfile can be usedto skip or customize tests that may be incompatiblewith a specificpackage.all tests on all
By default, the global configuration will try to runpackages.[here](https://nam06.safelinks.protection.outlook.com/?
## CI Test Types
All CI tests are instances of EDKII Tools plugins.
Documentation on the plugin system can be found
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-pytool-&data=02%7C01%7Csean.broga
n%40microsoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C
72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637045269452476994&sdat
a=1AM4geY%2BEFhekun7oQOYyVVt%2Bwn6CAF2hhtkS0sgwPU%3D&reserved=0extensions/blob/master/docs/usability/using_plugin_manager.md) and[here](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-&data=02%7C01%7Csean.brogan%40mic
rosoft.com%7C9e7d685fa1764cf500af08d73d4c1e1b%7C72f988b
f86f141af91ab2d
7cd011db47%7C1%7C0%7C637045269452476994&sdata=3vyPM
vrfG3V4doNM0%2BcKTpDSHEEEvkfxKAtw3eiJWsY%3D&reserved=0extensions/blob/master/docs/features/feature_plugin_man
pytool-ager.md). Upon invocation, each plugin will be passedthe path to thecurrent package under test and a dictionarycontaining its targetedconfiguration, as assembled from the command line,per-packageconfiguration, and global configuration.plugins and helper
Note: CI plugins are considered unique from buildplugins, even though some CI plugins may executesteps of a build.under test (in
In the example, these plugins live alongside the codethe `BaseTools` directory), but may be moved to the'edk2-test' repoif that location makes more sense for the community.and compares
### Module Inclusion Test - DscCompleteCheck
This test scans all available modules (via INF files)them to the package-level DSC file for the packageeach module iscontained within. The test considers it an error ifany module doesnot appear in the `Components` section of at leastone package-levelDSC (indicating that it would not be built if thepackage were built).modules would be
### Code Compilation Test - CompilerPlugin
Once the Module Inclusion Test has verified that allbuilt if all package-level DSCs were built, the CodeCompilation Testsimply runs through and builds every package-levelDSC on everytoolchain and for every architecture that issupported. Any modulethat fails to build is considered an error.and
### Host-Based UnitTests - HostUnitTestCompilerPluginHostUnitTestDscCompleteCheckmore detail on
The [Testing RFC doc](Readme-Testing-RFC.md) has muchthis, but the basic idea is that host- based unittests can becompiled against individual modules and libraries andrun on the buildagent (in this case, the Dev Ops build server). Thesuccessful andfailing test case results are collected and includedin the finalbuild report.rather than an
### GUID Uniqueness Test - GuidCheck
This test works on the collection of all packagesindividual package. It looks at all FILE_GUIDs andGUIDs declared inDEC files and ensures that they are unique for thecodebase. Thisprevents, for example, accidental duplication ofGUIDs when using anexisting INF as a template for a new module.INFs files for a
### Cross-Package Dependency Test - DependencyCheck
This test compares the list of all packages used ingiven package against a list of "alloweddependencies" in pluginconfiguration for that package. Any module thatdepends on adisallowed package will cause a test failure.a package's
### Library Declaration Test - LibraryClassCheck
This test looks at all library header files found in`Include/Library` directory and ensures that allfiles have a matchingLibraryClass declaration in the DEC file for thepackage. Any missingdeclarations will cause a failure.that there are no
### Invalid Character Test - CharEncodingCheck
This test scans all files in a package to make sureinvalid Unicode characters that may cause builderrors in somecharacter sets/localizations.existing RFCs from
## Next Steps
* Receive community feedback on RFC.
* Determine where this phase makes sense givenother TianoCore contributors.commits.
* Optimize testing beharior.
* Only run a subset of tests on PRs or individual* Run full testing either once per day or onceevery severalcommits.community feedback on
* Add more tests/capabilities.
* Continue to improve results formatting.
* Continue to improve CI documentation.
* Much of this documentation effort is pendingwhich parts are needed and what phases arepriorities.Behalf Of Michael D
Thanks
-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> OnKinney via Groups.IoIntegration Phase 1
Sent: Thursday, August 29, 2019 1:23 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Subject: [edk2-rfc] [RFC] EDK II Continuouscontinuous integration for
Hello,
This is a proposal for a first step towardsall TianoCore repositories to help improve to qualityof commits andautomate testing and release processes for all EDK IIpackages andplatforms.community members that
This is based on work from a number of EDK IIhave provide valuable input and evaluations.evaluation
* Rebecca Cran <mailto:rebecca@...> Jenkins* Laszlo Ersek <mailto:lersek@...> GitLabevaluation* Philippe Mathieu-Daudé <mailto:philmd@...>GitLab evaluation* Sean Brogan <mailto:sean.brogan@...>Azure Pipelines andHBFAthat contains a
* Bret Barkelew <mailto:Bret.Barkelew@...>
Azure Pipelines and HBFA
* Jiewen Yao <mailto:jiewen.yao@...> HBFA
The following link is a link to an EDK II WIKI pagesummary of the work to date. Please provide feedbackin the EDK IImailing lists. The WIKI pages will be updated withinput from theentire EDK II community.https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Continuous-Integration&data=02%7C01%7Csean.brogan%40microsoft.
com%7C6f67f169a6c746b4288608d72cbea7b6%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637027069686644659&sda
ta=GR9wN6gP3mJlCTopAaQ2rlzhby1nuF%2BwDVsfFIQAZjA%3D&reserved=0to the
Proposal
========
Phase 1 of adding continuous integration is limitededk2 repository. Additional repositories will beadded later.repository.
The following changes are proposed:
* Remove EDK II Maintainers write access to edk2Only EDK II Administrators will continue to havewriteaccess, and that should only be used to handleextraordinaryevents.instead of push
* EDK II Maintainers use a GitHub Pull Requestto commit a patch series to the edk2 repository.process. The
There are
no other changes to the development and reviewpatch series is prepared in an EDK II maintainerbranch withall commit message requirements met on each patchin the series.* The edk2 repository only accepts Pull Requests frommembersof the EDK II Maintainers team. Pull Requests fromanyone elseare rejected.series is auto
* Run pre-commit checks using Azure Pipelines
* If all pre-commit checks pass, then the patchcommitted. The result of this commit must matchthe contentsand commit history that would have occurred usingthe previouspush operation.submitter.
* If any pre-commit checks fail, then notify theA typical reason for a failure would be a mergeconflict withanother pull request that was just processed.minutes.
* Limit pre-commit checks execution time to 10* Provide on-demand builds to EDK II Maintainers thatto allowEDK II Maintainers to submit a branch through forthe sameset of pre-commit checks without submitting a pullrequest.ideas that could be
## Pre-Commit Checks in Phase 1
* Run and pass PatchCheck.py with no errors
=====================================================
The following are some additional pre-commit checkquickly added once the initial version usingPatchCheck.py is fullyfunctional.additional ones you
Please provide feedback on the ones you like andthink may improve the quality of the commits to theedk2 repository.with
## Proposed Pre-Commit Checks in Phase 2
* Verify Reviewed-by and Acked-by tags are presentcorrect maintainer email addressesmodified
* Verify no non-ASCII characters in modified files
* Verify no binary files in set of modified files
* Verify package dependency rules in modified files
## Proposed Pre-Commit Checks in Phase 3
* Run ECC on modified files
* Verify modified modules/libs build
* Run available host based tests (HBFA) againstmodules/libs
Best regards,
Mike
Sean
All,
Phase 1 CI content ready for full review and I would propose that if no additional comments this RFC can be completed.
Master branch of my edk2 fork located here.
https://github.com/spbrogan/edk2
Build badges in the readme will take you to the individual builds.
Packages being tested, issues, and test details located here
https://github.com/spbrogan/edk2/blob/master/ci/readme.md. Also available thru a link on the readme.
There are 24 commits to the latest edk2 to enable this.
The first 7 commits are enabling edk2 to leverage PyTools.
* Add pip requirements file to track pip versions
* Add files describing tools, versions, and paths.
* Add new tools_def for vs2019 and 17 and gcc5
* Add new build_rule to support MSFT AARM64 assembler
* Add python file for ci build
There are also two commits that add new NULL libraries to speed up build and minimize unnecessary cross package dependencies.
* Add BaseCryptLibNull
* Add TlsLibNull
The remaining commits are changes to each package to describe the configuration for each test and minor tweaks to enable the packages to build with the tested toolchains and architectures.
Finally the last commit adds the AzurePipeline Support files into a new ci folder.
Please review and provide feedback. Build times are under 10 minutes for Ubuntu and 12 minutes for Windows.
One more side note: Details are in the readme on how to run a spell checker and there is an online build that can be reviewed. There is some "noise" with valid words but there is a massive number of misspelled words that would be great to get cleaned up and I would like to put this test into place to avoid future errors. Mike - maybe you can add to phase 2 wiki.
Thanks
Sean
Phase 1 CI content ready for full review and I would propose that if no additional comments this RFC can be completed.
Master branch of my edk2 fork located here.
https://github.com/spbrogan/edk2
Build badges in the readme will take you to the individual builds.
Packages being tested, issues, and test details located here
https://github.com/spbrogan/edk2/blob/master/ci/readme.md. Also available thru a link on the readme.
There are 24 commits to the latest edk2 to enable this.
The first 7 commits are enabling edk2 to leverage PyTools.
* Add pip requirements file to track pip versions
* Add files describing tools, versions, and paths.
* Add new tools_def for vs2019 and 17 and gcc5
* Add new build_rule to support MSFT AARM64 assembler
* Add python file for ci build
There are also two commits that add new NULL libraries to speed up build and minimize unnecessary cross package dependencies.
* Add BaseCryptLibNull
* Add TlsLibNull
The remaining commits are changes to each package to describe the configuration for each test and minor tweaks to enable the packages to build with the tested toolchains and architectures.
Finally the last commit adds the AzurePipeline Support files into a new ci folder.
Please review and provide feedback. Build times are under 10 minutes for Ubuntu and 12 minutes for Windows.
One more side note: Details are in the readme on how to run a spell checker and there is an online build that can be reviewed. There is some "noise" with valid words but there is a massive number of misspelled words that would be great to get cleaned up and I would like to put this test into place to avoid future errors. Mike - maybe you can add to phase 2 wiki.
Thanks
Sean
rebecca@...
On 2019-10-02 15:47, Sean via Groups.Io wrote:
the Doxygen builds?
--
Rebecca Cran
Please review and provide feedback. Build times are under 10 minutes for Ubuntu and 12 minutes for Windows.Do you have a location set up for where you'll publish the results of
One more side note: Details are in the readme on how to run a spell checker and there is an online build that can be reviewed. There is some "noise" with valid words but there is a massive number of misspelled words that would be great to get cleaned up and I would like to put this test into place to avoid future errors. Mike - maybe you can add to phase 2 wiki.
the Doxygen builds?
--
Rebecca Cran
Sean
Rebecca,
I have not setup or integrated a Doxygen build as part of this. I think that is something that should happen and it should be done by a cloud build on a regular cadence but is independent of getting phase 1 CI up and running.
I would like to see a comprehensive plan for documentation. Spell check, image support, content other than code comments, and finally a defined expectation for contributions along with the code would be my initial requirements. Also hosting stable tag versions as well as master version. I think this can be its own RFC.
Thanks
Sean
I have not setup or integrated a Doxygen build as part of this. I think that is something that should happen and it should be done by a cloud build on a regular cadence but is independent of getting phase 1 CI up and running.
I would like to see a comprehensive plan for documentation. Spell check, image support, content other than code comments, and finally a defined expectation for contributions along with the code would be my initial requirements. Also hosting stable tag versions as well as master version. I think this can be its own RFC.
Thanks
Sean