RFC v2: Static Analysis in edk2 CI


Felix Polyudov
 

This is version 2 of the proposal that provides additional details regarding the bring up process.

The initial version is at https://edk2.groups.io/g/rfc/message/696

The goal of the proposal is integration of the static analysis (SA) into the edk2 workflow.

- Use Open Coverity SA service to scan edk2 repository. The service is free for open source projects.
edk2 Open Coverity project: https://scan.coverity.com/projects/tianocore-edk2
- Update edk2 CI scripts to run analysis once a week
- Perform analysis on all the edk2 packages using package DSC files that are used for CI build tests
(Coverity analysis is executed in the course of a specially instrumented project build).
- SA results are uploaded to scan.coverity.com. To access them one would need to register on the site and request tianocore-edk2 project access. The site can be used to triage the reported issues. Confirmed issues can be addressed using a standard edk2 process (Bugzilla, mailing list).
- During the initial bring up period, access to the SA results is restricted to stewards, maintainers, and members of the TianoCore InfoSec group, who are encouraged to review reported issues with the primary goal of identifying security-related issues. All such issues should be handled in accordance with the following guidelines:
https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
- The initial bring up period ends when embargo for all the identified security issues ends or after 30 days if no security issues have been identified
- Once brig up period is over, SA results access is open to everybody.
- The package maintainers should monitor weekly scan results for a newly reported issues and reach back to original patch submitters to resolve them. Package maintainers can revert the patch if no action is taken by the submitter.

-The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all copies of the transmission.


Michael D Kinney
 

+devel@edk2.groups.io

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix Polyudov via groups.io
Sent: Monday, June 13, 2022 10:48 AM
To: rfc@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>
Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

This is version 2 of the proposal that provides additional details regarding the bring up process.

The initial version is at https://edk2.groups.io/g/rfc/message/696

The goal of the proposal is integration of the static analysis (SA) into the edk2 workflow.

- Use Open Coverity SA service to scan edk2 repository. The service is free for open source projects.
edk2 Open Coverity project: https://scan.coverity.com/projects/tianocore-edk2
- Update edk2 CI scripts to run analysis once a week
- Perform analysis on all the edk2 packages using package DSC files that are used for CI build tests
(Coverity analysis is executed in the course of a specially instrumented project build).
- SA results are uploaded to scan.coverity.com. To access them one would need to register on the site and request tianocore-
edk2 project access. The site can be used to triage the reported issues. Confirmed issues can be addressed using a standard edk2
process (Bugzilla, mailing list).
- During the initial bring up period, access to the SA results is restricted to stewards, maintainers, and members of the
TianoCore InfoSec group, who are encouraged to review reported issues with the primary goal of identifying security-related
issues. All such issues should be handled in accordance with the following guidelines:
https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
- The initial bring up period ends when embargo for all the identified security issues ends or after 30 days if no security
issues have been identified
- Once brig up period is over, SA results access is open to everybody.
- The package maintainers should monitor weekly scan results for a newly reported issues and reach back to original patch
submitters to resolve them. Package maintainers can revert the patch if no action is taken by the submitter.

-The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication
is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this
message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly
prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all
copies of the transmission.




Pedro Falcato
 

(Replying under Mike for devel visibility)

Felix,

Why coverity? I feel like we could run something akin to LLVM's clang-tidy
+ scan-build; it's open source (transparent *and* we can improve it or add
UEFI quirks) and doesn't rely on a third-party service. I'm sure we could
figure something out for hosting the thing. Otherwise, looks good to me.

Thanks,
Pedro

On Mon, Jun 13, 2022 at 7:54 PM Michael D Kinney <michael.d.kinney@...>
wrote:

+devel@edk2.groups.io

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix
Polyudov via groups.io
Sent: Monday, June 13, 2022 10:48 AM
To: rfc@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>
Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

This is version 2 of the proposal that provides additional details
regarding the bring up process.

The initial version is at https://edk2.groups.io/g/rfc/message/696

The goal of the proposal is integration of the static analysis (SA) into
the edk2 workflow.

- Use Open Coverity SA service to scan edk2 repository. The service is
free for open source projects.
edk2 Open Coverity project:
https://scan.coverity.com/projects/tianocore-edk2
- Update edk2 CI scripts to run analysis once a week
- Perform analysis on all the edk2 packages using package DSC files
that are used for CI build tests
(Coverity analysis is executed in the course of a specially
instrumented project build).
- SA results are uploaded to scan.coverity.com. To access them one
would need to register on the site and request tianocore-
edk2 project access. The site can be used to triage the reported issues.
Confirmed issues can be addressed using a standard edk2
process (Bugzilla, mailing list).
- During the initial bring up period, access to the SA results is
restricted to stewards, maintainers, and members of the
TianoCore InfoSec group, who are encouraged to review reported issues
with the primary goal of identifying security-related
issues. All such issues should be handled in accordance with the
following guidelines:
https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
- The initial bring up period ends when embargo for all the identified
security issues ends or after 30 days if no security
issues have been identified
- Once brig up period is over, SA results access is open to everybody.
- The package maintainers should monitor weekly scan results for a newly
reported issues and reach back to original patch
submitters to resolve them. Package maintainers can revert the patch if
no action is taken by the submitter.

-The information contained in this message may be confidential and
proprietary to American Megatrends (AMI). This communication
is intended to be read only by the individual or entity to whom it is
addressed or by their designee. If the reader of this
message is not the intended recipient, you are on notice that any
distribution of this message, in any form, is strictly
prohibited. Please promptly notify the sender by reply e-mail or by
telephone at 770-246-8600, and then delete or destroy all
copies of the transmission.








--
Pedro Falcato


Rebecca Cran
 

LLVM's tools also appear to be much easier to review, for other people to run etc. I'd suggest at least starting with clang-tidy + scan-build and possibly adding Coverity later.

I've found the Coverity tools, while very powerful, tend to get ignored after a while because it's quite a process to keep it running, go through the issues it detects and keep the database up-to-date etc.


--

Rebecca Cran

On 6/13/22 15:54, Pedro Falcato wrote:
(Replying under Mike for devel visibility)

Felix,

Why coverity? I feel like we could run something akin to LLVM's clang-tidy
+ scan-build; it's open source (transparent *and* we can improve it or add
UEFI quirks) and doesn't rely on a third-party service. I'm sure we could
figure something out for hosting the thing. Otherwise, looks good to me.

Thanks,
Pedro

On Mon, Jun 13, 2022 at 7:54 PM Michael D Kinney <michael.d.kinney@...>
wrote:

+devel@edk2.groups.io

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix
Polyudov via groups.io
Sent: Monday, June 13, 2022 10:48 AM
To: rfc@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>
Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

This is version 2 of the proposal that provides additional details
regarding the bring up process.
The initial version is at https://edk2.groups.io/g/rfc/message/696

The goal of the proposal is integration of the static analysis (SA) into
the edk2 workflow.
- Use Open Coverity SA service to scan edk2 repository. The service is
free for open source projects.
edk2 Open Coverity project:
https://scan.coverity.com/projects/tianocore-edk2
- Update edk2 CI scripts to run analysis once a week
- Perform analysis on all the edk2 packages using package DSC files
that are used for CI build tests
(Coverity analysis is executed in the course of a specially
instrumented project build).
- SA results are uploaded to scan.coverity.com. To access them one
would need to register on the site and request tianocore-
edk2 project access. The site can be used to triage the reported issues.
Confirmed issues can be addressed using a standard edk2
process (Bugzilla, mailing list).
- During the initial bring up period, access to the SA results is
restricted to stewards, maintainers, and members of the
TianoCore InfoSec group, who are encouraged to review reported issues
with the primary goal of identifying security-related
issues. All such issues should be handled in accordance with the
following guidelines:
https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
- The initial bring up period ends when embargo for all the identified
security issues ends or after 30 days if no security
issues have been identified
- Once brig up period is over, SA results access is open to everybody.
- The package maintainers should monitor weekly scan results for a newly
reported issues and reach back to original patch
submitters to resolve them. Package maintainers can revert the patch if
no action is taken by the submitter.
-The information contained in this message may be confidential and
proprietary to American Megatrends (AMI). This communication
is intended to be read only by the individual or entity to whom it is
addressed or by their designee. If the reader of this
message is not the intended recipient, you are on notice that any
distribution of this message, in any form, is strictly
prohibited. Please promptly notify the sender by reply e-mail or by
telephone at 770-246-8600, and then delete or destroy all
copies of the transmission.








Felix Polyudov
 

Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it in the first version of the RFC.
CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open source front-end for the scan-build and clang-tidy.
It simplifies analyzer configuration and provides web-based report storage. However, it has to be hosted somewhere.
If somebody has an idea on how edk2 community can host the CodeChecker, that's definitely an option to consider.


Pedro Falcato
 

Just want to note that if we want to go ahead with fuzzing (I detailed a
possible plan to do so in the mailing list a month or so ago) we will
definitely need somewhere to run fuzzing (even if it's Google's syzbot).
Getting somewhere where we can run static analysis, fuzzing just makes
sense IMO (hell, who knows, maybe even CI or something like Gerrit for
mailing list-less code reviews).

On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io <felixp=
ami.com@groups.io> wrote:

Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it
in the first version of the RFC.
CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open
source front-end for the scan-build and clang-tidy.
It simplifies analyzer configuration and provides web-based report
storage. However, it has to be hosted somewhere.
If somebody has an idea on how edk2 community can host the CodeChecker,
that's definitely an option to consider.





--
Pedro Falcato


Pedro Falcato
 

(Re-adding devel@ since Felix dropped it)

On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato <pedro.falcato@...>
wrote:

Just want to note that if we want to go ahead with fuzzing (I detailed a
possible plan to do so in the mailing list a month or so ago) we will
definitely need somewhere to run fuzzing (even if it's Google's syzbot).
Getting somewhere where we can run static analysis, fuzzing just makes
sense IMO (hell, who knows, maybe even CI or something like Gerrit for
mailing list-less code reviews).

On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io <felixp=
ami.com@groups.io> wrote:

Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it
in the first version of the RFC.
CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open
source front-end for the scan-build and clang-tidy.
It simplifies analyzer configuration and provides web-based report
storage. However, it has to be hosted somewhere.
If somebody has an idea on how edk2 community can host the CodeChecker,
that's definitely an option to consider.





--
Pedro Falcato

--
Pedro Falcato


Michael D Kinney
 

I have Coverity scan builds running in a GitHub Action and then uploaded to Coverity.

We should be able to configure a GitHub Action to run other analyzers.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Pedro Falcato
Sent: Tuesday, June 14, 2022 1:00 PM
To: rfc@edk2.groups.io; POLUDOV, FELIX <felixp@...>
Cc: Rebecca Cran <rebecca@...>; edk2-devel-groups-io <devel@edk2.groups.io>
Subject: Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

(Re-adding devel@ since Felix dropped it)

On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato <pedro.falcato@...>
wrote:

Just want to note that if we want to go ahead with fuzzing (I detailed a
possible plan to do so in the mailing list a month or so ago) we will
definitely need somewhere to run fuzzing (even if it's Google's syzbot).
Getting somewhere where we can run static analysis, fuzzing just makes
sense IMO (hell, who knows, maybe even CI or something like Gerrit for
mailing list-less code reviews).

On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io <felixp=
ami.com@groups.io> wrote:

Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it
in the first version of the RFC.
CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open
source front-end for the scan-build and clang-tidy.
It simplifies analyzer configuration and provides web-based report
storage. However, it has to be hosted somewhere.
If somebody has an idea on how edk2 community can host the CodeChecker,
that's definitely an option to consider.





--
Pedro Falcato

--
Pedro Falcato




Felix Polyudov
 

Yes, we can run other analyzer; however, in case of CodeChecker we also need a server to upload the result to.

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Michael D
Kinney via groups.io
Sent: Thursday, June 23, 2022 9:30 PM
To: rfc@edk2.groups.io; pedro.falcato@...; Felix Polyudov
<Felixp@...>; Kinney, Michael D <michael.d.kinney@...>
Cc: Rebecca Cran <rebecca@...>; edk2-devel-groups-io
<devel@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI


**CAUTION: The e-mail below is from an external source. Please exercise
caution before opening attachments, clicking links, or following guidance.**

I have Coverity scan builds running in a GitHub Action and then uploaded to
Coverity.

We should be able to configure a GitHub Action to run other analyzers.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Pedro
Falcato
Sent: Tuesday, June 14, 2022 1:00 PM
To: rfc@edk2.groups.io; POLUDOV, FELIX <felixp@...>
Cc: Rebecca Cran <rebecca@...>; edk2-devel-groups-io
<devel@edk2.groups.io>
Subject: Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

(Re-adding devel@ since Felix dropped it)

On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato
<pedro.falcato@...>
wrote:

Just want to note that if we want to go ahead with fuzzing (I
detailed a possible plan to do so in the mailing list a month or so
ago) we will definitely need somewhere to run fuzzing (even if it's Google's
syzbot).
Getting somewhere where we can run static analysis, fuzzing just
makes sense IMO (hell, who knows, maybe even CI or something like
Gerrit for mailing list-less code reviews).

On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io
<felixp= ami.com@groups.io> wrote:

Yes, LLVM/CLANG Static Analyzer is another possibility. I've
mentioned it in the first version of the RFC.
CodeChecker
(https://codechecker.readthedocs.io/en/latest/) is an open source front-end
for the scan-build and clang-tidy.
It simplifies analyzer configuration and provides web-based report
storage. However, it has to be hosted somewhere.
If somebody has an idea on how edk2 community can host the
CodeChecker, that's definitely an option to consider.





--
Pedro Falcato
-The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all copies of the transmission.