Date   

Re: [PATCH v9 04/16] ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check

Ard Biesheuvel
 

On 8/11/20 9:01 AM, Shenglei Zhang wrote:
Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>

---
ArmVirtPkg/ArmVirtPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
index 27bb7ff49011..8b2c0305c175 100644
--- a/ArmVirtPkg/ArmVirtPkg.ci.yaml
+++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
@@ -5,6 +5,7 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -12,6 +13,17 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test


Re: [PATCH 3/3] MdeModulePkg/PartitionDxe: Fix the incorrect LBA size in child hander

Ni, Ray
 

Reviewed-by: Ray Ni <ray.ni@intel.com>

-----Original Message-----
From: Gao, Zhichao <zhichao.gao@intel.com>
Sent: Tuesday, August 11, 2020 2:43 PM
To: devel@edk2.groups.io
Cc: Wang, Jian J <jian.j.wang@intel.com>; Wu, Hao A <hao.a.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Gary Lin
<glin@suse.com>; Andrew Fish <afish@apple.com>
Subject: [PATCH 3/3] MdeModulePkg/PartitionDxe: Fix the incorrect LBA size in child hander

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2843

PartitionInstallChildHandle's parameters Start and End is counted
by the BlockSize, but in the implementation it uses the parent
device's BlockSize to calculate the new Start, End and LastBlock.
It would cause the driver report incorrect block scope and the file
system would fail to be found with right block scope.
So correct it to the right value.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Gary Lin <glin@suse.com>
Cc: Andrew Fish <afish@apple.com>
Signed-off-by: Zhichao Gao <zhichao.gao@intel.com>
---
MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
index 473e091320..f10ce7c65b 100644
--- a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
+++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
@@ -1149,8 +1149,8 @@ PartitionInstallChildHandle (

Private->Signature = PARTITION_PRIVATE_DATA_SIGNATURE;

- Private->Start = MultU64x32 (Start, ParentBlockIo->Media->BlockSize);
- Private->End = MultU64x32 (End + 1, ParentBlockIo->Media->BlockSize);
+ Private->Start = MultU64x32 (Start, BlockSize);
+ Private->End = MultU64x32 (End + 1, BlockSize);

Private->BlockSize = BlockSize;
Private->ParentBlockIo = ParentBlockIo;
@@ -1187,13 +1187,7 @@ PartitionInstallChildHandle (

Private->Media.IoAlign = 0;
Private->Media.LogicalPartition = TRUE;
- Private->Media.LastBlock = DivU64x32 (
- MultU64x32 (
- End - Start + 1,
- ParentBlockIo->Media->BlockSize
- ),
- BlockSize
- ) - 1;
+ Private->Media.LastBlock = End - Start;

Private->Media.BlockSize = (UINT32) BlockSize;

--
2.21.0.windows.1


Re: [PATCH 2/3] MdeModulePkg/PartitionDxe: Remove the check for special MBR

Ni, Ray
 

Zhichao,
Can you please just revert the fix you recently added?

-----Original Message-----
From: Gao, Zhichao <zhichao.gao@intel.com>
Sent: Tuesday, August 11, 2020 2:43 PM
To: devel@edk2.groups.io
Cc: Wang, Jian J <jian.j.wang@intel.com>; Wu, Hao A <hao.a.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Gary Lin
<glin@suse.com>; Andrew Fish <afish@apple.com>
Subject: [PATCH 2/3] MdeModulePkg/PartitionDxe: Remove the check for special MBR

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2823

Follow the spec definition, the ISO 9660 (and UDF) would be
checked before the MBR. So it is not required to skip such
MBR talbe that contian the entire block device.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Gary Lin <glin@suse.com>
Cc: Andrew Fish <afish@apple.com>
Signed-off-by: Zhichao Gao <zhichao.gao@intel.com>
---
.../Universal/Disk/PartitionDxe/Mbr.c | 19 -------------------
1 file changed, 19 deletions(-)

diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Mbr.c b/MdeModulePkg/Universal/Disk/PartitionDxe/Mbr.c
index 3830af1ea7..822bf03e92 100644
--- a/MdeModulePkg/Universal/Disk/PartitionDxe/Mbr.c
+++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Mbr.c
@@ -55,25 +55,6 @@ PartitionValidMbr (
StartingLBA = UNPACK_UINT32 (Mbr->Partition[Index1].StartingLBA);
SizeInLBA = UNPACK_UINT32 (Mbr->Partition[Index1].SizeInLBA);

- //
- // If the MBR with partition entry covering the ENTIRE disk, i.e. start at LBA0
- // with whole disk size, we treat it as an invalid MBR partition.
- //
- if ((StartingLBA == 0) &&
- (SizeInLBA == (LastLba + 1))) {
- //
- // Refer to the http://manpages.ubuntu.com/manpages/bionic/man8/mkudffs.8.html
- // "WHOLE DISK VS PARTITION"
- // Some linux ISOs may put the MBR table in the first 512 bytes for compatibility reasons with Windows.
- // Linux kernel ignores MBR table if contains partition which starts at sector 0.
- // Skip it because we don't have the partition check for UDF(El Torito compatible).
- // It would continue to do the whole disk check in the UDF routine.
- //
- DEBUG ((DEBUG_INFO, "PartitionValidMbr: MBR table has partition entry covering the ENTIRE disk. Don't treat it as a
valid MBR.\n"));
-
- return FALSE;
- }
-
if (Mbr->Partition[Index1].OSIndicator == 0x00 || SizeInLBA == 0) {
continue;
}
--
2.21.0.windows.1


Re: [PATCH 1/3] MdeModulePkg/PartitionDxe: Put the UDF check ahead of MBR

Ni, Ray
 

Zhichao,
Can you also add notes in the commit message describing that for some ISOs (better with more specific ISO info), the MBR information is not correct?

Thanks,
Ray

-----Original Message-----
From: Gao, Zhichao <zhichao.gao@intel.com>
Sent: Tuesday, August 11, 2020 2:43 PM
To: devel@edk2.groups.io
Cc: Wang, Jian J <jian.j.wang@intel.com>; Wu, Hao A <hao.a.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Gary Lin
<glin@suse.com>; Andrew Fish <afish@apple.com>
Subject: [PATCH 1/3] MdeModulePkg/PartitionDxe: Put the UDF check ahead of MBR

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2823

Refer to UEFI spec 2.8, Section 13.3.2, a block device should
be scanned as below order:
1. GPT
2. ISO 9660 (El Torito) (UDF should aslo be here)
3. MBR
4. no partition found
Note: UDF is using the same boot method as CD, so put it in
the same priority with ISO 9660.

This would also solve the issue that ISO image with MBR would
be treat as MBR device instead of CD/DVD. That would make the
behavior of the image boot different.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Gary Lin <glin@suse.com>
Cc: Andrew Fish <afish@apple.com>
Signed-off-by: Zhichao Gao <zhichao.gao@intel.com>
---
MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
index 6a43c3cafb..473e091320 100644
--- a/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
+++ b/MdeModulePkg/Universal/Disk/PartitionDxe/Partition.c
@@ -35,11 +35,19 @@ EFI_DRIVER_BINDING_PROTOCOL gPartitionDriverBinding = {

//
// Prioritized function list to detect partition table.
+// Refer to UEFI Spec 13.3.2 Partition Discovery, the block device
+// should be scanned in below order:
+// 1. GPT
+// 2. ISO 9660 (El Torito) (or UDF)
+// 3. MBR
+// 4. no partiton found
+// Note: UDF is using a same method as booting from CD-ROM, so put it along
+// with CD-ROM check.
//
PARTITION_DETECT_ROUTINE mPartitionDetectRoutineTable[] = {
PartitionInstallGptChildHandles,
- PartitionInstallMbrChildHandles,
PartitionInstallUdfChildHandles,
+ PartitionInstallMbrChildHandles,
NULL
};

--
2.21.0.windows.1


Re: [PATCH 0/3] MdeModulePkg/PartitionDxe: Make the parition driver match the spec

Gary Lin
 

On Tue, Aug 11, 2020 at 02:42:59PM +0800, Zhichao Gao wrote:
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2823
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2843

Refer to the UEFI spec 2.8, Section 13.3.2:
A block device should be scanned as below order:
1. GPT
2. ISO 9660 (El Torito) (UDF should aslo be here)
4. MBR
5. no partition found

But the code implementation is:
1. GPT
2. MBR
3. ISO 9660 (El Torito) (UDF)
4. no partition found

Which would cause the ISO 9960 image with MBR info be treated as MBR
device. That would cause unexpect behavior. So fix it to follow the spec
description.

The fix of the PartitionInstallChildHandle would change the boot behavior
of Linux ISO image with MBR table. So add it after the order adjustment
to make no impact of the boot.
Thanks for the patches. After applying this patch series, the firmware
recognizes openSUSE/SUSE iso images again.

Tested-by: Gary Lin <glin@suse.com>

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Gary Lin <glin@suse.com>
Cc: Andrew Fish <afish@apple.com>
Signed-off-by: Zhichao Gao <zhichao.gao@intel.com>
Zhichao Gao (3):
MdeModulePkg/PartitionDxe: Put the UDF check ahead of MBR
MdeModulePkg/PartitionDxe: Remove the check for special MBR
MdeModulePkg/PartitionDxe: Fix the incorrect LBA size in child hander

.../Universal/Disk/PartitionDxe/Mbr.c | 19 ----------------
.../Universal/Disk/PartitionDxe/Partition.c | 22 ++++++++++---------
2 files changed, 12 insertions(+), 29 deletions(-)

--
2.21.0.windows.1


[PATCH v9 16/16] UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.
Besides, add submodule path in IgnoreFiles section.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
index fc5b3b387664..9e9ffc1af113 100644
--- a/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
+++ b/UnitTestFrameworkPkg/UnitTestFrameworkPkg.ci.yaml
@@ -9,6 +9,18 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ "Library/CmockaLib/cmocka"
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "UnitTestFrameworkPkg.dsc"
--
2.18.0.windows.1


[PATCH v9 15/16] UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
UefiCpuPkg/UefiCpuPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
index d54651d43800..48ab710438a9 100644
--- a/UefiCpuPkg/UefiCpuPkg.ci.yaml
+++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for UefiCpuPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "UefiCpuPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 14/16] ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
ShellPkg/ShellPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml
index 565e08596b2f..30894d44bc3a 100644
--- a/ShellPkg/ShellPkg.ci.yaml
+++ b/ShellPkg/ShellPkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for ShellPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "ShellPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 13/16] SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
SecurityPkg/SecurityPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/SecurityPkg/SecurityPkg.ci.yaml b/SecurityPkg/SecurityPkg.ci.yaml
index 1ef1d2e67ead..03be2e94ca97 100644
--- a/SecurityPkg/SecurityPkg.ci.yaml
+++ b/SecurityPkg/SecurityPkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for SecurityPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "SecurityPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 12/16] PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 13 +++++++++++++
1 file changed, 13 insertions(+)

diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
index 5543d15c3248..61f3fd7b5074 100644
--- a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
+++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
@@ -2,12 +2,25 @@
# CI configuration for PcAtChipsetPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ ## options defined .pytool/Plugin/LicenseCheck
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "PcAtChipsetPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 11/16] OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
OvmfPkg/OvmfPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.ci.yaml b/OvmfPkg/OvmfPkg.ci.yaml
index ed342d7a3d08..3128aefe9ed1 100644
--- a/OvmfPkg/OvmfPkg.ci.yaml
+++ b/OvmfPkg/OvmfPkg.ci.yaml
@@ -5,6 +5,7 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -12,6 +13,17 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1


[PATCH v9 10/16] NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
NetworkPkg/NetworkPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/NetworkPkg/NetworkPkg.ci.yaml b/NetworkPkg/NetworkPkg.ci.yaml
index aa536830fc27..1a3ab71792d1 100644
--- a/NetworkPkg/NetworkPkg.ci.yaml
+++ b/NetworkPkg/NetworkPkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for NetworkPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "NetworkPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 09/16] MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
MdePkg/MdePkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/MdePkg/MdePkg.ci.yaml b/MdePkg/MdePkg.ci.yaml
index b088a4344908..15931eaccabf 100644
--- a/MdePkg/MdePkg.ci.yaml
+++ b/MdePkg/MdePkg.ci.yaml
@@ -2,6 +2,7 @@
# CI configuration for MdePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -9,6 +10,17 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
## options defined ci/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "MdePkg.dsc"
--
2.18.0.windows.1


[PATCH v9 08/16] FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Guomin Jiang <guomin.jiang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
FmpDevicePkg/FmpDevicePkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/FmpDevicePkg/FmpDevicePkg.ci.yaml b/FmpDevicePkg/FmpDevicePkg.ci.yaml
index d498ad7d21e1..1b3e2187356d 100644
--- a/FmpDevicePkg/FmpDevicePkg.ci.yaml
+++ b/FmpDevicePkg/FmpDevicePkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for FmpDevicePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "FmpDevicePkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 07/16] FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Guomin Jiang <guomin.jiang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
FatPkg/FatPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
index 489c4614e9e1..fe95f481b50c 100644
--- a/FatPkg/FatPkg.ci.yaml
+++ b/FatPkg/FatPkg.ci.yaml
@@ -2,12 +2,24 @@
# CI configuration for FatPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "FatPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 06/16] EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
EmulatorPkg/EmulatorPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/EmulatorPkg/EmulatorPkg.ci.yaml b/EmulatorPkg/EmulatorPkg.ci.yaml
index dead82f69745..980844c1fdd4 100644
--- a/EmulatorPkg/EmulatorPkg.ci.yaml
+++ b/EmulatorPkg/EmulatorPkg.ci.yaml
@@ -5,6 +5,7 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -12,6 +13,17 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1


[PATCH v9 05/16] CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.
Besides, add submodule path in IgnoreFiles section.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
CryptoPkg/CryptoPkg.ci.yaml | 13 +++++++++++++
1 file changed, 13 insertions(+)

diff --git a/CryptoPkg/CryptoPkg.ci.yaml b/CryptoPkg/CryptoPkg.ci.yaml
index e73b79e01fef..e2d190a90c51 100644
--- a/CryptoPkg/CryptoPkg.ci.yaml
+++ b/CryptoPkg/CryptoPkg.ci.yaml
@@ -2,12 +2,25 @@
# CI configuration for CryptoPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ "Library/OpensslLib/openssl"
+ ]
+ },
"CompilerPlugin": {
"DscPath": "CryptoPkg.dsc"
},
--
2.18.0.windows.1


[PATCH v9 04/16] ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
---
ArmVirtPkg/ArmVirtPkg.ci.yaml | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
index 27bb7ff49011..8b2c0305c175 100644
--- a/ArmVirtPkg/ArmVirtPkg.ci.yaml
+++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
@@ -5,6 +5,7 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -12,6 +13,17 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1


[PATCH v9 03/16] MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.
Besides, add submodule path in IgnoreFiles section.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
MdeModulePkg/MdeModulePkg.ci.yaml | 14 ++++++++++++++
1 file changed, 14 insertions(+)

diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml b/MdeModulePkg/MdeModulePkg.ci.yaml
index 0fc8c3633878..1a7e955185d8 100644
--- a/MdeModulePkg/MdeModulePkg.ci.yaml
+++ b/MdeModulePkg/MdeModulePkg.ci.yaml
@@ -2,6 +2,7 @@
# CI configuration for MdeModulePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
@@ -9,6 +10,19 @@
"LicenseCheck": {
"IgnoreFiles": []
},
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ ## Both file path and directory path are accepted.
+ "IgnoreFiles": [
+ "Library/BrotliCustomDecompressLib/brotli",
+ "Universal/RegularExpressionDxe/oniguruma"
+ ]
+ },
## options defined ci/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "MdeModulePkg.dsc"
--
2.18.0.windows.1


[PATCH v9 02/16] .pytool/Plugin: Add a plugin EccCheck

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
EccCheck is a plugin to report Ecc issues for code in pull request
, which will be run on open ci.
But note not each kind of issue could be reported out.
It can only handle the issues, whose line number in CSV report
accurately map with their code in source code files. And Ecc issues
about comments can also be handled.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
.pytool/Plugin/EccCheck/EccCheck.py | 302 ++++++++++++++++++
.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml | 11 +
.pytool/Plugin/EccCheck/Readme.md | 15 +
3 files changed, 328 insertions(+)
create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/EccCheck/Readme.md

diff --git a/.pytool/Plugin/EccCheck/EccCheck.py b/.pytool/Plugin/EccCheck/EccCheck.py
new file mode 100644
index 000000000000..249a16a7dddf
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/EccCheck.py
@@ -0,0 +1,302 @@
+# @file EccCheck.py
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import os
+import shutil
+import re
+import csv
+import xml.dom.minidom
+from typing import List, Dict, Tuple
+import logging
+from io import StringIO
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.utility_functions import RunCmd
+
+
+class EccCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that finds the Ecc issues of newly added code in pull request.
+
+ Configuration options:
+ "EccCheck": {
+ "ExceptionList": [],
+ "IgnoreFiles": []
+ },
+ """
+
+ ReModifyFile = re.compile(r'[B-Q,S-Z]+[\d]*\t(.*)')
+ FindModifyFile = re.compile(r'\+\+\+ b\/(.*)')
+ LineScopePattern = (r'@@ -\d*\,*\d* \+\d*\,*\d* @@.*')
+ LineNumRange = re.compile(r'@@ -\d*\,*\d* \+(\d*)\,*(\d*) @@.*')
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check for efi coding style for " + packagename, packagename + ".EccCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ edk2_path = Edk2pathObj.WorkspacePath
+ python_path = os.path.join(edk2_path, "BaseTools", "Source", "Python")
+ env = shell_environment.GetEnvironment()
+ env.set_shell_var('PYTHONPATH', python_path)
+ env.set_shell_var('WORKSPACE', edk2_path)
+ self.ECC_PASS = True
+ self.ApplyConfig(pkgconfig, edk2_path, packagename)
+ modify_dir_list = self.GetModifyDir(packagename)
+ patch = self.GetDiff(packagename)
+ ecc_diff_range = self.GetDiffRange(patch, packagename, edk2_path)
+ self.GenerateEccReport(modify_dir_list, ecc_diff_range, edk2_path)
+ ecc_log = os.path.join(edk2_path, "Ecc.log")
+ if self.ECC_PASS:
+ tc.SetSuccess()
+ self.RemoveFile(ecc_log)
+ return 0
+ else:
+ with open(ecc_log, encoding='utf8') as output:
+ ecc_output = output.readlines()
+ for line in ecc_output:
+ logging.error(line.strip())
+ self.RemoveFile(ecc_log)
+ tc.SetFailed("EccCheck failed for {0}".format(packagename), "Ecc detected issues")
+ return 1
+
+ def GetDiff(self, pkg: str) -> List[str]:
+ return_buffer = StringIO()
+ params = "diff --unified=0 origin/master HEAD"
+ RunCmd("git", params, outstream=return_buffer)
+ p = return_buffer.getvalue().strip()
+ patch = p.split("\n")
+ return_buffer.close()
+
+ return patch
+
+ def RemoveFile(self, file: str) -> None:
+ if os.path.exists(file):
+ os.remove(file)
+ return
+
+ def GetModifyDir(self, pkg: str) -> List[str]:
+ return_buffer = StringIO()
+ params = "diff --name-status" + ' HEAD' + ' origin/master'
+ RunCmd("git", params, outstream=return_buffer)
+ p1 = return_buffer.getvalue().strip()
+ dir_list = p1.split("\n")
+ return_buffer.close()
+ modify_dir_list = []
+ for modify_dir in dir_list:
+ file_path = self.ReModifyFile.findall(modify_dir)
+ if file_path:
+ file_dir = os.path.dirname(file_path[0])
+ else:
+ continue
+ if pkg in file_dir and file_dir != pkg:
+ modify_dir_list.append('%s' % file_dir)
+ else:
+ continue
+
+ modify_dir_list = list(set(modify_dir_list))
+ return modify_dir_list
+
+ def GetDiffRange(self, patch_diff: List[str], pkg: str, workingdir: str) -> Dict[str, List[Tuple[int, int]]]:
+ IsDelete = True
+ StartCheck = False
+ range_directory: Dict[str, List[Tuple[int, int]]] = {}
+ for line in patch_diff:
+ modify_file = self.FindModifyFile.findall(line)
+ if modify_file and pkg in modify_file[0] and not StartCheck and os.path.isfile(modify_file[0]):
+ modify_file_comment_dic = self.GetCommentRange(modify_file[0], workingdir)
+ IsDelete = False
+ StartCheck = True
+ modify_file_dic = modify_file[0]
+ modify_file_dic = modify_file_dic.replace("/", os.sep)
+ range_directory[modify_file_dic] = []
+ elif line.startswith('--- '):
+ StartCheck = False
+ elif re.match(self.LineScopePattern, line, re.I) and not IsDelete and StartCheck:
+ start_line = self.LineNumRange.search(line).group(1)
+ line_range = self.LineNumRange.search(line).group(2)
+ if not line_range:
+ line_range = '1'
+ range_directory[modify_file_dic].append((int(start_line), int(start_line) + int(line_range) - 1))
+ for i in modify_file_comment_dic:
+ if int(i[0]) <= int(start_line) <= int(i[1]):
+ range_directory[modify_file_dic].append(i)
+ return range_directory
+
+ def GetCommentRange(self, modify_file: str, workingdir: str) -> List[Tuple[int, int]]:
+ modify_file_path = os.path.join(workingdir, modify_file)
+ with open(modify_file_path) as f:
+ line_no = 1
+ comment_range: List[Tuple[int, int]] = []
+ Start = False
+ for line in f:
+ if line.startswith('/**'):
+ start_no = line_no
+ Start = True
+ if line.startswith('**/') and Start:
+ end_no = line_no
+ Start = False
+ comment_range.append((int(start_no), int(end_no)))
+ line_no += 1
+
+ if comment_range and comment_range[0][0] == 1:
+ del comment_range[0]
+ return comment_range
+
+ def GenerateEccReport(self, modify_dir_list: List[str], ecc_diff_range: Dict[str, List[Tuple[int, int]]],
+ edk2_path: str) -> None:
+ ecc_need = False
+ ecc_run = True
+ config = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "config.ini")
+ exception = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "exception.xml")
+ report = os.path.join(edk2_path, "Ecc.csv")
+ for modify_dir in modify_dir_list:
+ target = os.path.join(edk2_path, modify_dir)
+ logging.info('Run ECC tool for the commit in %s' % modify_dir)
+ ecc_need = True
+ ecc_params = "-c {0} -e {1} -t {2} -r {3}".format(config, exception, target, report)
+ return_code = RunCmd("Ecc", ecc_params, workingdir=edk2_path)
+ if return_code != 0:
+ ecc_run = False
+ break
+ if not ecc_run:
+ logging.error('Fail to run ECC tool')
+ self.ParseEccReport(ecc_diff_range, edk2_path)
+
+ if not ecc_need:
+ logging.info("Doesn't need run ECC check")
+
+ revert_params = "checkout -- {}".format(exception)
+ RunCmd("git", revert_params)
+ return
+
+ def ParseEccReport(self, ecc_diff_range: Dict[str, List[Tuple[int, int]]], edk2_path: str) -> None:
+ ecc_log = os.path.join(edk2_path, "Ecc.log")
+ ecc_csv = "Ecc.csv"
+ file = os.listdir(edk2_path)
+ row_lines = []
+ ignore_error_code = self.GetIgnoreErrorCode()
+ if ecc_csv in file:
+ with open(ecc_csv) as csv_file:
+ reader = csv.reader(csv_file)
+ for row in reader:
+ for modify_file in ecc_diff_range:
+ if modify_file in row[3]:
+ for i in ecc_diff_range[modify_file]:
+ line_no = int(row[4])
+ if i[0] <= line_no <= i[1] and row[1] not in ignore_error_code:
+ row[0] = '\nEFI coding style error'
+ row[1] = 'Error code: ' + row[1]
+ row[3] = 'file: ' + row[3]
+ row[4] = 'Line number: ' + row[4]
+ row_line = '\n *'.join(row)
+ row_lines.append(row_line)
+ break
+ break
+ if row_lines:
+ self.ECC_PASS = False
+
+ with open(ecc_log, 'a') as log:
+ all_line = '\n'.join(row_lines)
+ all_line = all_line + '\n'
+ log.writelines(all_line)
+ return
+
+ def ApplyConfig(self, pkgconfig: Dict[str, List[str]], edk2_path: str, pkg: str) -> None:
+ if "IgnoreFiles" in pkgconfig:
+ for a in pkgconfig["IgnoreFiles"]:
+ a = os.path.join(edk2_path, pkg, a)
+ a = a.replace(os.sep, "/")
+
+ logging.info("Ignoring Files {0}".format(a))
+ if os.path.exists(a):
+ if os.path.isfile(a):
+ self.RemoveFile(a)
+ elif os.path.isdir(a):
+ shutil.rmtree(a)
+ else:
+ logging.error("EccCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore files".format(a))
+
+ if "ExceptionList" in pkgconfig:
+ exception_list = pkgconfig["ExceptionList"]
+ exception_xml = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "exception.xml")
+ try:
+ logging.info("Appending exceptions")
+ self.AppendException(exception_list, exception_xml)
+ except Exception as e:
+ logging.error("Fail to apply exceptions")
+ raise e
+ return
+
+ def AppendException(self, exception_list: List[str], exception_xml: str) -> None:
+ error_code_list = exception_list[::2]
+ keyword_list = exception_list[1::2]
+ dom_tree = xml.dom.minidom.parse(exception_xml)
+ root_node = dom_tree.documentElement
+ for error_code, keyword in zip(error_code_list, keyword_list):
+ customer_node = dom_tree.createElement("Exception")
+ keyword_node = dom_tree.createElement("KeyWord")
+ keyword_node_text_value = dom_tree.createTextNode(keyword)
+ keyword_node.appendChild(keyword_node_text_value)
+ customer_node.appendChild(keyword_node)
+ error_code_node = dom_tree.createElement("ErrorID")
+ error_code_text_value = dom_tree.createTextNode(error_code)
+ error_code_node.appendChild(error_code_text_value)
+ customer_node.appendChild(error_code_node)
+ root_node.appendChild(customer_node)
+ with open(exception_xml, 'w') as f:
+ dom_tree.writexml(f, indent='', addindent='', newl='\n', encoding='UTF-8')
+ return
+
+ def GetIgnoreErrorCode(self) -> set:
+ """
+ Below are kinds of error code that are accurate in ecc scanning of edk2 level.
+ But EccCheck plugin is partial scanning so they are always false positive issues.
+ The mapping relationship of error code and error message is listed BaseTools/Sourc/Python/Ecc/EccToolError.py
+ """
+ ignore_error_code = {
+ "10000",
+ "10001",
+ "10002",
+ "10003",
+ "10004",
+ "10005",
+ "10006",
+ "10007",
+ "10008",
+ "10009",
+ "10010",
+ "10011",
+ "10012",
+ "10013",
+ "10015",
+ "10016",
+ "10017",
+ "10022",
+ }
+ return ignore_error_code
diff --git a/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml b/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
new file mode 100644
index 000000000000..0d121ecb1861
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+## @file
+# CiBuildPlugin used to check Ecc issues
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "EccCheck Test",
+ "module": "EccCheck"
+}
diff --git a/.pytool/Plugin/EccCheck/Readme.md b/.pytool/Plugin/EccCheck/Readme.md
new file mode 100644
index 000000000000..85c439a5edf1
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/Readme.md
@@ -0,0 +1,15 @@
+# EFI Coding style Check Plugin
+
+This CiBuildPlugin finds the Ecc issues of newly added code in pull request.
+
+## Configuration
+
+The plugin can be configured to ignore certain files and issues.
+
+"EccCheck": {
+ "ExceptionList": [],
+ "IgnoreFiles": []
+ },
+ """
+
+OPTIONAL List of file to ignore.
--
2.18.0.windows.1