Date   
[PATCH v6 12/16] PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Ray Ni <ray.ni@...>
---
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
index be470807bd9e..c59287064d65 100644
--- a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
+++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for PcAtChipsetPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "PcAtChipsetPkg.dsc"
},
--
2.18.0.windows.1

[PATCH v6 11/16] OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jordan Justen <jordan.l.justen@...>
Cc: Laszlo Ersek <lersek@...>
Cc: Ard Biesheuvel <ard.biesheuvel@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Laszlo Ersek <lersek@...>
---
OvmfPkg/OvmfPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.ci.yaml b/OvmfPkg/OvmfPkg.ci.yaml
index 98992f0429ff..e151cebf7020 100644
--- a/OvmfPkg/OvmfPkg.ci.yaml
+++ b/OvmfPkg/OvmfPkg.ci.yaml
@@ -5,9 +5,20 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1

[PATCH v6 10/16] NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Maciej Rabeda <maciej.rabeda@...>
Cc: Jiaxin Wu <jiaxin.wu@...>
Cc: Siyuan Fu <@sfu5>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Maciej Rabeda <maciej.rabeda@...>
---
NetworkPkg/NetworkPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/NetworkPkg/NetworkPkg.ci.yaml b/NetworkPkg/NetworkPkg.ci.yaml
index 70f2e1014748..d45faf0ac967 100644
--- a/NetworkPkg/NetworkPkg.ci.yaml
+++ b/NetworkPkg/NetworkPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for NetworkPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "NetworkPkg.dsc"
},
--
2.18.0.windows.1

[PATCH v6 09/16] MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
MdePkg/MdePkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/MdePkg/MdePkg.ci.yaml b/MdePkg/MdePkg.ci.yaml
index 3268f1535499..b6d7c57de83c 100644
--- a/MdePkg/MdePkg.ci.yaml
+++ b/MdePkg/MdePkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for MdePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined ci/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "MdePkg.dsc"
--
2.18.0.windows.1

[PATCH v6 08/16] FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Liming Gao <liming.gao@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Guomin Jiang <guomin.jiang@...>
---
FmpDevicePkg/FmpDevicePkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/FmpDevicePkg/FmpDevicePkg.ci.yaml b/FmpDevicePkg/FmpDevicePkg.ci.yaml
index 74a0aefe8e49..b84400f06ffd 100644
--- a/FmpDevicePkg/FmpDevicePkg.ci.yaml
+++ b/FmpDevicePkg/FmpDevicePkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for FmpDevicePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "FmpDevicePkg.dsc"
},
--
2.18.0.windows.1

[PATCH v6 07/16] FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Guomin Jiang <guomin.jiang@...>
---
FatPkg/FatPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
index 8b0fb1d4fcd5..e6a4c810976f 100644
--- a/FatPkg/FatPkg.ci.yaml
+++ b/FatPkg/FatPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for FatPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "FatPkg.dsc"
},
--
2.18.0.windows.1

[PATCH v6 06/16] EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jordan Justen <jordan.l.justen@...>
Cc: Andrew Fish <afish@...>
Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Ray Ni <ray.ni@...>
---
EmulatorPkg/EmulatorPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/EmulatorPkg/EmulatorPkg.ci.yaml b/EmulatorPkg/EmulatorPkg.ci.yaml
index 81f81780ec76..a664f4462e5d 100644
--- a/EmulatorPkg/EmulatorPkg.ci.yaml
+++ b/EmulatorPkg/EmulatorPkg.ci.yaml
@@ -5,9 +5,20 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1

[PATCH v6 05/16] CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jian J Wang <jian.j.wang@...>
Cc: Xiaoyu Lu <xiaoyux.lu@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Guomin Jiang <guomin.jiang@...>
---
CryptoPkg/CryptoPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/CryptoPkg/CryptoPkg.ci.yaml b/CryptoPkg/CryptoPkg.ci.yaml
index f54ebfb22e70..c60d8bac6dee 100644
--- a/CryptoPkg/CryptoPkg.ci.yaml
+++ b/CryptoPkg/CryptoPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for CryptoPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "CryptoPkg.dsc"
},
--
2.18.0.windows.1

[PATCH v6 04/16] ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Laszlo Ersek <lersek@...>
Cc: Ard Biesheuvel <ard.biesheuvel@...>
Cc: Leif Lindholm <leif@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Laszlo Ersek <lersek@...>
---
ArmVirtPkg/ArmVirtPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
index 4553725ee528..3b321c806a62 100644
--- a/ArmVirtPkg/ArmVirtPkg.ci.yaml
+++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
@@ -5,9 +5,20 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1

[PATCH v6 03/16] MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check

Zhang, Shenglei
 

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Change-Id: I55c78eed392a8354a7201d17a591e4ff8dc3d6e9
Cc: Jian J Wang <jian.j.wang@...>
Cc: Hao A Wu <hao.a.wu@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
MdeModulePkg/MdeModulePkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml b/MdeModulePkg/MdeModulePkg.ci.yaml
index 1cfc1328390e..0ed929855417 100644
--- a/MdeModulePkg/MdeModulePkg.ci.yaml
+++ b/MdeModulePkg/MdeModulePkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for MdeModulePkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined ci/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "MdeModulePkg.dsc"
--
2.18.0.windows.1

[PATCH v6 02/16] .pytool/Plugin: Add a plugin EccCheck

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
EccCheck is a plugin to report Ecc issues for code in pull request
, which will be run on open ci.
But note not each kind of issue could be reported out.
It can only handle the issues, whose line number in CSV report
accurately map with their code in source code files. And Ecc issues
about comments can also be handled.

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
.pytool/Plugin/EccCheck/EccCheck.py | 268 ++++++++++++++++++
.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml | 11 +
.pytool/Plugin/EccCheck/Readme.md | 15 +
3 files changed, 294 insertions(+)
create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/EccCheck/Readme.md

diff --git a/.pytool/Plugin/EccCheck/EccCheck.py b/.pytool/Plugin/EccCheck/EccCheck.py
new file mode 100644
index 000000000000..ccdb66be616d
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/EccCheck.py
@@ -0,0 +1,268 @@
+# @file EccCheck.py
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import os
+import re
+import csv
+import xml.dom.minidom
+from typing import List, Dict, Tuple
+import logging
+from io import StringIO
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.utility_functions import RunCmd
+
+
+class EccCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that finds the Ecc issues of newly added code in pull request.
+
+ Configuration options:
+ "EccCheck": {
+ "ExceptionList": [],
+ "IgnoreFiles": []
+ },
+ """
+
+ ReModifyFile = re.compile(r'[B-Q,S-Z]+[\d]*\t(.*)')
+ FindModifyFile = re.compile(r'\+\+\+ b\/(.*)')
+ LineScopePattern = (r'@@ -\d*\,*\d* \+\d*\,*\d* @@.*')
+ LineNumRange = re.compile(r'@@ -\d*\,*\d* \+(\d*)\,*(\d*) @@.*')
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check for efi coding style for " + packagename, packagename + ".EccCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ edk2_path = Edk2pathObj.WorkspacePath
+ python_path = os.path.join(edk2_path, "BaseTools", "Source", "Python")
+ env = shell_environment.GetEnvironment()
+ env.set_shell_var('PYTHONPATH', python_path)
+ env.set_shell_var('WORKSPACE', edk2_path)
+ self.ECC_PASS = True
+ self.ApplyConfig(pkgconfig, edk2_path, packagename)
+ modify_dir_list = self.GetModifyDir(packagename)
+ patch = self.GetDiff(packagename)
+ ecc_diff_range = self.GetDiffRange(patch, packagename, edk2_path)
+ self.GenerateEccReport(modify_dir_list, ecc_diff_range, edk2_path)
+ ecc_log = os.path.join(edk2_path, "ecc.log")
+ if self.ECC_PASS:
+ tc.SetSuccess()
+ self.RemoveFile(ecc_log)
+ return 0
+ else:
+ with open(ecc_log, encoding='utf8') as output:
+ ecc_output = output.readlines()
+ for line in ecc_output:
+ logging.error(line)
+ self.RemoveFile(ecc_log)
+ tc.SetFailed("EccCheck failed for {0}".format(packagename), "Ecc detected issues")
+ return 1
+
+ def GetDiff(self, pkg: str) -> List[str]:
+ return_buffer = StringIO()
+ params = "diff --unified=0 origin/master HEAD"
+ RunCmd("git", params, outstream=return_buffer)
+ p = return_buffer.getvalue().strip()
+ patch = p.split("\n")
+ return_buffer.close()
+
+ return patch
+
+ def RemoveFile(self, file: str) -> None:
+ if os.path.exists(file):
+ os.remove(file)
+ return
+
+ def GetModifyDir(self, pkg: str) -> List[str]:
+ return_buffer = StringIO()
+ params = "diff --name-status" + ' HEAD' + ' origin/master'
+ RunCmd("git", params, outstream=return_buffer)
+ p1 = return_buffer.getvalue().strip()
+ dir_list = p1.split("\n")
+ return_buffer.close()
+ modify_dir_list = []
+ for modify_dir in dir_list:
+ file_path = self.ReModifyFile.findall(modify_dir)
+ if file_path:
+ file_dir = os.path.dirname(file_path[0])
+ else:
+ continue
+ if pkg in file_dir and file_dir != pkg:
+ modify_dir_list.append('%s' % file_dir)
+ else:
+ continue
+
+ modify_dir_list = list(set(modify_dir_list))
+ return modify_dir_list
+
+ def GetDiffRange(self, patch_diff: List[str], pkg: str, workingdir: str) -> Dict[str, List[Tuple[int, int]]]:
+ IsDelete = True
+ StartCheck = False
+ range_directory: Dict[str, List[Tuple[int, int]]] = {}
+ for line in patch_diff:
+ modify_file = self.FindModifyFile.findall(line)
+ if modify_file and pkg in modify_file[0] and not StartCheck and os.path.isfile(modify_file[0]):
+ modify_file_comment_dic = self.GetCommentRange(modify_file[0], workingdir)
+ IsDelete = False
+ StartCheck = True
+ modify_file_dic = modify_file[0]
+ modify_file_dic = modify_file_dic.replace("/", "\\")
+ range_directory[modify_file_dic] = []
+ elif line.startswith('--- '):
+ StartCheck = False
+ elif re.match(self.LineScopePattern, line, re.I) and not IsDelete and StartCheck:
+ start_line = self.LineNumRange.search(line).group(1)
+ line_range = self.LineNumRange.search(line).group(2)
+ if not line_range:
+ line_range = '1'
+ range_directory[modify_file_dic].append((int(start_line), int(start_line) + int(line_range) - 1))
+ for i in modify_file_comment_dic:
+ if int(i[0]) <= int(start_line) <= int(i[1]):
+ range_directory[modify_file_dic].append(i)
+ return range_directory
+
+ def GetCommentRange(self, modify_file: str, workingdir: str) -> List[Tuple[int, int]]:
+ modify_file_path = os.path.join(workingdir, modify_file)
+ with open(modify_file_path) as f:
+ line_no = 1
+ comment_range: List[Tuple[int, int]] = []
+ Start = False
+ for line in f:
+ if line.startswith('/**'):
+ start_no = line_no
+ Start = True
+ if line.startswith('**/') and Start:
+ end_no = line_no
+ Start = False
+ comment_range.append((int(start_no), int(end_no)))
+ line_no += 1
+
+ if comment_range and comment_range[0][0] == 1:
+ del comment_range[0]
+ return comment_range
+
+ def GenerateEccReport(self, modify_dir_list: List[str], ecc_diff_range: Dict[str, List[Tuple[int, int]]],
+ edk2_path: str) -> None:
+ ecc_need = False
+ ecc_run = True
+ ecc_tool = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "EccMain.py")
+ config = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "config.ini")
+ exception = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "exception.xml")
+ report = os.path.join(edk2_path, "Ecc.csv")
+ for modify_dir in modify_dir_list:
+ target = os.path.join(edk2_path, modify_dir)
+ logging.info('Run ECC tool for the commit in %s' % modify_dir)
+ ecc_need = True
+ ecc_params = "-3 {0} -c {1} -e {2} -t {3} -r {4}".format(ecc_tool, config, exception, target, report)
+ return_code = RunCmd("py", ecc_params, workingdir=edk2_path)
+ if return_code != 0:
+ ecc_run = False
+ break
+ if not ecc_run:
+ logging.error('Fail to run ECC tool')
+ self.ParseEccReport(ecc_diff_range, edk2_path)
+
+ if not ecc_need:
+ logging.info("Doesn't need run ECC check")
+
+ revert_params = "checkout -- {}".format(exception)
+ RunCmd("git", revert_params)
+ return
+
+ def ParseEccReport(self, ecc_diff_range: Dict[str, List[Tuple[int, int]]], edk2_path: str) -> None:
+ ecc_log = os.path.join(edk2_path, "Ecc.log")
+ ecc_csv = "Ecc.csv"
+ file = os.listdir(edk2_path)
+ row_lines = []
+ if ecc_csv in file:
+ with open(ecc_csv) as csv_file:
+ reader = csv.reader(csv_file)
+ for row in reader:
+ for modify_file in ecc_diff_range:
+ if modify_file in row[3]:
+ for i in ecc_diff_range[modify_file]:
+ line_no = int(row[4])
+ if i[0] <= line_no <= i[1]:
+ row[0] = '\nEFI coding style error'
+ row[1] = 'Error code: ' + row[1]
+ row[3] = 'file: ' + row[3]
+ row[4] = 'Line number: ' + row[4]
+ row_line = '\n *'.join(row)
+ row_lines.append(row_line)
+ break
+ break
+ if row_lines:
+ self.ECC_PASS = False
+
+ with open(ecc_log, 'a') as log:
+ all_line = '\n'.join(row_lines)
+ all_line = all_line + '\n'
+ log.writelines(all_line)
+ return
+
+ def ApplyConfig(self, pkgconfig: Dict[str, List[str]], edk2_path: str, pkg: str) -> None:
+ if "IgnoreFiles" in pkgconfig:
+ for a in pkgconfig["IgnoreFiles"]:
+ a = os.path.join(edk2_path, pkg, a)
+ a = a.replace(os.sep, "/")
+ try:
+ logging.info("Ignoring File {0}".format(a))
+ self.RemoveFile(a)
+ except Exception:
+ logging.info("EccCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ if "ExceptionList" in pkgconfig:
+ exception_list = pkgconfig["ExceptionList"]
+ exception_xml = os.path.join(edk2_path, "BaseTools", "Source", "Python", "Ecc", "exception.xml")
+ try:
+ logging.info("Appending exceptions")
+ self.AppendException(exception_list, exception_xml)
+ except Exception as e:
+ logging.error("Fail to apply exceptions")
+ raise e
+ return
+
+ def AppendException(self, exception_list: List[str], exception_xml: str) -> None:
+ error_code_list = exception_list[::2]
+ keyword_list = exception_list[1::2]
+ dom_tree = xml.dom.minidom.parse(exception_xml)
+ root_node = dom_tree.documentElement
+ for error_code, keyword in zip(error_code_list, keyword_list):
+ customer_node = dom_tree.createElement("Exception")
+ keyword_node = dom_tree.createElement("KeyWord")
+ keyword_node_text_value = dom_tree.createTextNode(keyword)
+ keyword_node.appendChild(keyword_node_text_value)
+ customer_node.appendChild(keyword_node)
+ error_code_node = dom_tree.createElement("ErrorID")
+ error_code_text_value = dom_tree.createTextNode(error_code)
+ error_code_node.appendChild(error_code_text_value)
+ customer_node.appendChild(error_code_node)
+ root_node.appendChild(customer_node)
+ with open(exception_xml, 'w') as f:
+ dom_tree.writexml(f, indent='', addindent='', newl='\n', encoding='UTF-8')
+ return
diff --git a/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml b/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
new file mode 100644
index 000000000000..0d121ecb1861
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+## @file
+# CiBuildPlugin used to check Ecc issues
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "EccCheck Test",
+ "module": "EccCheck"
+}
diff --git a/.pytool/Plugin/EccCheck/Readme.md b/.pytool/Plugin/EccCheck/Readme.md
new file mode 100644
index 000000000000..85c439a5edf1
--- /dev/null
+++ b/.pytool/Plugin/EccCheck/Readme.md
@@ -0,0 +1,15 @@
+# EFI Coding style Check Plugin
+
+This CiBuildPlugin finds the Ecc issues of newly added code in pull request.
+
+## Configuration
+
+The plugin can be configured to ignore certain files and issues.
+
+"EccCheck": {
+ "ExceptionList": [],
+ "IgnoreFiles": []
+ },
+ """
+
+OPTIONAL List of file to ignore.
--
2.18.0.windows.1

[PATCH v6 01/16] pip-requirements.txt: Add Ecc required lib

Zhang, Shenglei
 

antlr4-python3-runtime is a lib to support Ecc run with Py3.x.

Cc: Sean Brogan <sean.brogan@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
pip-requirements.txt | 1 +
1 file changed, 1 insertion(+)

diff --git a/pip-requirements.txt b/pip-requirements.txt
index 574dac43b1a6..0fecd37f2a83 100644
--- a/pip-requirements.txt
+++ b/pip-requirements.txt
@@ -14,3 +14,4 @@

edk2-pytool-library==0.10.*
edk2-pytool-extensions~=0.13.3
+antlr4-python3-runtime==4.7.1
--
2.18.0.windows.1

[PATCH v6 00/16] Add a plugin to check Ecc issues for edk2 on open ci

Zhang, Shenglei
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2606
As planed we will enable Ecc check for edk2 on open ci. And they are
ready now. I appreciate receiving feedback and comments if someone
find errors or false positive issues.

I created a pipline of EccCheck for my forked edk2. Welcome everyone to
create pull request to test the quality of this plugin.
My forked tree: https://github.com/shenglei10/edk2

And I also created some test cases for ECC plugin. Below are test cases.
https://github.com/shenglei10/edk2/tree/ECC
Results can be view in below azure server.
https://dev.azure.com/shengleizhang/shengleizhang/_build?definitionId=12&_a=summary

Patches
1/16: It's a lib necessary for py3 to run Ecc on azure servers.

2/16: EccCheck.py is a plugin to report Ecc issues for commits. It can be run
on azure servers for open ci, or a local virtual environment.

3/16~16/16: We consider some cases that will report out Ecc issues but they won't
be fixed, like submodule and industry standard related things. So we
add two configuration fields "Exception" and "IgnoreFiles" for people
to use. These patches add configuration in yaml files for Ecc check.

Cc: Bob Feng <bob.c.feng@...>
Cc: Bret Barkelew <Bret.Barkelew@...>
Cc: Michael D Kinney <michael.d.kinney@...>
Cc: Liming Gao <liming.gao@...>
Cc: Sean Brogan <sean.brogan@...>

v2: Update 1/17, fix the bug that the script can't hanlde multiple commits.

v3: Update 1/17, set the only workalbe workspace is edk2 root directory.
Update 2/17, designate the version of antlr4 is 4.7.1.
Add 4/17~17/17.

v4. Update 1/17, remove the function EdksetupRebuild(), instead add
function SetupEnvironment(). Update variables' format and type hints
to pass flake8 and mypy.

v5. Conver the former method to plugin solution, to align with
other check points on open ci.

v6. The 1/16 patch is missed in v5 series. Now add it in v6.

Shenglei Zhang (16):
pip-requirements.txt: Add Ecc required lib
.pytool/Plugin: Add a plugin EccCheck
MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for Ecc check
ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for Ecc check
CryptoPkg/CryptoPkg.ci.yaml: Add configuration for Ecc check
EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check
FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check
FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for Ecc check
MdePkg/MdePkg.ci.yaml: Add configuration for Ecc check
NetworkPkg/NetworkPkg.ci.yaml: Add configuration for Ecc check
OvmfPkg/OvmfPkg.ci.yaml: Add configuration for Ecc check
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check
SecurityPkg/SecurityPkg.ci.yaml: Add configuration for Ecc check
ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check
UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check
UnitTestFrameworkPkg: Add configuration for Ecc check in yaml file

.pytool/Plugin/EccCheck/EccCheck.py | 268 ++++++++++++++++++
.pytool/Plugin/EccCheck/EccCheck_plug_in.yaml | 11 +
.pytool/Plugin/EccCheck/Readme.md | 15 +
ArmVirtPkg/ArmVirtPkg.ci.yaml | 11 +
CryptoPkg/CryptoPkg.ci.yaml | 11 +
EmulatorPkg/EmulatorPkg.ci.yaml | 11 +
FatPkg/FatPkg.ci.yaml | 11 +
FmpDevicePkg/FmpDevicePkg.ci.yaml | 11 +
MdeModulePkg/MdeModulePkg.ci.yaml | 11 +
MdePkg/MdePkg.ci.yaml | 11 +
NetworkPkg/NetworkPkg.ci.yaml | 11 +
OvmfPkg/OvmfPkg.ci.yaml | 11 +
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 11 +
SecurityPkg/SecurityPkg.ci.yaml | 11 +
ShellPkg/ShellPkg.ci.yaml | 11 +
UefiCpuPkg/UefiCpuPkg.ci.yaml | 11 +
.../UnitTestFrameworkPkg.ci.yaml | 10 +
pip-requirements.txt | 1 +
18 files changed, 448 insertions(+)
create mode 100644 .pytool/Plugin/EccCheck/EccCheck.py
create mode 100644 .pytool/Plugin/EccCheck/EccCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/EccCheck/Readme.md

--
2.18.0.windows.1

Upcoming Event: TianoCore Bug Triage - APAC / NAMO - Tue, 06/30/2020 6:30pm-7:30pm #cal-reminder

devel@edk2.groups.io Calendar <devel@...>
 

Reminder: TianoCore Bug Triage - APAC / NAMO

When: Tuesday, 30 June 2020, 6:30pm to 7:30pm, (GMT-07:00) America/Los Angeles

Where:https://bluejeans.com/889357567?src=join_info

View Event

Organizer: Brian Richardson brian.richardson@...

Description:

https://www.tianocore.org/bug-triage

 

Meeting URL

https://bluejeans.com/889357567?src=join_info

 

Meeting ID

889 357 567

 

Want to dial in from a phone?

Dial one of the following numbers:

+1.408.740.7256 (US (San Jose))

+1.408.317.9253 (US (Primary, San Jose))

 

(see all numbers - https://www.bluejeans.com/numbers)

Enter the meeting ID and passcode followed by #

Re: [PATCH v5 11/15] PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check

Ni, Ray
 

Acked-by: Ray Ni <ray.ni@...>

-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@...>
Sent: Tuesday, June 30, 2020 8:00 PM
To: devel@edk2.groups.io
Cc: Ni, Ray <ray.ni@...>
Subject: [PATCH v5 11/15] PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for Ecc check

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Guomin Jiang <guomin.jiang@...>
---
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
index be470807bd9e..c59287064d65 100644
--- a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
+++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for PcAtChipsetPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "PcAtChipsetPkg.dsc"
},
--
2.18.0.windows.1

Re: [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Bob Feng
 

Hi Pierre,

Yes. if the patch set can't pass CI, it can't be merged.

I think we can add a check for the file path. The scripts under BinWrappers\PosixLike are all linux shell scripts.

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Tuesday, June 30, 2020 9:56 PM
To: Feng, Bob C <bob.c.feng@...>; devel@edk2.groups.io
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

I agree that PatchCheck.py needs to be updated. However, this is difficult for PatchCheck.py to know whether a file should be allowed to have LF line endings or not. Some files currently in edk2 have LF line endings and:
- have a ".sh" extension (EmulatorPkg/build.sh), some other don't (BaseTools/BinWrappers/PosixLike/GenFv);
- start with "#!/usr/bin/env bash" (BaseTools/BinWrappers/PosixLike/GenFv), some other don't (edksetup.sh); I don't see any criteria that would allow to identify whether a file is supposed to have LF line endings or not. This is why I think creating a ".gitattribute" file in edk2 would help. In this file, people could identify files/folders being allowed to have LF line endings. PatchCheck.py would parse this file to exclude them from the CRLF check. Plus it would enforce the line ending on local copies of edk2, thus preventing mistakes like I did when sending a bash script with CRLF line endings.
Do you agree with this solution?

Another question: does the CRLF error triggered by PatchCheck.py imply delaying this current patch set? Indeed I'm scared trying to add a ".gitattribute" file might start long discussions.

Regards,
Pierre

-----Original Message-----
From: Feng, Bob C <bob.c.feng@...>
Sent: 30 June 2020 11:59
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@...>
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Hi Pierre,

I think we need to update the PatchCheck.py script. The comments says, "Do not enforce CR/LF line endings for linux shell scripts.", but that only check the file ext is .sh is not enough.

if self.filename.endswith('.sh'):
#
# Do not enforce CR/LF line endings for linux shell scripts.
#
self.force_crlf = False

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, June 29, 2020 10:33 PM
To: Feng, Bob C <bob.c.feng@...>; devel@edk2.groups.io
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Hello Bob,
I actually found more files having a LF line ending by running the following command:
for file in `git ls-files`; do if [ "${file##*.}" != bin ] && [ "${file##*.}" != bmp ] && [ "${file##*.}" != a ] && [ "${file##*.}" != lib ] && [ "${file##*.}" != cer ] && [ "${file##*.}" != pyd ] &&[ "${file##*.}" != pem ] && [ "${file##*.}" != raw ]; then file $file | grep -v "CRLF" | grep -v ": directory"; fi; done The command list files referenced in git, then excludes files with specific extensions, excludes directories, and excludes files with CRLF line endings.

I believe some of them should have CRLF line endings if this is the default in edk2. I am mainly thinking about the ".rtf", ".py" and ".txt" files. For the actual bash scripts, maybe a ".gitattributes" file can be created in edk2 to explicitly make them use LF line endings. This can be done by setting this attribute: "eol=lf".
This ".gitattributes" file would then be parsed by the PatchCHeck.py script not to trigger an error for files having the later attribute.

Regards,
Pierre

List of files obtained by running the command above, (maybe it needs some more filters):
BaseTools/Bin/CYGWIN_NT-5.1-i686/BootSectImage: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/BuildEnv: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/Ecc: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiLdrImage: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/EfiRom: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenCrc32: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenDepex: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFds: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFfs: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFv: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenFw: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenPage: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenSec: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GenVtf: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/GnuGenBootSector: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaCompress: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/LzmaF86Compress: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/RunBinToolFromBuildDir: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/RunToolFromSource: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/Split: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/TargetTool: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/TianoCompress: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/Trim: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/VfrCompile: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/VolInfo: Bourne-Again shell script, ASCII text executable
BaseTools/Bin/CYGWIN_NT-5.1-i686/build: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/BPDG: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Brotli: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/BrotliCompress: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/DevicePath: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Ecc: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/EfiRom: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenCrc32: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenDepex: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenFds: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenFfs: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenFv: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenFw: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenPatchPcdTable: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenSec: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/GenerateCapsule: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/LzmaCompress: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/LzmaF86Compress: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/PatchPcdValue: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Pkcs7Sign: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Rsa2048Sha256GenerateKeys: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Rsa2048Sha256Sign: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Split: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/TargetTool: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/TianoCompress: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/Trim: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/UPT: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/VfrCompile: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/VolInfo: Bourne-Again shell script, ASCII text executable
BaseTools/BinWrappers/PosixLike/build: Bourne-Again shell script, ASCII text executable
BaseTools/BuildEnv: ASCII text
BaseTools/Source/C/VfrCompile/Pccts/KNOWN_PROBLEMS.txt: data
BaseTools/Source/C/VfrCompile/Pccts/history.ps: PostScript document text conforming DSC level 3.0
BaseTools/Source/Python/Ecc/CParser3/__init__.py: empty
BaseTools/Source/Python/Ecc/CParser4/__init__.py: empty
BaseTools/Source/Python/Eot/CParser3/__init__.py: empty
BaseTools/Source/Python/Eot/CParser4/__init__.py: empty
BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gEfiSecurityPkgTokenSpaceGuid.PcdPkcs7CertBuffer.inc: ASCII text, with very long lines, with no line terminators
BaseTools/Source/Python/Pkcs7Sign/TestRoot.cer.gFmpDevicePkgTokenSpaceGuid.PcdFmpDevicePkcs7CertBufferXdr.inc: ASCII text, with very long lines, with no line terminators
BaseTools/UserManuals/Build_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/EfiRom_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenCrc32_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenDepex_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenFds_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenFfs_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenFv_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenFw_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenPatchPcdTable_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/GenSec_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/Intel_UEFI_Packaging_Tool_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/LzmaCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/PatchPcdValue_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/SplitFile_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/TargetTool_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/TianoCompress_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/Trim_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/UtilityManPage_template.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/VfrCompiler_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
BaseTools/UserManuals/VolInfo_Utility_Man_Page.rtf: Rich Text Format data, version 1, unknown character set
CryptoPkg/Library/Include/internal/dso_conf.h: C source, ASCII text
CryptoPkg/Library/OpensslLib/process_files.pl: Perl script text executable
EmbeddedPkg/Library/FdtLib/TODO: ASCII text
EmbeddedPkg/Library/FdtLib/fdt_overlay.c: C source, ASCII text
EmulatorPkg/Unix/.gdbinit: ASCII text
EmulatorPkg/Unix/GdbRun.sh: ASCII text
EmulatorPkg/Unix/Host/X11IncludeHack: ASCII text, with no line terminators
EmulatorPkg/Unix/Xcode/xcode_project32/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable
EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/default.pbxuser: ASCII text
EmulatorPkg/Unix/Xcode/xcode_project32/xcode_project.xcodeproj/project.pbxproj: ASCII text
EmulatorPkg/Unix/Xcode/xcode_project64/XcodeBuild.sh: Bourne-Again shell script, ASCII text executable
EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/default.pbxuser: ASCII text
EmulatorPkg/Unix/Xcode/xcode_project64/xcode_project.xcodeproj/project.pbxproj: ASCII text
EmulatorPkg/Unix/lldbinit: ASCII text
EmulatorPkg/build.sh: Bourne-Again shell script, ASCII text executable
OvmfPkg/QemuVideoDxe/VbeShim.sh: POSIX shell script, ASCII text executable
OvmfPkg/build.sh: Bourne-Again shell script, ASCII text executable
edksetup.sh: ASCII text


-----Original Message-----
From: Feng, Bob C <bob.c.feng@...>
Sent: 28 June 2020 08:34
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@...>
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: RE: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Hi Pierre,

I met a problem when I push your patch set.

If I change the CRLF to a unix format EOL, the patch can't pass PatchCheck.py.
If I don't change the CRLF, The build on Linux will fail.
So I can't make the patch set pass the CI.

Could you share me how did you do to make your PR 729 pass?

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Thursday, June 25, 2020 5:31 PM
To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@...>
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Hello Bob,
I believe the line endings of the BaseTools/BinWrappers/PosixLike/AmlToC script have been modified to CRLF when I sent the patch. I created a pull request from the linked github branch noted in the patches. It is available at https://github.com/PierreARM/edk2/commits/803_Compile_AML_bytecode_array_into_OBJ_file_v3 . The pull request is available here (to show the result of the CI tests) https://github.com/tianocore/edk2/pull/729 .
Do you want a v4 or is it possible to pull the patches from the github repository?
Sorry for the inconvenience.

Regards,
Pierre

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng via groups.io
Sent: 24 June 2020 16:16
To: devel@edk2.groups.io; Pierre Gondois <Pierre.Gondois@...>
Cc: Sami Mujawar <Sami.Mujawar@...>; Tomas Pilar <Tomas.Pilar@...>; Gao, Liming <liming.gao@...>; nd <nd@...>
Subject: Re: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Hi Pierre,

There are some build failed in OpenCI. Would you please check it?
https://github.com/tianocore/edk2/pull/727

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Wednesday, June 24, 2020 5:09 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@...>; sami.mujawar@...; tomas.pilar@...; Feng, Bob C <bob.c.feng@...>; Gao, Liming <liming.gao@...>; nd@...
Subject: [edk2-devel] [PATCH v3 0/4] Compile AML bytecode array into OBJ file

Following the BZ at https://bugzilla.tianocore.org/show_bug.cgi?id=2425
This patch serie is a another way to solve the dependency of C files over ASL files. With this new method, the dependency is resolved at the linking stage.

The last method to solve this dependency was to add the possibility to modify INF files to depict such a dependency. This method was not accepted. The discussion is available at https://edk2.groups.io/g/devel/topic/72655342#56658

The last patch modifying the INF specification and INF parsing are available at:
https://edk2.groups.io/g/devel/topic/72655342#56658
https://edk2.groups.io/g/devel/topic/72656060#56662

Pierre Gondois (4):
BaseTools: Generate multiple rules when multiple output files
BaseTools: Rename AmlToHex script to AmlToC
BaseTools: Compile AML bytecode arrays into .obj file
BaseTools: Fix string concatenation

BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} | 28 +++----
BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} | 0
BaseTools/Conf/build_rule.template | 15 +++-
BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} | 82 ++++++++------------
BaseTools/Source/Python/AutoGen/BuildEngine.py | 2 +-
BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++
BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 +++++----
7 files changed, 89 insertions(+), 82 deletions(-) rename BaseTools/BinWrappers/PosixLike/{AmlToHex => AmlToC} (97%) rename BaseTools/BinWrappers/WindowsLike/{AmlToHex.bat => AmlToC.bat} (100%) rename BaseTools/Source/Python/{AmlToHex/AmlToHex.py => AmlToC/AmlToC.py} (52%)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'

Re: [PATCH v5 14/15] UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check

Ni, Ray
 

Acked-by: Ray Ni <ray.ni@...>

-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@...>
Sent: Tuesday, June 30, 2020 8:00 PM
To: devel@edk2.groups.io
Cc: Dong, Eric <eric.dong@...>; Ni, Ray <ray.ni@...>; Laszlo Ersek <lersek@...>
Subject: [PATCH v5 14/15] UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for Ecc check

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Eric Dong <eric.dong@...>
Cc: Ray Ni <ray.ni@...>
Cc: Laszlo Ersek <lersek@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Laszlo Ersek <lersek@...>
---
UefiCpuPkg/UefiCpuPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
index 99e460a8b090..0e216344cd53 100644
--- a/UefiCpuPkg/UefiCpuPkg.ci.yaml
+++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for UefiCpuPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "UefiCpuPkg.dsc"
},
--
2.18.0.windows.1

Re: [PATCH v5 13/15] ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check

Ni, Ray
 

Acked-by: Ray Ni <ray.ni@...>

-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@...>
Sent: Tuesday, June 30, 2020 8:00 PM
To: devel@edk2.groups.io
Cc: Ni, Ray <ray.ni@...>; Gao, Zhichao <zhichao.gao@...>
Subject: [PATCH v5 13/15] ShellPkg/ShellPkg.ci.yaml: Add configuration for Ecc check

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@...>
Cc: Zhichao Gao <zhichao.gao@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Acked-by: Zhichao gao <zhichao.gao@...>
---
ShellPkg/ShellPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml
index 67de34a2556e..eedc173738dc 100644
--- a/ShellPkg/ShellPkg.ci.yaml
+++ b/ShellPkg/ShellPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for ShellPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "ShellPkg.dsc"
},
--
2.18.0.windows.1

Re: [PATCH v5 06/15] FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check

Ni, Ray
 

Acked-by: Ray Ni <ray.ni@...>

-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@...>
Sent: Tuesday, June 30, 2020 8:00 PM
To: devel@edk2.groups.io
Cc: Ni, Ray <ray.ni@...>
Subject: [PATCH v5 06/15] FatPkg/FatPkg.ci.yaml: Add configuration for Ecc check

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
Reviewed-by: Guomin Jiang <guomin.jiang@...>
---
FatPkg/FatPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
index 8b0fb1d4fcd5..e6a4c810976f 100644
--- a/FatPkg/FatPkg.ci.yaml
+++ b/FatPkg/FatPkg.ci.yaml
@@ -2,9 +2,20 @@
# CI configuration for FatPkg
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
"CompilerPlugin": {
"DscPath": "FatPkg.dsc"
},
--
2.18.0.windows.1

Re: [PATCH v5 05/15] EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check

Ni, Ray
 

Acked-by: Ray Ni <ray.ni@...>

-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@...>
Sent: Tuesday, June 30, 2020 8:00 PM
To: devel@edk2.groups.io
Cc: Justen, Jordan L <jordan.l.justen@...>; Andrew Fish <afish@...>; Ni, Ray <ray.ni@...>
Subject: [PATCH v5 05/15] EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for Ecc check

Add configuration ExceptionList and IgnoreFiles for package config
files. So users can rely on this to ignore some Ecc issues.

Cc: Jordan Justen <jordan.l.justen@...>
Cc: Andrew Fish <afish@...>
Cc: Ray Ni <ray.ni@...>
Signed-off-by: Shenglei Zhang <shenglei.zhang@...>
---
EmulatorPkg/EmulatorPkg.ci.yaml | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/EmulatorPkg/EmulatorPkg.ci.yaml b/EmulatorPkg/EmulatorPkg.ci.yaml
index 81f81780ec76..a664f4462e5d 100644
--- a/EmulatorPkg/EmulatorPkg.ci.yaml
+++ b/EmulatorPkg/EmulatorPkg.ci.yaml
@@ -5,9 +5,20 @@
# used for code analysis.
#
# Copyright (c) Microsoft Corporation
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
# SPDX-License-Identifier: BSD-2-Clause-Patent
##
{
+ "EccCheck": {
+ ## Exception sample looks like below:
+ ## "ExceptionList": [
+ ## "<ErrorID>", "<KeyWord>"
+ ## ]
+ "ExceptionList": [
+ ],
+ "IgnoreFiles": [
+ ]
+ },
## options defined .pytool/Plugin/CompilerPlugin
"CompilerPlugin": {
"DscPath": "" # Don't support this test
--
2.18.0.windows.1