Date   

Re: [PATCH 1/3 v6] Drivers/OpTeeRpmb: Add an OP-TEE backed RPMB driver

Ilias Apalodimas
 

Hi Pierre,

On Wed, Mar 10, 2021 at 09:58:19AM +0000, Pierre wrote:
Hi Ilias,
Just minor coding style nit picking:

* Drivers/OpTee/OpteeRpmbPkg/FixupPcd.c:FixPcdMemory() I think the
error codes are missing in the function header
Ah you mean the return values of locate protocol?

* Thanks for adding the IN/OUT function parameter descriptors. Is it
possible to also add them in the function headers [1] ?
Sure, I'll send a v7 anyways since I managed to mess up the maintainers patch
somehow...

I hope I haven't missed any of your other requests.


About the FFA/SVC call:

If this is an FFA call, is it possible to:
  - put a reference in the header to the spec (it should be similar to the
one at
edk2/ArmPkg/Library/StandaloneMmMmuLib/AArch64/ArmMmuStandaloneMmLib.c)
  - check the return status of the SVC call against the ones available at
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h
  - if possible, remove the dependency to <IndustryStandard/ArmMmSvc.h>
The call is technically an FFA one but at the moment OP-TEE returns the
StMM
return code which is defined in the last header you mention.
The relevant code is in ./core/arch/arm/kernel/stmm_sp.c function
tee2stmm_ret_val().
So unless we redefine that in OP-TEE or (better imho), wait for a full FFA
mechanism to be in place, I'd prefer leaving it as is.
Keep in mind that adding the full FFA will also get rid of the hardcoded
IDs
on the beginning of the file.
The description of a FFA_MSG_SEND_DIRECT_REQ (s10.2 of [2]) doesn't seem to
return the same error codes as the ones optee returns (in
optee_os/core/arch/arm/kernel/stmm_sp.c:tee2stmm_ret_val()). I am not sure a
new FFA specification will change these error codes.
Another thing is that I think the mMemMgrId variable described in
edk2-platforms/Drivers/OpTee/OpteeRpmbPkg/OpTeeRpmbFvb.c is currently
defined as edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h:ARM_FFA_DESTINATION_ENDPOINT_ID
(the name seems to be misleading).
I think it would be better to:

* align the optee error codes with what is in the FFA spec
* handle these error codes in edk2 with what is in
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h and remove the
dependendy to edk2/ArmPkg/Include/IndustryStandard/ArmMmSvc.h if
possible
* rename
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h:ARM_FFA_DESTINATION_ENDPOINT_ID
define to a proper name, according to what is in
optee_os/core/arch/arm/kernel/stmm_sp.c, and add one define for
'mem_mgr_id'
* remove the mMemMgrId and mStorageId variables from
edk2-platforms/Drivers/OpTee/OpteeRpmbPkg/OpTeeRpmbFvb.c and use the
newly created defines from
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h

This would allow to be aligned with the current FFA spec and when a new one
comes, these endpoints IDs (mMemMgrId/mStorageId) can just be removed from
one location (as you said).
In the end it seems you and Sami will maintain this, so I will let you
decide what is best.
I get the whole point, and for the record you are technically right.

But there's something (once again) that's 'weird' here.
This StandAloneMM that's compiling over here is only used by OP-TEE.
In order to use that you need to call an SMC into OP-TEE (not FFA) from
the non-secure world to initiate it.
There's an OP-TEE PTA (pseudo-TA), that then converts the message to MM and
sends it over to StandAloneMM. There are no FFA manifests yet, that's why the
get/set memory attributes code is still running, to set up page permissions
as well.

The FFA mechanism you are seeing right now, is just the internal contract
between OP-TEE and this driver. We did some of the calls depend on FFA since
this was a good way to start introducing FFA code into EDK2 (which will
eventually be needed), without being too intrusive.

In the long run OP-TEE will be fully converted into FFA the changes you are
talking about make sense. In fact there's a ./core/arch/arm/kernel/secure_partition.c
in OP-TEE doing exactly that but it's not yet complete.
I tried to describe the entire situation here [1].

If anyone feels really strong about this, we can go and change it. The changes
aren't too big to begin with. That being said I'd prefer keeping it as is,
since this will naturally evolve to a complete FFA spec, but the mechanisms
are still missing from OP-TEE. Last but not least when OP-TEE gets that's FFA
support you won't bundle StandAloneMM with the driver right? You'd have 2
discrete Secure partitions, one dealing with variables and one dealing with
storage.

[1] https://apalos.github.io/Protected%20UEFI%20variables%20with%20U-Boot.html#Protected%20UEFI%20variables%20with%20U-Boot


Thanks
/Ilias

[1] https://edk2-docs.gitbook.io/edk-ii-c-coding-standards-specification/6_documenting_software/610_special_commands#6-10-4-param-in-out-parameter-description
[2] https://developer.arm.com/documentation/den0077/a

Regards,
Pierre


Re: [PATCH v4] UefiCpuPkg/PiSmmCpu: Don't allocate Token for SmmStartupThisAp

Laszlo Ersek
 

On 03/10/21 11:12, Ray Ni wrote:
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3199

When Token points to mSmmStartupThisApToken, this routine is called
from SmmStartupThisAp() in non-blocking mode due to
PcdCpuSmmBlockStartupThisAp == FALSE.

In this case, caller wants to startup AP procedure in non-blocking
mode and cannot get the completion status from the Token because there
is no way to return the Token to caller from SmmStartupThisAp().
Caller needs to use its specific way to query the completion status.

There is no need to allocate a token for such case so the 3 overheads
can be avoided:
1. Call AllocateTokenBuffer() when there is no free token.
2. Get a free token from the token buffer.
3. Call ReleaseToken() in APHandler().

Signed-off-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
---
UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 30 ++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)
Thanks for the update, Ray!
Laszlo


diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c
index 6227b2428a..fd6583f9d1 100644
--- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c
+++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c
@@ -1,7 +1,7 @@
/** @file
SMM MP service implementation

-Copyright (c) 2009 - 2020, Intel Corporation. All rights reserved.<BR>
+Copyright (c) 2009 - 2021, Intel Corporation. All rights reserved.<BR>
Copyright (c) 2017, AMD Incorporated. All rights reserved.<BR>

SPDX-License-Identifier: BSD-2-Clause-Patent
@@ -22,6 +22,7 @@ UINTN mSemaphoreSize;
SPIN_LOCK *mPFLock = NULL;
SMM_CPU_SYNC_MODE mCpuSmmSyncMode;
BOOLEAN mMachineCheckSupported = FALSE;
+MM_COMPLETION mSmmStartupThisApToken;

extern UINTN mSmmShadowStackSize;

@@ -1240,9 +1241,26 @@ InternalSmmStartupThisAp (
mSmmMpSyncData->CpuData[CpuIndex].Procedure = Procedure;
mSmmMpSyncData->CpuData[CpuIndex].Parameter = ProcArguments;
if (Token != NULL) {
- ProcToken= GetFreeToken (1);
- mSmmMpSyncData->CpuData[CpuIndex].Token = ProcToken;
- *Token = (MM_COMPLETION)ProcToken->SpinLock;
+ if (Token != &mSmmStartupThisApToken) {
+ //
+ // When Token points to mSmmStartupThisApToken, this routine is called
+ // from SmmStartupThisAp() in non-blocking mode (PcdCpuSmmBlockStartupThisAp == FALSE).
+ //
+ // In this case, caller wants to startup AP procedure in non-blocking
+ // mode and cannot get the completion status from the Token because there
+ // is no way to return the Token to caller from SmmStartupThisAp().
+ // Caller needs to use its implementation specific way to query the completion status.
+ //
+ // There is no need to allocate a token for such case so the 3 overheads
+ // can be avoided:
+ // 1. Call AllocateTokenBuffer() when there is no free token.
+ // 2. Get a free token from the token buffer.
+ // 3. Call ReleaseToken() in APHandler().
+ //
+ ProcToken = GetFreeToken (1);
+ mSmmMpSyncData->CpuData[CpuIndex].Token = ProcToken;
+ *Token = (MM_COMPLETION)ProcToken->SpinLock;
+ }
}
mSmmMpSyncData->CpuData[CpuIndex].Status = CpuStatus;
if (mSmmMpSyncData->CpuData[CpuIndex].Status != NULL) {
@@ -1474,8 +1492,6 @@ SmmStartupThisAp (
IN OUT VOID *ProcArguments OPTIONAL
)
{
- MM_COMPLETION Token;
-
gSmmCpuPrivate->ApWrapperFunc[CpuIndex].Procedure = Procedure;
gSmmCpuPrivate->ApWrapperFunc[CpuIndex].ProcedureArgument = ProcArguments;

@@ -1486,7 +1502,7 @@ SmmStartupThisAp (
ProcedureWrapper,
CpuIndex,
&gSmmCpuPrivate->ApWrapperFunc[CpuIndex],
- FeaturePcdGet (PcdCpuSmmBlockStartupThisAp) ? NULL : &Token,
+ FeaturePcdGet (PcdCpuSmmBlockStartupThisAp) ? NULL : &mSmmStartupThisApToken,
0,
NULL
);


[PATCH v4] UefiCpuPkg/PiSmmCpu: Don't allocate Token for SmmStartupThisAp

Ni, Ray
 

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3199

When Token points to mSmmStartupThisApToken, this routine is called
from SmmStartupThisAp() in non-blocking mode due to
PcdCpuSmmBlockStartupThisAp =3D=3D FALSE.

In this case, caller wants to startup AP procedure in non-blocking
mode and cannot get the completion status from the Token because there
is no way to return the Token to caller from SmmStartupThisAp().
Caller needs to use its specific way to query the completion status.

There is no need to allocate a token for such case so the 3 overheads
can be avoided:
1. Call AllocateTokenBuffer() when there is no free token.
2. Get a free token from the token buffer.
3. Call ReleaseToken() in APHandler().

Signed-off-by: Ray Ni <ray.ni@intel.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
---
UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 30 ++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)

diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxe=
Smm/MpService.c
index 6227b2428a..fd6583f9d1 100644
--- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c
+++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c
@@ -1,7 +1,7 @@
/** @file=0D
SMM MP service implementation=0D
=0D
-Copyright (c) 2009 - 2020, Intel Corporation. All rights reserved.<BR>=0D
+Copyright (c) 2009 - 2021, Intel Corporation. All rights reserved.<BR>=0D
Copyright (c) 2017, AMD Incorporated. All rights reserved.<BR>=0D
=0D
SPDX-License-Identifier: BSD-2-Clause-Patent=0D
@@ -22,6 +22,7 @@ UINTN mSemaphoreSiz=
e;
SPIN_LOCK *mPFLock =3D NULL;=0D
SMM_CPU_SYNC_MODE mCpuSmmSyncMode;=0D
BOOLEAN mMachineCheckSupported =3D FAL=
SE;=0D
+MM_COMPLETION mSmmStartupThisApToken;=0D
=0D
extern UINTN mSmmShadowStackSize;=0D
=0D
@@ -1240,9 +1241,26 @@ InternalSmmStartupThisAp (
mSmmMpSyncData->CpuData[CpuIndex].Procedure =3D Procedure;=0D
mSmmMpSyncData->CpuData[CpuIndex].Parameter =3D ProcArguments;=0D
if (Token !=3D NULL) {=0D
- ProcToken=3D GetFreeToken (1);=0D
- mSmmMpSyncData->CpuData[CpuIndex].Token =3D ProcToken;=0D
- *Token =3D (MM_COMPLETION)ProcToken->SpinLock;=0D
+ if (Token !=3D &mSmmStartupThisApToken) {=0D
+ //=0D
+ // When Token points to mSmmStartupThisApToken, this routine is call=
ed=0D
+ // from SmmStartupThisAp() in non-blocking mode (PcdCpuSmmBlockStart=
upThisAp =3D=3D FALSE).=0D
+ //=0D
+ // In this case, caller wants to startup AP procedure in non-blockin=
g=0D
+ // mode and cannot get the completion status from the Token because =
there=0D
+ // is no way to return the Token to caller from SmmStartupThisAp().=
=0D
+ // Caller needs to use its implementation specific way to query the =
completion status.=0D
+ //=0D
+ // There is no need to allocate a token for such case so the 3 overh=
eads=0D
+ // can be avoided:=0D
+ // 1. Call AllocateTokenBuffer() when there is no free token.=0D
+ // 2. Get a free token from the token buffer.=0D
+ // 3. Call ReleaseToken() in APHandler().=0D
+ //=0D
+ ProcToken =3D GetFreeToken (1);=0D
+ mSmmMpSyncData->CpuData[CpuIndex].Token =3D ProcToken;=0D
+ *Token =3D (MM_COMPLETION)ProcToken->SpinLock;=0D
+ }=0D
}=0D
mSmmMpSyncData->CpuData[CpuIndex].Status =3D CpuStatus;=0D
if (mSmmMpSyncData->CpuData[CpuIndex].Status !=3D NULL) {=0D
@@ -1474,8 +1492,6 @@ SmmStartupThisAp (
IN OUT VOID *ProcArguments OPTIONAL=0D
)=0D
{=0D
- MM_COMPLETION Token;=0D
-=0D
gSmmCpuPrivate->ApWrapperFunc[CpuIndex].Procedure =3D Procedure;=0D
gSmmCpuPrivate->ApWrapperFunc[CpuIndex].ProcedureArgument =3D ProcArgume=
nts;=0D
=0D
@@ -1486,7 +1502,7 @@ SmmStartupThisAp (
ProcedureWrapper,=0D
CpuIndex,=0D
&gSmmCpuPrivate->ApWrapperFunc[CpuIndex],=0D
- FeaturePcdGet (PcdCpuSmmBlockStartupThisAp) ? NULL : &Token,=0D
+ FeaturePcdGet (PcdCpuSmmBlockStartupThisAp) ? NULL : &mSmmStartupThisA=
pToken,=0D
0,=0D
NULL=0D
);=0D
--=20
2.27.0.windows.1


Re: [PATCH 1/3 v6] Drivers/OpTeeRpmb: Add an OP-TEE backed RPMB driver

PierreGondois
 

Hi Ilias,
Just minor coding style nit picking:

* Drivers/OpTee/OpteeRpmbPkg/FixupPcd.c:FixPcdMemory() I think the
error codes are missing in the function header
* Thanks for adding the IN/OUT function parameter descriptors. Is it
possible to also add them in the function headers [1] ?

About the FFA/SVC call:

If this is an FFA call, is it possible to:
  - put a reference in the header to the spec (it should be similar
to the
one at
edk2/ArmPkg/Library/StandaloneMmMmuLib/AArch64/ArmMmuStandaloneMmLib.c)
  - check the return status of the SVC call against the ones available at
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h
  - if possible, remove the dependency to <IndustryStandard/ArmMmSvc.h>
The call is technically an FFA one but at the moment OP-TEE returns
the StMM
return code which is defined in the last header you mention.
The relevant code is in ./core/arch/arm/kernel/stmm_sp.c function
tee2stmm_ret_val().
So unless we redefine that in OP-TEE or (better imho), wait for a
full FFA
mechanism to be in place, I'd prefer leaving it as is.
Keep in mind that adding the full FFA will also get rid of the
hardcoded IDs
on the beginning of the file.
The description of a FFA_MSG_SEND_DIRECT_REQ (s10.2 of [2]) doesn't seem to return the same error codes as the ones optee returns (in optee_os/core/arch/arm/kernel/stmm_sp.c:tee2stmm_ret_val()). I am not sure a new FFA specification will change these error codes.
Another thing is that I think the mMemMgrId variable described in edk2-platforms/Drivers/OpTee/OpteeRpmbPkg/OpTeeRpmbFvb.c is currently defined as edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h:ARM_FFA_DESTINATION_ENDPOINT_ID (the name seems to be misleading).
I think it would be better to:

* align the optee error codes with what is in the FFA spec
* handle these error codes in edk2 with what is in
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h and remove the
dependendy to edk2/ArmPkg/Include/IndustryStandard/ArmMmSvc.h if
possible
* rename
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h:ARM_FFA_DESTINATION_ENDPOINT_ID
define to a proper name, according to what is in
optee_os/core/arch/arm/kernel/stmm_sp.c, and add one define for
'mem_mgr_id'
* remove the mMemMgrId and mStorageId variables from
edk2-platforms/Drivers/OpTee/OpteeRpmbPkg/OpTeeRpmbFvb.c and use the
newly created defines from
edk2/ArmPkg/Include/IndustryStandard/ArmFfaSvc.h

This would allow to be aligned with the current FFA spec and when a new one comes, these endpoints IDs (mMemMgrId/mStorageId) can just be removed from one location (as you said).
In the end it seems you and Sami will maintain this, so I will let you decide what is best.

[1] https://edk2-docs.gitbook.io/edk-ii-c-coding-standards-specification/6_documenting_software/610_special_commands#6-10-4-param-in-out-parameter-description
[2] https://developer.arm.com/documentation/den0077/a

Regards,
Pierre


Re: [Patch edk2-platforms V2] Intel/BoardModulePkg: sort load option in the first boot

Ni, Ray
 

1. DataSIze should be set to sizeof (BOOLEAN) before calling GetVariable()

+ Status = gRT->GetVariable (
+ L"IsFirstBoot",
2. Can you please define a macro in this C file for IsFirstBoot string?
e.g.: #define IS_FIRST_BOOT_VAR_NAME L"IsFirstBoot"

+ if (IsFirstBoot == TRUE) {
3. Please remove "== TRUE". Just use "If (IsFirstBoot)".

+ L"IsFirstBoot",
4. Please use the macro defined as above.


+ &gEfiCallerIdGuid,

+ EFI_VARIABLE_NON_VOLATILE |
EFI_VARIABLE_RUNTIME_ACCESS | EFI_VARIABLE_BOOTSERVICE_ACCESS,
5. Please remove "EFI_VARIABLE_RUNTIME_ACCESS".

+ 1,
6. Please use sizeof (BOOLEAN) instead of "1".


Re: TianoCore Open Design Meeting

Ni, Ray
 

Please ignore this meeting request sent from Outlook. I will create meetings in groups.io.


Updated Event: TianoCore Design Meeting - APAC/NAMO - Friday, 19 March 2021 #cal-invite

devel@edk2.groups.io Calendar <noreply@...>
 

TianoCore Design Meeting - APAC/NAMO

When:
Friday, 19 March 2021
9:30am to 10:30am
(UTC+08:00) Asia/Shanghai

Where:
Microsoft Teams Meeting

Organizer: Ray Ni ray.ni@...

Description:

TOPIC

  1. NA

For more info, see here: https://www.tianocore.org/design-meeting/


Microsoft Teams meeting

Join on your computer or mobile app

Click here to join the meeting

Join with a video conferencing device

teams@...

Video Conference ID: 119 699 030 8

Alternate VTC dialing instructions

Learn More | Meeting options


Canceled: TianoCore Open Design Meeting

Ni, Ray
 

 
________________________________________________________________________________
Microsoft Teams meeting
Join on your computer or mobile app
Join with a video conferencing device
teams@...
Video Conference ID: 119 715 416 0
________________________________________________________________________________
 
 


Updated Event: TianoCore Design Meeting - APAC/NAMO #cal-invite

devel@edk2.groups.io Calendar <noreply@...>
 

TianoCore Design Meeting - APAC/NAMO

When:
Friday, 19 March 2021
9:30am to 10:30am
(UTC+08:00) Asia/Shanghai
Repeats: Every 2 weeks on Friday, through Thursday, 1 April 2021

Where:
https://intel.webex.com/intel/j.php?MTID=me894224717bbf0af843f2ff7502c98cb

Organizer: Ray Ni ray.ni@...

Description:

TOPIC

  • TBD

For more info, see here: https://www.tianocore.org/design-meeting/

Join Webex Meeting

Meeting link: https://intel.webex.com/intel/j.php?MTID=me894224717bbf0af843f2ff7502c98cb

Meeting number: 130 603 1757

Password: C3tRPJPV86@

Host key: 991045

More ways to join

Join by video system Dial 1306031757@...

You can also dial 173.243.2.68 and enter your meeting number. Join by phone

+1-210-795-1110 US Toll

+1-866-662-9987 US Toll Free

Access code: 130 603 1757

(Webex can be downloaded from https://www.webex.com/downloads.html/.)


Event: TianoCore Design Meeting - APAC/NAMO #cal-invite

devel@edk2.groups.io Calendar <noreply@...>
 

TianoCore Design Meeting - APAC/NAMO

When:
Friday, 19 March 2021
9:30am to 10:30am
(UTC+08:00) Asia/Shanghai
Repeats: Every 2 weeks on Friday

Where:
Microsoft Teams

Organizer: Ray Ni ray.ni@...

Description:

TOPIC

  1. NA

For more info, see here: https://www.tianocore.org/design-meeting/


Microsoft Teams meeting

Join on your computer or mobile app

Click here to join the meeting

Join with a video conferencing device

teams@...

Video Conference ID: 119 715 416 0

Alternate VTC dialing instructions

Learn More | Meeting options


Updated Event: TianoCore Design Meeting - APAC/NAMO - Friday, 19 March 2021 #cal-invite

devel@edk2.groups.io Calendar <noreply@...>
 

TianoCore Design Meeting - APAC/NAMO

When:
Friday, 19 March 2021
9:30am to 10:30am
(UTC+08:00) Asia/Shanghai

Where:
Microsoft Teams

Organizer: Ray Ni ray.ni@...

Description:

TOPIC

TBD

For more info, see here: https://www.tianocore.org/design-meeting/


Microsoft Teams meeting

Join on your computer or mobile app

Click here to join the meeting

Join with a video conferencing device

teams@...

Video Conference ID: 119 715 416 0

Alternate VTC dialing instructions

Learn More | Meeting options


TianoCore Open Design Meeting

Ni, Ray
 

 
________________________________________________________________________________
Microsoft Teams meeting
Join on your computer or mobile app
Join with a video conferencing device
Video Conference ID: 119 715 416 0
________________________________________________________________________________
 
 


TianoCore Open Design Meeting

Ni, Ray
 

 
 


[PATCH] IntelFsp2Pkg: Add Config Editor tool support

 

This is a GUI interface that can be used by users who
would like to change configuration settings directly
from the interface without having to modify the source.

This tool depends on Python GUI tool kit Tkinter.
It runs on both Windows and Linux.

The user needs to load the YAML file along with DLT file
for a specific board into the ConfigEditor, change the desired
configuration values. Finally, generate a new configuration delta
file or a config binary blob for the newly changed values to take
effect. These will be the inputs to the merge tool or the stitch
tool so that new config changes can be merged and stitched into
the final configuration blob.

Running Configuration Editor:
python ConfigEditor.py

Co-authored-by: Maurice Ma <maurice.ma@intel.com>
Cc: Maurice Ma <maurice.ma@intel.com>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Chasel Chiu <chasel.chiu@intel.com>
Signed-off-by: Loo Tung Lun <tung.lun.loo@intel.com>
---
IntelFsp2Pkg/Tools/ConfigEditor/CommonUtility.py | 456 ++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
IntelFsp2Pkg/Tools/ConfigEditor/ConfigEditor.py | 941 ++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
IntelFsp2Pkg/Tools/ConfigEditor/GenCfgData.py | 1962 ++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
++++++++++++++++++++++++
IntelFsp2Pkg/Tools/ConfigEditor/SingleSign.py | 301 ++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
IntelFsp2Pkg/Tools/UserManuals/ConfigEditorUserManual.md | 19 ++++++++++
5 files changed, 3679 insertions(+)

diff --git a/IntelFsp2Pkg/Tools/ConfigEditor/CommonUtility.py b/IntelFsp2Pk=
g/Tools/ConfigEditor/CommonUtility.py
new file mode 100644
index 0000000000..32b3e510c7
--- /dev/null
+++ b/IntelFsp2Pkg/Tools/ConfigEditor/CommonUtility.py
@@ -0,0 +1,456 @@
+#!/usr/bin/env python=0D
+## @ CommonUtility.py=0D
+# Common utility script=0D
+#=0D
+# Copyright (c) 2016 - 2020, Intel Corporation. All rights reserved.<BR>=0D
+# SPDX-License-Identifier: BSD-2-Clause-Patent=0D
+#=0D
+##=0D
+=0D
+##=0D
+# Import Modules=0D
+#=0D
+import os=0D
+import sys=0D
+import re=0D
+import shutil=0D
+import subprocess=0D
+import struct=0D
+import hashlib=0D
+import string=0D
+from ctypes import *=0D
+from functools import reduce=0D
+from importlib.machinery import SourceFileLoader=0D
+from SingleSign import *=0D
+=0D
+=0D
+# Key types defined should match with cryptolib.h=0D
+PUB_KEY_TYPE =3D {=0D
+ # key_type : key_val=0D
+ "RSA" : 1,=0D
+ "ECC" : 2,=0D
+ "DSA" : 3,=0D
+ }=0D
+=0D
+# Signing type schemes defined should match with cryptolib.h=0D
+SIGN_TYPE_SCHEME =3D {=0D
+ # sign_type : key_val=0D
+ "RSA_PKCS1" : 1,=0D
+ "RSA_PSS" : 2,=0D
+ "ECC" : 3,=0D
+ "DSA" : 4,=0D
+ }=0D
+=0D
+# Hash values defined should match with cryptolib.h=0D
+HASH_TYPE_VALUE =3D {=0D
+ # Hash_string : Hash_Value=0D
+ "SHA2_256" : 1,=0D
+ "SHA2_384" : 2,=0D
+ "SHA2_512" : 3,=0D
+ "SM3_256" : 4,=0D
+ }=0D
+=0D
+# Hash values defined should match with cryptolib.h=0D
+HASH_VAL_STRING =3D dict(map(reversed, HASH_TYPE_VALUE.items()))=0D
+=0D
+AUTH_TYPE_HASH_VALUE =3D {=0D
+ # Auth_type : Hash_type=0D
+ "SHA2_256" : 1,=0D
+ "SHA2_384" : 2,=0D
+ "SHA2_512" : 3,=0D
+ "SM3_256" : 4,=0D
+ "RSA2048SHA256" : 1,=0D
+ "RSA3072SHA384" : 2,=0D
+ }=0D
+=0D
+HASH_DIGEST_SIZE =3D {=0D
+ # Hash_string : Hash_Size=0D
+ "SHA2_256" : 32,=0D
+ "SHA2_384" : 48,=0D
+ "SHA2_512" : 64,=0D
+ "SM3_256" : 32,=0D
+ }=0D
+=0D
+class PUB_KEY_HDR (Structure):=0D
+ _pack_ =3D 1=0D
+ _fields_ =3D [=0D
+ ('Identifier', ARRAY(c_char, 4)), #signature ('P', 'U', 'B', =
'K')=0D
+ ('KeySize', c_uint16), #Length of Public Key=0D
+ ('KeyType', c_uint8), #RSA or ECC=0D
+ ('Reserved', ARRAY(c_uint8, 1)),=0D
+ ('KeyData', ARRAY(c_uint8, 0)), #Pubic key data with KeySize =
bytes for RSA_KEY() format=0D
+ ]=0D
+=0D
+ def __init__(self):=0D
+ self.Identifier =3D b'PUBK'=0D
+=0D
+class SIGNATURE_HDR (Structure):=0D
+ _pack_ =3D 1=0D
+ _fields_ =3D [=0D
+ ('Identifier', ARRAY(c_char, 4)), #signature Identifier('S', =
'I', 'G', 'N')=0D
+ ('SigSize', c_uint16), #Length of signature 2K and=
3K in bytes=0D
+ ('SigType', c_uint8), #PKCSv1.5 or RSA-PSS or ECC=
=0D
+ ('HashAlg', c_uint8), #Hash Alg for signingh SHA2=
56, 384=0D
+ ('Signature', ARRAY(c_uint8, 0)), #Signature length defined b=
y SigSize bytes=0D
+ ]=0D
+=0D
+ def __init__(self):=0D
+ self.Identifier =3D b'SIGN'=0D
+=0D
+class LZ_HEADER(Structure):=0D
+ _pack_ =3D 1=0D
+ _fields_ =3D [=0D
+ ('signature', ARRAY(c_char, 4)),=0D
+ ('compressed_len', c_uint32),=0D
+ ('length', c_uint32),=0D
+ ('version', c_uint16),=0D
+ ('svn', c_uint8),=0D
+ ('attribute', c_uint8)=0D
+ ]=0D
+ _compress_alg =3D {=0D
+ b'LZDM' : 'Dummy',=0D
+ b'LZ4 ' : 'Lz4',=0D
+ b'LZMA' : 'Lzma',=0D
+ }=0D
+=0D
+def print_bytes (data, indent=3D0, offset=3D0, show_ascii =3D False):=0D
+ bytes_per_line =3D 16=0D
+ printable =3D ' ' + string.ascii_letters + string.digits + string.punc=
tuation=0D
+ str_fmt =3D '{:s}{:04x}: {:%ds} {:s}' % (bytes_per_line * 3)=0D
+ bytes_per_line=0D
+ data_array =3D bytearray(data)=0D
+ for idx in range(0, len(data_array), bytes_per_line):=0D
+ hex_str =3D ' '.join('%02X' % val for val in data_array[idx:idx + =
bytes_per_line])=0D
+ asc_str =3D ''.join('%c' % (val if (chr(val) in printable) else '.=
')=0D
+ for val in data_array[idx:idx + bytes_per_line])=
=0D
+ print (str_fmt.format(indent * ' ', offset + idx, hex_str, ' ' + a=
sc_str if show_ascii else ''))=0D
+=0D
+def get_bits_from_bytes (bytes, start, length):=0D
+ if length =3D=3D 0:=0D
+ return 0=0D
+ byte_start =3D (start) // 8=0D
+ byte_end =3D (start + length - 1) // 8=0D
+ bit_start =3D start & 7=0D
+ mask =3D (1 << length) - 1=0D
+ val =3D bytes_to_value (bytes[byte_start:byte_end + 1])=0D
+ val =3D (val >> bit_start) & mask=0D
+ return val=0D
+=0D
+def set_bits_to_bytes (bytes, start, length, bvalue):=0D
+ if length =3D=3D 0:=0D
+ return=0D
+ byte_start =3D (start) // 8=0D
+ byte_end =3D (start + length - 1) // 8=0D
+ bit_start =3D start & 7=0D
+ mask =3D (1 << length) - 1=0D
+ val =3D bytes_to_value (bytes[byte_start:byte_end + 1])=0D
+ val &=3D ~(mask << bit_start)=0D
+ val |=3D ((bvalue & mask) << bit_start)=0D
+ bytes[byte_start:byte_end+1] =3D value_to_bytearray (val, byte_end + 1=
- byte_start)=0D
+=0D
+def value_to_bytes (value, length):=0D
+ return value.to_bytes(length, 'little')=0D
+=0D
+def bytes_to_value (bytes):=0D
+ return int.from_bytes (bytes, 'little')=0D
+=0D
+def value_to_bytearray (value, length):=0D
+ return bytearray(value_to_bytes(value, length))=0D
+=0D
+def value_to_bytearray (value, length):=0D
+ return bytearray(value_to_bytes(value, length))=0D
+=0D
+def get_aligned_value (value, alignment =3D 4):=0D
+ if alignment !=3D (1 << (alignment.bit_length() - 1)):=0D
+ raise Exception ('Alignment (0x%x) should to be power of 2 !' % al=
ignment)=0D
+ value =3D (value + (alignment - 1)) & ~(alignment - 1)=0D
+ return value=0D
+=0D
+def get_padding_length (data_len, alignment =3D 4):=0D
+ new_data_len =3D get_aligned_value (data_len, alignment)=0D
+ return new_data_len - data_len=0D
+=0D
+def get_file_data (file, mode =3D 'rb'):=0D
+ return open(file, mode).read()=0D
+=0D
+def gen_file_from_object (file, object):=0D
+ open (file, 'wb').write(object)=0D
+=0D
+def gen_file_with_size (file, size):=0D
+ open (file, 'wb').write(b'\xFF' * size);=0D
+=0D
+def check_files_exist (base_name_list, dir =3D '', ext =3D ''):=0D
+ for each in base_name_list:=0D
+ if not os.path.exists (os.path.join (dir, each + ext)):=0D
+ return False=0D
+ return True=0D
+=0D
+def load_source (name, filepath):=0D
+ mod =3D SourceFileLoader (name, filepath).load_module()=0D
+ return mod=0D
+=0D
+def get_openssl_path ():=0D
+ if os.name =3D=3D 'nt':=0D
+ if 'OPENSSL_PATH' not in os.environ:=0D
+ openssl_dir =3D "C:\\Openssl\\bin\\"=0D
+ if os.path.exists (openssl_dir):=0D
+ os.environ['OPENSSL_PATH'] =3D openssl_dir=0D
+ else:=0D
+ os.environ['OPENSSL_PATH'] =3D "C:\\Openssl\\"=0D
+ if 'OPENSSL_CONF' not in os.environ:=0D
+ openssl_cfg =3D "C:\\Openssl\\openssl.cfg"=0D
+ if os.path.exists(openssl_cfg):=0D
+ os.environ['OPENSSL_CONF'] =3D openssl_cfg=0D
+ openssl =3D os.path.join(os.environ.get ('OPENSSL_PATH', ''), 'ope=
nssl.exe')=0D
+ else:=0D
+ # Get openssl path for Linux cases=0D
+ openssl =3D shutil.which('openssl')=0D
+=0D
+ return openssl=0D
+=0D
+def run_process (arg_list, print_cmd =3D False, capture_out =3D False):=0D
+ sys.stdout.flush()=0D
+ if os.name =3D=3D 'nt' and os.path.splitext(arg_list[0])[1] =3D=3D '' =
and \=0D
+ os.path.exists (arg_list[0] + '.exe'):=0D
+ arg_list[0] +=3D '.exe'=0D
+ if print_cmd:=0D
+ print (' '.join(arg_list))=0D
+=0D
+ exc =3D None=0D
+ result =3D 0=0D
+ output =3D ''=0D
+ try:=0D
+ if capture_out:=0D
+ output =3D subprocess.check_output(arg_list).decode()=0D
+ else:=0D
+ result =3D subprocess.call (arg_list)=0D
+ except Exception as ex:=0D
+ result =3D 1=0D
+ exc =3D ex=0D
+=0D
+ if result:=0D
+ if not print_cmd:=0D
+ print ('Error in running process:\n %s' % ' '.join(arg_list))=
=0D
+ if exc is None:=0D
+ sys.exit(1)=0D
+ else:=0D
+ raise exc=0D
+=0D
+ return output=0D
+=0D
+# Adjust hash type algorithm based on Public key file=0D
+def adjust_hash_type (pub_key_file):=0D
+ key_type =3D get_key_type (pub_key_file)=0D
+ if key_type =3D=3D 'RSA2048':=0D
+ hash_type =3D 'SHA2_256'=0D
+ elif key_type =3D=3D 'RSA3072':=0D
+ hash_type =3D 'SHA2_384'=0D
+ else:=0D
+ hash_type =3D None=0D
+=0D
+ return hash_type=0D
+=0D
+def rsa_sign_file (priv_key, pub_key, hash_type, sign_scheme, in_file, out=
_file, inc_dat =3D False, inc_key =3D False):=0D
+=0D
+ bins =3D bytearray()=0D
+ if inc_dat:=0D
+ bins.extend(get_file_data(in_file))=0D
+=0D
+ single_sign_file(priv_key, hash_type, sign_scheme, in_file, out_file)=
=0D
+=0D
+ out_data =3D get_file_data(out_file)=0D
+=0D
+ sign =3D SIGNATURE_HDR()=0D
+ sign.SigSize =3D len(out_data)=0D
+ sign.SigType =3D SIGN_TYPE_SCHEME[sign_scheme]=0D
+ sign.HashAlg =3D HASH_TYPE_VALUE[hash_type]=0D
+=0D
+ bins.extend(bytearray(sign) + out_data)=0D
+ if inc_key:=0D
+ key =3D gen_pub_key (priv_key, pub_key)=0D
+ bins.extend(key)=0D
+=0D
+ if len(bins) !=3D len(out_data):=0D
+ gen_file_from_object (out_file, bins)=0D
+=0D
+def get_key_type (in_key):=0D
+=0D
+ # Check in_key is file or key Id=0D
+ if not os.path.exists(in_key):=0D
+ key =3D bytearray(gen_pub_key (in_key))=0D
+ else:=0D
+ # Check for public key in binary format.=0D
+ key =3D bytearray(get_file_data(in_key))=0D
+=0D
+ pub_key_hdr =3D PUB_KEY_HDR.from_buffer(key)=0D
+ if pub_key_hdr.Identifier !=3D b'PUBK':=0D
+ pub_key =3D gen_pub_key (in_key)=0D
+ pub_key_hdr =3D PUB_KEY_HDR.from_buffer(pub_key)=0D
+=0D
+ key_type =3D next((key for key, value in PUB_KEY_TYPE.items() if value=
=3D=3D pub_key_hdr.KeyType))=0D
+ return '%s%d' % (key_type, (pub_key_hdr.KeySize - 4) * 8)=0D
+=0D
+=0D
+def get_auth_hash_type (key_type, sign_scheme):=0D
+ if key_type =3D=3D "RSA2048" and sign_scheme =3D=3D "RSA_PKCS1":=0D
+ hash_type =3D 'SHA2_256'=0D
+ auth_type =3D 'RSA2048_PKCS1_SHA2_256'=0D
+ elif key_type =3D=3D "RSA3072" and sign_scheme =3D=3D "RSA_PKCS1":=0D
+ hash_type =3D 'SHA2_384'=0D
+ auth_type =3D 'RSA3072_PKCS1_SHA2_384'=0D
+ elif key_type =3D=3D "RSA2048" and sign_scheme =3D=3D "RSA_PSS":=0D
+ hash_type =3D 'SHA2_256'=0D
+ auth_type =3D 'RSA2048_PSS_SHA2_256'=0D
+ elif key_type =3D=3D "RSA3072" and sign_scheme =3D=3D "RSA_PSS":=0D
+ hash_type =3D 'SHA2_384'=0D
+ auth_type =3D 'RSA3072_PSS_SHA2_384'=0D
+ else:=0D
+ hash_type =3D ''=0D
+ auth_type =3D ''=0D
+ return auth_type, hash_type=0D
+=0D
+def gen_pub_key (in_key, pub_key =3D None):=0D
+=0D
+ keydata =3D single_sign_gen_pub_key (in_key, pub_key)=0D
+=0D
+ publickey =3D PUB_KEY_HDR()=0D
+ publickey.KeySize =3D len(keydata)=0D
+ publickey.KeyType =3D PUB_KEY_TYPE['RSA']=0D
+=0D
+ key =3D bytearray(publickey) + keydata=0D
+=0D
+ if pub_key:=0D
+ gen_file_from_object (pub_key, key)=0D
+=0D
+ return key=0D
+=0D
+def decompress (in_file, out_file, tool_dir =3D ''):=0D
+ if not os.path.isfile(in_file):=0D
+ raise Exception ("Invalid input file '%s' !" % in_file)=0D
+=0D
+ # Remove the Lz Header=0D
+ fi =3D open(in_file,'rb')=0D
+ di =3D bytearray(fi.read())=0D
+ fi.close()=0D
+=0D
+ lz_hdr =3D LZ_HEADER.from_buffer (di)=0D
+ offset =3D sizeof (lz_hdr)=0D
+ if lz_hdr.signature =3D=3D b"LZDM" or lz_hdr.compressed_len =3D=3D 0:=
=0D
+ fo =3D open(out_file,'wb')=0D
+ fo.write(di[offset:offset + lz_hdr.compressed_len])=0D
+ fo.close()=0D
+ return=0D
+=0D
+ temp =3D os.path.splitext(out_file)[0] + '.tmp'=0D
+ if lz_hdr.signature =3D=3D b"LZMA":=0D
+ alg =3D "Lzma"=0D
+ elif lz_hdr.signature =3D=3D b"LZ4 ":=0D
+ alg =3D "Lz4"=0D
+ else:=0D
+ raise Exception ("Unsupported compression '%s' !" % lz_hdr.signatu=
re)=0D
+=0D
+ fo =3D open(temp, 'wb')=0D
+ fo.write(di[offset:offset + lz_hdr.compressed_len])=0D
+ fo.close()=0D
+=0D
+ compress_tool =3D "%sCompress" % alg=0D
+ if alg =3D=3D "Lz4":=0D
+ try:=0D
+ cmdline =3D [=0D
+ os.path.join (tool_dir, compress_tool),=0D
+ "-d",=0D
+ "-o", out_file,=0D
+ temp]=0D
+ run_process (cmdline, False, True)=0D
+ except:=0D
+ print("Could not find/use CompressLz4 tool, trying with python=
lz4...")=0D
+ try:=0D
+ import lz4.block=0D
+ if lz4.VERSION !=3D '3.1.1':=0D
+ print("Recommended lz4 module version is '3.1.1', '%s'=
is currently installed." % lz4.VERSION)=0D
+ except ImportError:=0D
+ print("Could not import lz4, use 'python -m pip install lz=
4=3D=3D3.1.1' to install it.")=0D
+ exit(1)=0D
+ decompress_data =3D lz4.block.decompress(get_file_data(temp))=
=0D
+ with open(out_file, "wb") as lz4bin:=0D
+ lz4bin.write(decompress_data)=0D
+ else:=0D
+ cmdline =3D [=0D
+ os.path.join (tool_dir, compress_tool),=0D
+ "-d",=0D
+ "-o", out_file,=0D
+ temp]=0D
+ run_process (cmdline, False, True)=0D
+ os.remove(temp)=0D
+=0D
+def compress (in_file, alg, svn=3D0, out_path =3D '', tool_dir =3D ''):=0D
+ if not os.path.isfile(in_file):=0D
+ raise Exception ("Invalid input file '%s' !" % in_file)=0D
+=0D
+ basename, ext =3D os.path.splitext(os.path.basename (in_file))=0D
+ if out_path:=0D
+ if os.path.isdir (out_path):=0D
+ out_file =3D os.path.join(out_path, basename + '.lz')=0D
+ else:=0D
+ out_file =3D os.path.join(out_path)=0D
+ else:=0D
+ out_file =3D os.path.splitext(in_file)[0] + '.lz'=0D
+=0D
+ if alg =3D=3D "Lzma":=0D
+ sig =3D "LZMA"=0D
+ elif alg =3D=3D "Tiano":=0D
+ sig =3D "LZUF"=0D
+ elif alg =3D=3D "Lz4":=0D
+ sig =3D "LZ4 "=0D
+ elif alg =3D=3D "Dummy":=0D
+ sig =3D "LZDM"=0D
+ else:=0D
+ raise Exception ("Unsupported compression '%s' !" % alg)=0D
+=0D
+ in_len =3D os.path.getsize(in_file)=0D
+ if in_len > 0:=0D
+ compress_tool =3D "%sCompress" % alg=0D
+ if sig =3D=3D "LZDM":=0D
+ shutil.copy(in_file, out_file)=0D
+ compress_data =3D get_file_data(out_file)=0D
+ elif sig =3D=3D "LZ4 ":=0D
+ try:=0D
+ cmdline =3D [=0D
+ os.path.join (tool_dir, compress_tool),=0D
+ "-e",=0D
+ "-o", out_file,=0D
+ in_file]=0D
+ run_process (cmdline, False, True)=0D
+ compress_data =3D get_file_data(out_file)=0D
+ except:=0D
+ print("Could not find/use CompressLz4 tool, trying with py=
thon lz4...")=0D
+ try:=0D
+ import lz4.block=0D
+ if lz4.VERSION !=3D '3.1.1':=0D
+ print("Recommended lz4 module version is '3.1.1', =
'%s' is currently installed." % lz4.VERSION)=0D
+ except ImportError:=0D
+ print("Could not import lz4, use 'python -m pip instal=
l lz4=3D=3D3.1.1' to install it.")=0D
+ exit(1)=0D
+ compress_data =3D lz4.block.compress(get_file_data(in_file=
), mode=3D'high_compression')=0D
+ elif sig =3D=3D "LZMA":=0D
+ cmdline =3D [=0D
+ os.path.join (tool_dir, compress_tool),=0D
+ "-e",=0D
+ "-o", out_file,=0D
+ in_file]=0D
+ run_process (cmdline, False, True)=0D
+ compress_data =3D get_file_data(out_file)=0D
+ else:=0D
+ compress_data =3D bytearray()=0D
+=0D
+ lz_hdr =3D LZ_HEADER ()=0D
+ lz_hdr.signature =3D sig.encode()=0D
+ lz_hdr.svn =3D svn=0D
+ lz_hdr.compressed_len =3D len(compress_data)=0D
+ lz_hdr.length =3D os.path.getsize(in_file)=0D
+ data =3D bytearray ()=0D
+ data.extend (lz_hdr)=0D
+ data.extend (compress_data)=0D
+ gen_file_from_object (out_file, data)=0D
+=0D
+ return out_file=0D
diff --git a/IntelFsp2Pkg/Tools/ConfigEditor/ConfigEditor.py b/IntelFsp2Pkg=
/Tools/ConfigEditor/ConfigEditor.py
new file mode 100644
index 0000000000..29d24c9bc8
--- /dev/null
+++ b/IntelFsp2Pkg/Tools/ConfigEditor/ConfigEditor.py
@@ -0,0 +1,941 @@
+## @ ConfigEditor.py=0D
+#=0D
+# Copyright (c) 2018 - 2020, Intel Corporation. All rights reserved.<BR>=0D
+# SPDX-License-Identifier: BSD-2-Clause-Patent=0D
+#=0D
+##=0D
+=0D
+import os=0D
+import re=0D
+import sys=0D
+import marshal=0D
+from pathlib import Path=0D
+=0D
+sys.dont_write_bytecode =3D True=0D
+=0D
+import tkinter=0D
+import tkinter.ttk as ttk=0D
+import tkinter.messagebox as messagebox=0D
+import tkinter.filedialog as filedialog=0D
+=0D
+from GenCfgData import CGenCfgData, bytes_to_value, bytes_to_bracket_str, =
value_to_bytes, array_str_to_value=0D
+=0D
+class create_tool_tip(object):=0D
+ '''=0D
+ create a tooltip for a given widget=0D
+ '''=0D
+ in_progress =3D False=0D
+=0D
+ def __init__(self, widget, text=3D''):=0D
+ self.top_win =3D None=0D
+ self.widget =3D widget=0D
+ self.text =3D text=0D
+ self.widget.bind("<Enter>", self.enter)=0D
+ self.widget.bind("<Leave>", self.leave)=0D
+=0D
+ def enter(self, event=3DNone):=0D
+ if self.in_progress:=0D
+ return=0D
+ if self.widget.winfo_class() =3D=3D 'Treeview':=0D
+ # Only show help when cursor is on row header.=0D
+ rowid =3D self.widget.identify_row(event.y)=0D
+ if rowid !=3D '':=0D
+ return=0D
+ else:=0D
+ x, y, cx, cy =3D self.widget.bbox("insert")=0D
+=0D
+ cursor =3D self.widget.winfo_pointerxy()=0D
+ x =3D self.widget.winfo_rootx() + 35=0D
+ y =3D self.widget.winfo_rooty() + 20=0D
+ if cursor[1] > y and cursor[1] < y + 20:=0D
+ y +=3D 20=0D
+=0D
+ # creates a toplevel window=0D
+ self.top_win =3D tkinter.Toplevel(self.widget)=0D
+ # Leaves only the label and removes the app window=0D
+ self.top_win.wm_overrideredirect(True)=0D
+ self.top_win.wm_geometry("+%d+%d" % (x, y))=0D
+ label =3D tkinter.Message(self.top_win,=0D
+ text=3Dself.text,=0D
+ justify=3D'left',=0D
+ background=3D'bisque',=0D
+ relief=3D'solid',=0D
+ borderwidth=3D1,=0D
+ font=3D("times", "10", "normal"))=0D
+ label.pack(ipadx=3D1)=0D
+ self.in_progress =3D True=0D
+=0D
+ def leave(self, event=3DNone):=0D
+ if self.top_win:=0D
+ self.top_win.destroy()=0D
+ self.in_progress =3D False=0D
+=0D
+=0D
+class validating_entry(tkinter.Entry):=0D
+ def __init__(self, master, **kw):=0D
+ tkinter.Entry.__init__(*(self, master), **kw)=0D
+ self.parent =3D master=0D
+ self.old_value =3D ''=0D
+ self.last_value =3D ''=0D
+ self.variable =3D tkinter.StringVar()=0D
+ self.variable.trace("w", self.callback)=0D
+ self.config(textvariable=3Dself.variable)=0D
+ self.config({"background": "#c0c0c0"})=0D
+ self.bind("<Return>", self.move_next)=0D
+ self.bind("<Tab>", self.move_next)=0D
+ self.bind("<Escape>", self.cancel)=0D
+ for each in ['BackSpace', 'Delete']:=0D
+ self.bind("<%s>" % each, self.ignore)=0D
+ self.display (None)=0D
+=0D
+ def ignore (self, even):=0D
+ return "break"=0D
+=0D
+ def move_next (self, event):=0D
+ if self.row < 0:=0D
+ return=0D
+ row, col =3D self.row, self.col=0D
+ txt, row_id, col_id =3D self.parent.get_next_cell (row, col)=0D
+ self.display (txt, row_id, col_id)=0D
+ return "break"=0D
+=0D
+ def cancel (self, event):=0D
+ self.variable.set(self.old_value)=0D
+ self.display (None)=0D
+=0D
+ def display (self, txt, row_id =3D '', col_id =3D ''):=0D
+ if txt is None:=0D
+ self.row =3D -1=0D
+ self.col =3D -1=0D
+ self.place_forget()=0D
+ else:=0D
+ row =3D int('0x' + row_id[1:], 0) - 1=0D
+ col =3D int(col_id[1:]) - 1=0D
+ self.row =3D row=0D
+ self.col =3D col=0D
+ self.old_value =3D txt=0D
+ self.last_value =3D txt=0D
+ x, y, width, height =3D self.parent.bbox(row_id, col)=0D
+ self.place(x=3Dx, y=3Dy, w=3Dwidth)=0D
+ self.variable.set(txt)=0D
+ self.focus_set()=0D
+ self.icursor(0)=0D
+=0D
+ def callback(self, *Args):=0D
+ cur_val =3D self.variable.get()=0D
+ new_val =3D self.validate(cur_val)=0D
+ if new_val is not None and self.row >=3D 0:=0D
+ self.last_value =3D new_val=0D
+ self.parent.set_cell (self.row , self.col, new_val)=0D
+ self.variable.set(self.last_value)=0D
+=0D
+=0D
+ def validate(self, value):=0D
+ if len(value) > 0:=0D
+ try:=0D
+ int(value, 16)=0D
+ except:=0D
+ return None=0D
+=0D
+ # Normalize the cell format=0D
+ self.update()=0D
+ cell_width =3D self.winfo_width ()=0D
+ max_len =3D custom_table.to_byte_length(cell_width) * 2=0D
+ cur_pos =3D self.index("insert")=0D
+ if cur_pos =3D=3D max_len + 1:=0D
+ value =3D value[-max_len:]=0D
+ else:=0D
+ value =3D value[:max_len]=0D
+ if value =3D=3D '':=0D
+ value =3D '0'=0D
+ fmt =3D '%%0%dX' % max_len=0D
+ return fmt % int(value, 16)=0D
+=0D
+=0D
+class custom_table(ttk.Treeview):=0D
+ _Padding =3D 20=0D
+ _Char_width =3D 6=0D
+=0D
+ def __init__(self, parent, col_hdr, bins):=0D
+ cols =3D len(col_hdr)=0D
+=0D
+ col_byte_len =3D []=0D
+ for col in range(cols): #Columns=0D
+ col_byte_len.append(int(col_hdr[col].split(':')[1]))=0D
+=0D
+ byte_len =3D sum(col_byte_len)=0D
+ rows =3D (len(bins) + byte_len - 1) // byte_len=0D
+=0D
+ self.rows =3D rows=0D
+ self.cols =3D cols=0D
+ self.col_byte_len =3D col_byte_len=0D
+ self.col_hdr =3D col_hdr=0D
+=0D
+ self.size =3D len(bins)=0D
+ self.last_dir =3D ''=0D
+=0D
+ style =3D ttk.Style()=0D
+ style.configure("Custom.Treeview.Heading", font=3D('calibri', 10, =
'bold'), foreground=3D"blue")=0D
+ ttk.Treeview.__init__(self, parent, height=3Drows, columns=3D[''] =
+ col_hdr, show=3D'headings', style=3D"Custom.Treeview", selectmode=3D'none=
')=0D
+ self.bind("<Button-1>", self.click)=0D
+ self.bind("<FocusOut>", self.focus_out)=0D
+ self.entry =3D validating_entry(self, width=3D4, justify=3Dtkinte=
r.CENTER)=0D
+=0D
+ self.heading(0, text=3D'LOAD')=0D
+ self.column (0, width=3D60, stretch=3D0, anchor=3Dtkinter.CENTER)=
=0D
+=0D
+ for col in range(cols): #Columns=0D
+ text =3D col_hdr[col].split(':')[0]=0D
+ byte_len =3D int(col_hdr[col].split(':')[1])=0D
+ self.heading(col+1, text=3Dtext)=0D
+ self.column(col+1, width=3Dself.to_cell_width(byte_len), stret=
ch=3D0, anchor=3Dtkinter.CENTER)=0D
+=0D
+ idx =3D 0=0D
+ for row in range(rows): #Rows=0D
+ text =3D '%04X' % (row * len(col_hdr))=0D
+ vals =3D ['%04X:' % (cols * row)]=0D
+ for col in range(cols): #Columns=0D
+ if idx >=3D len(bins):=0D
+ break=0D
+ byte_len =3D int(col_hdr[col].split(':')[1])=0D
+ value =3D bytes_to_value (bins[idx:idx+byte_len])=0D
+ hex =3D ("%%0%dX" % (byte_len * 2) ) % value=0D
+ vals.append (hex)=0D
+ idx +=3D byte_len=0D
+ self.insert('', 'end', values=3Dtuple(vals))=0D
+ if idx >=3D len(bins):=0D
+ break=0D
+=0D
+ @staticmethod=0D
+ def to_cell_width(byte_len):=0D
+ return byte_len * 2 * custom_table._Char_width + custom_table._Pad=
ding=0D
+=0D
+ @staticmethod=0D
+ def to_byte_length(cell_width):=0D
+ return (cell_width - custom_table._Padding) // (2 * custom_table._=
Char_width)=0D
+=0D
+ def focus_out (self, event):=0D
+ self.entry.display (None)=0D
+=0D
+ def refresh_bin (self, bins):=0D
+ if not bins:=0D
+ return=0D
+=0D
+ # Reload binary into widget=0D
+ bin_len =3D len(bins)=0D
+ for row in range(self.rows):=0D
+ iid =3D self.get_children()[row]=0D
+ for col in range(self.cols):=0D
+ idx =3D row * sum(self.col_byte_len) + sum(self.col_byte_l=
en[:col])=0D
+ byte_len =3D self.col_byte_len[col]=0D
+ if idx + byte_len <=3D self.size:=0D
+ byte_len =3D int(self.col_hdr[col].split(':')[1])=0D
+ if idx + byte_len > bin_len:=0D
+ val =3D 0=0D
+ else:=0D
+ val =3D bytes_to_value (bins[idx:idx+byte_len])=0D
+ hex_val =3D ("%%0%dX" % (byte_len * 2) ) % val=0D
+ self.set (iid, col + 1, hex_val)=0D
+=0D
+ def get_cell (self, row, col):=0D
+ iid =3D self.get_children()[row]=0D
+ txt =3D self.item(iid, 'values')[col]=0D
+ return txt=0D
+=0D
+ def get_next_cell (self, row, col):=0D
+ rows =3D self.get_children()=0D
+ col +=3D 1=0D
+ if col > self.cols:=0D
+ col =3D 1=0D
+ row +=3D1=0D
+ cnt =3D row * sum(self.col_byte_len) + sum(self.col_byte_len[:col]=
)=0D
+ if cnt > self.size:=0D
+ # Reached the last cell, so roll back to beginning=0D
+ row =3D 0=0D
+ col =3D 1=0D
+=0D
+ txt =3D self.get_cell(row, col)=0D
+ row_id =3D rows[row]=0D
+ col_id =3D '#%d' % (col + 1)=0D
+ return (txt, row_id, col_id)=0D
+=0D
+ def set_cell (self, row, col, val):=0D
+ iid =3D self.get_children()[row]=0D
+ self.set (iid, col, val)=0D
+=0D
+ def load_bin (self):=0D
+ # Load binary from file=0D
+ path =3D filedialog.askopenfilename(=0D
+ initialdir=3Dself.last_dir,=0D
+ title=3D"Load binary file",=0D
+ filetypes=3D(("Binary files", "*.bin"), (=0D
+ "binary files", "*.bin")))=0D
+ if path:=0D
+ self.last_dir =3D os.path.dirname(path)=0D
+ fd =3D open(path, 'rb')=0D
+ bins =3D bytearray(fd.read())[:self.size]=0D
+ fd.close()=0D
+ bins.extend (b'\x00' * (self.size - len(bins)))=0D
+ return bins=0D
+=0D
+ return None=0D
+=0D
+ def click (self, event):=0D
+ row_id =3D self.identify_row(event.y)=0D
+ col_id =3D self.identify_column(event.x)=0D
+ if row_id =3D=3D '' and col_id =3D=3D '#1':=0D
+ # Clicked on "LOAD" cell=0D
+ bins =3D self.load_bin ()=0D
+ self.refresh_bin (bins)=0D
+ return=0D
+=0D
+ if col_id =3D=3D '#1':=0D
+ # Clicked on column 1 (Offset column)=0D
+ return=0D
+=0D
+ item =3D self.identify('item', event.x, event.y)=0D
+ if not item or not col_id:=0D
+ # Not clicked on valid cell=0D
+ return=0D
+=0D
+ # Clicked cell=0D
+ row =3D int('0x' + row_id[1:], 0) - 1=0D
+ col =3D int(col_id[1:]) - 1=0D
+ if row * self.cols + col > self.size:=0D
+ return=0D
+=0D
+ vals =3D self.item(item, 'values')=0D
+ if col < len(vals):=0D
+ txt =3D self.item(item, 'values')[col]=0D
+ self.entry.display (txt, row_id, col_id)=0D
+=0D
+ def get(self):=0D
+ bins =3D bytearray()=0D
+ row_ids =3D self.get_children()=0D
+ for row_id in row_ids:=0D
+ row =3D int('0x' + row_id[1:], 0) - 1=0D
+ for col in range(self.cols):=0D
+ idx =3D row * sum(self.col_byte_len) + sum(self.col_byte_len=
[:col])=0D
+ byte_len =3D self.col_byte_len[col]=0D
+ if idx + byte_len > self.size:=0D
+ break=0D
+ hex =3D self.item(row_id, 'values')[col + 1]=0D
+ values =3D value_to_bytes (int(hex, 16) & ((1 << byte_len * =
8) - 1), byte_len)=0D
+ bins.extend(values)=0D
+ return bins=0D
+=0D
+class state:=0D
+ def __init__(self):=0D
+ self.state =3D False=0D
+=0D
+ def set(self, value):=0D
+ self.state =3D value=0D
+=0D
+ def get(self):=0D
+ return self.state=0D
+=0D
+class application(tkinter.Frame):=0D
+ def __init__(self, master=3DNone):=0D
+ root =3D master=0D
+=0D
+ self.debug =3D True=0D
+ self.page_id =3D ''=0D
+ self.page_list =3D {}=0D
+ self.conf_list =3D {}=0D
+ self.cfg_data_obj =3D None=0D
+ self.org_cfg_data_bin =3D None=0D
+ self.in_left =3D state()=0D
+ self.in_right =3D state()=0D
+=0D
+ # Check if current directory contains a file with a .yaml extensio=
n=0D
+ # if not default self.last_dir to a Platform directory where it is=
easier to locate *BoardPkg\CfgData\*Def.yaml files=0D
+ self.last_dir =3D '.'=0D
+ if not any(fname.endswith('.yaml') for fname in os.listdir('.')):=
=0D
+ platform_path =3D Path(os.path.realpath(__file__)).parents[2].=
joinpath('Platform')=0D
+ if platform_path.exists():=0D
+ self.last_dir =3D platform_path=0D
+=0D
+ tkinter.Frame.__init__(self, master, borderwidth=3D2)=0D
+=0D
+ self.menu_string =3D [=0D
+ 'Save Config Data to Binary', 'Load Config Data from Binary',=
=0D
+ 'Load Config Changes from Delta File',=0D
+ 'Save Config Changes to Delta File',=0D
+ 'Save Full Config Data to Delta File'=0D
+ ]=0D
+=0D
+ root.geometry("1200x800")=0D
+=0D
+ paned =3D ttk.Panedwindow(root, orient=3Dtkinter.HORIZONTAL)=0D
+ paned.pack(fill=3Dtkinter.BOTH, expand=3DTrue, padx=3D(4, 4))=0D
+=0D
+ status =3D tkinter.Label(master, text=3D"", bd=3D1, relief=3Dtkint=
er.SUNKEN, anchor=3Dtkinter.W)=0D
+ status.pack(side=3Dtkinter.BOTTOM, fill=3Dtkinter.X)=0D
+=0D
+ frame_left =3D ttk.Frame(paned, height=3D800, relief=3D"groove")=0D
+=0D
+ self.left =3D ttk.Treeview(frame_left, show=3D"tree")=0D
+=0D
+ # Set up tree HScroller=0D
+ pady =3D (10, 10)=0D
+ self.tree_scroll =3D ttk.Scrollbar(frame_left,=0D
+ orient=3D"vertical",=0D
+ command=3Dself.left.yview)=0D
+ self.left.configure(yscrollcommand=3Dself.tree_scroll.set)=0D
+ self.left.bind("<<TreeviewSelect>>", self.on_config_page_select_ch=
ange)=0D
+ self.left.bind("<Enter>", lambda e: self.in_left.set(True))=0D
+ self.left.bind("<Leave>", lambda e: self.in_left.set(False))=0D
+ self.left.bind("<MouseWheel>", self.on_tree_scroll)=0D
+=0D
+ self.left.pack(side=3D'left',=0D
+ fill=3Dtkinter.BOTH,=0D
+ expand=3DTrue,=0D
+ padx=3D(5, 0),=0D
+ pady=3Dpady)=0D
+ self.tree_scroll.pack(side=3D'right', fill=3Dtkinter.Y, pady=3Dpad=
y, padx=3D(0, 5))=0D
+=0D
+ frame_right =3D ttk.Frame(paned, relief=3D"groove")=0D
+ self.frame_right =3D frame_right=0D
+=0D
+ self.conf_canvas =3D tkinter.Canvas(frame_right, highlightthicknes=
s=3D0)=0D
+ self.page_scroll =3D ttk.Scrollbar(frame_right,=0D
+ orient=3D"vertical",=0D
+ command=3Dself.conf_canvas.yview)=
=0D
+ self.right_grid =3D ttk.Frame(self.conf_canvas)=0D
+ self.conf_canvas.configure(yscrollcommand=3Dself.page_scroll.set)=
=0D
+ self.conf_canvas.pack(side=3D'left',=0D
+ fill=3Dtkinter.BOTH,=0D
+ expand=3DTrue,=0D
+ pady=3Dpady,=0D
+ padx=3D(5, 0))=0D
+ self.page_scroll.pack(side=3D'right', fill=3Dtkinter.Y, pady=3Dpad=
y, padx=3D(0, 5))=0D
+ self.conf_canvas.create_window(0, 0, window=3Dself.right_grid, anc=
hor=3D'nw')=0D
+ self.conf_canvas.bind('<Enter>', lambda e: self.in_right.set(True=
))=0D
+ self.conf_canvas.bind('<Leave>', lambda e: self.in_right.set(Fals=
e))=0D
+ self.conf_canvas.bind("<Configure>", self.on_canvas_configure)=0D
+ self.conf_canvas.bind_all("<MouseWheel>", self.on_page_scroll)=0D
+=0D
+ paned.add(frame_left, weight=3D2)=0D
+ paned.add(frame_right, weight=3D10)=0D
+=0D
+ style =3D ttk.Style()=0D
+ style.layout("Treeview", [('Treeview.treearea', {'sticky': 'nswe'}=
)])=0D
+=0D
+ menubar =3D tkinter.Menu(root)=0D
+ file_menu =3D tkinter.Menu(menubar, tearoff=3D0)=0D
+ file_menu.add_command(label=3D"Open Config YAML file...",=0D
+ command=3Dself.load_from_yaml)=0D
+ file_menu.add_command(label=3Dself.menu_string[0],=0D
+ command=3Dself.save_to_bin,=0D
+ state=3D'disabled')=0D
+ file_menu.add_command(label=3Dself.menu_string[1],=0D
+ command=3Dself.load_from_bin,=0D
+ state=3D'disabled')=0D
+ file_menu.add_command(label=3Dself.menu_string[2],=0D
+ command=3Dself.load_from_delta,=0D
+ state=3D'disabled')=0D
+ file_menu.add_command(label=3Dself.menu_string[3],=0D
+ command=3Dself.save_to_delta,=0D
+ state=3D'disabled')=0D
+ file_menu.add_command(label=3Dself.menu_string[4],=0D
+ command=3Dself.save_full_to_delta,=0D
+ state=3D'disabled')=0D
+ file_menu.add_command(label=3D"About", command=3Dself.about)=0D
+ menubar.add_cascade(label=3D"File", menu=3Dfile_menu)=0D
+ self.file_menu =3D file_menu=0D
+=0D
+ root.config(menu=3Dmenubar)=0D
+=0D
+ if len(sys.argv) > 1:=0D
+ path =3D sys.argv[1]=0D
+ if not path.endswith('.yaml') and not path.endswith('.pkl'):=0D
+ messagebox.showerror('LOADING ERROR', "Unsupported file '%=
s' !" % path)=0D
+ return=0D
+ else:=0D
+ self.load_cfg_file (path)=0D
+=0D
+ if len(sys.argv) > 2:=0D
+ path =3D sys.argv[2]=0D
+ if path.endswith('.dlt'):=0D
+ self.load_delta_file (path)=0D
+ elif path.endswith('.bin'):=0D
+ self.load_bin_file (path)=0D
+ else:=0D
+ messagebox.showerror('LOADING ERROR', "Unsupported file '%=
s' !" % path)=0D
+ return=0D
+=0D
+ def set_object_name(self, widget, name):=0D
+ self.conf_list[id(widget)] =3D name=0D
+=0D
+ def get_object_name(self, widget):=0D
+ if id(widget) in self.conf_list:=0D
+ return self.conf_list[id(widget)]=0D
+ else:=0D
+ return None=0D
+=0D
+ def limit_entry_size(self, variable, limit):=0D
+ value =3D variable.get()=0D
+ if len(value) > limit:=0D
+ variable.set(value[:limit])=0D
+=0D
+ def on_canvas_configure(self, event):=0D
+ self.right_grid.grid_columnconfigure(0, minsize=3Devent.width)=0D
+=0D
+ def on_tree_scroll(self, event):=0D
+ if not self.in_left.get() and self.in_right.get():=0D
+ # This prevents scroll event from being handled by both left a=
nd=0D
+ # right frame at the same time.=0D
+ self.on_page_scroll (event)=0D
+ return 'break'=0D
+=0D
+ def on_page_scroll(self, event):=0D
+ if self.in_right.get():=0D
+ # Only scroll when it is in active area=0D
+ min, max =3D self.page_scroll.get()=0D
+ if not ((min =3D=3D 0.0) and (max =3D=3D 1.0)):=0D
+ self.conf_canvas.yview_scroll(-1 * int(event.delta / 120),=
'units')=0D
+=0D
+ def update_visibility_for_widget(self, widget, args):=0D
+=0D
+ visible =3D True=0D
+ item =3D self.get_config_data_item_from_widget(widget, True)=0D
+ if item is None:=0D
+ return visible=0D
+ elif not item:=0D
+ return visible=0D
+=0D
+ result =3D 1=0D
+ if item['condition']:=0D
+ result =3D self.evaluate_condition(item)=0D
+ if result =3D=3D 2:=0D
+ # Gray=0D
+ widget.configure(state=3D'disabled')=0D
+ elif result =3D=3D 0:=0D
+ # Hide=0D
+ visible =3D False=0D
+ widget.grid_remove()=0D
+ else:=0D
+ # Show=0D
+ widget.grid()=0D
+ widget.configure(state=3D'normal')=0D
+=0D
+ return visible=0D
+=0D
+ def update_widgets_visibility_on_page(self):=0D
+ self.walk_widgets_in_layout(self.right_grid,=0D
+ self.update_visibility_for_widget)=0D
+=0D
+ def combo_select_changed(self, event):=0D
+ self.update_config_data_from_widget(event.widget, None)=0D
+ self.update_widgets_visibility_on_page()=0D
+=0D
+ def edit_num_finished(self, event):=0D
+ widget =3D event.widget=0D
+ item =3D self.get_config_data_item_from_widget(widget)=0D
+ if not item:=0D
+ return=0D
+ parts =3D item['type'].split(',')=0D
+ if len(parts) > 3:=0D
+ min =3D parts[2].lstrip()[1:]=0D
+ max =3D parts[3].rstrip()[:-1]=0D
+ min_val =3D array_str_to_value(min)=0D
+ max_val =3D array_str_to_value(max)=0D
+ text =3D widget.get()=0D
+ if ',' in text:=0D
+ text =3D '{ %s }' % text=0D
+ try:=0D
+ value =3D array_str_to_value(text)=0D
+ if value < min_val or value > max_val:=0D
+ raise Exception('Invalid input!')=0D
+ self.set_config_item_value(item, text)=0D
+ except Exception as e:=0D
+ pass=0D
+=0D
+ text =3D item['value'].strip('{').strip('}').strip()=0D
+ widget.delete(0, tkinter.END)=0D
+ widget.insert(0, text)=0D
+=0D
+ self.update_widgets_visibility_on_page()=0D
+=0D
+ def update_page_scroll_bar(self):=0D
+ # Update scrollbar=0D
+ self.frame_right.update()=0D
+ self.conf_canvas.config(scrollregion=3Dself.conf_canvas.bbox("all"=
))=0D
+=0D
+=0D
+ def on_config_page_select_change(self, event):=0D
+ self.update_config_data_on_page()=0D
+ sel =3D self.left.selection()=0D
+ if len(sel) > 0:=0D
+ page_id =3D sel[0]=0D
+ self.build_config_data_page(page_id)=0D
+ self.update_widgets_visibility_on_page()=0D
+ self.update_page_scroll_bar()=0D
+=0D
+ def walk_widgets_in_layout(self, parent, callback_function, args=3DNon=
e):=0D
+ for widget in parent.winfo_children():=0D
+ callback_function(widget, args)=0D
+=0D
+ def clear_widgets_inLayout(self, parent=3DNone):=0D
+ if parent is None:=0D
+ parent =3D self.right_grid=0D
+=0D
+ for widget in parent.winfo_children():=0D
+ widget.destroy()=0D
+=0D
+ parent.grid_forget()=0D
+ self.conf_list.clear()=0D
+=0D
+ def build_config_page_tree(self, cfg_page, parent):=0D
+ for page in cfg_page['child']:=0D
+ page_id =3D next(iter(page))=0D
+ # Put CFG items into related page list=0D
+ self.page_list[page_id] =3D self.cfg_data_obj.get_cfg_list (pa=
ge_id)=0D
+ self.page_list[page_id].sort (key=3Dlambda x: x['order'])=0D
+ page_name =3D self.cfg_data_obj.get_page_title(page_id)=0D
+ child =3D self.left.insert(=0D
+ parent, 'end',=0D
+ iid=3Dpage_id, text=3Dpage_name,=0D
+ value=3D0)=0D
+ if len(page[page_id]) > 0:=0D
+ self.build_config_page_tree(page[page_id], child)=0D
+=0D
+=0D
+ def is_config_data_loaded(self):=0D
+ return True if len(self.page_list) else False=0D
+=0D
+ def set_current_config_page(self, page_id):=0D
+ self.page_id =3D page_id=0D
+=0D
+ def get_current_config_page(self):=0D
+ return self.page_id=0D
+=0D
+ def get_current_config_data(self):=0D
+ page_id =3D self.get_current_config_page()=0D
+ if page_id in self.page_list:=0D
+ return self.page_list[page_id]=0D
+ else:=0D
+ return []=0D
+=0D
+ def build_config_data_page(self, page_id):=0D
+ self.clear_widgets_inLayout()=0D
+ self.set_current_config_page(page_id)=0D
+ disp_list =3D []=0D
+ for item in self.get_current_config_data():=0D
+ disp_list.append(item)=0D
+ row =3D 0=0D
+ disp_list.sort(key=3Dlambda x:x['order'])=0D
+ for item in disp_list:=0D
+ self.add_config_item (item, row)=0D
+ row +=3D 2=0D
+=0D
+ def load_config_data(self, file_name):=0D
+ gen_cfg_data =3D CGenCfgData()=0D
+ if file_name.endswith('.pkl'):=0D
+ with open(file_name, "rb") as pkl_file:=0D
+ gen_cfg_data.__dict__ =3D marshal.load(pkl_file)=0D
+ gen_cfg_data.prepare_marshal (False)=0D
+ elif file_name.endswith('.yaml'):=0D
+ if gen_cfg_data.load_yaml(file_name) !=3D 0:=0D
+ raise Exception(gen_cfg_data.get_last_error())=0D
+ else:=0D
+ raise Exception('Unsupported file "%s" !' % file_name)=0D
+ return gen_cfg_data=0D
+=0D
+ def about(self):=0D
+ msg =3D 'Configuration Editor\n--------------------------------\nV=
ersion 0.8\n2020'=0D
+ lines =3D msg.split('\n')=0D
+ width =3D 30=0D
+ text =3D []=0D
+ for line in lines:=0D
+ text.append(line.center(width, ' '))=0D
+ messagebox.showinfo('Config Editor', '\n'.join(text))=0D
+=0D
+ def update_last_dir (self, path):=0D
+ self.last_dir =3D os.path.dirname(path)=0D
+=0D
+ def get_open_file_name(self, ftype):=0D
+ if self.is_config_data_loaded():=0D
+ if ftype =3D=3D 'dlt':=0D
+ question =3D ''=0D
+ elif ftype =3D=3D 'bin':=0D
+ question =3D 'All configuration will be reloaded from BIN =
file, continue ?'=0D
+ elif ftype =3D=3D 'yaml':=0D
+ question =3D ''=0D
+ else:=0D
+ raise Exception('Unsupported file type !')=0D
+ if question:=0D
+ reply =3D messagebox.askquestion('', question, icon=3D'war=
ning')=0D
+ if reply =3D=3D 'no':=0D
+ return None=0D
+=0D
+ if ftype =3D=3D 'yaml':=0D
+ file_type =3D 'YAML or PKL'=0D
+ file_ext =3D 'pkl *Def.yaml'=0D
+ else:=0D
+ file_type =3D ftype.upper()=0D
+ file_ext =3D ftype=0D
+=0D
+ path =3D filedialog.askopenfilename(=0D
+ initialdir=3Dself.last_dir,=0D
+ title=3D"Load file",=0D
+ filetypes=3D(("%s files" % file_type, "*.%s" % file_ext), (=0D
+ "all files", "*.*")))=0D
+ if path:=0D
+ self.update_last_dir (path)=0D
+ return path=0D
+ else:=0D
+ return None=0D
+=0D
+=0D
+ def load_from_delta(self):=0D
+ path =3D self.get_open_file_name('dlt')=0D
+ if not path:=0D
+ return=0D
+ self.load_delta_file (path)=0D
+=0D
+ def load_delta_file (self, path):=0D
+ self.reload_config_data_from_bin(self.org_cfg_data_bin)=0D
+ try:=0D
+ self.cfg_data_obj.override_default_value(path)=0D
+ except Exception as e:=0D
+ messagebox.showerror('LOADING ERROR', str(e))=0D
+ return=0D
+ self.update_last_dir (path)=0D
+ self.refresh_config_data_page()=0D
+=0D
+ def load_from_bin(self):=0D
+ path =3D self.get_open_file_name('bin')=0D
+ if not path:=0D
+ return=0D
+ self.load_bin_file (path)=0D
+=0D
+ def load_bin_file (self, path):=0D
+ with open(path, 'rb') as fd:=0D
+ bin_data =3D bytearray(fd.read())=0D
+ if len(bin_data) < len(self.org_cfg_data_bin):=0D
+ messagebox.showerror('Binary file size is smaller than what YA=
ML requires !')=0D
+ return=0D
+=0D
+ try:=0D
+ self.reload_config_data_from_bin(bin_data)=0D
+ except Exception as e:=0D
+ messagebox.showerror('LOADING ERROR', str(e))=0D
+ return=0D
+=0D
+ def load_cfg_file(self, path):=0D
+ # Save current values in widget and clear database=0D
+ self.clear_widgets_inLayout()=0D
+ self.left.delete(*self.left.get_children())=0D
+=0D
+ self.cfg_data_obj =3D self.load_config_data(path)=0D
+=0D
+ self.update_last_dir (path)=0D
+ self.org_cfg_data_bin =3D self.cfg_data_obj.generate_binary_array(=
)=0D
+ self.build_config_page_tree(self.cfg_data_obj.get_cfg_page()['root=
'], '')=0D
+=0D
+ for menu in self.menu_string:=0D
+ self.file_menu.entryconfig(menu, state=3D"normal")=0D
+=0D
+ return 0=0D
+=0D
+ def load_from_yaml(self):=0D
+ path =3D self.get_open_file_name('yaml')=0D
+ if not path:=0D
+ return=0D
+=0D
+ self.load_cfg_file(path)=0D
+=0D
+ def get_save_file_name (self, extension):=0D
+ path =3D filedialog.asksaveasfilename(=0D
+ initialdir=3Dself.last_dir,=0D
+ title=3D"Save file",=0D
+ defaultextension=3Dextension)=0D
+ if path:=0D
+ self.last_dir =3D os.path.dirname(path)=0D
+ return path=0D
+ else:=0D
+ return None=0D
+=0D
+ def save_delta_file(self, full=3DFalse):=0D
+ path =3D self.get_save_file_name (".dlt")=0D
+ if not path:=0D
+ return=0D
+=0D
+ self.update_config_data_on_page()=0D
+ new_data =3D self.cfg_data_obj.generate_binary_array()=0D
+ self.cfg_data_obj.generate_delta_file_from_bin (path, self.org_cfg=
_data_bin, new_data, full)=0D
+=0D
+ def save_to_delta(self):=0D
+ self.save_delta_file()=0D
+=0D
+ def save_full_to_delta(self):=0D
+ self.save_delta_file(True)=0D
+=0D
+ def save_to_bin(self):=0D
+ path =3D self.get_save_file_name (".bin")=0D
+ if not path:=0D
+ return=0D
+=0D
+ self.update_config_data_on_page()=0D
+ with open(path, 'wb') as fd:=0D
+ bins =3D self.cfg_data_obj.generate_binary_array()=0D
+ fd.write(bins)=0D
+=0D
+=0D
+ def refresh_config_data_page(self):=0D
+ self.clear_widgets_inLayout()=0D
+ self.on_config_page_select_change(None)=0D
+=0D
+ def reload_config_data_from_bin(self, bin_dat):=0D
+ self.cfg_data_obj.load_default_from_bin(bin_dat)=0D
+ self.refresh_config_data_page()=0D
+=0D
+ def set_config_item_value(self, item, value_str):=0D
+ itype =3D item['type'].split(',')[0]=0D
+ if itype =3D=3D "Table":=0D
+ new_value =3D value_str=0D
+ elif itype =3D=3D "EditText":=0D
+ length =3D (self.cfg_data_obj.get_cfg_item_length(item) + 7) /=
/ 8=0D
+ new_value =3D value_str[:length]=0D
+ if item['value'].startswith("'"):=0D
+ new_value =3D "'%s'" % new_value=0D
+ else:=0D
+ try:=0D
+ new_value =3D self.cfg_data_obj.reformat_value_str (value_=
str, self.cfg_data_obj.get_cfg_item_length(item), item['value'])=0D
+ except:=0D
+ print("WARNING: Failed to format value string '%s' for '%s=
' !" % (value_str, item['path']))=0D
+ new_value =3D item['value']=0D
+=0D
+ if item['value'] !=3D new_value:=0D
+ if self.debug:=0D
+ print('Update %s from %s to %s !' % (item['cname'], item[=
'value'], new_value))=0D
+ item['value'] =3D new_value=0D
+=0D
+=0D
+ def get_config_data_item_from_widget(self, widget, label=3DFalse):=0D
+ name =3D self.get_object_name(widget)=0D
+ if not name or not len(self.page_list):=0D
+ return None=0D
+=0D
+ if name.startswith('LABEL_'):=0D
+ if label:=0D
+ path =3D name[6:]=0D
+ else:=0D
+ return None=0D
+ else:=0D
+ path =3D name=0D
+ item =3D self.cfg_data_obj.get_item_by_path (path)=0D
+ return item=0D
+=0D
+=0D
+ def update_config_data_from_widget(self, widget, args):=0D
+ item =3D self.get_config_data_item_from_widget(widget)=0D
+ if item is None:=0D
+ return=0D
+ elif not item:=0D
+ if isinstance(widget, tkinter.Label):=0D
+ return=0D
+ raise Exception('Failed to find "%s" !' % self.get_object_name=
(widget))=0D
+=0D
+ itype =3D item['type'].split(',')[0]=0D
+ if itype =3D=3D "Combo":=0D
+ opt_list =3D self.cfg_data_obj.get_cfg_item_options (item)=0D
+ tmp_list =3D [opt[0] for opt in opt_list]=0D
+ idx =3D widget.current()=0D
+ self.set_config_item_value(item, tmp_list[idx])=0D
+ elif itype in ["EditNum", "EditText"]:=0D
+ self.set_config_item_value(item, widget.get())=0D
+ elif itype in ["Table"]:=0D
+ new_value =3D bytes_to_bracket_str(widget.get())=0D
+ self.set_config_item_value(item, new_value)=0D
+=0D
+=0D
+ def evaluate_condition(self, item):=0D
+ try:=0D
+ result =3D self.cfg_data_obj.evaluate_condition(item)=0D
+ except:=0D
+ print("WARNING: Condition '%s' is invalid for '%s' !" % (item[=
'condition'], item['path']))=0D
+ result =3D 1=0D
+ return result=0D
+=0D
+=0D
+ def add_config_item(self, item, row):=0D
+ parent =3D self.right_grid=0D
+=0D
+ name =3D tkinter.Label(parent, text=3Ditem['name'], anchor=3D"w")=
=0D
+=0D
+ parts =3D item['type'].split(',')=0D
+ itype =3D parts[0]=0D
+ widget =3D None=0D
+=0D
+ if itype =3D=3D "Combo":=0D
+ # Build=0D
+ opt_list =3D self.cfg_data_obj.get_cfg_item_options (item)=0D
+ current_value =3D self.cfg_data_obj.get_cfg_item_value (item, =
False)=0D
+ option_list =3D []=0D
+ current =3D None=0D
+=0D
+ for idx, option in enumerate(opt_list):=0D
+ option_str =3D option[0]=0D
+ try:=0D
+ option_value =3D self.cfg_data_obj.get_value(option_st=
r, len(option_str), False)=0D
+ except:=0D
+ option_value =3D 0=0D
+ print('WARNING: Option "%s" has invalid format for "%s=
" !' % (option_str, item['path']))=0D
+ if option_value =3D=3D current_value:=0D
+ current =3D idx=0D
+ option_list.append(option[1])=0D
+=0D
+=0D
+ widget =3D ttk.Combobox(parent, value=3Doption_list, state=3D"=
readonly")=0D
+ widget.bind("<<ComboboxSelected>>", self.combo_select_changed)=
=0D
+ widget.unbind_class("TCombobox", "<MouseWheel>")=0D
+=0D
+ if current is None:=0D
+ print('WARNING: Value "%s" is an invalid option for "%s" !=
' %=0D
+ (current_value, item['path']))=0D
+ else:=0D
+ widget.current(current)=0D
+=0D
+ elif itype in ["EditNum", "EditText"]:=0D
+ txt_val =3D tkinter.StringVar()=0D
+ widget =3D tkinter.Entry(parent, textvariable=3Dtxt_val)=0D
+ value =3D item['value'].strip("'")=0D
+ if itype in ["EditText"]:=0D
+ txt_val.trace(=0D
+ 'w',=0D
+ lambda *args: self.limit_entry_size(txt_val, (self.cfg=
_data_obj.get_cfg_item_length(item) + 7) // 8))=0D
+ elif itype in ["EditNum"]:=0D
+ value =3D item['value'].strip("{").strip("}").strip()=0D
+ widget.bind("<FocusOut>", self.edit_num_finished)=0D
+ txt_val.set(value)=0D
+=0D
+ elif itype in ["Table"]:=0D
+ bins =3D self.cfg_data_obj.get_cfg_item_value(item, True)=0D
+ col_hdr =3D item['option'].split(',')=0D
+ widget =3D custom_table(parent, col_hdr, bins)=0D
+=0D
+ else:=0D
+ if itype and itype not in ["Reserved"]:=0D
+ print ("WARNING: Type '%s' is invalid for '%s' !" % (itype=
, item['path']))=0D
+=0D
+ if widget:=0D
+ ttp =3D create_tool_tip(widget, item['help'])=0D
+ self.set_object_name(name, 'LABEL_' + item['path'])=0D
+ self.set_object_name(widget, item['path'])=0D
+ name.grid(row=3Drow, column=3D0, padx=3D10, pady=3D5, sticky=
=3D"nsew")=0D
+ widget.grid(row=3Drow + 1, rowspan=3D1, column=3D0, padx=3D10,=
pady=3D5, sticky=3D"nsew")=0D
+=0D
+ def update_config_data_on_page(self):=0D
+ self.walk_widgets_in_layout(self.right_grid,=0D
+ self.update_config_data_from_widget)=0D
+=0D
+=0D
+=0D
+if __name__ =3D=3D '__main__':=0D
+ root =3D tkinter.Tk()=0D
+ app =3D application(master=3Droot)=0D
+ root.title("Config Editor")=0D
+ root.mainloop()=0D
+=0D
diff --git a/IntelFsp2Pkg/Tools/ConfigEditor/GenCfgData.py b/IntelFsp2Pkg/T=
ools/ConfigEditor/GenCfgData.py
new file mode 100644
index 0000000000..de02974cce
--- /dev/null
+++ b/IntelFsp2Pkg/Tools/ConfigEditor/GenCfgData.py
@@ -0,0 +1,1962 @@
+## @ GenCfgData.py=0D
+#=0D
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>=0D
+# SPDX-License-Identifier: BSD-2-Clause-Patent=0D
+#=0D
+##=0D
+=0D
+import os=0D
+import sys=0D
+import re=0D
+import struct=0D
+import marshal=0D
+import pprint=0D
+import string=0D
+import operator as op=0D
+import ast=0D
+import binascii=0D
+from datetime import date=0D
+from collections import OrderedDict=0D
+=0D
+from CommonUtility import *=0D
+=0D
+# Generated file copyright header=0D
+__copyright_tmp__ =3D """/** @file=0D
+=0D
+ Platform Configuration %s File.=0D
+=0D
+ Copyright (c) %4d, Intel Corporation. All rights reserved.<BR>=0D
+ SPDX-License-Identifier: BSD-2-Clause-Patent=0D
+=0D
+ This file is automatically generated. Please do NOT modify !!!=0D
+=0D
+**/=0D
+"""=0D
+=0D
+def get_copyright_header (file_type, allow_modify =3D False):=0D
+ file_description =3D {=0D
+ 'yaml': 'Boot Setting',=0D
+ 'dlt' : 'Delta',=0D
+ 'inc' : 'C Binary Blob',=0D
+ 'h' : 'C Struct Header'=0D
+ }=0D
+ if file_type in ['yaml', 'dlt']:=0D
+ comment_char =3D '#'=0D
+ else:=0D
+ comment_char =3D ''=0D
+ lines =3D __copyright_tmp__.split('\n')=0D
+ if allow_modify:=0D
+ lines =3D [line for line in lines if 'Please do NOT modify' not in l=
ine]=0D
+ copyright_hdr =3D '\n'.join('%s%s' % (comment_char, line) for line in =
lines)[:-1] + '\n'=0D
+ return copyright_hdr % (file_description[file_type], date.today().year=
)=0D
+=0D
+def check_quote (text):=0D
+ if (text[0] =3D=3D "'" and text[-1] =3D=3D "'") or (text[0] =3D=3D '"'=
and text[-1] =3D=3D '"'):=0D
+ return True=0D
+ return False=0D
+=0D
+def strip_quote (text):=0D
+ new_text =3D text.strip()=0D
+ if check_quote (new_text):=0D
+ return new_text[1:-1]=0D
+ return text=0D
+=0D
+def strip_delimiter (text, delim):=0D
+ new_text =3D text.strip()=0D
+ if new_text:=0D
+ if new_text[0] =3D=3D delim[0] and new_text[-1] =3D=3D delim[-1]:=
=0D
+ return new_text[1:-1]=0D
+ return text=0D
+=0D
+def bytes_to_bracket_str (bytes):=0D
+ return '{ %s }' % (', '.join('0x%02x' % i for i in bytes))=0D
+=0D
+def array_str_to_value (val_str):=0D
+ val_str =3D val_str.strip()=0D
+ val_str =3D strip_delimiter (val_str, '{}')=0D
+ val_str =3D strip_quote (val_str)=0D
+ value =3D 0=0D
+ for each in val_str.split(',')[::-1]:=0D
+ each =3D each.strip()=0D
+ value =3D (value << 8) | int(each, 0)=0D
+ return value=0D
+=0D
+def write_lines (lines, file):=0D
+ fo =3D open(file, "w")=0D
+ fo.write (''.join ([x[0] for x in lines]))=0D
+ fo.close ()=0D
+=0D
+def read_lines (file):=0D
+ if not os.path.exists(file):=0D
+ test_file =3D os.path.basename(file)=0D
+ if os.path.exists(test_file):=0D
+ file =3D test_file=0D
+ fi =3D open (file, 'r')=0D
+ lines =3D fi.readlines ()=0D
+ fi.close ()=0D
+ return lines=0D
+=0D
+def expand_file_value (path, value_str):=0D
+ result =3D bytearray()=0D
+ match =3D re.match("\{\s*FILE:(.+)\}", value_str)=0D
+ if match:=0D
+ file_list =3D match.group(1).split(',')=0D
+ for file in file_list:=0D
+ file =3D file.strip()=0D
+ bin_path =3D os.path.join(path, file)=0D
+ result.extend(bytearray(open(bin_path, 'rb').read()))=0D
+ return result=0D
+=0D
+class ExpressionEval(ast.NodeVisitor):=0D
+ operators =3D {=0D
+ ast.Add: op.add,=0D
+ ast.Sub: op.sub,=0D
+ ast.Mult: op.mul,=0D
+ ast.Div: op.floordiv,=0D
+ ast.Mod: op.mod,=0D
+ ast.Eq: op.eq,=0D
+ ast.NotEq: op.ne,=0D
+ ast.Gt: op.gt,=0D
+ ast.Lt: op.lt,=0D
+ ast.GtE: op.ge,=0D
+ ast.LtE: op.le,=0D
+ ast.BitXor: op.xor,=0D
+ ast.BitAnd: op.and_,=0D
+ ast.BitOr: op.or_,=0D
+ ast.Invert: op.invert,=0D
+ ast.USub: op.neg=0D
+ }=0D
+=0D
+=0D
+ def __init__(self):=0D
+ self._debug =3D False=0D
+ self._expression =3D ''=0D
+ self._namespace =3D {}=0D
+ self._get_variable =3D None=0D
+=0D
+ def eval(self, expr, vars=3D{}):=0D
+ self._expression =3D expr=0D
+ if type(vars) is dict:=0D
+ self._namespace =3D vars=0D
+ self._get_variable =3D None=0D
+ else:=0D
+ self._namespace =3D {}=0D
+ self._get_variable =3D vars=0D
+ node =3D ast.parse(self._expression, mode=3D'eval')=0D
+ result =3D self.visit(node.body)=0D
+ if self._debug:=0D
+ print ('EVAL [ %s ] =3D %s' % (expr, str(result)))=0D
+ return result=0D
+=0D
+ def visit_Name(self, node):=0D
+ if self._get_variable is not None:=0D
+ return self._get_variable(node.id)=0D
+ else:=0D
+ return self._namespace[node.id]=0D
+=0D
+ def visit_Num(self, node):=0D
+ return node.n=0D
+=0D
+ def visit_NameConstant(self, node):=0D
+ return node.value=0D
+=0D
+ def visit_BoolOp(self, node):=0D
+ result =3D False=0D
+ if isinstance(node.op, ast.And):=0D
+ for value in node.values:=0D
+ result =3D self.visit(value)=0D
+ if not result:=0D
+ break=0D
+ elif isinstance(node.op, ast.Or):=0D
+ for value in node.values:=0D
+ result =3D self.visit(value)=0D
+ if result:=0D
+ break=0D
+ return True if result else False=0D
+=0D
+ def visit_UnaryOp(self, node):=0D
+ val =3D self.visit(node.operand)=0D
+ return operators[type(node.op)](val)=0D
+=0D
+ def visit_BinOp(self, node):=0D
+ lhs =3D self.visit(node.left)=0D
+ rhs =3D self.visit(node.right)=0D
+ return ExpressionEval.operators[type(node.op)](lhs, rhs)=0D
+=0D
+ def visit_Compare(self, node):=0D
+ right =3D self.visit(node.left)=0D
+ result =3D True=0D
+ for operation, comp in zip(node.ops, node.comparators):=0D
+ if not result:=0D
+ break=0D
+ left =3D right=0D
+ right =3D self.visit(comp)=0D
+ result =3D ExpressionEval.operators[type(operation)](left, rig=
ht)=0D
+ return result=0D
+=0D
+ def visit_Call(self, node):=0D
+ if node.func.id in ['ternary']:=0D
+ condition =3D self.visit (node.args[0])=0D
+ val_true =3D self.visit (node.args[1])=0D
+ val_false =3D self.visit (node.args[2])=0D
+ return val_true if condition else val_false=0D
+ elif node.func.id in ['offset', 'length']:=0D
+ if self._get_variable is not None:=0D
+ return self._get_variable(node.args[0].s, node.func.id)=0D
+ else:=0D
+ raise ValueError("Unsupported function: " + repr(node))=0D
+=0D
+ def generic_visit(self, node):=0D
+ raise ValueError("malformed node or string: " + repr(node))=0D
+=0D
+=0D
+class CFG_YAML():=0D
+ TEMPLATE =3D 'template'=0D
+ CONFIGS =3D 'configs'=0D
+ VARIABLE =3D 'variable'=0D
+=0D
+ def __init__ (self):=0D
+ self.log_line =3D False=0D
+ self.allow_template =3D False=0D
+ self.cfg_tree =3D None=0D
+ self.tmp_tree =3D None=0D
+ self.var_dict =3D None=0D
+ self.def_dict =3D {}=0D
+ self.yaml_path =3D ''=0D
+ self.lines =3D []=0D
+ self.full_lines =3D []=0D
+ self.index =3D 0=0D
+ self.re_expand =3D re.compile (r'(.+:\s+|\s*\-\s*)!expand\s+\{\s*=
(\w+_TMPL)\s*:\s*\[(.+)]\s*\}')=0D
+ self.re_include =3D re.compile (r'(.+:\s+|\s*\-\s*)!include\s+(.+)=
')=0D
+=0D
+ @staticmethod=0D
+ def count_indent (line):=0D
+ return next((i for i, c in enumerate(line) if not c.isspace()), le=
n(line))=0D
+=0D
+ @staticmethod=0D
+ def substitue_args (text, arg_dict):=0D
+ for arg in arg_dict:=0D
+ text =3D text.replace ('$' + arg, arg_dict[arg])=0D
+ return text=0D
+=0D
+ @staticmethod=0D
+ def dprint (*args):=0D
+ pass=0D
+=0D
+ def process_include (self, line, insert =3D True):=0D
+ match =3D self.re_include.match (line)=0D
+ if not match:=0D
+ raise Exception ("Invalid !include format '%s' !" % line.strip=
())=0D
+=0D
+ prefix =3D match.group(1)=0D
+ include =3D match.group(2)=0D
+ if prefix.strip() =3D=3D '-':=0D
+ prefix =3D ''=0D
+ adjust =3D 0=0D
+ else:=0D
+ adjust =3D 2=0D
+=0D
+ include =3D strip_quote (include)=0D
+ request =3D CFG_YAML.count_indent (line) + adjust=0D
+=0D
+ if self.log_line:=0D
+ # remove the include line itself=0D
+ del self.full_lines[-1]=0D
+=0D
+ inc_path =3D os.path.join (self.yaml_path, include)=0D
+ if not os.path.exists(inc_path):=0D
+ # try relative path to project root=0D
+ try_path =3D os.path.join(os.path.dirname (os.path.realpath(__=
file__)), "../..", include)=0D
+ if os.path.exists(try_path):=0D
+ inc_path =3D try_path=0D
+ else:=0D
+ raise Exception ("ERROR: Cannot open file '%s'." % inc_pat=
h)=0D
+=0D
+ lines =3D read_lines (inc_path)=0D
+=0D
+ current =3D 0=0D
+ same_line =3D False=0D
+ for idx, each in enumerate (lines):=0D
+ start =3D each.lstrip()=0D
+ if start =3D=3D '' or start[0] =3D=3D '#':=0D
+ continue=0D
+=0D
+ if start[0] =3D=3D '>':=0D
+ # append the content directly at the same line=0D
+ same_line =3D True=0D
+=0D
+ start =3D idx=0D
+ current =3D CFG_YAML.count_indent (each)=0D
+ break=0D
+=0D
+ lines =3D lines[start+1:] if same_line else lines[start:]=0D
+ leading =3D ''=0D
+ if same_line:=0D
+ request =3D len(prefix)=0D
+ leading =3D '>'=0D
+=0D
+ lines =3D [prefix + '%s\n' % leading] + [' ' * request + i[current=
:] for i in lines]=0D
+ if insert:=0D
+ self.lines =3D lines + self.lines=0D
+=0D
+ return lines=0D
+=0D
+ def process_expand (self, line):=0D
+ match =3D self.re_expand.match(line)=0D
+ if not match:=0D
+ raise Exception ("Invalid !expand format '%s' !" % line.strip(=
))=0D
+ lines =3D []=0D
+ prefix =3D match.group(1)=0D
+ temp_name =3D match.group(2)=0D
+ args =3D match.group(3)=0D
+=0D
+ if prefix.strip() =3D=3D '-':=0D
+ indent =3D 0=0D
+ else:=0D
+ indent =3D 2=0D
+ lines =3D self.process_expand_template (temp_name, prefix, ar=
gs, indent)=0D
+ self.lines =3D lines + self.lines=0D
+=0D
+=0D
+ def process_expand_template (self, temp_name, prefix, args, indent =3D=
2):=0D
+ # expand text with arg substitution=0D
+ if temp_name not in self.tmp_tree:=0D
+ raise Exception ("Could not find template '%s' !" % temp_name)=
=0D
+ parts =3D args.split(',')=0D
+ parts =3D [i.strip() for i in parts]=0D
+ num =3D len(parts)=0D
+ arg_dict =3D dict(zip( ['(%d)' % (i + 1) for i in range(num)], par=
ts))=0D
+ str_data =3D self.tmp_tree[temp_name]=0D
+ text =3D DefTemplate(str_data).safe_substitute(self.def_dict)=0D
+ text =3D CFG_YAML.substitue_args (text, arg_dict)=0D
+ target =3D CFG_YAML.count_indent (prefix) + indent=0D
+ current =3D CFG_YAML.count_indent (text)=0D
+ padding =3D target * ' '=0D
+ if indent =3D=3D 0:=0D
+ leading =3D []=0D
+ else:=0D
+ leading =3D [prefix + '\n']=0D
+ text =3D leading + [(padding + i + '\n')[current:] for i in text.s=
plitlines()]=0D
+ return text=0D
+=0D
+=0D
+ def load_file (self, yaml_file):=0D
+ self.index =3D 0=0D
+ self.lines =3D read_lines (yaml_file)=0D
+=0D
+=0D
+ def peek_line (self):=0D
+ if len(self.lines) =3D=3D 0:=0D
+ return None=0D
+ else:=0D
+ return self.lines[0]=0D
+=0D
+=0D
+ def put_line (self, line):=0D
+ self.lines.insert (0, line)=0D
+ if self.log_line:=0D
+ del self.full_lines[-1]=0D
+=0D
+=0D
+ def get_line (self):=0D
+ if len(self.lines) =3D=3D 0:=0D
+ return None=0D
+ else:=0D
+ line =3D self.lines.pop(0)=0D
+ if self.log_line:=0D
+ self.full_lines.append (line.rstrip())=0D
+ return line=0D
+=0D
+=0D
+ def get_multiple_line (self, indent):=0D
+ text =3D ''=0D
+ newind =3D indent + 1=0D
+ while True:=0D
+ line =3D self.peek_line ()=0D
+ if line is None:=0D
+ break=0D
+ sline =3D line.strip()=0D
+ if sline !=3D '':=0D
+ newind =3D CFG_YAML.count_indent(line)=0D
+ if newind <=3D indent:=0D
+ break=0D
+ self.get_line ()=0D
+ if sline !=3D '':=0D
+ text =3D text + line=0D
+ return text=0D
+=0D
+=0D
+ def traverse_cfg_tree (self, handler):=0D
+ def _traverse_cfg_tree (root, level =3D 0):=0D
+ # config structure=0D
+ for key in root:=0D
+ if type(root[key]) is OrderedDict:=0D
+ level +=3D 1=0D
+ handler (key, root[key], level)=0D
+ _traverse_cfg_tree (root[key], level)=0D
+ level -=3D 1=0D
+ _traverse_cfg_tree (self.cfg_tree)=0D
+=0D
+=0D
+ def count (self):=0D
+ def _count (name, cfgs, level):=0D
+ num[0] +=3D 1=0D
+ num =3D [0]=0D
+ self.traverse_cfg_tree (_count)=0D
+ return num[0]=0D
+=0D
+=0D
+ def parse (self, parent_name =3D '', curr =3D None, level =3D 0):=0D
+ child =3D None=0D
+ last_indent =3D None=0D
+ temp_chk =3D {}=0D
+=0D
+ while True:=0D
+ line =3D self.get_line ()=0D
+ if line is None:=0D
+ break=0D
+=0D
+ curr_line =3D line.strip()=0D
+ if curr_line =3D=3D '' or curr_line[0] =3D=3D '#':=0D
+ continue=0D
+=0D
+ indent =3D CFG_YAML.count_indent(line)=0D
+ if last_indent is None:=0D
+ last_indent =3D indent=0D
+=0D
+ if indent !=3D last_indent:=0D
+ # outside of current block, put the line back to queue=0D
+ self.put_line (' ' * indent + curr_line)=0D
+=0D
+ if curr_line.endswith (': >'):=0D
+ # multiline marker=0D
+ old_count =3D len(self.full_lines)=0D
+ line =3D self.get_multiple_line (indent)=0D
+ if self.log_line and not self.allow_template and '!include=
' in line:=0D
+ # expand include in template=0D
+ new_lines =3D []=0D
+ lines =3D line.splitlines()=0D
+ for idx, each in enumerate(lines):=0D
+ if '!include ' in each:=0D
+ new_line =3D ''.join(self.process_include (eac=
h, False))=0D
+ new_lines.append(new_line)=0D
+ else:=0D
+ new_lines.append(each)=0D
+ self.full_lines =3D self.full_lines[:old_count] + new_=
lines=0D
+ curr_line =3D curr_line + line=0D
+=0D
+ if indent > last_indent:=0D
+ # child nodes=0D
+ if child is None:=0D
+ raise Exception ('Unexpected format at line: %s' % (cu=
rr_line))=0D
+=0D
+ level +=3D 1=0D
+ self.parse (key, child, level)=0D
+ level -=3D 1=0D
+=0D
+ line =3D self.peek_line ()=0D
+ if line is not None:=0D
+ curr_line =3D line.strip()=0D
+ indent =3D CFG_YAML.count_indent(line)=0D
+ if indent >=3D last_indent:=0D
+ # consume the line=0D
+ self.get_line ()=0D
+ else:=0D
+ # end of file=0D
+ indent =3D -1=0D
+=0D
+ if curr is None:=0D
+ curr =3D OrderedDict()=0D
+=0D
+ if indent < last_indent:=0D
+ return curr=0D
+=0D
+ marker1 =3D curr_line[0]=0D
+ marker2 =3D curr_line[-1]=0D
+ start =3D 1 if marker1 =3D=3D '-' else 0=0D
+ pos =3D curr_line.find(': ')=0D
+ if pos > 0:=0D
+ child =3D None=0D
+ key =3D curr_line[start:pos].strip()=0D
+ if curr_line[pos + 2] =3D=3D '>':=0D
+ curr[key] =3D curr_line[pos + 3:]=0D
+ else:=0D
+ # XXXX: !include / !expand=0D
+ if '!include ' in curr_line:=0D
+ self.process_include (line)=0D
+ elif '!expand ' in curr_line:=0D
+ if self.allow_template and not self.log_line:=0D
+ self.process_expand (line)=0D
+ else:=0D
+ value_str =3D curr_line[pos + 2:].strip()=0D
+ curr[key] =3D value_str=0D
+ if self.log_line and value_str[0] =3D=3D '{':=0D
+ # expand {FILE: xxxx} format in the log line=0D
+ if value_str[1:].rstrip().startswith('FILE:'):=
=0D
+ value_bytes =3D expand_file_value (self.ya=
ml_path, value_str)=0D
+ value_str =3D bytes_to_bracket_str (value_=
bytes)=0D
+ self.full_lines[-1] =3D line[:indent] + cu=
rr_line[:pos + 2] + value_str=0D
+=0D
+ elif marker2 =3D=3D ':':=0D
+ child =3D OrderedDict()=0D
+ key =3D curr_line[start:-1].strip()=0D
+ if key =3D=3D '$ACTION':=0D
+ # special virtual nodes, rename to ensure unique key=0D
+ key =3D '$ACTION_%04X' % self.index=0D
+ self.index +=3D 1=0D
+ if key in curr:=0D
+ if key not in temp_chk:=0D
+ # check for duplicated keys at same level=0D
+ temp_chk[key] =3D 1=0D
+ else:=0D
+ raise Exception ("Duplicated item '%s:%s' found !"=
% (parent_name, key))=0D
+=0D
+ curr[key] =3D child=0D
+ if self.var_dict is None and key =3D=3D CFG_YAML.VARIABLE:=
=0D
+ self.var_dict =3D child=0D
+ if self.tmp_tree is None and key =3D=3D CFG_YAML.TEMPLATE:=
=0D
+ self.tmp_tree =3D child=0D
+ if self.var_dict:=0D
+ for each in self.var_dict:=0D
+ txt =3D self.var_dict[each]=0D
+ if type(txt) is str:=0D
+ self.def_dict['(%s)' % each] =3D txt=0D
+ if self.tmp_tree and key =3D=3D CFG_YAML.CONFIGS:=0D
+ # apply template for the main configs=0D
+ self.allow_template =3D True=0D
+ else:=0D
+ child =3D None=0D
+ # - !include cfg_opt.yaml=0D
+ if '!include ' in curr_line:=0D
+ self.process_include (line)=0D
+=0D
+ return curr=0D
+=0D
+=0D
+ def load_yaml (self, opt_file):=0D
+ self.var_dict =3D None=0D
+ self.yaml_path =3D os.path.dirname (opt_file)=0D
+ self.load_file (opt_file)=0D
+ yaml_tree =3D self.parse ()=0D
+ self.tmp_tree =3D yaml_tree[CFG_YAML.TEMPLATE]=0D
+ self.cfg_tree =3D yaml_tree[CFG_YAML.CONFIGS]=0D
+ return self.cfg_tree=0D
+=0D
+=0D
+ def expand_yaml (self, opt_file):=0D
+ self.log_line =3D True=0D
+ self.load_yaml (opt_file)=0D
+ self.log_line =3D False=0D
+ text =3D '\n'.join (self.full_lines)=0D
+ self.full_lines =3D []=0D
+ return text=0D
+=0D
+=0D
+class DefTemplate(string.Template):=0D
+ idpattern =3D '\([_A-Z][_A-Z0-9]*\)|[_A-Z][_A-Z0-9]*'=0D
+=0D
+=0D
+class CGenCfgData:=0D
+ STRUCT =3D '$STRUCT'=0D
+ bits_width =3D {'b':1, 'B':8, 'W':16, 'D':32, 'Q':64}=0D
+ builtin_option =3D {'$EN_DIS' : [('0', 'Disable'), ('1', 'Enable')]}=0D
+ exclude_struct =3D ['GPIO_GPP_*', 'GPIO_CFG_DATA', 'GpioConfPad*', 'G=
pioPinConfig',=0D
+ 'BOOT_OPTION*', 'PLATFORMID_CFG_DATA', '\w+_Half[01]=
']=0D
+ include_tag =3D ['GPIO_CFG_DATA']=0D
+ keyword_set =3D set(['name', 'type', 'option', 'help', 'length', 'v=
alue', 'order', 'struct', 'condition'])=0D
+=0D
+ def __init__(self):=0D
+ self.initialize ()=0D
+=0D
+=0D
+ def initialize (self):=0D
+ self._cfg_tree =3D {}=0D
+ self._tmp_tree =3D {}=0D
+ self._cfg_list =3D []=0D
+ self._cfg_page =3D {'root': {'title': '', 'child': []}}=0D
+ self._cur_page =3D ''=0D
+ self._var_dict =3D {}=0D
+ self._def_dict =3D {}=0D
+ self._yaml_path =3D ''=0D
+=0D
+=0D
+ @staticmethod=0D
+ def deep_convert_dict (layer):=0D
+ # convert OrderedDict to list + dict=0D
+ new_list =3D layer=0D
+ if isinstance(layer, OrderedDict):=0D
+ new_list =3D list (layer.items())=0D
+ for idx, pair in enumerate (new_list):=0D
+ new_node =3D CGenCfgData.deep_convert_dict (pair[1])=0D
+ new_list[idx] =3D dict({pair[0] : new_node})=0D
+ return new_list=0D
+=0D
+=0D
+ @staticmethod=0D
+ def deep_convert_list (layer):=0D
+ if isinstance(layer, list):=0D
+ od =3D OrderedDict({})=0D
+ for each in layer:=0D
+ if isinstance(each, dict):=0D
+ key =3D next(iter(each))=0D
+ od[key] =3D CGenCfgData.deep_convert_list(each[key])=0D
+ return od=0D
+ else:=0D
+ return layer=0D
+=0D
+=0D
+ @staticmethod=0D
+ def expand_include_files (file_path, cur_dir =3D ''):=0D
+ if cur_dir =3D=3D '':=0D
+ cur_dir =3D os.path.dirname(file_path)=0D
+ file_path =3D os.path.basename(file_path)=0D
+=0D
+ input_file_path =3D os.path.join(cur_dir, file_path)=0D
+ file =3D open(input_file_path, "r")=0D
+ lines =3D file.readlines()=0D
+ file.close()=0D
+=0D
+ new_lines =3D []=0D
+ for line_num, line in enumerate(lines):=0D
+ match =3D re.match("^!include\s*(.+)?$", line.strip())=0D
+ if match:=0D
+ inc_path =3D match.group(1)=0D
+ tmp_path =3D os.path.join(cur_dir, inc_path)=0D
+ org_path =3D tmp_path=0D
+ if not os.path.exists(tmp_path):=0D
+ cur_dir =3D os.path.join(os.path.dirname (os.path.real=
path(__file__)), "..", "..")=0D
+ tmp_path =3D os.path.join(cur_dir, inc_path)=0D
+ if not os.path.exists(tmp_path):=0D
+ raise Exception ("ERROR: Cannot open include file '%s'=
." % org_path)=0D
+ else:=0D
+ new_lines.append (('# Included from file: %s\n' % inc_=
path, tmp_path, 0))=0D
+ new_lines.append (('# %s\n' % ('=3D' * 80), tmp_path, =
0))=0D
+ new_lines.extend (CGenCfgData.expand_include_files (in=
c_path, cur_dir))=0D
+ else:=0D
+ new_lines.append ((line, input_file_path, line_num))=0D
+=0D
+ return new_lines=0D
+=0D
+=0D
+ @staticmethod=0D
+ def format_struct_field_name (input, count =3D 0):=0D
+ name =3D ''=0D
+ cap =3D True=0D
+ if '_' in input:=0D
+ input =3D input.lower()=0D
+ for each in input:=0D
+ if each =3D=3D '_':=0D
+ cap =3D True=0D
+ continue=0D
+ elif cap:=0D
+ each =3D each.upper()=0D
+ cap =3D False=0D
+ name =3D name + each=0D
+=0D
+ if count > 1:=0D
+ name =3D '%s[%d]' % (name, count)=0D
+=0D
+ return name=0D
+=0D
+ def get_last_error (self):=0D
+ return ''=0D
+=0D
+=0D
+ def get_variable (self, var, attr =3D 'value'):=0D
+ if var in self._var_dict:=0D
+ var =3D self._var_dict[var]=0D
+ return var=0D
+=0D
+ item =3D self.locate_cfg_item (var, False)=0D
+ if item is None:=0D
+ raise ValueError ("Cannot find variable '%s' !" % var)=0D
+=0D
+ if item:=0D
+ if 'indx' in item:=0D
+ item =3D self.get_item_by_index (item['indx'])=0D
+ if attr =3D=3D 'offset':=0D
+ var =3D item['offset']=0D
+ elif attr =3D=3D 'length':=0D
+ var =3D item['length']=0D
+ elif attr =3D=3D 'value':=0D
+ var =3D self.get_cfg_item_value (item)=0D
+ else:=0D
+ raise ValueError ("Unsupported variable attribute '%s' !" =
% attr)=0D
+ return var=0D
+=0D
+=0D
+ def eval (self, expr):=0D
+ def _handler (pattern):=0D
+ if pattern.group(1):=0D
+ target =3D 1=0D
+ else:=0D
+ target =3D 2=0D
+ result =3D self.get_variable(pattern.group(target))=0D
+ if result is None:=0D
+ raise ValueError('Unknown variable $(%s) !' % pattern.grou=
p(target))=0D
+ return hex(result)=0D
+=0D
+ expr_eval =3D ExpressionEval ()=0D
+ if '$' in expr:=0D
+ # replace known variable first=0D
+ expr =3D re.sub(r'\$\(([_a-zA-Z][\w\.]*)\)|\$([_a-zA-Z][\w\.]*=
)', _handler, expr)=0D
+ return expr_eval.eval(expr, self.get_variable)=0D
+=0D
+=0D
+ def get_cfg_list (self, page_id =3D None):=0D
+ if page_id is None:=0D
+ # return full list=0D
+ return self._cfg_list=0D
+ else:=0D
+ # build a new list for items under a page ID=0D
+ cfgs =3D [i for i in self._cfg_list if i['cname'] and (i['pag=
e'] =3D=3D page_id)]=0D
+ return cfgs=0D
+=0D
+=0D
+ def get_cfg_page (self):=0D
+ return self._cfg_page=0D
+=0D
+ def get_cfg_item_length (self, item):=0D
+ return item['length']=0D
+=0D
+ def get_cfg_item_value (self, item, array =3D False):=0D
+ value_str =3D item['value']=0D
+ length =3D item['length']=0D
+ return self.get_value (value_str, length, array)=0D
+=0D
+=0D
+ def format_value_to_str (self, value, bit_length, old_value =3D ''):=0D
+ # value is always int=0D
+ length =3D (bit_length + 7) // 8=0D
+ fmt =3D ''=0D
+ if old_value.startswith ('0x'):=0D
+ fmt =3D '0x'=0D
+ elif old_value and (old_value[0] in ['"', "'", '{']):=0D
+ fmt =3D old_value[0]=0D
+ else:=0D
+ fmt =3D ''=0D
+=0D
+ bvalue =3D value_to_bytearray (value, length)=0D
+ if fmt in ['"', "'"]:=0D
+ svalue =3D bvalue.rstrip(b'\x00').decode()=0D
+ value_str =3D fmt + svalue + fmt=0D
+ elif fmt =3D=3D "{":=0D
+ value_str =3D '{ ' + ', '.join(['0x%02x' % i for i in bvalue])=
+ ' }'=0D
+ elif fmt =3D=3D '0x':=0D
+ hex_len =3D length * 2=0D
+ if len(old_value) =3D=3D hex_len + 2:=0D
+ fstr =3D '0x%%0%dX' % hex_len=0D
+ else:=0D
+ fstr =3D '0x%X'=0D
+ value_str =3D fstr % value=0D
+ else:=0D
+ if length <=3D 2:=0D
+ value_str =3D '%d' % value=0D
+ elif length <=3D 8:=0D
+ value_str =3D '0x%x' % value=0D
+ else:=0D
+ value_str =3D '{ ' + ', '.join(['0x%02x' % i for i in bval=
ue]) + ' }'=0D
+ return value_str=0D
+=0D
+=0D
+ def reformat_value_str (self, value_str, bit_length, old_value =3D Non=
e):=0D
+ value =3D self.parse_value (value_str, bit_length, False)=0D
+ if old_value is None:=0D
+ old_value =3D value_str=0D
+ new_value =3D self.format_value_to_str (value, bit_length, old_val=
ue)=0D
+ return new_value=0D
+=0D
+=0D
+ def get_value (self, value_str, bit_length, array =3D True):=0D
+ value_str =3D value_str.strip()=0D
+ if value_str[0] =3D=3D "'" and value_str[-1] =3D=3D "'" or \=0D
+ value_str[0] =3D=3D '"' and value_str[-1] =3D=3D '"':=0D
+ value_str =3D value_str[1:-1]=0D
+ bvalue =3D bytearray (value_str.encode())=0D
+ if len(bvalue) =3D=3D 0:=0D
+ bvalue =3D bytearray(b'\x00')=0D
+ if array:=0D
+ return bvalue=0D
+ else:=0D
+ return bytes_to_value (bvalue)=0D
+ else:=0D
+ if value_str[0] in '{' :=0D
+ value_str =3D value_str[1:-1].strip()=0D
+ value =3D 0=0D
+ for each in value_str.split(',')[::-1]:=0D
+ each =3D each.strip()=0D
+ value =3D (value << 8) | int(each, 0)=0D
+ if array:=0D
+ length =3D (bit_length + 7) // 8=0D
+ return value_to_bytearray (value, length)=0D
+ else:=0D
+ return value=0D
+=0D
+=0D
+ def parse_value (self, value_str, bit_length, array =3D True):=0D
+ length =3D (bit_length + 7) // 8=0D
+ if check_quote(value_str):=0D
+ value_str =3D bytes_to_bracket_str(value_str[1:-1].encode())=0D
+ elif (',' in value_str) and (value_str[0] !=3D '{'):=0D
+ value_str =3D '{ %s }' % value_str=0D
+ if value_str[0] =3D=3D '{':=0D
+ result =3D expand_file_value (self._yaml_path, value_str)=0D
+ if len(result) =3D=3D 0 :=0D
+ bin_list =3D value_str[1:-1].split(',')=0D
+ value =3D 0=0D
+ bit_len =3D 0=0D
+ unit_len =3D 1=0D
+ for idx, element in enumerate(bin_list):=0D
+ each =3D element.strip()=0D
+ if len(each) =3D=3D 0:=0D
+ continue=0D
+=0D
+ in_bit_field =3D False=0D
+ if each[0] in "'" + '"':=0D
+ each_value =3D bytearray(each[1:-1], 'utf-8')=0D
+ elif ':' in each:=0D
+ match =3D re.match("^(.+):(\d+)([b|B|W|D|Q])$",=
each)=0D
+ if match is None:=0D
+ raise SystemExit("Exception: Invald value list=
format '%s' !" % each)=0D
+ if match.group(1) =3D=3D '0' and match.group(2) =
=3D=3D '0':=0D
+ unit_len =3D CGenCfgData.bits_width[match.grou=
p(3)] // 8=0D
+ cur_bit_len =3D int(match.group(2)) * CGenCfgData.=
bits_width[match.group(3)]=0D
+ value +=3D ((self.eval(match.group(1)) & (1<<cur=
_bit_len) - 1)) << bit_len=0D
+ bit_len +=3D cur_bit_len=0D
+ each_value =3D bytearray()=0D
+ if idx + 1 < len(bin_list):=0D
+ in_bit_field =3D True=0D
+ else:=0D
+ try:=0D
+ each_value =3D value_to_bytearray(self.eval(ea=
ch.strip()), unit_len)=0D
+ except:=0D
+ raise SystemExit("Exception: Value cannot fit =
into %s bytes !" % (each, unit_len))=0D
+=0D
+ if not in_bit_field:=0D
+ if bit_len > 0:=0D
+ if bit_len % 8 !=3D 0:=0D
+ raise SystemExit("Exception: Invalid bit f=
ield alignment '%s' !" % value_str)=0D
+ result.extend(value_to_bytes(value, bit_len //=
8))=0D
+ value =3D 0=0D
+ bit_len =3D 0=0D
+=0D
+ result.extend(each_value)=0D
+=0D
+ elif check_quote (value_str):=0D
+ result =3D bytearray(value_str[1:-1], 'utf-8') # Excluding qu=
otes=0D
+ else:=0D
+ result =3D value_to_bytearray (self.eval(value_str), length)=0D
+=0D
+ if len(result) < length:=0D
+ result.extend(b'\x00' * (length - len(result)))=0D
+ elif len(result) > length:=0D
+ raise SystemExit ("Exception: Value '%s' is too big to fit int=
o %d bytes !" % (value_str, length))=0D
+=0D
+ if array:=0D
+ return result=0D
+ else:=0D
+ return bytes_to_value(result)=0D
+=0D
+ return result=0D
+=0D
+=0D
+ def get_cfg_item_options (self, item):=0D
+ tmp_list =3D []=0D
+ if item['type'] =3D=3D "Combo":=0D
+ if item['option'] in CGenCfgData.builtin_option:=0D
+ for op_val, op_str in CGenCfgData.builtin_option[item['opt=
ion']]:=0D
+ tmp_list.append((op_val, op_str))=0D
+ else:=0D
+ opt_list =3D item['option'].split(',')=0D
+ for option in opt_list:=0D
+ option =3D option.strip()=0D
+ try:=0D
+ (op_val, op_str) =3D option.split(':')=0D
+ except:=0D
+ raise SystemExit ("Exception: Invalide option form=
at '%s' !" % option)=0D
+ tmp_list.append((op_val, op_str))=0D
+ return tmp_list=0D
+=0D
+=0D
+ def get_page_title(self, page_id, top =3D None):=0D
+ if top is None:=0D
+ top =3D self.get_cfg_page()['root']=0D
+ for node in top['child']:=0D
+ page_key =3D next(iter(node))=0D
+ if page_id =3D=3D page_key:=0D
+ return node[page_key]['title']=0D
+ else:=0D
+ result =3D self.get_page_title (page_id, node[page_key])=0D
+ if result is not None:=0D
+ return result=0D
+ return None=0D
+=0D
+=0D
+ def print_pages(self, top=3DNone, level=3D0):=0D
+ if top is None:=0D
+ top =3D self.get_cfg_page()['root']=0D
+ for node in top['child']:=0D
+ page_id =3D next(iter(node))=0D
+ print('%s%s: %s' % (' ' * level, page_id, node[page_id]['titl=
e']))=0D
+ level +=3D 1=0D
+ self.print_pages(node[page_id], level)=0D
+ level -=3D 1=0D
+=0D
+=0D
+ def get_item_by_index (self, index):=0D
+ return self._cfg_list[index]=0D
+=0D
+=0D
+ def get_item_by_path (self, path):=0D
+ node =3D self.locate_cfg_item (path)=0D
+ if node:=0D
+ return self.get_item_by_index (node['indx'])=0D
+ else:=0D
+ return None=0D
+=0D
+ def locate_cfg_path (self, item):=0D
+ def _locate_cfg_path (root, level =3D 0):=0D
+ # config structure=0D
+ if item is root:=0D
+ return path=0D
+ for key in root:=0D
+ if type(root[key]) is OrderedDict:=0D
+ level +=3D 1=0D
+ path.append(key)=0D
+ ret =3D _locate_cfg_path (root[key], level)=0D
+ if ret:=0D
+ return ret=0D
+ path.pop()=0D
+ return None=0D
+ path =3D []=0D
+ return _locate_cfg_path (self._cfg_tree)=0D
+=0D
+=0D
+ def locate_cfg_item (self, path, allow_exp =3D True):=0D
+ def _locate_cfg_item (root, path, level =3D 0):=0D
+ if len(path) =3D=3D level:=0D
+ return root=0D
+ next_root =3D root.get(path[level], None)=0D
+ if next_root is None:=0D
+ if allow_exp:=0D
+ raise Exception ('Not a valid CFG config option path: =
%s' % '.'.join(path[:level+1]))=0D
+ else:=0D
+ return None=0D
+ return _locate_cfg_item (next_root, path, level + 1)=0D
+=0D
+ path_nodes =3D path.split('.')=0D
+ return _locate_cfg_item (self._cfg_tree, path_nodes)=0D
+=0D
+=0D
+ def traverse_cfg_tree (self, handler, top =3D None):=0D
+ def _traverse_cfg_tree (root, level =3D 0):=0D
+ # config structure=0D
+ for key in root:=0D
+ if type(root[key]) is OrderedDict:=0D
+ level +=3D 1=0D
+ handler (key, root[key], level)=0D
+ _traverse_cfg_tree (root[key], level)=0D
+ level -=3D 1=0D
+=0D
+ if top is None:=0D
+ top =3D self._cfg_tree=0D
+ _traverse_cfg_tree (top)=0D
+=0D
+=0D
+ def print_cfgs(self, root =3D None, short =3D True, print_level =3D 25=
6):=0D
+ def _print_cfgs (name, cfgs, level):=0D
+=0D
+ if 'indx' in cfgs:=0D
+ act_cfg =3D self.get_item_by_index (cfgs['indx'])=0D
+ else:=0D
+ offset =3D 0=0D
+ length =3D 0=0D
+ value =3D ''=0D
+ path=3D''=0D
+ if CGenCfgData.STRUCT in cfgs:=0D
+ cfg =3D cfgs[CGenCfgData.STRUCT]=0D
+ offset =3D int(cfg['offset'])=0D
+ length =3D int(cfg['length'])=0D
+ if 'value' in cfg:=0D
+ value =3D cfg['value']=0D
+ if length =3D=3D 0:=0D
+ return=0D
+ act_cfg =3D dict({'value' : value, 'offset' : offset, 'len=
gth' : length})=0D
+ value =3D act_cfg['value']=0D
+ bit_len =3D act_cfg['length']=0D
+ offset =3D (act_cfg['offset'] + 7) // 8=0D
+ if value !=3D '':=0D
+ try:=0D
+ value =3D self.reformat_value_str (act_cfg['value'], a=
ct_cfg['length'])=0D
+ except:=0D
+ value =3D act_cfg['value']=0D
+ length =3D bit_len // 8=0D
+ bit_len =3D '(%db)' % bit_len if bit_len % 8 else '' * 4=0D
+ if level <=3D print_level:=0D
+ if short and len(value) > 40:=0D
+ value =3D '%s ... %s' % (value[:20] , value[-20:])=0D
+ print('%04X:%04X%-6s %s%s : %s' % (offset, length, bit_len=
, ' ' * level, name, value))=0D
+=0D
+ self.traverse_cfg_tree (_print_cfgs)=0D
+=0D
+=0D
+ def build_var_dict (self):=0D
+ def _build_var_dict (name, cfgs, level):=0D
+ if level <=3D 2:=0D
+ if CGenCfgData.STRUCT in cfgs:=0D
+ struct_info =3D cfgs[CGenCfgData.STRUCT]=0D
+ self._var_dict['_LENGTH_%s_' % name] =3D struct_info['=
length'] // 8=0D
+ self._var_dict['_OFFSET_%s_' % name] =3D struct_info['=
offset'] // 8=0D
+=0D
+ self._var_dict =3D {}=0D
+ self.traverse_cfg_tree (_build_var_dict)=0D
+ self._var_dict['_LENGTH_'] =3D self._cfg_tree[CGenCfgData.STRUCT][=
'length'] // 8=0D
+ return 0=0D
+=0D
+=0D
+ def add_cfg_page(self, child, parent, title=3D''):=0D
+ def _add_cfg_page(cfg_page, child, parent):=0D
+ key =3D next(iter(cfg_page))=0D
+ if parent =3D=3D key:=0D
+ cfg_page[key]['child'].append({child: {'title': title,=0D
+ 'child': []}})=0D
+ return True=0D
+ else:=0D
+ result =3D False=0D
+ for each in cfg_page[key]['child']:=0D
+ if _add_cfg_page(each, child, parent):=0D
+ result =3D True=0D
+ break=0D
+ return result=0D
+=0D
+ return _add_cfg_page(self._cfg_page, child, parent)=0D
+=0D
+=0D
+ def set_cur_page(self, page_str):=0D
+ if not page_str:=0D
+ return=0D
+=0D
+ if ',' in page_str:=0D
+ page_list =3D page_str.split(',')=0D
+ else:=0D
+ page_list =3D [page_str]=0D
+ for page_str in page_list:=0D
+ parts =3D page_str.split(':')=0D
+ if len(parts) in [1, 3]:=0D
+ page =3D parts[0].strip()=0D
+ if len(parts) =3D=3D 3:=0D
+ # it is a new page definition, add it into tree=0D
+ parent =3D parts[1] if parts[1] else 'root'=0D
+ parent =3D parent.strip()=0D
+ if parts[2][0] =3D=3D '"' and parts[2][-1] =3D=3D '"':=
=0D
+ parts[2] =3D parts[2][1:-1]=0D
+=0D
+ if not self.add_cfg_page(page, parent, parts[2]):=0D
+ raise SystemExit("Error: Cannot find parent page '=
%s'!" % parent)=0D
+ else:=0D
+ raise SystemExit("Error: Invalid page format '%s' !" % pag=
e_str)=0D
+ self._cur_page =3D page=0D
+=0D
+=0D
+ def extend_variable (self, line):=0D
+ # replace all variables=0D
+ if line =3D=3D '':=0D
+ return line=0D
+ loop =3D 2=0D
+ while loop > 0:=0D
+ line_after =3D DefTemplate(line).safe_substitute(self._def_dic=
t)=0D
+ if line =3D=3D line_after:=0D
+ break=0D
+ loop -=3D 1=0D
+ line =3D line_after=0D
+ return line_after=0D
+=0D
+ def reformat_number_per_type (self, itype, value):=0D
+ if check_quote(value) or value.startswith('{'):=0D
+ return value=0D
+ parts =3D itype.split(',')=0D
+ if len(parts) > 3 and parts[0] =3D=3D 'EditNum':=0D
+ num_fmt =3D parts[1].strip()=0D
+ else:=0D
+ num_fmt =3D ''=0D
+ if num_fmt =3D=3D 'HEX' and not value.startswith('0x'):=0D
+ value =3D '0x%X' % int(value, 10)=0D
+ elif num_fmt =3D=3D 'DEC' and value.startswith('0x'):=0D
+ value =3D '%d' % int(value, 16)=0D
+ return value=0D
+=0D
+ def add_cfg_item(self, name, item, offset, path):=0D
+=0D
+ self.set_cur_page (item.get('page', ''))=0D
+=0D
+ if name[0] =3D=3D '$':=0D
+ # skip all virtual node=0D
+ return 0=0D
+=0D
+=0D
+ if not set(item).issubset(CGenCfgData.keyword_set):=0D
+ for each in list(item):=0D
+ if each not in CGenCfgData.keyword_set:=0D
+ raise Exception ("Invalid attribute '%s' for '%s'!" % =
(each, '.'.join(path)))=0D
+=0D
+ length =3D item.get('length', 0)=0D
+ if type(length) is str:=0D
+ match =3D re.match("^(\d+)([b|B|W|D|Q])([B|W|D|Q]?)\s*$", leng=
th)=0D
+ if match:=0D
+ unit_len =3D CGenCfgData.bits_width[match.group(2)]=0D
+ length =3D int(match.group(1), 10) * unit_len=0D
+ else:=0D
+ try:=0D
+ length =3D int(length, 0) * 8=0D
+ except:=0D
+ raise Exception ("Invalid length field '%s' for '%s' !=
" % (length, '.'.join(path)))=0D
+=0D
+ if offset % 8 > 0:=0D
+ raise Exception ("Invalid alignment for field '%s' for=
'%s' !" % (name, '.'.join(path)))=0D
+ else:=0D
+ # define is length in bytes=0D
+ length =3D length * 8=0D
+=0D
+ if not name.isidentifier():=0D
+ raise Exception ("Invalid config name '%s' for '%s' !" % (name=
, '.'.join(path)))=0D
+=0D
+=0D
+ itype =3D str(item.get('type', 'Reserved'))=0D
+ value =3D str(item.get('value', ''))=0D
+ if value:=0D
+ if not (check_quote(value) or value.startswith('{')):=0D
+ if ',' in value:=0D
+ value =3D '{ %s }' % value=0D
+ else:=0D
+ value =3D self.reformat_number_per_type (itype, value)=
=0D
+=0D
+ help =3D str(item.get('help', ''))=0D
+ if '\n' in help:=0D
+ help =3D ' '.join ([i.strip() for i in help.splitlines()])=0D
+=0D
+ option =3D str(item.get('option', ''))=0D
+ if '\n' in option:=0D
+ option =3D ' '.join ([i.strip() for i in option.splitlines()])=
=0D
+=0D
+ # extend variables for value and condition=0D
+ condition =3D str(item.get('condition', ''))=0D
+ if condition:=0D
+ condition =3D self.extend_variable (condition)=0D
+ value =3D self.extend_variable (value)=0D
+=0D
+ order =3D str(item.get('order', ''))=0D
+ if order:=0D
+ if '.' in order:=0D
+ (major, minor) =3D order.split('.')=0D
+ order =3D int (major, 16)=0D
+ else:=0D
+ order =3D int (order, 16)=0D
+ else:=0D
+ order =3D offset=0D
+=0D
+ cfg_item =3D dict()=0D
+ cfg_item['length'] =3D length=0D
+ cfg_item['offset'] =3D offset=0D
+ cfg_item['value'] =3D value=0D
+ cfg_item['type'] =3D itype=0D
+ cfg_item['cname'] =3D str(name)=0D
+ cfg_item['name'] =3D str(item.get('name', ''))=0D
+ cfg_item['help'] =3D help=0D
+ cfg_item['option'] =3D option=0D
+ cfg_item['page'] =3D self._cur_page=0D
+ cfg_item['order'] =3D order=0D
+ cfg_item['path'] =3D '.'.join(path)=0D
+ cfg_item['condition'] =3D condition=0D
+ if 'struct' in item:=0D
+ cfg_item['struct'] =3D item['struct']=0D
+ self._cfg_list.append(cfg_item)=0D
+=0D
+ item['indx'] =3D len(self._cfg_list) - 1=0D
+=0D
+ # remove used info for reducing pkl size=0D
+ item.pop('option', None)=0D
+ item.pop('condition', None)=0D
+ item.pop('help', None)=0D
+ item.pop('name', None)=0D
+ item.pop('page', None)=0D
+=0D
+ return length=0D
+=0D
+=0D
+ def build_cfg_list (self, cfg_name =3D'', top =3D None, path =3D [], i=
nfo =3D {'offset': 0}):=0D
+ if top is None:=0D
+ top =3D self._cfg_tree=0D
+=0D
+ start =3D info['offset']=0D
+ is_leaf =3D True=0D
+ for key in top:=0D
+ path.append(key)=0D
+ if type(top[key]) is OrderedDict:=0D
+ is_leaf =3D False=0D
+ self.build_cfg_list(key, top[key], path, info)=0D
+ path.pop()=0D
+=0D
+ if is_leaf:=0D
+ length =3D self.add_cfg_item(cfg_name, top, info['offset'], pa=
th)=0D
+ info['offset'] +=3D length=0D
+ elif cfg_name =3D=3D '' or (cfg_name and cfg_name[0] !=3D '$'):=0D
+ # check first element for struct=0D
+ first =3D next(iter(top))=0D
+ struct_str =3D CGenCfgData.STRUCT=0D
+ if first !=3D struct_str:=0D
+ struct_node =3D OrderedDict({})=0D
+ top[struct_str] =3D struct_node=0D
+ top.move_to_end (struct_str, False)=0D
+ else:=0D
+ struct_node =3D top[struct_str]=0D
+ struct_node['offset'] =3D start=0D
+ struct_node['length'] =3D info['offset'] - start=0D
+ if struct_node['length'] % 8 !=3D 0:=0D
+ raise SystemExit("Error: Bits length not aligned for %s !"=
% str(path))=0D
+=0D
+=0D
+ def get_field_value (self, top =3D None):=0D
+ def _get_field_value (name, cfgs, level):=0D
+ if 'indx' in cfgs:=0D
+ act_cfg =3D self.get_item_by_index (cfgs['indx'])=0D
+ if act_cfg['length'] =3D=3D 0:=0D
+ return=0D
+ value =3D self.get_value (act_cfg['value'], act_cfg['lengt=
h'], False)=0D
+ set_bits_to_bytes (result, act_cfg['offset'] - struct_info=
['offset'], act_cfg['length'], value)=0D
+=0D
+ if top is None:=0D
+ top =3D self._cfg_tree=0D
+ struct_info =3D top[CGenCfgData.STRUCT]=0D
+ result =3D bytearray ((struct_info['length'] + 7) // 8)=0D
+ self.traverse_cfg_tree (_get_field_value, top)=0D
+ return result=0D
+=0D
+=0D
+ def set_field_value (self, top, value_bytes, force =3D False):=0D
+ def _set_field_value (name, cfgs, level):=0D
+ if 'indx' not in cfgs:=0D
+ return=0D
+ act_cfg =3D self.get_item_by_index (cfgs['indx'])=0D
+ if force or act_cfg['value'] =3D=3D '':=0D
+ value =3D get_bits_from_bytes (full_bytes, act_cfg['offset=
'] - struct_info['offset'], act_cfg['length'])=0D
+ act_val =3D act_cfg['value']=0D
+ if act_val =3D=3D '':=0D
+ act_val =3D '%d' % value=0D
+ act_val =3D self.reformat_number_per_type (act_cfg['type']=
, act_val)=0D
+ act_cfg['value'] =3D self.format_value_to_str (value, act_=
cfg['length'], act_val)=0D
+=0D
+ if 'indx' in top:=0D
+ # it is config option=0D
+ value =3D bytes_to_value (value_bytes)=0D
+ act_cfg =3D self.get_item_by_index (top['indx'])=0D
+ act_cfg['value'] =3D self.format_value_to_str (value, act_cfg[=
'length'], act_cfg['value'])=0D
+ else:=0D
+ # it is structure=0D
+ struct_info =3D top[CGenCfgData.STRUCT]=0D
+ length =3D struct_info['length'] // 8=0D
+ full_bytes =3D bytearray(value_bytes[:length])=0D
+ if len(full_bytes) < length:=0D
+ full_bytes.extend(bytearray(length - len(value_bytes)))=0D
+ self.traverse_cfg_tree (_set_field_value, top)=0D
+=0D
+=0D
+ def update_def_value (self):=0D
+ def _update_def_value (name, cfgs, level):=0D
+ if 'indx' in cfgs:=0D
+ act_cfg =3D self.get_item_by_index (cfgs['indx'])=0D
+ if act_cfg['value'] !=3D '' and act_cfg['length'] > 0:=0D
+ try:=0D
+ act_cfg['value'] =3D self.reformat_value_str (act_=
cfg['value'], act_cfg['length'])=0D
+ except:=0D
+ raise Exception ("Invalid value expression '%s' fo=
r '%s' !" % (act_cfg['value'], act_cfg['path']))=0D
+ else:=0D
+ if CGenCfgData.STRUCT in cfgs and 'value' in cfgs[CGenCfgD=
ata.STRUCT]:=0D
+ curr =3D cfgs[CGenCfgData.STRUCT]=0D
+ value_bytes =3D value_to_bytearray (self.eval(curr['va=
lue']), (curr['length'] + 7) // 8)=0D
+ self.set_field_value (cfgs, value_bytes)=0D
+=0D
+ self.traverse_cfg_tree (_update_def_value, self._cfg_tree)=0D
+=0D
+=0D
+ def evaluate_condition (self, item):=0D
+ expr =3D item['condition']=0D
+ result =3D self.parse_value (expr, 1, False)=0D
+ return result=0D
+=0D
+=0D
+ def load_default_from_bin (self, bin_data):=0D
+ self.set_field_value(self._cfg_tree, bin_data, True)=0D
+=0D
+=0D
+ def generate_binary_array (self):=0D
+ return self.get_field_value()=0D
+=0D
+ def generate_binary (self, bin_file_name):=0D
+ bin_file =3D open(bin_file_name, "wb")=0D
+ bin_file.write (self.generate_binary_array ())=0D
+ bin_file.close()=0D
+ return 0=0D
+=0D
+ def write_delta_file (self, out_file, platform_id, out_lines):=0D
+ dlt_fd =3D open (out_file, "w")=0D
+ dlt_fd.write ("%s\n" % get_copyright_header('dlt', True))=0D
+ dlt_fd.write ('#\n')=0D
+ dlt_fd.write ('# Delta configuration values for platform ID 0x%04X=
\n' % platform_id)=0D
+ dlt_fd.write ('#\n\n')=0D
+ for line in out_lines:=0D
+ dlt_fd.write ('%s\n' % line)=0D
+ dlt_fd.close()=0D
+=0D
+=0D
+ def override_default_value(self, dlt_file):=0D
+ error =3D 0=0D
+ dlt_lines =3D CGenCfgData.expand_include_files(dlt_file)=0D
+=0D
+ platform_id =3D None=0D
+ for line, file_path, line_num in dlt_lines:=0D
+ line =3D line.strip()=0D
+ if not line or line.startswith('#'):=0D
+ continue=0D
+ match =3D re.match("\s*([\w\.]+)\s*\|\s*(.+)", line)=0D
+ if not match:=0D
+ raise Exception("Unrecognized line '%s' (File:'%s' Line:%d=
) !" %=0D
+ (line, file_path, line_num + 1))=0D
+=0D
+ path =3D match.group(1)=0D
+ value_str =3D match.group(2)=0D
+ top =3D self.locate_cfg_item (path)=0D
+ if not top:=0D
+ raise Exception(=0D
+ "Invalid configuration '%s' (File:'%s' Line:%d) !" %=0D
+ (path, file_path, line_num + 1))=0D
+=0D
+ if 'indx' in top:=0D
+ act_cfg =3D self.get_item_by_index (top['indx'])=0D
+ bit_len =3D act_cfg['length']=0D
+ else:=0D
+ struct_info =3D top[CGenCfgData.STRUCT]=0D
+ bit_len =3D struct_info['length']=0D
+=0D
+ value_bytes =3D self.parse_value (value_str, bit_len)=0D
+ self.set_field_value (top, value_bytes, True)=0D
+=0D
+ if path =3D=3D 'PLATFORMID_CFG_DATA.PlatformId':=0D
+ platform_id =3D value_str=0D
+=0D
+ if platform_id is None:=0D
+ raise Exception(=0D
+ "PLATFORMID_CFG_DATA.PlatformId is missing in file '%s' !"=
%=0D
+ (dlt_file))=0D
+=0D
+ return error=0D
+=0D
+=0D
+ def generate_delta_file_from_bin (self, delta_file, old_data, new_data=
, full=3DFalse):=0D
+ self.load_default_from_bin (new_data)=0D
+ lines =3D []=0D
+ tag_name =3D ''=0D
+ level =3D 0=0D
+ platform_id =3D None=0D
+ def_platform_id =3D 0=0D
+=0D
+ for item in self._cfg_list:=0D
+ old_val =3D get_bits_from_bytes (old_data, item['offset'], it=
em['length'])=0D
+ new_val =3D get_bits_from_bytes (new_data, item['offset'], it=
em['length'])=0D
+=0D
+ full_name =3D item['path']=0D
+ if 'PLATFORMID_CFG_DATA.PlatformId' =3D=3D full_name:=0D
+ def_platform_id =3D old_val=0D
+ platform_id =3D new_val=0D
+ elif item['type'] !=3D 'Reserved' and ((new_val !=3D old_val) =
or full):=0D
+ val_str =3D self.reformat_value_str (item['value'], item['=
length'])=0D
+ text =3D '%-40s | %s' % (full_name, val_str)=0D
+ lines.append(text)=0D
+=0D
+ if platform_id is None or def_platform_id =3D=3D platform_id:=0D
+ platform_id =3D def_platform_id=0D
+=0D
+ lines.insert(0, '%-40s | %s\n\n' %=0D
+ ('PLATFORMID_CFG_DATA.PlatformId', '0x%04X' % platfor=
m_id))=0D
+=0D
+ self.write_delta_file (delta_file, platform_id, lines)=0D
+ return 0=0D
+=0D
+=0D
+ def generate_delta_file(self, delta_file, bin_file, bin_file2, full=3D=
False):=0D
+ fd =3D open (bin_file, 'rb')=0D
+ new_data =3D bytearray(fd.read())=0D
+ fd.close()=0D
+=0D
+ if bin_file2 =3D=3D '':=0D
+ old_data =3D self.generate_binary_array()=0D
+ else:=0D
+ old_data =3D new_data=0D
+ fd =3D open (bin_file2, 'rb')=0D
+ new_data =3D bytearray(fd.read())=0D
+ fd.close()=0D
+=0D
+ return self.generate_delta_file_from_bin (delta_file, old_data, ne=
w_data, full)=0D
+=0D
+=0D
+ def prepare_marshal (self, is_save):=0D
+ if is_save:=0D
+ # Ordered dict is not marshallable, convert to list=0D
+ self._cfg_tree =3D CGenCfgData.deep_convert_dict (self._cfg_tr=
ee)=0D
+ else:=0D
+ # Revert it back=0D
+ self._cfg_tree =3D CGenCfgData.deep_convert_list (self._cfg_tr=
ee)=0D
+=0D
+ def generate_yml_file (self, in_file, out_file):=0D
+ cfg_yaml =3D CFG_YAML()=0D
+ text =3D cfg_yaml.expand_yaml (in_file)=0D
+ yml_fd =3D open(out_file, "w")=0D
+ yml_fd.write (text)=0D
+ yml_fd.close ()=0D
+ return 0=0D
+=0D
+=0D
+ def write_cfg_header_file (self, hdr_file_name, tag_mode, tag_dict, st=
ruct_list):=0D
+ lines =3D []=0D
+ lines.append ('\n\n')=0D
+ tag_list =3D sorted(list(tag_dict.items()), key=3Dlambda x: x[1])=
=0D
+ for tagname, tagval in tag_list:=0D
+ if (tag_mode =3D=3D 0 and tagval >=3D 0x100) or (tag_mode =3D=
=3D 1 and tagval < 0x100):=0D
+ continue=0D
+ lines.append ('#define %-30s 0x%03X\n' % ('CDATA_%s_TAG' % =
tagname[:-9], tagval))=0D
+ lines.append ('\n\n')=0D
+=0D
+ name_dict =3D {}=0D
+ new_dict =3D {}=0D
+ for each in struct_list:=0D
+ if (tag_mode =3D=3D 0 and each['tag'] >=3D 0x100) or (tag_mode=
=3D=3D 1 and each['tag'] < 0x100):=0D
+ continue=0D
+ new_dict[each['name']] =3D (each['alias'], each['count'])=0D
+ if each['alias'] not in name_dict:=0D
+ name_dict[each['alias']] =3D 1=0D
+ lines.extend(self.create_struct (each['alias'], each['node=
'], new_dict))=0D
+=0D
+=0D
+ self.write_header_file (lines, hdr_file_name)=0D
+=0D
+=0D
+ def write_header_file (self, txt_body, file_name, type =3D 'h'):=0D
+ file_name_def =3D os.path.basename(file_name).replace ('.', '_')=0D
+ file_name_def =3D re.sub('(.)([A-Z][a-z]+)', r'\1_\2', file_name_d=
ef)=0D
+ file_name_def =3D re.sub('([a-z0-9])([A-Z])', r'\1_\2', file_name_=
def).upper()=0D
+=0D
+ lines =3D []=0D
+ lines.append ("%s\n" % get_copyright_header(type))=0D
+ lines.append ("#ifndef __%s__\n" % file_name_def)=0D
+ lines.append ("#define __%s__\n\n" % file_name_def)=0D
+ if type =3D=3D 'h':=0D
+ lines.append ("#pragma pack(1)\n\n")=0D
+ lines.extend (txt_body)=0D
+ if type =3D=3D 'h':=0D
+ lines.append ("#pragma pack()\n\n")=0D
+ lines.append ("#endif\n")=0D
+=0D
+ # Don't rewrite if the contents are the same=0D
+ create =3D True=0D
+ if os.path.exists(file_name):=0D
+ hdr_file =3D open(file_name, "r")=0D
+ org_txt =3D hdr_file.read()=0D
+ hdr_file.close()=0D
+=0D
+ new_txt =3D ''.join(lines)=0D
+ if org_txt =3D=3D new_txt:=0D
+ create =3D False=0D
+=0D
+ if create:=0D
+ hdr_file =3D open(file_name, "w")=0D
+ hdr_file.write (''.join(lines))=0D
+ hdr_file.close()=0D
+=0D
+=0D
+ def generate_data_inc_file (self, dat_inc_file_name, bin_file =3D None=
):=0D
+ # Put a prefix GUID before CFGDATA so that it can be located later=
on=0D
+ prefix =3D b'\xa7\xbd\x7f\x73\x20\x1e\x46\xd6\xbe\x8f\x64\x12\x0=
5\x8d\x0a\xa8'=0D
+ if bin_file:=0D
+ fin =3D open (bin_file, 'rb')=0D
+ bin_dat =3D prefix + bytearray(fin.read())=0D
+ fin.close()=0D
+ else:=0D
+ bin_dat =3D prefix + self.generate_binary_array ()=0D
+=0D
+ file_name =3D os.path.basename(dat_inc_file_name).upper()=0D
+ file_name =3D file_name.replace('.', '_')=0D
+=0D
+ txt_lines =3D []=0D
+=0D
+ txt_lines.append ("UINT8 mConfigDataBlob[%d] =3D {\n" % len(bin_d=
at))=0D
+ count =3D 0=0D
+ line =3D [' ']=0D
+ for each in bin_dat:=0D
+ line.append('0x%02X, ' % each)=0D
+ count =3D count + 1=0D
+ if (count & 0x0F) =3D=3D 0:=0D
+ line.append('\n')=0D
+ txt_lines.append (''.join(line))=0D
+ line =3D [' ']=0D
+ if len(line) > 1:=0D
+ txt_lines.append (''.join(line) + '\n')=0D
+=0D
+ txt_lines.append ("};\n\n")=0D
+=0D
+ self.write_header_file (txt_lines, dat_inc_file_name, 'inc')=0D
+=0D
+ return 0=0D
+=0D
+=0D
+ def get_struct_array_info (self, input):=0D
+ parts =3D input.split(':')=0D
+ if len(parts) > 1:=0D
+ var =3D parts[1]=0D
+ input =3D parts[0]=0D
+ else:=0D
+ var =3D ''=0D
+ array_str =3D input.split('[')=0D
+ name =3D array_str[0]=0D
+ if len(array_str) > 1:=0D
+ num_str =3D ''.join(c for c in array_str[-1] if c.isdigit())=0D
+ num_str =3D '1000' if len(num_str) =3D=3D 0 else num_str=0D
+ array_num =3D int(num_str)=0D
+ else:=0D
+ array_num =3D 0=0D
+ return name, array_num, var=0D
+=0D
+=0D
+ def process_multilines (self, string, max_char_length):=0D
+ multilines =3D ''=0D
+ string_length =3D len(string)=0D
+ current_string_start =3D 0=0D
+ string_offset =3D 0=0D
+ break_line_dict =3D []=0D
+ if len(string) <=3D max_char_length:=0D
+ while (string_offset < string_length):=0D
+ if string_offset >=3D 1:=0D
+ if string[string_offset - 1] =3D=3D '\\' and string[st=
ring_offset] =3D=3D 'n':=0D
+ break_line_dict.append (string_offset + 1)=0D
+ string_offset +=3D 1=0D
+ if break_line_dict !=3D []:=0D
+ for each in break_line_dict:=0D
+ multilines +=3D " %s\n" % string[current_string_start=
:each].lstrip()=0D
+ current_string_start =3D each=0D
+ if string_length - current_string_start > 0:=0D
+ multilines +=3D " %s\n" % string[current_string_start=
:].lstrip()=0D
+ else:=0D
+ multilines =3D " %s\n" % string=0D
+ else:=0D
+ new_line_start =3D 0=0D
+ new_line_count =3D 0=0D
+ found_space_char =3D False=0D
+ while (string_offset < string_length):=0D
+ if string_offset >=3D 1:=0D
+ if new_line_count >=3D max_char_length - 1:=0D
+ if string[string_offset] =3D=3D ' ' and string_len=
gth - string_offset > 10:=0D
+ break_line_dict.append (new_line_start + new_l=
ine_count)=0D
+ new_line_start =3D new_line_start + new_line_c=
ount=0D
+ new_line_count =3D 0=0D
+ found_space_char =3D True=0D
+ elif string_offset =3D=3D string_length - 1 and fo=
und_space_char =3D=3D False:=0D
+ break_line_dict.append (0)=0D
+ if string[string_offset - 1] =3D=3D '\\' and string[st=
ring_offset] =3D=3D 'n':=0D
+ break_line_dict.append (string_offset + 1)=0D
+ new_line_start =3D string_offset + 1=0D
+ new_line_count =3D 0=0D
+ string_offset +=3D 1=0D
+ new_line_count +=3D 1=0D
+ if break_line_dict !=3D []:=0D
+ break_line_dict.sort ()=0D
+ for each in break_line_dict:=0D
+ if each > 0:=0D
+ multilines +=3D " %s\n" % string[current_string_s=
tart:each].lstrip()=0D
+ current_string_start =3D each=0D
+ if string_length - current_string_start > 0:=0D
+ multilines +=3D " %s\n" % string[current_string_start=
:].lstrip()=0D
+ return multilines=0D
+=0D
+=0D
+ def create_field (self, item, name, length, offset, struct, bsf_name, =
help, option, bits_length =3D None):=0D
+ pos_name =3D 28=0D
+ pos_comment =3D 30=0D
+ name_line=3D''=0D
+ help_line=3D''=0D
+ option_line=3D''=0D
+=0D
+ if length =3D=3D 0 and name =3D=3D 'dummy':=0D
+ return '\n'=0D
+=0D
+ if bits_length =3D=3D 0:=0D
+ return '\n'=0D
+=0D
+ is_array =3D False=0D
+ if length in [1,2,4,8]:=0D
+ type =3D "UINT%d" % (length * 8)=0D
+ else:=0D
+ is_array =3D True=0D
+ type =3D "UINT8"=0D
+=0D
+ if item and item['value'].startswith('{'):=0D
+ type =3D "UINT8"=0D
+ is_array =3D True=0D
+=0D
+ if struct !=3D '':=0D
+ struct_base =3D struct.rstrip('*')=0D
+ name =3D '*' * (len(struct) - len(struct_base)) + name=0D
+ struct =3D struct_base=0D
+ type =3D struct=0D
+ if struct in ['UINT8','UINT16','UINT32','UINT64']:=0D
+ is_array =3D True=0D
+ unit =3D int(type[4:]) // 8=0D
+ length =3D length / unit=0D
+ else:=0D
+ is_array =3D False=0D
+=0D
+ if is_array:=0D
+ name =3D name + '[%d]' % length=0D
+=0D
+ if len(type) < pos_name:=0D
+ space1 =3D pos_name - len(type)=0D
+ else:=0D
+ space1 =3D 1=0D
+=0D
+ if bsf_name !=3D '':=0D
+ name_line=3D" %s\n" % bsf_name=0D
+ else:=0D
+ name_line=3D"N/A\n"=0D
+=0D
+ if help !=3D '':=0D
+ help_line =3D self.process_multilines (help, 80)=0D
+=0D
+ if option !=3D '':=0D
+ option_line =3D self.process_multilines (option, 80)=0D
+=0D
+ if offset is None:=0D
+ offset_str =3D '????'=0D
+ else:=0D
+ offset_str =3D '0x%04X' % offset=0D
+=0D
+ if bits_length is None:=0D
+ bits_length =3D ''=0D
+ else:=0D
+ bits_length =3D ' : %d' % bits_length=0D
+=0D
+ #return "\n/** %s%s%s**/\n %s%s%s%s;\n" % (name_line, help_line, =
option_line, type, ' ' * space1, name, bits_length)=0D
+ return "\n /* %s */\n %s%s%s%s;\n" % (name_line.strip(), type, '=
' * space1, name, bits_length)=0D
+=0D
+=0D
+ def create_struct (self, cname, top, struct_dict):=0D
+ index =3D 0=0D
+ last =3D ''=0D
+ lines =3D []=0D
+ lines.append ('\ntypedef struct {\n')=0D
+ for field in top:=0D
+ if field[0] =3D=3D '$':=0D
+ continue=0D
+=0D
+ index +=3D 1=0D
+=0D
+ t_item =3D top[field]=0D
+ if 'indx' not in t_item:=0D
+ if CGenCfgData.STRUCT not in top[field]:=0D
+ continue=0D
+=0D
+ if struct_dict[field][1] =3D=3D 0:=0D
+ continue=0D
+=0D
+ append =3D True=0D
+ struct_info =3D top[field][CGenCfgData.STRUCT]=0D
+=0D
+ if 'struct' in struct_info:=0D
+ struct, array_num, var =3D self.get_struct_array_info =
(struct_info['struct'])=0D
+ if array_num > 0:=0D
+ if last =3D=3D struct:=0D
+ append =3D False=0D
+ last =3D struct=0D
+ if var =3D=3D '':=0D
+ var =3D field=0D
+=0D
+ field =3D CGenCfgData.format_struct_field_name (v=
ar, struct_dict[field][1])=0D
+ else:=0D
+ struct =3D struct_dict[field][0]=0D
+ field =3D CGenCfgData.format_struct_field_name (field=
, struct_dict[field][1])=0D
+=0D
+ if append:=0D
+ line =3D self.create_field (None, field, 0, 0, struct,=
'', '', '')=0D
+ lines.append (' %s' % line)=0D
+ last =3D struct=0D
+ continue=0D
+=0D
+ item =3D self.get_item_by_index (t_item['indx'])=0D
+ if item['cname'] =3D=3D 'CfgHeader' and index =3D=3D 1 or (it=
em['cname'] =3D=3D 'CondValue' and index =3D=3D 2):=0D
+ continue=0D
+=0D
+ bit_length =3D None=0D
+ length =3D (item['length'] + 7) // 8=0D
+ match =3D re.match("^(\d+)([b|B|W|D|Q])([B|W|D|Q]?)", t_item[=
'length'])=0D
+ if match and match.group(2) =3D=3D 'b':=0D
+ bit_length =3D int(match.group(1))=0D
+ if match.group(3) !=3D '':=0D
+ length =3D CGenCfgData.bits_width[match.group(3)] // 8=
=0D
+ else:=0D
+ length =3D 4=0D
+ offset =3D item['offset'] // 8=0D
+ struct =3D item.get('struct', '')=0D
+ name =3D field=0D
+ prompt =3D item['name']=0D
+ help =3D item['help']=0D
+ option =3D item['option']=0D
+ line =3D self.create_field (item, name, length, offset, struct=
, prompt, help, option, bit_length)=0D
+ lines.append (' %s' % line)=0D
+ last =3D struct=0D
+=0D
+ lines.append ('\n} %s;\n\n' % cname)=0D
+=0D
+ return lines=0D
+=0D
+=0D
+ def create_header_file (self, hdr_file_name, com_hdr_file_name =3D '')=
:=0D
+ def _build_header_struct (name, cfgs, level):=0D
+ if CGenCfgData.STRUCT in cfgs:=0D
+ if 'CfgHeader' in cfgs:=0D
+ # collect CFGDATA TAG IDs=0D
+ cfghdr =3D self.get_item_by_index (cfgs['CfgHeader'][=
'indx'])=0D
+ tag_val =3D array_str_to_value(cfghdr['value']) >> 20=
=0D
+ tag_dict[name] =3D tag_val=0D
+ if level =3D=3D 1:=0D
+ tag_curr[0] =3D tag_val=0D
+ struct_dict[name] =3D (level, tag_curr[0], cfgs)=0D
+=0D
+ tag_curr =3D [0]=0D
+ tag_dict =3D {}=0D
+ struct_dict =3D {}=0D
+ self.traverse_cfg_tree (_build_header_struct)=0D
+=0D
+ if tag_curr[0] =3D=3D 0:=0D
+ hdr_mode =3D 2=0D
+ else:=0D
+ hdr_mode =3D 1=0D
+=0D
+ # filter out the items to be built for tags and structures=0D
+ struct_list =3D []=0D
+ for each in struct_dict:=0D
+ match =3D False=0D
+ for check in CGenCfgData.exclude_struct:=0D
+ if re.match (check, each):=0D
+ match =3D True=0D
+ if each in tag_dict:=0D
+ if each not in CGenCfgData.include_tag:=0D
+ del tag_dict[each]=0D
+ break=0D
+ if not match:=0D
+ struct_list.append ({'name':each, 'alias':'', 'count' : 0,=
'level':struct_dict[each][0],=0D
+ 'tag':struct_dict[each][1], 'node':st=
ruct_dict[each][2]})=0D
+=0D
+ # sort by level so that the bottom level struct will be build firs=
t to satisfy dependencies=0D
+ struct_list =3D sorted(struct_list, key=3Dlambda x: x['level'], re=
verse=3DTrue)=0D
+=0D
+ # Convert XXX_[0-9]+ to XXX as an array hint=0D
+ for each in struct_list:=0D
+ cfgs =3D each['node']=0D
+ if 'struct' in cfgs['$STRUCT']:=0D
+ each['alias'], array_num, var =3D self.get_struct_array_in=
fo (cfgs['$STRUCT']['struct'])=0D
+ else:=0D
+ match =3D re.match('(\w+)(_\d+)', each['name'])=0D
+ if match:=0D
+ each['alias'] =3D match.group(1)=0D
+ else:=0D
+ each['alias'] =3D each['name']=0D
+=0D
+ # count items for array build=0D
+ for idx, each in enumerate(struct_list):=0D
+ if idx > 0:=0D
+ last_struct =3D struct_list[idx-1]['node']['$STRUCT']=0D
+ curr_struct =3D each['node']['$STRUCT']=0D
+ if struct_list[idx-1]['alias'] =3D=3D each['alias'] and \=
=0D
+ curr_struct['length'] =3D=3D last_struct['length'] and =
\=0D
+ curr_struct['offset'] =3D=3D last_struct['offset'] + la=
st_struct['length']:=0D
+ for idx2 in range (idx-1, -1, -1):=0D
+ if struct_list[idx2]['count'] > 0:=0D
+ struct_list[idx2]['count'] +=3D 1=0D
+ break=0D
+ continue=0D
+ each['count'] =3D 1=0D
+=0D
+ # generate common header=0D
+ if com_hdr_file_name:=0D
+ self.write_cfg_header_file (com_hdr_file_name, 0, tag_dict, st=
ruct_list)=0D
+=0D
+ # generate platform header=0D
+ self.write_cfg_header_file (hdr_file_name, hdr_mode, tag_dict, str=
uct_list)=0D
+=0D
+ return 0=0D
+=0D
+=0D
+ def load_yaml (self, cfg_file):=0D
+ cfg_yaml =3D CFG_YAML()=0D
+ self.initialize ()=0D
+ self._cfg_tree =3D cfg_yaml.load_yaml (cfg_file)=0D
+ self._def_dict =3D cfg_yaml.def_dict=0D
+ self._yaml_path =3D os.path.dirname(cfg_file)=0D
+ self.build_cfg_list()=0D
+ self.build_var_dict()=0D
+ self.update_def_value()=0D
+ return 0=0D
+=0D
+=0D
+def usage():=0D
+ print ('\n'.join([=0D
+ "GenCfgData Version 0.50",=0D
+ "Usage:",=0D
+ " GenCfgData GENINC BinFile IncOutFile",=0D
+ " GenCfgData GENPKL YamlFile PklOutFile",=0D
+ " GenCfgData GENBIN YamlFile[;DltFile] BinOutFile",=0D
+ " GenCfgData GENDLT YamlFile[;BinFile] DltOutFile",=0D
+ " GenCfgData GENYML YamlFile YamlOutFile",=0D
+ " GenCfgData GENHDR YamlFile HdrOutFile"=0D
+ ]))=0D
+=0D
+=0D
+def main():=0D
+ # Parse the options and args=0D
+ argc =3D len(sys.argv)=0D
+ if argc < 4 or argc > 5:=0D
+ usage()=0D
+ return 1=0D
+=0D
+ gen_cfg_data =3D CGenCfgData()=0D
+ command =3D sys.argv[1].upper()=0D
+ out_file =3D sys.argv[3]=0D
+=0D
+ file_list =3D sys.argv[2].split(';')=0D
+ if len(file_list) >=3D 2:=0D
+ yml_file =3D file_list[0]=0D
+ dlt_file =3D file_list[1]=0D
+ elif len(file_list) =3D=3D 1:=0D
+ yml_file =3D file_list[0]=0D
+ dlt_file =3D ''=0D
+ else:=0D
+ raise Exception ("ERROR: Invalid parameter '%s' !" % sys.argv[2])=
=0D
+=0D
+ if command =3D=3D "GENDLT" and yml_file.endswith('.dlt'):=0D
+ # It needs to expand an existing DLT file=0D
+ dlt_file =3D yml_file=0D
+ lines =3D gen_cfg_data.expand_include_files (dlt_file)=0D
+ write_lines (lines, out_file)=0D
+ return 0=0D
+=0D
+ if command =3D=3D "GENYML":=0D
+ if not yml_file.lower().endswith('.yaml'):=0D
+ raise Exception ('Only YAML file is supported !')=0D
+ gen_cfg_data.generate_yml_file(yml_file, out_file)=0D
+ return 0=0D
+=0D
+ bin_file =3D ''=0D
+ if (yml_file.lower().endswith('.bin')) and (command =3D=3D "GENINC"):=
=0D
+ # It is binary file=0D
+ bin_file =3D yml_file=0D
+ yml_file =3D ''=0D
+=0D
+ if bin_file:=0D
+ gen_cfg_data.generate_data_inc_file(out_file, bin_file)=0D
+ return 0=0D
+=0D
+ cfg_bin_file =3D ''=0D
+ cfg_bin_file2 =3D ''=0D
+ if dlt_file:=0D
+ if command =3D=3D "GENDLT":=0D
+ cfg_bin_file =3D dlt_file=0D
+ dlt_file =3D ''=0D
+ if len(file_list) >=3D 3:=0D
+ cfg_bin_file2 =3D file_list[2]=0D
+=0D
+ if yml_file.lower().endswith('.pkl'):=0D
+ with open(yml_file, "rb") as pkl_file:=0D
+ gen_cfg_data.__dict__ =3D marshal.load(pkl_file)=0D
+ gen_cfg_data.prepare_marshal (False)=0D
+ else:=0D
+ gen_cfg_data.load_yaml (yml_file)=0D
+ if command =3D=3D 'GENPKL':=0D
+ gen_cfg_data.prepare_marshal (True)=0D
+ with open(out_file, "wb") as pkl_file:=0D
+ marshal.dump(gen_cfg_data.__dict__, pkl_file)=0D
+ json_file =3D os.path.splitext(out_file)[0] + '.json'=0D
+ fo =3D open (json_file, 'w')=0D
+ path_list =3D []=0D
+ cfgs =3D {'_cfg_page' : gen_cfg_data._cfg_page, '_cfg_list':ge=
n_cfg_data._cfg_list, '_path_list' : path_list}=0D
+ # optimize to reduce size=0D
+ path =3D None=0D
+ for each in cfgs['_cfg_list']:=0D
+ new_path =3D each['path'][:-len(each['cname'])-1]=0D
+ if path !=3D new_path:=0D
+ path =3D new_path=0D
+ each['path'] =3D path=0D
+ path_list.append(path)=0D
+ else:=0D
+ del each['path']=0D
+ if each['order'] =3D=3D each['offset']:=0D
+ del each['order']=0D
+ del each['offset']=0D
+=0D
+ # value is just used to indicate display type=0D
+ value =3D each['value']=0D
+ if value.startswith ('0x'):=0D
+ hex_len =3D ((each['length'] + 7) // 8) * 2=0D
+ if len(value) =3D=3D hex_len:=0D
+ value =3D 'x%d' % hex_len=0D
+ else:=0D
+ value =3D 'x'=0D
+ each['value'] =3D value=0D
+ elif value and value[0] in ['"', "'", '{']:=0D
+ each['value'] =3D value[0]=0D
+ else:=0D
+ del each['value']=0D
+=0D
+ fo.write(repr(cfgs))=0D
+ fo.close ()=0D
+ return 0=0D
+=0D
+ if dlt_file:=0D
+ gen_cfg_data.override_default_value(dlt_file)=0D
+=0D
+ if command =3D=3D "GENBIN":=0D
+ if len(file_list) =3D=3D 3:=0D
+ old_data =3D gen_cfg_data.generate_binary_array()=0D
+ fi =3D open (file_list[2], 'rb')=0D
+ new_data =3D bytearray (fi.read ())=0D
+ fi.close ()=0D
+ if len(new_data) !=3D len(old_data):=0D
+ raise Exception ("Binary file '%s' length does not match, =
ignored !" % file_list[2])=0D
+ else:=0D
+ gen_cfg_data.load_default_from_bin (new_data)=0D
+ gen_cfg_data.override_default_value(dlt_file)=0D
+=0D
+ gen_cfg_data.generate_binary(out_file)=0D
+=0D
+ elif command =3D=3D "GENDLT":=0D
+ gen_cfg_data.generate_delta_file (out_file, cfg_bin_file, cfg_bin_=
file2)=0D
+=0D
+ elif command =3D=3D "GENHDR":=0D
+ out_files =3D out_file.split(';')=0D
+ brd_out_file =3D out_files[0].strip()=0D
+ if len(out_files) > 1:=0D
+ com_out_file =3D out_files[1].strip()=0D
+ else:=0D
+ com_out_file =3D ''=0D
+ gen_cfg_data.create_header_file(brd_out_file, com_out_file)=0D
+=0D
+ elif command =3D=3D "GENINC":=0D
+ gen_cfg_data.generate_data_inc_file(out_file)=0D
+=0D
+ elif command =3D=3D "DEBUG":=0D
+ gen_cfg_data.print_cfgs()=0D
+=0D
+ else:=0D
+ raise Exception ("Unsuported command '%s' !" % command)=0D
+=0D
+ return 0=0D
+=0D
+=0D
+if __name__ =3D=3D '__main__':=0D
+ sys.exit(main())=0D
+=0D
diff --git a/IntelFsp2Pkg/Tools/ConfigEditor/SingleSign.py b/IntelFsp2Pkg/T=
ools/ConfigEditor/SingleSign.py
new file mode 100644
index 0000000000..be9ba7dd72
--- /dev/null
+++ b/IntelFsp2Pkg/Tools/ConfigEditor/SingleSign.py
@@ -0,0 +1,301 @@
+#!/usr/bin/env python=0D
+## @ SingleSign.py=0D
+# Single signing script=0D
+#=0D
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>=0D
+# SPDX-License-Identifier: BSD-2-Clause-Patent=0D
+#=0D
+##=0D
+=0D
+##=0D
+# Import Modules=0D
+#=0D
+import os=0D
+import sys=0D
+import re=0D
+import shutil=0D
+import subprocess=0D
+import struct=0D
+import hashlib=0D
+import string=0D
+=0D
+SIGNING_KEY =3D {=0D
+ # Key Id | Key File Name start |=0D
+ # =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0D
+ # KEY_ID_MASTER is used for signing Slimboot Key Hash Manifest contain=
er (KEYH Component)=0D
+ "KEY_ID_MASTER_RSA2048" : "MasterTestKey_Priv_RSA2048.pem"=
,=0D
+ "KEY_ID_MASTER_RSA3072" : "MasterTestKey_Priv_RSA3072.pem"=
,=0D
+=0D
+ # KEY_ID_CFGDATA is used for signing external Config data blob)=0D
+ "KEY_ID_CFGDATA_RSA2048" : "ConfigTestKey_Priv_RSA2048.pem"=
,=0D
+ "KEY_ID_CFGDATA_RSA3072" : "ConfigTestKey_Priv_RSA3072.pem"=
,=0D
+=0D
+ # KEY_ID_FIRMWAREUPDATE is used for signing capsule firmware update im=
age)=0D
+ "KEY_ID_FIRMWAREUPDATE_RSA2048" : "FirmwareUpdateTestKey_Priv_RSA2=
048.pem",=0D
+ "KEY_ID_FIRMWAREUPDATE_RSA3072" : "FirmwareUpdateTestKey_Priv_RSA3=
072.pem",=0D
+=0D
+ # KEY_ID_CONTAINER is used for signing container header with mono sign=
ature=0D
+ "KEY_ID_CONTAINER_RSA2048" : "ContainerTestKey_Priv_RSA2048.p=
em",=0D
+ "KEY_ID_CONTAINER_RSA3072" : "ContainerTestKey_Priv_RSA3072.p=
em",=0D
+=0D
+ # CONTAINER_COMP1_KEY_ID is used for signing container components=0D
+ "KEY_ID_CONTAINER_COMP_RSA2048" : "ContainerCompTestKey_Priv_RSA204=
8.pem",=0D
+ "KEY_ID_CONTAINER_COMP_RSA3072" : "ContainerCompTestKey_Priv_RSA307=
2.pem",=0D
+=0D
+ # KEY_ID_OS1_PUBLIC, KEY_ID_OS2_PUBLIC is used for referencing Boot OS=
public keys=0D
+ "KEY_ID_OS1_PUBLIC_RSA2048" : "OS1_TestKey_Pub_RSA2048.pem",=0D
+ "KEY_ID_OS1_PUBLIC_RSA3072" : "OS1_TestKey_Pub_RSA3072.pem",=0D
+=0D
+ "KEY_ID_OS2_PUBLIC_RSA2048" : "OS2_TestKey_Pub_RSA2048.pem",=0D
+ "KEY_ID_OS2_PUBLIC_RSA3072" : "OS2_TestKey_Pub_RSA3072.pem",=0D
+=0D
+ }=0D
+=0D
+MESSAGE_SBL_KEY_DIR =3D (=0D
+ "!!! PRE-REQUISITE: Path to SBL_KEY_DIR has to be set with SBL=
KEYS DIRECTORY !!! \n"=0D
+ "!!! Generate keys using GenerateKeys.py available in Bootload=
erCorePkg/Tools directory !!! \n"=0D
+ "!!! Run $python BootloaderCorePkg/Tools/GenerateKeys.py -k $P=
ATH_TO_SBL_KEY_DIR !!!\n"=0D
+ "!!! Set SBL_KEY_DIR environ with path to SBL KEYS DIR !!!\n"=
=0D
+ "!!! Windows $set SBL_KEY_DIR=3D$PATH_TO_SBL_KEY_DIR !!!\n"=
=0D
+ "!!! Linux $export SBL_KEY_DIR=3D$PATH_TO_SBL_KEY_DIR !!!\n"=
=0D
+ )=0D
+=0D
+def get_openssl_path ():=0D
+ if os.name =3D=3D 'nt':=0D
+ if 'OPENSSL_PATH' not in os.environ:=0D
+ openssl_dir =3D "C:\\Openssl\\bin\\"=0D
+ if os.path.exists (openssl_dir):=0D
+ os.environ['OPENSSL_PATH'] =3D openssl_dir=0D
+ else:=0D
+ os.environ['OPENSSL_PATH'] =3D "C:\\Openssl\\"=0D
+ if 'OPENSSL_CONF' not in os.environ:=0D
+ openssl_cfg =3D "C:\\Openssl\\openssl.cfg"=0D
+ if os.path.exists(openssl_cfg):=0D
+ os.environ['OPENSSL_CONF'] =3D openssl_cfg=0D
+ openssl =3D os.path.join(os.environ.get ('OPENSSL_PATH', ''), 'ope=
nssl.exe')=0D
+ else:=0D
+ # Get openssl path for Linux cases=0D
+ openssl =3D shutil.which('openssl')=0D
+=0D
+ return openssl=0D
+=0D
+def run_process (arg_list, print_cmd =3D False, capture_out =3D False):=0D
+ sys.stdout.flush()=0D
+ if print_cmd:=0D
+ print (' '.join(arg_list))=0D
+=0D
+ exc =3D None=0D
+ result =3D 0=0D
+ output =3D ''=0D
+ try:=0D
+ if capture_out:=0D
+ output =3D subprocess.check_output(arg_list).decode()=0D
+ else:=0D
+ result =3D subprocess.call (arg_list)=0D
+ except Exception as ex:=0D
+ result =3D 1=0D
+ exc =3D ex=0D
+=0D
+ if result:=0D
+ if not print_cmd:=0D
+ print ('Error in running process:\n %s' % ' '.join(arg_list))=
=0D
+ if exc is None:=0D
+ sys.exit(1)=0D
+ else:=0D
+ raise exc=0D
+=0D
+ return output=0D
+=0D
+def check_file_pem_format (priv_key):=0D
+ # Check for file .pem format=0D
+ key_name =3D os.path.basename(priv_key)=0D
+ if os.path.splitext(key_name)[1] =3D=3D ".pem":=0D
+ return True=0D
+ else:=0D
+ return False=0D
+=0D
+def get_key_id (priv_key):=0D
+ # Extract base name if path is provided.=0D
+ key_name =3D os.path.basename(priv_key)=0D
+ # Check for KEY_ID in key naming.=0D
+ if key_name.startswith('KEY_ID'):=0D
+ return key_name=0D
+ else:=0D
+ return None=0D
+=0D
+def get_sbl_key_dir ():=0D
+ # Check Key store setting SBL_KEY_DIR path=0D
+ if 'SBL_KEY_DIR' not in os.environ:=0D
+ raise Exception ("ERROR: SBL_KEY_DIR is not defined. Set SBL_KEY_D=
IR with SBL Keys directory!!\n"=0D
+ + MESSAGE_=
SBL_KEY_DIR)=0D
+=0D
+ sbl_key_dir =3D os.environ.get('SBL_KEY_DIR')=0D
+ if not os.path.exists(sbl_key_dir):=0D
+ raise Exception (("ERROR:SBL_KEY_DIR set %s is not valid. Set the =
correct SBL_KEY_DIR path !!\n"=0D
+ + =
MESSAGE_SBL_KEY_DIR) % sbl_key_dir)=0D
+ else:=0D
+ return sbl_key_dir=0D
+=0D
+def get_key_from_store (in_key):=0D
+=0D
+ #Check in_key is path to key=0D
+ if os.path.exists(in_key):=0D
+ return in_key=0D
+=0D
+ # Get Slimboot key dir path=0D
+ sbl_key_dir =3D get_sbl_key_dir()=0D
+=0D
+ # Extract if in_key is key_id=0D
+ priv_key =3D get_key_id (in_key)=0D
+ if priv_key is not None:=0D
+ if (priv_key in SIGNING_KEY):=0D
+ # Generate key file name from key id=0D
+ priv_key_file =3D SIGNING_KEY[priv_key]=0D
+ else:=0D
+ raise Exception('KEY_ID %s is not found in supported KEY IDs!!=
' % priv_key)=0D
+ elif check_file_pem_format(in_key) =3D=3D True:=0D
+ # check if file name is provided in pem format=0D
+ priv_key_file =3D in_key=0D
+ else:=0D
+ priv_key_file =3D None=0D
+ raise Exception('key provided %s is not valid!' % in_key)=0D
+=0D
+ # Create a file path=0D
+ # Join Key Dir and priv_key_file=0D
+ try:=0D
+ priv_key =3D os.path.join (sbl_key_dir, priv_key_file)=0D
+ except:=0D
+ raise Exception('priv_key is not found %s!' % priv_key)=0D
+=0D
+ # Check for priv_key construted based on KEY ID exists in specified pa=
th=0D
+ if not os.path.isfile(priv_key):=0D
+ raise Exception (("!!! ERROR: Key file corresponding to '%s' do no=
t exist in Sbl key directory at '%s' !!! \n" + MESSAGE_SBL_KEY_DIR) % (in_=
key, sbl_key_dir))=0D
+=0D
+ return priv_key=0D
+=0D
+#=0D
+# Sign an file using openssl=0D
+#=0D
+# priv_key [Input] Key Id or Path to Private key=0D
+# hash_type [Input] Signing hash=0D
+# sign_scheme[Input] Sign/padding scheme=0D
+# in_file [Input] Input file to be signed=0D
+# out_file [Input/Output] Signed data file=0D
+#=0D
+=0D
+def single_sign_file (priv_key, hash_type, sign_scheme, in_file, out_file)=
:=0D
+=0D
+ _hash_type_string =3D {=0D
+ "SHA2_256" : 'sha256',=0D
+ "SHA2_384" : 'sha384',=0D
+ "SHA2_512" : 'sha512',=0D
+ }=0D
+=0D
+ _hash_digest_Size =3D {=0D
+ # Hash_string : Hash_Size=0D
+ "SHA2_256" : 32,=0D
+ "SHA2_384" : 48,=0D
+ "SHA2_512" : 64,=0D
+ "SM3_256" : 32,=0D
+ }=0D
+=0D
+ _sign_scheme_string =3D {=0D
+ "RSA_PKCS1" : 'pkcs1',=0D
+ "RSA_PSS" : 'pss',=0D
+ }=0D
+=0D
+ priv_key =3D get_key_from_store(priv_key)=0D
+=0D
+ # Temporary files to store hash generated=0D
+ hash_file_tmp =3D out_file+'.hash.tmp'=0D
+ hash_file =3D out_file+'.hash'=0D
+=0D
+ # Generate hash using openssl dgst in hex format=0D
+ cmdargs =3D [get_openssl_path(), 'dgst', '-'+'%s' % _hash_type_string[=
hash_type], '-out', '%s' % hash_file_tmp, '%s' % in_file]=0D
+ run_process (cmdargs)=0D
+=0D
+ # Extract hash form dgst command output and convert to ascii=0D
+ with open(hash_file_tmp, 'r') as fin:=0D
+ hashdata =3D fin.read()=0D
+ fin.close()=0D
+=0D
+ try:=0D
+ hashdata =3D hashdata.rsplit('=3D', 1)[1].strip()=0D
+ except:=0D
+ raise Exception('Hash Data not found for signing!')=0D
+=0D
+ if len(hashdata) !=3D (_hash_digest_Size[hash_type] * 2):=0D
+ raise Exception('Hash Data size do match with for hash type!')=0D
+=0D
+ hashdata_bytes =3D bytearray.fromhex(hashdata)=0D
+ open (hash_file, 'wb').write(hashdata_bytes)=0D
+=0D
+ print ("Key used for Singing %s !!" % priv_key)=0D
+=0D
+ # sign using Openssl pkeyutl=0D
+ cmdargs =3D [get_openssl_path(), 'pkeyutl', '-sign', '-in', '%s' % has=
h_file, '-inkey', '%s' % priv_key,=0D
+ '-out', '%s' % out_file, '-pkeyopt', 'digest:%s' % _hash_ty=
pe_string[hash_type],=0D
+ '-pkeyopt', 'rsa_padding_mode:%s' % _sign_scheme_string[sig=
n_scheme]]=0D
+=0D
+ run_process (cmdargs)=0D
+=0D
+ return=0D
+=0D
+#=0D
+# Extract public key using openssl=0D
+#=0D
+# in_key [Input] Private key or public key in pem format=0D
+# pub_key_file [Input/Output] Public Key to a file=0D
+#=0D
+# return keydata (mod, exp) in bin format=0D
+#=0D
+=0D
+def single_sign_gen_pub_key (in_key, pub_key_file =3D None):=0D
+=0D
+ in_key =3D get_key_from_store(in_key)=0D
+=0D
+ # Expect key to be in PEM format=0D
+ is_prv_key =3D False=0D
+ cmdline =3D [get_openssl_path(), 'rsa', '-pubout', '-text', '-noout', =
'-in', '%s' % in_key]=0D
+ # Check if it is public key or private key=0D
+ text =3D open(in_key, 'r').read()=0D
+ if '-BEGIN RSA PRIVATE KEY-' in text:=0D
+ is_prv_key =3D True=0D
+ elif '-BEGIN PUBLIC KEY-' in text:=0D
+ cmdline.extend (['-pubin'])=0D
+ else:=0D
+ raise Exception('Unknown key format "%s" !' % in_key)=0D
+=0D
+ if pub_key_file:=0D
+ cmdline.extend (['-out', '%s' % pub_key_file])=0D
+ capture =3D False=0D
+ else:=0D
+ capture =3D True=0D
+=0D
+ output =3D run_process (cmdline, capture_out =3D capture)=0D
+ if not capture:=0D
+ output =3D text =3D open(pub_key_file, 'r').read()=0D
+ data =3D output.replace('\r', '')=0D
+ data =3D data.replace('\n', '')=0D
+ data =3D data.replace(' ', '')=0D
+=0D
+ # Extract the modulus=0D
+ if is_prv_key:=0D
+ match =3D re.search('modulus(.*)publicExponent:\s+(\d+)\s+', data)=
=0D
+ else:=0D
+ match =3D re.search('Modulus(?:.*?):(.*)Exponent:\s+(\d+)\s+', dat=
a)=0D
+ if not match:=0D
+ raise Exception('Public key not found!')=0D
+ modulus =3D match.group(1).replace(':', '')=0D
+ exponent =3D int(match.group(2))=0D
+=0D
+ mod =3D bytearray.fromhex(modulus)=0D
+ # Remove the '00' from the front if the MSB is 1=0D
+ if mod[0] =3D=3D 0 and (mod[1] & 0x80):=0D
+ mod =3D mod[1:]=0D
+ exp =3D bytearray.fromhex('{:08x}'.format(exponent))=0D
+=0D
+ keydata =3D mod + exp=0D
+=0D
+ return keydata=0D
+=0D
diff --git a/IntelFsp2Pkg/Tools/UserManuals/ConfigEditorUserManual.md b/Int=
elFsp2Pkg/Tools/UserManuals/ConfigEditorUserManual.md
new file mode 100644
index 0000000000..aeb3c4bdf1
--- /dev/null
+++ b/IntelFsp2Pkg/Tools/UserManuals/ConfigEditorUserManual.md
@@ -0,0 +1,19 @@
+# ConfigEditor.py is a python script with a GUI interface that can support=
changing configuration settings directly from the interface without having=
to modify the source.=0D
+=0D
+This tool depends on Python GUI tool kit Tkinter. It runs on both Windows =
and Linux.=0D
+=0D
+It supports:=0D
+=0D
+- Load the YAML file for a FSP UPD into the ConfigEditor to change the des=
ired configuration values=0D
+=0D
+- Generate a new configuration delta file or a config binary blob for the =
newly changed values to take effect=0D
+=0D
+## Running Configuration Editor:=0D
+=0D
+ **python ConfigEditor.py** =0D
+=0D
+For example:=0D
+=0D
+![Example ConfigEditor 1](https://slimbootloader.github.io/_images/CfgEdit=
Open.png) =0D
+=0D
+![Example ConfigEditor 2](https://slimbootloader.github.io/_images/CfgEdit=
DefYaml.png)=20=20
\ No newline at end of file
--=20
2.28.0.windows.1


[Patch edk2-platforms V2] Intel/BoardModulePkg: sort load option in the first boot

Zhiguang Liu
 

Currently, load option is only sorted when setup is the first priority in b=
oot
option. However, Below change in UefiBootManagerLib puts setup in the end, =
which
causes the sort function won't be called.
MdeModulePkg/UefiBootManagerLib: Put BootMenu at the end of BootOrder
SHA-1: 7f34681c488aee2563eaa2afcc6a2c8aa7c5b912

This patch will set a NV variable in the first boot, and sort the boot opti=
on
in first boot.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>

Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
---
Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib.c | =
70 +++++++++++++++++++++++++++++++++++-----------------------------------
1 file changed, 35 insertions(+), 35 deletions(-)

diff --git a/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBds=
HookLib.c b/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsH=
ookLib.c
index d7612fb80a..5b65e5f3e9 100644
--- a/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib=
.c
+++ b/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib=
.c
@@ -992,37 +992,6 @@ ConnectSequence (
EfiBootManagerConnectAll ();=0D
}=0D
=0D
-=0D
-/**=0D
- The function is to consider the boot order which is not in our expectati=
on.=0D
- In the case that we need to re-sort the boot option.=0D
-=0D
- @retval TRUE Need to sort Boot Option.=0D
- @retval FALSE Don't need to sort Boot Option.=0D
-**/=0D
-BOOLEAN=0D
-IsNeedSortBootOption (=0D
- VOID=0D
- )=0D
-{=0D
- EFI_BOOT_MANAGER_LOAD_OPTION *BootOptions;=0D
- UINTN BootOptionCount;=0D
-=0D
- BootOptions =3D EfiBootManagerGetLoadOptions (&BootOptionCount, LoadOpti=
onTypeBoot);=0D
-=0D
- //=0D
- // If setup is the first priority in boot option, we need to sort boot o=
ption.=0D
- //=0D
- if ((BootOptionCount > 1) &&=0D
- (((StrnCmp (BootOptions->Description, L"Enter Setup", StrLen (L"Enter =
Setup"))) =3D=3D 0) ||=0D
- ((StrnCmp (BootOptions->Description, L"BootManagerMenuApp", StrLen (L"=
BootManagerMenuApp"))) =3D=3D 0))) {=0D
- return TRUE;=0D
- }=0D
-=0D
- return FALSE;=0D
-}=0D
-=0D
-=0D
/**=0D
Connects Root Bridge=0D
**/=0D
@@ -1332,6 +1301,9 @@ BdsAfterConsoleReadyBeforeBootOptionCallback (
)=0D
{=0D
EFI_BOOT_MODE LocalBootMode;=0D
+ EFI_STATUS Status;=0D
+ BOOLEAN IsFirstBoot;=0D
+ UINTN DataSize;=0D
=0D
DEBUG ((DEBUG_INFO, "Event gBdsAfterConsoleReadyBeforeBootOptionEvent ca=
llback starts\n"));=0D
//=0D
@@ -1376,14 +1348,42 @@ BdsAfterConsoleReadyBeforeBootOptionCallback (
//=0D
// PXE boot option may appear after boot option enumeration=0D
//=0D
+=0D
+ EfiBootManagerRefreshAllBootOption ();=0D
+=0D
+ Status =3D gRT->GetVariable (=0D
+ L"IsFirstBoot",=0D
+ &gEfiCallerIdGuid,=0D
+ NULL,=0D
+ &DataSize,=0D
+ &IsFirstBoot=0D
+ );=0D
+ if (EFI_ERROR (Status)) {=0D
+ //=0D
+ // If can't find the variable, see it as the first boot=0D
+ //=0D
+ IsFirstBoot =3D TRUE;=0D
+ }=0D
+=0D
+ if (IsFirstBoot =3D=3D TRUE) {=0D
+ //=0D
+ // In the first boot, sort the boot option=0D
+ //=0D
+ EfiBootManagerSortLoadOptionVariable (LoadOptionTypeBoot, CompareB=
ootOption);=0D
+ IsFirstBoot =3D FALSE;=0D
+ Status =3D gRT->SetVariable (=0D
+ L"IsFirstBoot",=0D
+ &gEfiCallerIdGuid,=0D
+ EFI_VARIABLE_NON_VOLATILE | EFI_VARIABLE_RUNTIME_A=
CCESS | EFI_VARIABLE_BOOTSERVICE_ACCESS,=0D
+ 1,=0D
+ &IsFirstBoot=0D
+ );=0D
+ }=0D
+=0D
break;=0D
}=0D
=0D
Print (L"Press F7 for BootMenu!\n");=0D
=0D
- EfiBootManagerRefreshAllBootOption ();=0D
=0D
- if (IsNeedSortBootOption()) {=0D
- EfiBootManagerSortLoadOptionVariable (LoadOptionTypeBoot, CompareBootO=
ption);=0D
- }=0D
}=0D
--=20
2.30.0.windows.2


[PATCH] [edk2-platforms]Intel/BoardModulePkg: sort load option in the first boot

Zhiguang Liu
 

Currently, load option is only sorted when setup is the first priority in b=
oot
option. However, Below change in UefiBootManagerLib puts setup in the end, =
which
causes the sort function won't be called.
MdeModulePkg/UefiBootManagerLib: Put BootMenu at the end of BootOrder
SHA-1: 7f34681c488aee2563eaa2afcc6a2c8aa7c5b912

This patch will set a NV variable in the first boot, and sort the boot opti=
on
in first boot.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>

Change-Id: Idd4e9547e9ae8aad8296f2f71954951ad3093576
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
---
Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib.c | =
70 +++++++++++++++++++++++++++++++++++-----------------------------------
1 file changed, 35 insertions(+), 35 deletions(-)

diff --git a/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBds=
HookLib.c b/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsH=
ookLib.c
index d7612fb8..5b65e5f3 100644
--- a/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib=
.c
+++ b/Platform/Intel/BoardModulePkg/Library/BoardBdsHookLib/BoardBdsHookLib=
.c
@@ -992,37 +992,6 @@ ConnectSequence (
EfiBootManagerConnectAll ();=0D
}=0D
=0D
-=0D
-/**=0D
- The function is to consider the boot order which is not in our expectati=
on.=0D
- In the case that we need to re-sort the boot option.=0D
-=0D
- @retval TRUE Need to sort Boot Option.=0D
- @retval FALSE Don't need to sort Boot Option.=0D
-**/=0D
-BOOLEAN=0D
-IsNeedSortBootOption (=0D
- VOID=0D
- )=0D
-{=0D
- EFI_BOOT_MANAGER_LOAD_OPTION *BootOptions;=0D
- UINTN BootOptionCount;=0D
-=0D
- BootOptions =3D EfiBootManagerGetLoadOptions (&BootOptionCount, LoadOpti=
onTypeBoot);=0D
-=0D
- //=0D
- // If setup is the first priority in boot option, we need to sort boot o=
ption.=0D
- //=0D
- if ((BootOptionCount > 1) &&=0D
- (((StrnCmp (BootOptions->Description, L"Enter Setup", StrLen (L"Enter =
Setup"))) =3D=3D 0) ||=0D
- ((StrnCmp (BootOptions->Description, L"BootManagerMenuApp", StrLen (L"=
BootManagerMenuApp"))) =3D=3D 0))) {=0D
- return TRUE;=0D
- }=0D
-=0D
- return FALSE;=0D
-}=0D
-=0D
-=0D
/**=0D
Connects Root Bridge=0D
**/=0D
@@ -1332,6 +1301,9 @@ BdsAfterConsoleReadyBeforeBootOptionCallback (
)=0D
{=0D
EFI_BOOT_MODE LocalBootMode;=0D
+ EFI_STATUS Status;=0D
+ BOOLEAN IsFirstBoot;=0D
+ UINTN DataSize;=0D
=0D
DEBUG ((DEBUG_INFO, "Event gBdsAfterConsoleReadyBeforeBootOptionEvent ca=
llback starts\n"));=0D
//=0D
@@ -1376,14 +1348,42 @@ BdsAfterConsoleReadyBeforeBootOptionCallback (
//=0D
// PXE boot option may appear after boot option enumeration=0D
//=0D
+=0D
+ EfiBootManagerRefreshAllBootOption ();=0D
+=0D
+ Status =3D gRT->GetVariable (=0D
+ L"IsFirstBoot",=0D
+ &gEfiCallerIdGuid,=0D
+ NULL,=0D
+ &DataSize,=0D
+ &IsFirstBoot=0D
+ );=0D
+ if (EFI_ERROR (Status)) {=0D
+ //=0D
+ // If can't find the variable, see it as the first boot=0D
+ //=0D
+ IsFirstBoot =3D TRUE;=0D
+ }=0D
+=0D
+ if (IsFirstBoot =3D=3D TRUE) {=0D
+ //=0D
+ // In the first boot, sort the boot option=0D
+ //=0D
+ EfiBootManagerSortLoadOptionVariable (LoadOptionTypeBoot, CompareB=
ootOption);=0D
+ IsFirstBoot =3D FALSE;=0D
+ Status =3D gRT->SetVariable (=0D
+ L"IsFirstBoot",=0D
+ &gEfiCallerIdGuid,=0D
+ EFI_VARIABLE_NON_VOLATILE | EFI_VARIABLE_RUNTIME_A=
CCESS | EFI_VARIABLE_BOOTSERVICE_ACCESS,=0D
+ 1,=0D
+ &IsFirstBoot=0D
+ );=0D
+ }=0D
+=0D
break;=0D
}=0D
=0D
Print (L"Press F7 for BootMenu!\n");=0D
=0D
- EfiBootManagerRefreshAllBootOption ();=0D
=0D
- if (IsNeedSortBootOption()) {=0D
- EfiBootManagerSortLoadOptionVariable (LoadOptionTypeBoot, CompareBootO=
ption);=0D
- }=0D
}=0D
--=20
2.30.0.windows.2


Conflicting virtual addresses causing Runtime Services issues

Jon Nettleton
 

I am debugging a failure that I am seeing while using the HoneyComb's
spi-nor flash for runtime variable storage. I am hoping someone on
the list can give me some insight as what may be the problem.

The problem showed up when we switched the MMIO region for the fspi
flash device to be marked as non executable. reading variables is
fine, however writes began throwing an error.

[ 556.709828] Unable to handle kernel execute from non-executable
memory at virtual address 00000000206a3968

I have patched the kernel and removed the X86 requirement and enabled
the sysfs runtime mappings kernel config so I can get an easy view of
the mappings the kernel carries for runtime services. I then track
that virtual address to the MMIO region of nor flash, which makes
sense that region is marked as non executable. The question is why is
code being executed from this address range

attribute
::::::::::::::
0x8000000000000001
::::::::::::::
num_pages
::::::::::::::
0x40
::::::::::::::
phys_addr
::::::::::::::
0x20500000
::::::::::::::
type
::::::::::::::
0xb
::::::::::::::
virt_addr
::::::::::::::
0x20680000

So then I patched the PL011 serial driver to be able to log to the
console in runtime and I track down the access to Status =
Fvb->GetPhysicalAddress(Fvb, &FvVolHdr); in UpdateVariableStore().
What I don't understand is why EfiConvertPointer is mapping that
pointer into the Virtual address space occupied by the runtime mmio of
the flash. The pointer is being properly remapped. Here are the
pointer addresses in EFI and Kernel Runtime

EFI:
UpdateVariableStore:156 ECE33968
FvbGetPhysicalAddress(BaseAddress=0x20000000)

KERNEL:
UpdateVariableStore:156 206A3968
[ 556.709828] Unable to handle kernel execute from non-executable
memory at virtual address 00000000206a3968

Any insight that anyone could provide would be much appreciated.

Thanks,
Jon


Google Summer of Code (GSoC) 2021!

Nate DeSimone
 

Hi Everyone,

I am pleased to announce that TianoCore has been accepted to participate in GSoC 2021! Student applications will start rolling in on March 29th, so now we need some volunteers to be mentors. Thank you to all who have volunteered! Mentors will need to be able to commit to the following:

* Generally, we prefer mentors who have been long term contributors to the TianoCore project
* During the student application period (March 29 - April 13), mentors need to be available to help review student applications.
* On average, we expect each student will need roughly 5 hours of help each week during the 10 week development period (June 7 - August 16). I know that most people here have day jobs (including myself) so I'd like to assign 2 mentors per student whenever possible, this will reduce the time investment to ~2.5 hours per week.
* The student and their mentors should meet for 30min at absolute minimum on a weekly basis. The mentors should be able to make themselves available for more time if the student requires additional mentoring. In general, mentors should help their students succeed.
* Help the students get involved the rest of the TianoCore community. A lot of times, students have no prior open source experience and can at times be shy. One of the goals for GSoC is to give students new experiences and exposure to the open source community. Mentors should help students present their project at one of the TianoCore Design Meetings and help them solicit feedback from other community members on the mailing list.
* Mentors will need to write up 2 evaluations (July and August) for the students they are mentoring.

If you are a student and are interested in participating this year, feel free to introduce yourself on the mailing list and discuss potential project proposals.

If you are interested, please send me an off-list/private email.

Thanks!
Nate


Re: [PATCH 0/2] Update Maintainers.txt for TDX and Confidential Computing

Yao, Jiewen
 

Both 1 and 2 - Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

-----Original Message-----
From: Xu, Min M <min.m.xu@intel.com>
Sent: Wednesday, March 10, 2021 10:56 AM
To: devel@edk2.groups.io
Cc: Xu, Min M <min.m.xu@intel.com>; Andrew Fish <afish@apple.com>; Laszlo
Ersek <lersek@redhat.com>; Leif Lindholm <leif@nuviainc.com>; Kinney,
Michael D <michael.d.kinney@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>
Subject: [PATCH 0/2] Update Maintainers.txt for TDX and Confidential
Computing

Register reviewers for the TDX-related and Confidential Computing related
files in OvmfPkg.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>

Min Xu (2):
Maintainers.txt: Add reviewers for the OvmfPkg TDX-related files
Maintainers.txt: Add reviewers for Confidential Computing related
modules

Maintainers.txt | 7 +++++++
1 file changed, 7 insertions(+)

--
2.29.2.windows.2


[PATCH 1/2] Maintainers.txt: Add reviewers for the OvmfPkg TDX-related files

Min Xu
 

Register reviewers for the TDX-related files in OvmfPkg.

Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
Maintainers.txt | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/Maintainers.txt b/Maintainers.txt
index e38497123857..7d9fe89d6d28 100644
--- a/Maintainers.txt
+++ b/Maintainers.txt
@@ -465,6 +465,10 @@ F: OvmfPkg/PlatformPei/AmdSev.c
R: Tom Lendacky <thomas.lendacky@amd.com>
R: Brijesh Singh <brijesh.singh@amd.com>

+OvmfPkg: TDX-related modules
+R: Jiewen Yao <jiewen.yao@intel.com>
+R: Min Xu <min.m.xu@intel.com>
+
OvmfPkg: TCG- and TPM2-related modules
F: OvmfPkg/Include/IndustryStandard/QemuTpm.h
F: OvmfPkg/Library/PlatformBootManagerLib/BdsPlatform.c
--
2.29.2.windows.2

13441 - 13460 of 85992