[edk2-staging/OpenSSL11_EOL][PATCH 4/4] Readme: 0322 update


Li, Yi
 

Signed-off-by: Yi Li <yi1.li@...>
---
CryptoPkg/Readme-OpenSSL3.0.md | 65 +++++++++++++++++++++++++++++-----
1 file changed, 57 insertions(+), 8 deletions(-)

diff --git a/CryptoPkg/Readme-OpenSSL3.0.md b/CryptoPkg/Readme-OpenSSL3.0.md
index 85ed37b61d..fc5d24d074 100644
--- a/CryptoPkg/Readme-OpenSSL3.0.md
+++ b/CryptoPkg/Readme-OpenSSL3.0.md
@@ -17,19 +17,26 @@ The branch owner: Li Yi <yi1.li@...>

## Latest update
Will update latest result here (Build based on Intel platform).
+Binaries mode (use crypto drivers)
| Driver | 1.1.1 | 3.0 | percent |
|-----------------|------------|------------|------------|
|CryptoPei | 386 | 398 | 3.1% |
|CryptoPeiPreMem | 31 | 31 | 0% |
-|CryptoDxeFull | 1014 | 1031 | 1.7% |
-|CryptoDxe | 804 | 886 | 10.1% |
-|CryptoSmm | 558 | 604 | 8.2% |
-
+|CryptoDxeFull | 1014 | 997 | -1.6% |
+|CryptoDxe | 804 | 871 | 8.3% |
+|CryptoSmm | 558 | 581 | 4.1% |
+
| LZMA Compressed | 1.1.1 | 3.0 | percent |
|-----------------|------------|------------|------------|
-|CryptoDxe | 311 | 350 | 12.2% |
-|CryptoSmm | 211 | 238 | 12.8% |
-|FV (Dxe+Smm) | 357 | 412 | 15.4% |
+|CryptoDxe | 311 | 346 | 11.2% |
+|CryptoSmm | 211 | 233 | 10.4% |
+|FV (Dxe+Smm) | 357 | 406 | 13.7% |
+
+Library mode (use crypto library)
+| Driver | 1.1.1 | 3.0 | delta |
+|--------------------|------------|------------|------------|
+| FV | 2377 | 2639 | 262 |
+|SecurityStubDxe.efi | 562 | 605 | 43 |

## Limitation

@@ -49,7 +56,7 @@ More complex API: There will be two code paths supporting 1.1.1 legacy and 3.0 p
### 1.Cut Provider
As CryptoPkg\Library\OpensslLib\OpensslStub\uefiprov.c

-### Remove unnecessary module
+### 2.Remove unnecessary module
SM2,
SM3 - 12KB,
MD5 - 8KB,
@@ -105,5 +112,47 @@ remove unused bio prov - 4KB
#### Risk:
This is workaround.

+## Openssl code change summary
+### Level 1: Reasonable changes to reduce size
+1. Add macro such like OPENSSL_NO_ECX OPENSSL_NO_ECD to remove ecx and ecd feature,
+will reduce size about 104KB.
+(commit: ec: disable ecx and ecd)
+https://github.com/liyi77/openssl/commit/2b0a888c3623e1dc0637fbe0c5dcc1211b4d0545
+
+2. Avoid build error when sm3 disabled.
+(commit: sm3: avoid build error after sm3 disabled)
+https://github.com/liyi77/openssl/commit/df92e440e45667da6ca1f9013f015e6d18981f2e
+
+### Level 2: A bit like workaround, with possibility of upstream to openssl
+1. Enable the legacy path for X509 pubkey decode and pmeth initialization,
+The purpose is to avoid the use of EN/DECODE and Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd6ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1bfe1d0263f074b
+
+2. Add 'type' field back to enable OPENSSL_NO_AUTOALGINIT, will reduce size about 27KB.
+issue: https://github.com/openssl/openssl/issues/20221
+(commit: evp: add type filed back)
+https://github.com/liyi77/openssl/commit/9c68a18a3a1967baf8d93eacadac9f0e14523715
+
+### Level 3: Totally workaround and hard to upstream to openssl, may need scripts to apply them inside EDK2
+1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-staging/commit/c3a5b69d8a3465259cfdca8f38b0dc7683b3690e
+
+2. Cut Name/NID mapping, will reduce size about 70KB.
+(commit: CryptoPkg: trim obj_dat.h)
+https://github.com/liyi77/edk2-staging/commit/6874485ebf89959953f7094990c7123e19748527
+
+3. Cut unnecessary API in structure.
+(commit: evp: cut bio_enc func 3KB)
+https://github.com/liyi77/openssl/commit/3a2331133c2e3bda3e9bdb861ea97e5d3969fb2d
+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e2955d7ff4306
+(commit: rsa: remove unused rsa ameth 7KB)
+https://github.com/liyi77/openssl/commit/8488c75701cdd5e626785e6d9d002f6fb30ae0ff
+(commit: x509: remove unused extentions 19KB)
+https://github.com/liyi77/openssl/commit/c27b3428708eb240b626946ce10d4219806d8adf
## Timeline
Target for 2023 Q1
\ No newline at end of file
--
2.31.1.windows.1


Gerd Hoffmann
 

Hi,

+### Level 2: A bit like workaround, with possibility of upstream to openssl
+1. Enable the legacy path for X509 pubkey decode and pmeth initialization,
+The purpose is to avoid the use of EN/DECODE and Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd6ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1bfe1d0263f074b
I suspect that is not going to work well long-term, probably openssl
will remove the code paths they consider being "legacy" at some point
in the future. Probably not 3.0.x but maybe in 3.1 branch.

+### Level 3: Totally workaround and hard to upstream to openssl, may need scripts to apply them inside EDK2
+1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-staging/commit/c3a5b69d8a3465259cfdca8f38b0dc7683b3690e
Allow people implement their own providers looks like an openssl feature
to me. So I don't think this will be a big problem to maintain, I
expect they try to keep the interfaces stable to not break apps doing
so.

The only little detail we do differently here is to remove the default
providers so LTO can actually remove the unused code.

+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e2955d7ff4306
Did you double-check this doesn't break something?

It did for me, due to some code in openssl depending on a
working bio_sprintf() implementation.

take care,
Gerd


Li, Yi
 

Hi Gerd,

Thanks for review,

+### Level 2: A bit like workaround, with possibility of upstream to
+openssl 1. Enable the legacy path for X509 pubkey decode and pmeth
+initialization, The purpose is to avoid the use of EN/DECODE and Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd6
+ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1bf
+e1d0263f074b
I suspect that is not going to work well long-term, probably openssl will remove the code paths they consider being "legacy" at some point in the future. Probably not 3.0.x but maybe in 3.1 branch.
Yes, I think in long-term the better way is to remove all legacy code paths, this will also help reduce the size.
The problem is that a large number of legacy APIs are currently used in the EDK2 code.
In the future, it may be a big update to throw all the legacy code.

+### Level 3: Totally workaround and hard to upstream to openssl, may
+need scripts to apply them inside EDK2 1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-staging/commit/c3a5b69d8a3465259cfdca8
+f38b0dc7683b3690e
Allow people implement their own providers looks like an openssl feature to me. So I don't think this will be a big problem to maintain, I expect they try to keep the interfaces stable to not break apps doing so.
The only little detail we do differently here is to remove the default providers so LTO can actually remove the unused code.
+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e
+2955d7ff4306
Did you double-check this doesn't break something?
It did for me, due to some code in openssl depending on a working bio_sprintf() implementation.
I don't do any more test than unit test.
I am sick of this part, but I currently have no other way to reduce the size. I would like to drop those changes first if i find another way.

Regards,
Yi


Yao, Jiewen
 

We have 2 level APIs.

1) EDKII other code -> CryptoPkg.
2) CryptoPkg -> Openssl.

Current strategy of openssl 3.0 update is to keep both 1) and 2). That is minimal impact.

Do you think if we can keep 1) and only update 2) to use new API in openssl 3.0?

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Li, Yi
Sent: Friday, March 24, 2023 9:47 AM
To: Gerd Hoffmann <kraxel@...>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4]
Readme: 0322 update

Hi Gerd,

Thanks for review,

+### Level 2: A bit like workaround, with possibility of upstream to
+openssl 1. Enable the legacy path for X509 pubkey decode and pmeth
+initialization, The purpose is to avoid the use of EN/DECODE and
Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd
6
+ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1
bf
+e1d0263f074b
I suspect that is not going to work well long-term, probably openssl will
remove the code paths they consider being "legacy" at some point in the
future. Probably not 3.0.x but maybe in 3.1 branch.

Yes, I think in long-term the better way is to remove all legacy code paths,
this will also help reduce the size.
The problem is that a large number of legacy APIs are currently used in the
EDK2 code.
In the future, it may be a big update to throw all the legacy code.

+### Level 3: Totally workaround and hard to upstream to openssl, may
+need scripts to apply them inside EDK2 1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-
staging/commit/c3a5b69d8a3465259cfdca8
+f38b0dc7683b3690e
Allow people implement their own providers looks like an openssl feature to
me. So I don't think this will be a big problem to maintain, I expect they try
to keep the interfaces stable to not break apps doing so.

The only little detail we do differently here is to remove the default
providers so LTO can actually remove the unused code.

+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e
+2955d7ff4306
Did you double-check this doesn't break something?
It did for me, due to some code in openssl depending on a working
bio_sprintf() implementation.

I don't do any more test than unit test.
I am sick of this part, but I currently have no other way to reduce the size. I
would like to drop those changes first if i find another way.

Regards,
Yi





Li, Yi
 

Not easy, I have tried to update, but blocked at the RSA and MAC part, there will be many strange problems such as:
the context generated by RsaNew and RsaSetKey cannot be used for sign/verify,
the hmac_duplicate (*src,*dst) function needs to expose the openssl structure details...
https://github.com/liyi77/libspdm/tree/openssl-3-rsa
https://github.com/liyi77/libspdm/tree/openssl-3-work-mac

I didn't have enough time to debug it.

Even if the API is updated, we still need to delete legacy code inside openssl to reduce size.

Regards,
Yi

-----Original Message-----
From: Yao, Jiewen <jiewen.yao@...>
Sent: Friday, March 24, 2023 2:11 PM
To: devel@edk2.groups.io; Li, Yi1 <yi1.li@...>; Gerd Hoffmann <kraxel@...>
Subject: RE: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4] Readme: 0322 update

We have 2 level APIs.

1) EDKII other code -> CryptoPkg.
2) CryptoPkg -> Openssl.

Current strategy of openssl 3.0 update is to keep both 1) and 2). That is minimal impact.

Do you think if we can keep 1) and only update 2) to use new API in openssl 3.0?



-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Li, Yi
Sent: Friday, March 24, 2023 9:47 AM
To: Gerd Hoffmann <kraxel@...>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4]
Readme: 0322 update

Hi Gerd,

Thanks for review,

+### Level 2: A bit like workaround, with possibility of upstream
+to openssl 1. Enable the legacy path for X509 pubkey decode and
+pmeth initialization, The purpose is to avoid the use of EN/DECODE
+and
Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd
6
+ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1
bf
+e1d0263f074b
I suspect that is not going to work well long-term, probably openssl
will
remove the code paths they consider being "legacy" at some point in
the future. Probably not 3.0.x but maybe in 3.1 branch.

Yes, I think in long-term the better way is to remove all legacy code
paths, this will also help reduce the size.
The problem is that a large number of legacy APIs are currently used
in the
EDK2 code.
In the future, it may be a big update to throw all the legacy code.

+### Level 3: Totally workaround and hard to upstream to openssl,
+may need scripts to apply them inside EDK2 1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-
staging/commit/c3a5b69d8a3465259cfdca8
+f38b0dc7683b3690e
Allow people implement their own providers looks like an openssl
feature to
me. So I don't think this will be a big problem to maintain, I expect
they try to keep the interfaces stable to not break apps doing so.

The only little detail we do differently here is to remove the
default
providers so LTO can actually remove the unused code.

+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e
+2955d7ff4306
Did you double-check this doesn't break something?
It did for me, due to some code in openssl depending on a working
bio_sprintf() implementation.

I don't do any more test than unit test.
I am sick of this part, but I currently have no other way to reduce
the size. I would like to drop those changes first if i find another way.

Regards,
Yi





Yao, Jiewen
 

Thanks. That means we need revisit the API in crypto library.

The original idea of crypt API is suitable for all crypto implementation.
But if they cannot be implemented with opensssl 3.0 mode, the crypto API design is problematic... Sigh...

-----Original Message-----
From: Li, Yi1 <yi1.li@...>
Sent: Friday, March 24, 2023 3:51 PM
To: Yao, Jiewen <jiewen.yao@...>; devel@edk2.groups.io; Gerd
Hoffmann <kraxel@...>
Subject: RE: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4]
Readme: 0322 update

Not easy, I have tried to update, but blocked at the RSA and MAC part, there
will be many strange problems such as:
the context generated by RsaNew and RsaSetKey cannot be used for
sign/verify,
the hmac_duplicate (*src,*dst) function needs to expose the openssl
structure details...
https://github.com/liyi77/libspdm/tree/openssl-3-rsa
https://github.com/liyi77/libspdm/tree/openssl-3-work-mac

I didn't have enough time to debug it.

Even if the API is updated, we still need to delete legacy code inside openssl
to reduce size.

Regards,
Yi

-----Original Message-----
From: Yao, Jiewen <jiewen.yao@...>
Sent: Friday, March 24, 2023 2:11 PM
To: devel@edk2.groups.io; Li, Yi1 <yi1.li@...>; Gerd Hoffmann
<kraxel@...>
Subject: RE: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4]
Readme: 0322 update

We have 2 level APIs.

1) EDKII other code -> CryptoPkg.
2) CryptoPkg -> Openssl.

Current strategy of openssl 3.0 update is to keep both 1) and 2). That is
minimal impact.

Do you think if we can keep 1) and only update 2) to use new API in openssl
3.0?



-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Li, Yi
Sent: Friday, March 24, 2023 9:47 AM
To: Gerd Hoffmann <kraxel@...>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-staging/OpenSSL11_EOL][PATCH 4/4]
Readme: 0322 update

Hi Gerd,

Thanks for review,

+### Level 2: A bit like workaround, with possibility of upstream
+to openssl 1. Enable the legacy path for X509 pubkey decode and
+pmeth initialization, The purpose is to avoid the use of EN/DECODE
+and
Signature provider, will reduce size about 90KB.
+(commit: x509: enable legacy path in pub decode)
+https://github.com/liyi77/openssl/commit/8780956da77c949ca42f6c4c3fd
6
+ef7045646ef0
+(commit: evp: enable legacy pmeth)
+https://github.com/liyi77/openssl/commit/a2232b35aa308198b61c5734c1
bf
+e1d0263f074b
I suspect that is not going to work well long-term, probably openssl
will
remove the code paths they consider being "legacy" at some point in
the future. Probably not 3.0.x but maybe in 3.1 branch.

Yes, I think in long-term the better way is to remove all legacy code
paths, this will also help reduce the size.
The problem is that a large number of legacy APIs are currently used
in the
EDK2 code.
In the future, it may be a big update to throw all the legacy code.

+### Level 3: Totally workaround and hard to upstream to openssl,
+may need scripts to apply them inside EDK2 1. Provider cut.
+(commit: CryptoPkg: add own openssl provider)
+https://github.com/liyi77/edk2-
staging/commit/c3a5b69d8a3465259cfdca8
+f38b0dc7683b3690e
Allow people implement their own providers looks like an openssl
feature to
me. So I don't think this will be a big problem to maintain, I expect
they try to keep the interfaces stable to not break apps doing so.

The only little detail we do differently here is to remove the
default
providers so LTO can actually remove the unused code.

+(commit: x509: remove print function 7KB)
+https://github.com/liyi77/openssl/commit/faa5d6781c3af601bcbc11ff199e
+2955d7ff4306
Did you double-check this doesn't break something?
It did for me, due to some code in openssl depending on a working
bio_sprintf() implementation.

I don't do any more test than unit test.
I am sick of this part, but I currently have no other way to reduce
the size. I would like to drop those changes first if i find another way.

Regards,
Yi