Bob, how long would it take to resolve a tool issue if all users use base tool from pip?

From: gaoliming <gaoliming@...>
Sent: Saturday, October 8, 2022 10:39 AM
To: Feng, Bob C <bob.c.feng@...>; discuss@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>; Chen, Christine <yuwei.chen@...>; Huang, Long1 <long1.huang@...>; Ni, Ray <ray.ni@...>
Subject: 回复: Follow-up for the issue about DSC LibraryClass precedence rule

Bob:
This is a good enhancement in BaseTools development process. Will you also update TianoCore Wiki?

Thanks
Liming
发件人: Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>
发送时间: 2022年10月2日 23:01
收件人: discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
抄送: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Gao, Liming <gaoliming@...<mailto:gaoliming@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>; Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
主题: Follow-up for the issue about DSC LibraryClass precedence rule

Hello,

On 30th June, we merged a BaseTools patch “Fix DSC LibraryClass precedence rule” to edk2 repo. This patch changed the DSC LibraryClass precedence rule to make the basetools behavior consistent with DSC spec. About 21th Aug, it was found that that patch introduced a regression issue that cause MdeModulePkg gcc build to fail. And after syncing that basetools patch to edk2-repo and updating edk2 pip-requirements.txt, that regression issue was also caught by edk2 CI. Finally, that patch was reverted from edk2 repo and edk2-basetools repo.

But the issue about that the DSC LibraryClass precedence rule is inconsistent in BaseTools implementation and DSC spec still exists. As the follow-up, the bug submitter, me and other stakeholders did an offline discussion, we got aligned that we will update DSC spec to make basetools implantation and DSC spec consistent. If there is no objection from the community, we will update DSC spec and reuse bug https://bugzilla.tianocore.org/show_bug.cgi?id=3965 to track DSC spec update.

Further, to avoid similar issues happening, I’d suggest the improvement in basetools python tools work process. We must merge the basetools changes to edk2-basetools first and then sync them to edk2 repo.

1. Maintain process

1. After the patch gets reviewed, the maintainer creates a PR to edk-basetools repo, and merges it into edk2-basetools repo if the CI checks pass
2. Wait for the new version pip module generated in pypi.org
3. Update edk2-basetools value to the latest basetools pip module version from edk2/pip-requirements.txt file. Create a PR to edk2 repo to trigger edk2 CI to do the packages build tests.
Notes: The above 3 steps is to make edk2 CI test the basetools changes.

1. Create a pip-requirement patch and send it to community review.
2. Get the Reviewed-by
Notes: Can we remove the above 2 steps to optimize the process since the basetools patch has been got reviewed and tested?

1. Create a PR and merge the pip-requirement change to edk2 repo
2. Create a PR and merge the basetools patch to edk2 repo

1. edk2-basetools repo CI enhancement

1. enable the edk2 packages build test in edk2-basetools CI, so that we can remove the step c from the above maintain process.

Also, we need every edk2 downstream consumer to be ready to use basetools pip module so that we can remove the python code from edk2 repo and reduce the edk2 basetools maintenance efforts.


Thanks,
Bob

Hi Ray,

When we start to use basetools pip module, the maintenance process would be like this,


1. Reviewers review basetools patch via PR for edk2-basetools repository
2. Maintainer push the code to edk2-basetools repository
3. Github automatically create new version of basetools pip module and upload it to pypi.org. This action will take few minutes right after code being pushed to repo.
4. Maintainer will send a edk2 patch to update basetools version in pip-requirement.txt
5. After the pip-requirement.txt being reviewed and pushed to edk2 repo, the new version of basetools will work.

Step 4 and step 5 are quite like the process that we are doing for basetools patch now.


Thanks,
Bob

From: Ni, Ray <ray.ni@...>
Sent: Saturday, October 8, 2022 1:59 PM
To: Gao, Liming <gaoliming@...>; Feng, Bob C <bob.c.feng@...>; discuss@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>; Chen, Christine <yuwei.chen@...>; Huang, Long1 <long1.huang@...>
Subject: RE: Follow-up for the issue about DSC LibraryClass precedence rule

Bob, how long would it take to resolve a tool issue if all users use base tool from pip?


From: gaoliming <gaoliming@...<mailto:gaoliming@...>>
Sent: Saturday, October 8, 2022 10:39 AM
To: Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>; discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
Cc: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>; Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
Subject: 回复: Follow-up for the issue about DSC LibraryClass precedence rule

Bob:
This is a good enhancement in BaseTools development process. Will you also update TianoCore Wiki?

Thanks
Liming
发件人: Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>
发送时间: 2022年10月2日 23:01
收件人: discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
抄送: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Gao, Liming <gaoliming@...<mailto:gaoliming@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>; Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
主题: Follow-up for the issue about DSC LibraryClass precedence rule

Hello,

On 30th June, we merged a BaseTools patch “Fix DSC LibraryClass precedence rule” to edk2 repo. This patch changed the DSC LibraryClass precedence rule to make the basetools behavior consistent with DSC spec. About 21th Aug, it was found that that patch introduced a regression issue that cause MdeModulePkg gcc build to fail. And after syncing that basetools patch to edk2-repo and updating edk2 pip-requirements.txt, that regression issue was also caught by edk2 CI. Finally, that patch was reverted from edk2 repo and edk2-basetools repo.

But the issue about that the DSC LibraryClass precedence rule is inconsistent in BaseTools implementation and DSC spec still exists. As the follow-up, the bug submitter, me and other stakeholders did an offline discussion, we got aligned that we will update DSC spec to make basetools implantation and DSC spec consistent. If there is no objection from the community, we will update DSC spec and reuse bug https://bugzilla.tianocore.org/show_bug.cgi?id=3965 to track DSC spec update.

Further, to avoid similar issues happening, I’d suggest the improvement in basetools python tools work process. We must merge the basetools changes to edk2-basetools first and then sync them to edk2 repo.

1. Maintain process

1. After the patch gets reviewed, the maintainer creates a PR to edk-basetools repo, and merges it into edk2-basetools repo if the CI checks pass
2. Wait for the new version pip module generated in pypi.org
3. Update edk2-basetools value to the latest basetools pip module version from edk2/pip-requirements.txt file. Create a PR to edk2 repo to trigger edk2 CI to do the packages build tests.
Notes: The above 3 steps is to make edk2 CI test the basetools changes.

1. Create a pip-requirement patch and send it to community review.
2. Get the Reviewed-by
Notes: Can we remove the above 2 steps to optimize the process since the basetools patch has been got reviewed and tested?

1. Create a PR and merge the pip-requirement change to edk2 repo
2. Create a PR and merge the basetools patch to edk2 repo

1. edk2-basetools repo CI enhancement

1. enable the edk2 packages build test in edk2-basetools CI, so that we can remove the step c from the above maintain process.

Also, we need every edk2 downstream consumer to be ready to use basetools pip module so that we can remove the python code from edk2 repo and reduce the edk2 basetools maintenance efforts.


Thanks,
Bob

Liming,

Yes, I’ll update the TianoCore Wiki.

Since there is no objection to the basetools maintenance process proposal, I’ll start to follow that process for those reviewed python code patches.

Thanks,
Bob

From: Feng, Bob C
Sent: Tuesday, October 11, 2022 10:16 AM
To: Ni, Ray <ray.ni@...>; Gao, Liming <gaoliming@...>; discuss@edk2.groups.io
Cc: Kinney, Michael D <michael.d.kinney@...>; Chen, Christine <Yuwei.Chen@...>; Huang, Long1 <long1.huang@...>
Subject: RE: Follow-up for the issue about DSC LibraryClass precedence rule

Hi Ray,

When we start to use basetools pip module, the maintenance process would be like this,


1. Reviewers review basetools patch via PR for edk2-basetools repository
2. Maintainer push the code to edk2-basetools repository
3. Github automatically create new version of basetools pip module and upload it to pypi.org. This action will take few minutes right after code being pushed to repo.
4. Maintainer will send a edk2 patch to update basetools version in pip-requirement.txt
5. After the pip-requirement.txt being reviewed and pushed to edk2 repo, the new version of basetools will work.

Step 4 and step 5 are quite like the process that we are doing for basetools patch now.


Thanks,
Bob

From: Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
Sent: Saturday, October 8, 2022 1:59 PM
To: Gao, Liming <gaoliming@...<mailto:gaoliming@...>>; Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>; discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
Cc: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>
Subject: RE: Follow-up for the issue about DSC LibraryClass precedence rule

Bob, how long would it take to resolve a tool issue if all users use base tool from pip?


From: gaoliming <gaoliming@...<mailto:gaoliming@...>>
Sent: Saturday, October 8, 2022 10:39 AM
To: Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>; discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
Cc: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>; Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
Subject: 回复: Follow-up for the issue about DSC LibraryClass precedence rule

Bob:
This is a good enhancement in BaseTools development process. Will you also update TianoCore Wiki?

Thanks
Liming
发件人: Feng, Bob C <bob.c.feng@...<mailto:bob.c.feng@...>>
发送时间: 2022年10月2日 23:01
收件人: discuss@edk2.groups.io<mailto:discuss@edk2.groups.io>
抄送: Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>; Gao, Liming <gaoliming@...<mailto:gaoliming@...>>; Chen, Christine <yuwei.chen@...<mailto:yuwei.chen@...>>; Huang, Long1 <long1.huang@...<mailto:long1.huang@...>>; Ni, Ray <ray.ni@...<mailto:ray.ni@...>>
主题: Follow-up for the issue about DSC LibraryClass precedence rule

Hello,

On 30th June, we merged a BaseTools patch “Fix DSC LibraryClass precedence rule” to edk2 repo. This patch changed the DSC LibraryClass precedence rule to make the basetools behavior consistent with DSC spec. About 21th Aug, it was found that that patch introduced a regression issue that cause MdeModulePkg gcc build to fail. And after syncing that basetools patch to edk2-repo and updating edk2 pip-requirements.txt, that regression issue was also caught by edk2 CI. Finally, that patch was reverted from edk2 repo and edk2-basetools repo.

But the issue about that the DSC LibraryClass precedence rule is inconsistent in BaseTools implementation and DSC spec still exists. As the follow-up, the bug submitter, me and other stakeholders did an offline discussion, we got aligned that we will update DSC spec to make basetools implantation and DSC spec consistent. If there is no objection from the community, we will update DSC spec and reuse bug https://bugzilla.tianocore.org/show_bug.cgi?id=3965 to track DSC spec update.

Further, to avoid similar issues happening, I’d suggest the improvement in basetools python tools work process. We must merge the basetools changes to edk2-basetools first and then sync them to edk2 repo.

1. Maintain process

1. After the patch gets reviewed, the maintainer creates a PR to edk-basetools repo, and merges it into edk2-basetools repo if the CI checks pass
2. Wait for the new version pip module generated in pypi.org
3. Update edk2-basetools value to the latest basetools pip module version from edk2/pip-requirements.txt file. Create a PR to edk2 repo to trigger edk2 CI to do the packages build tests.
Notes: The above 3 steps is to make edk2 CI test the basetools changes.

1. Create a pip-requirement patch and send it to community review.
2. Get the Reviewed-by
Notes: Can we remove the above 2 steps to optimize the process since the basetools patch has been got reviewed and tested?

1. Create a PR and merge the pip-requirement change to edk2 repo
2. Create a PR and merge the basetools patch to edk2 repo

1. edk2-basetools repo CI enhancement

1. enable the edk2 packages build test in edk2-basetools CI, so that we can remove the step c from the above maintain process.

Also, we need every edk2 downstream consumer to be ready to use basetools pip module so that we can remove the python code from edk2 repo and reduce the edk2 basetools maintenance efforts.


Thanks,
Bob