Date   

Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Bret Barkelew <bret.barkelew@...>
 

That link didn’t work for me.

- Bret

From: Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>
Sent: Monday, September 14, 2020 8:59 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] RE: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

No, EFI REST JSON Structure DXE Driver (UEF spec section 29.7.3) is a centralized manager to manage “EFI Redfish JSON resource to C structure Converter libraries/drivers” for converting Redfish resource in the specific schema from JSON format to the C structure or vice versa.
EFI REST JSON Structure DXE Driver itself doesn’t use JSON library, however “EFI Redfish JSON resource to C structure Converter libraries/drivers” do use open source jansson library to parse JSON payload. Furthermore, “EFI Redfish JSON resource to C structure Converter libraries/drivers” are generated by tool based on the published Redfish schemas. https://github.com/DMTF/Redfish-Schema-C-Struct-Generator<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDMTF%2FRedfish-Schema-C-Struct-Generator&data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1b8115973f364cea229008d8592bc486%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357391811797643&sdata=LdHAPly4oeVhyC6Xk7p0G0r1y8qVxrLfGZrN9EW5WpE%3D&reserved=0>

From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Bret Barkelew via groups.io
Sent: Tuesday, September 15, 2020 11:41 AM
To: Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; devel@edk2.groups.io; rfc@edk2.groups.io
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>; Chen, Aaron <aaron.chen@hpe.com>; siyuan.fu@intel.com; Wang, Fan <fan.wang@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

JSON Structure would probably be worth discussing. I know that there are other places I’ve seen JSON used and it may end up that we want common business logic (similar to using Oniguruma for regex). Do you parse/format JSON in that one?

- Bret

From: Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>
Sent: Monday, September 14, 2020 8:33 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] RE: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Not many drivers fall in edk2 repo so far, those are drivers with the corresponding definitions in UEFI spec.

* EFI REST EX UEFI Driver for Redfish service
* EFI Redfish Discover UEFI Driver
* EFI REST JSON Structure DXE Driver

All others have to go through code first policy, will be in edk2-staging repo.

-Abner

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Bret Barkelew via groups.io
Sent: Tuesday, September 15, 2020 11:19 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com<mailto:abner.chang@hpe.com>>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

I think code review works. I’m primarily interested in seeing how much code falls under the “edk2” vs “edk2-staging” repos.

- Bret

From: Abner Chang via groups.io<mailto:abner.chang=hpe.com@groups.io>
Sent: Monday, September 14, 2020 8:12 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Seems no one has comment on this topic. Let’s just go through the code review process.
Thanks

Abner

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Wednesday, September 9, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>>; Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Add [RFC] to the subject, add Ray and Mike to the loop.

From: Chang, Abner (HPS SW/FW Technologist)
Sent: Tuesday, September 8, 2020 12:06 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com<mailto:abner.chang@hpe.com>>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>
Subject: RE: Request for the new package "RedfishPkg" under edk2 repo

This is the RFC for the new package "RedfishPkg" introduced to edk2 repo, I thought mailing system will add [RFC] prefix to the subject. Sorry for the inconvenience.

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Tuesday, September 8, 2020 11:48 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Request for the new package "RedfishPkg" under edk2 repo

Hi everyone,
Given that we are going to contribute code of UEFI Redfish edk2 solution, a new package “RedfishPkg” under edk2 repo is necessary for accommodating the UEFI Redfish driver stacks, that includes

* EFI Redfish Host Interface DXE Driver
* EFI Refish Credential DXE Driver
* EFI REST EX UEFI Driver for Redfish service
* EFI Redfish Discover UEFI Driver
* EFI Redfish Discover Protocol
* EFI Redfish Config UEFI Driver
* EFI BIOS Config To Redfish Dxe Driver
* EFI REST JSON Structure DXE Driver
* EFI Source Coding DXE Driver
* EFI BIOS Resource Provision Generation Protocol
* EFI BIOS Resource Provision Transport Layer Protocol

The architecture have been discussing in TianoCore Design meeting and the corresponding BZ were created as well.
The code we will start to contribute includes

* Contribute to edk2 repo for those drivers already have the corresponding definitions in UEFI spec.
* Contribute code to edk2-staging/UEFI _Redfish for those drivers do not have the corresponding definitions in UEFI spec. This is for the evaluation and require ECR to USWG if community agree with having this driver for Redfish edk2 solution.

Please refer to below link for the details, https://github.com/tianocore/edk2-staging/blob/UEFI_Redfish/Readme.md<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttps-2D3A-5F-5Fnam06.safelinks.protection.outlook.com-5F-2D3Furl-2D3Dhttps-2D253A-2D252F-2D252Fgithub.com-2D252Ftianocore-2D252Fedk2-2D2Dstaging-2D252Fblob-2D252FUEFI-2D5FRedfish-2D252FReadme.md-2D26data-2D3D02-2D257C01-2D257Cbret.barkelew-2D2540microsoft.com-2D257Cec6961ac4b3143f196be08d859251f68-2D257C72f988bf86f141af91ab2d7cd011db47-2D257C1-2D257C0-2D257C637357363278947284-2D26sdata-2D3Dkt66JYtpN1X1hCrt5cQY3btyQEdoqZYkVPW5J7w8dws-2D253D-2D26reserved-2D3D0-2526d-253DDwMF-2Dg-2526c-253DC5b8zRQO1miGmBeVZ2LFWg-2526r-253D-5FSN6FZBN4Vgi4Ulkskz6qU3NYRO03nHp9P7Z5q59A3E-2526m-253DvNotrFyeoRyYey-2D0DOEVOLlZ7unqNGts5l1lH-2D4MzqM-2526s-253DBO3e8WR8joHCC9lD6Guk5Q2XN8DJ0JCOTy2AfB279q8-2526e-253D-26data-3D02-257C01-257Cbret.barkelew-2540microsoft.com-257C6bd38bf379f64b06f6b808d859282266-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C637357376203679617-26sdata-3D-252Fc1YADqJZbAxtJEfc7R4LRToIVVG-252F8P5K9XRzp2RTio-253D-26reserved-3D0%26d%3DDwMF-g%26c%3DC5b8zRQO1miGmBeVZ2LFWg%26r%3D_SN6FZBN4Vgi4Ulkskz6qU3NYRO03nHp9P7Z5q59A3E%26m%3DDkSlNNriVFNl3jnCeMG8vtCRlB3CgfREapKoapz-cx0%26s%3DvXm2LZsMsTXRCNZ9IZvid63RGiDHFF5aL_2JBwiL7kg%26e%3D&data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1b8115973f364cea229008d8592bc486%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357391811802634&sdata=r9JTbIGJH3tmabZ%2BOtYAjVm5PRXWvuK0Wm2v2NByE1M%3D&reserved=0>

Thanks
Abner


Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Bret Barkelew <bret.barkelew@...>
 

JSON Structure would probably be worth discussing. I know that there are other places I’ve seen JSON used and it may end up that we want common business logic (similar to using Oniguruma for regex). Do you parse/format JSON in that one?

- Bret

From: Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>
Sent: Monday, September 14, 2020 8:33 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@microsoft.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] RE: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Not many drivers fall in edk2 repo so far, those are drivers with the corresponding definitions in UEFI spec.

* EFI REST EX UEFI Driver for Redfish service
* EFI Redfish Discover UEFI Driver
* EFI REST JSON Structure DXE Driver

All others have to go through code first policy, will be in edk2-staging repo.

-Abner

From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Bret Barkelew via groups.io
Sent: Tuesday, September 15, 2020 11:19 AM
To: devel@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; rfc@edk2.groups.io
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>; Chen, Aaron <aaron.chen@hpe.com>; siyuan.fu@intel.com; Wang, Fan <fan.wang@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

I think code review works. I’m primarily interested in seeing how much code falls under the “edk2” vs “edk2-staging” repos.

- Bret

From: Abner Chang via groups.io<mailto:abner.chang=hpe.com@groups.io>
Sent: Monday, September 14, 2020 8:12 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Seems no one has comment on this topic. Let’s just go through the code review process.
Thanks

Abner

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Wednesday, September 9, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Ni, Ray <ray.ni@intel.com<mailto:ray.ni@intel.com>>; Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Add [RFC] to the subject, add Ray and Mike to the loop.

From: Chang, Abner (HPS SW/FW Technologist)
Sent: Tuesday, September 8, 2020 12:06 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com<mailto:abner.chang@hpe.com>>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>
Subject: RE: Request for the new package "RedfishPkg" under edk2 repo

This is the RFC for the new package "RedfishPkg" introduced to edk2 repo, I thought mailing system will add [RFC] prefix to the subject. Sorry for the inconvenience.

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Tuesday, September 8, 2020 11:48 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Request for the new package "RedfishPkg" under edk2 repo

Hi everyone,
Given that we are going to contribute code of UEFI Redfish edk2 solution, a new package “RedfishPkg” under edk2 repo is necessary for accommodating the UEFI Redfish driver stacks, that includes

* EFI Redfish Host Interface DXE Driver
* EFI Refish Credential DXE Driver
* EFI REST EX UEFI Driver for Redfish service
* EFI Redfish Discover UEFI Driver
* EFI Redfish Discover Protocol
* EFI Redfish Config UEFI Driver
* EFI BIOS Config To Redfish Dxe Driver
* EFI REST JSON Structure DXE Driver
* EFI Source Coding DXE Driver
* EFI BIOS Resource Provision Generation Protocol
* EFI BIOS Resource Provision Transport Layer Protocol

The architecture have been discussing in TianoCore Design meeting and the corresponding BZ were created as well.
The code we will start to contribute includes

* Contribute to edk2 repo for those drivers already have the corresponding definitions in UEFI spec.
* Contribute code to edk2-staging/UEFI _Redfish for those drivers do not have the corresponding definitions in UEFI spec. This is for the evaluation and require ECR to USWG if community agree with having this driver for Redfish edk2 solution.

Please refer to below link for the details, https://github.com/tianocore/edk2-staging/blob/UEFI_Redfish/Readme.md<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__nam06.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Ftianocore-252Fedk2-2Dstaging-252Fblob-252FUEFI-5FRedfish-252FReadme.md-26data-3D02-257C01-257Cbret.barkelew-2540microsoft.com-257Cec6961ac4b3143f196be08d859251f68-257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-257C637357363278947284-26sdata-3Dkt66JYtpN1X1hCrt5cQY3btyQEdoqZYkVPW5J7w8dws-253D-26reserved-3D0%26d%3DDwMF-g%26c%3DC5b8zRQO1miGmBeVZ2LFWg%26r%3D_SN6FZBN4Vgi4Ulkskz6qU3NYRO03nHp9P7Z5q59A3E%26m%3DvNotrFyeoRyYey-0DOEVOLlZ7unqNGts5l1lH-4MzqM%26s%3DBO3e8WR8joHCC9lD6Guk5Q2XN8DJ0JCOTy2AfB279q8%26e%3D&data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6bd38bf379f64b06f6b808d859282266%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357376203679617&sdata=%2Fc1YADqJZbAxtJEfc7R4LRToIVVG%2F8P5K9XRzp2RTio%3D&reserved=0>

Thanks
Abner


Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Bret Barkelew <bret.barkelew@...>
 

I think code review works. I’m primarily interested in seeing how much code falls under the “edk2” vs “edk2-staging” repos.

- Bret

From: Abner Chang via groups.io<mailto:abner.chang=hpe.com@groups.io>
Sent: Monday, September 14, 2020 8:12 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist)<mailto:abner.chang@hpe.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW)<mailto:nickle.wang@hpe.com>; Chen, Aaron<mailto:aaron.chen@hpe.com>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan<mailto:fan.wang@intel.com>; Wu, Jiaxin<mailto:jiaxin.wu@intel.com>; Ni, Ray<mailto:ray.ni@intel.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Seems no one has comment on this topic. Let’s just go through the code review process.
Thanks

Abner

From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Wednesday, September 9, 2020 11:02 AM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com>; Chen, Aaron <aaron.chen@hpe.com>; siyuan.fu@intel.com; Wang, Fan <fan.wang@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>; Ni, Ray <ray.ni@intel.com>; Michael D Kinney <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [RFC] Request for the new package "RedfishPkg" under edk2 repo

Add [RFC] to the subject, add Ray and Mike to the loop.

From: Chang, Abner (HPS SW/FW Technologist)
Sent: Tuesday, September 8, 2020 12:06 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com<mailto:abner.chang@hpe.com>>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>
Subject: RE: Request for the new package "RedfishPkg" under edk2 repo

This is the RFC for the new package "RedfishPkg" introduced to edk2 repo, I thought mailing system will add [RFC] prefix to the subject. Sorry for the inconvenience.

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> [mailto:devel@edk2.groups.io] On Behalf Of Abner Chang
Sent: Tuesday, September 8, 2020 11:48 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Cc: Wang, Nickle (HPS SW) <nickle.wang@hpe.com<mailto:nickle.wang@hpe.com>>; Chen, Aaron <aaron.chen@hpe.com<mailto:aaron.chen@hpe.com>>; siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>; Wang, Fan <fan.wang@intel.com<mailto:fan.wang@intel.com>>; Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Request for the new package "RedfishPkg" under edk2 repo

Hi everyone,
Given that we are going to contribute code of UEFI Redfish edk2 solution, a new package “RedfishPkg” under edk2 repo is necessary for accommodating the UEFI Redfish driver stacks, that includes

* EFI Redfish Host Interface DXE Driver
* EFI Refish Credential DXE Driver
* EFI REST EX UEFI Driver for Redfish service
* EFI Redfish Discover UEFI Driver
* EFI Redfish Discover Protocol
* EFI Redfish Config UEFI Driver
* EFI BIOS Config To Redfish Dxe Driver
* EFI REST JSON Structure DXE Driver
* EFI Source Coding DXE Driver
* EFI BIOS Resource Provision Generation Protocol
* EFI BIOS Resource Provision Transport Layer Protocol

The architecture have been discussing in TianoCore Design meeting and the corresponding BZ were created as well.
The code we will start to contribute includes

* Contribute to edk2 repo for those drivers already have the corresponding definitions in UEFI spec.
* Contribute code to edk2-staging/UEFI _Redfish for those drivers do not have the corresponding definitions in UEFI spec. This is for the evaluation and require ECR to USWG if community agree with having this driver for Redfish edk2 solution.

Please refer to below link for the details, https://github.com/tianocore/edk2-staging/blob/UEFI_Redfish/Readme.md<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging%2Fblob%2FUEFI_Redfish%2FReadme.md&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cec6961ac4b3143f196be08d859251f68%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637357363278947284&sdata=kt66JYtpN1X1hCrt5cQY3btyQEdoqZYkVPW5J7w8dws%3D&reserved=0>

Thanks
Abner


[RFC] Proposal to move all FSP 1.x binaries to "legacy" branch

Nate DeSimone
 

Hi Everyone,



Given that we have removed FSP 1.x support from TianoCore in edk2-stable201908, it seems that the 1.x FSP binaries at https://github.com/intel/FSP have become increasingly limited in their usefulness. For this reason, I would like to move those older binaries off master branch and create a new “legacy” branch to store them. I'll be sure to mention the legacy branch in the readme.md file in master. Any concerns with this change?



Thanks,

Nate


Re: [edk2-devel] Basetools as a pip module

Matthew Carlson <matthewfcarlson@...>
 

Hey Andrew,

Your comments were helpful and valid 😊

Part of the appeal of making it into a pip module is that it doesn’t have a specific file system path. Python itself take cares of it and it can be addressed by the module name rather than adding it to the path. So you hit the nail on the head, EDK_TOOL_PATH doesn’t really make sense in an all pip module world. It will probably be deprecated. You can see here: https://github.com/matthewfcarlson/edk2/blob/feature/pip-basetools/BaseTools/BinWrappers/WindowsLike/build.bat that we call directly into the module without having to setup the python path. The file here doesn’t use the BaseTools inside EDK2, that still need to built.

You have some great questions and I’ll do my best to answer with what has been discussed on the original RFC.

* if I run build from the command line how did my path get set?
* The pip system adds it to the python patch for you where ever you have it installed (locally, user, system, etc). In addition, pip module offers console scripts, which are small hooks that basically do what the build.bat script does: start the python interpreter and call the build module. So in the current pip module there are a few new commands, one of them being edk2_build which is available anywhere (within the scope of your pip environment). Alternatively, you could use the BinWrappers and let the edksetup do the heavy lifting for you. It should all just work out of the box (fingers crossed).
* How do I configure different versions of BaseTools?
* So Lazlo pointed out that you could use virtual environments, one for each edk2 clone on your system. It has the added benefit of making your system state much simpler and more reproducible since the pip module are for that repo rather than your whole system. Another option is just to install a local copy of the pip module using pip install -e . which will stay in sync with whatever you have checked out. This does mean an extra step of making sure you have the right commit checked out and probably isn’t advisable since it’s easy to forget. But since I only am ever have one clone open at a time, it’s not bad for me personally.
* I guess my repo could have a yml file that points to the version of the tools that I use, but that seems like a per user, not per repo config
* There’s a pip requirement file that keeps track of the exact version of the pip module you need (EDK2 already has one for CI). For a given commit in EDK2, you’d have the exact version of basetools that you know works.
* It seems to me we could have developers that want to contribute to edk2 and work on their own code base and that could rehire (require?) two different Basetool versions installed on the system, and I think we need a story for that?
* Yeah- I agree that story isn’t as fleshed out. In some way it’s better than EDK2’s current story as the basetools can be consumed separately from EDK2 itself. You could setup your own codebase to leverage the basetools as is or you could point it at your own fork of the BaseTools. But I can see folks who have forked EDK2 and have tweaks in BaseTools having to push those changes back into BaseTools. I believe that moving it into it’s own project will make it easier for fixes and other projects to be built off the BaseTools as it makes it easy to include without needing the rest of EDK2 with it. For example, easy to use and easy to install capsule generation tools for FMP devices.
* For the macOS Xcode compiler you can install as many versions as you want and there is a command line tool to let you set the current version of Xcode, and to show you the currently select versions. Basically the tools in the magic location in your path are just redirectors to the currently selected tools.
* I think that’s possible inside pip to have sort of the functionality you describe but generally it would be through virtual environments and perhaps installing it in different places (system vs user for example). With pip, you can also do a pip freeze to see the state of your pip system in the given environment. It’s likely not as robust/fully-featured as the XCode system, though it’s been a few years since I was a serious XCode user and my recollection of exactly the experience is a little fuzzy.

Thanks again for your feedback and hopefully that addresses your questions.

*From:* Andrew Fish ( afish@apple.com )
*Sent:* Wednesday, September 2, 2020 7:14 PM
*To:* Matthew Carlson ( matthewfcarlson@gmail.com )
*Cc:* Laszlo Ersek ( lersek@redhat.com ) ; edk2-devel-groups-io ( devel@edk2.groups.io ) ; rfc@edk2.groups.io
*Subject:* Re: [edk2-devel] Basetools as a pip module

Matthew,

I did not meant to imply we should optimize for the current installed base, I just think it is useful to think about them. I think Lazlo is pointing out what is best for the project and that should have more weight than the installed base, but it is always good to think how things impact different groups.

In terms of the UI I was thinking of pointing to the pip install location vs. your git repo not so much a boolean. I actually don’t quite understand what EDK_TOOL_PATH actually means if BaseTools is a pip module. If that is an obsolete concept then we should remove it, and replace with some kind of statement that the pip installed BaseTools are being used. Another question if I run build from the command line how did my path get set? For example my user account has 5 different versions of edk2 in it how do I configure different versions of BaseTools? How do I pip multiple versions on to my system? I guess my repo could have a yml file that points to the version of the tools that I use, but that seems like a per user, not per repo config? It seems to me we could have developers that want to contribute to edk2 and work on their own code base and that could rehire two different Basetool versions installed on the system, and I think we need a story for that.

For the macOS Xcode compiler you can install as many versions as you want and there is a command line tool to let you set the current version of Xcode, and to show you the currently select versions. Basically the tools in the magic location in your path are just redirectors to the currently selected tools.

Thanks,

Andrew Fish




On Sep 2, 2020, at 12:06 PM, Matthew Carlson < matthewfcarlson@gmail.com >
wrote:








Andrew:









I think leveraging the existing edksetup is a great idea. Using the
existing EDK_TOOL_PATH variable could work but it seems clunky. Since the
pip module wouldn't be a path, it seems strange to put a boolean value in
a variable meant to hold a path. I definitely think that the scripts could
print whether they're using the pip modules or the in-source tools. Since
Lazlo suggested that pip will be the default, we could have the in-source
modules notify of the fact that you're using the in-source modules. An
additional feature for the pip module could be printing the version that
they are (since you can use the pip infrastructure to query the version of
a given module within a python script). Another option would be simply
trying the pip module first and then falling back to the in-source module.
There would be a slight speed penalty (likely around 10ms) but since this
would only apply to trim and build, it should have relatively low impact.









Lazlo:




Thank you for the excellent summary of the different pieces of the
discussion along with the links. To answer your first point, I think what
a developer does with their pip module is largely up to them. They could
do a virtual environment, they could just do what the requirements state,
or pip install from a checked out basetools.I do think there are some
variables that the virtual environment sets up that would be a good signal
whether you're in a virtual environment or not. I agree with your approach
of basetools development going into the out of edk2 repo and the
importance of making sure package maintainers test and validate their
areas with the new setup. I would personally try to get this early into
the development cycle, (just after this next stable tag this week) to give
the community and developers the most amount of time to get used to
things. A trial period of one release makes sense.









I also agree that the gateway is important in maintaining cohesion between
the new and the old. Hopefully that's nearing completion.









Hopefully other stewards will weigh in but otherwise we'll move ahead with
a proposed solution in patches next week.









-Matthew Carlson













On Wed, Sep 2, 2020 at 1:49 AM Laszlo Ersek < lersek@redhat.com > wrote:





On 09/02/20 02:49, Andrew Fish via groups.io ( http://groups.io/ ) wrote:


On Sep 1, 2020, at 4:35 PM, Matthew Carlson < matthewfcarlson@gmail.com
wrote:

Hello all,

A recent topic on the RFC mailing list went out and the work on moving
Basetools/Sources/Python to a separate repo has started. See the RFC
conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270 < https://edk2.groups.io/g/rfc/topic/74009714#270


The repo in question is here: https://github.com/tianocore/edk2-basetools
< https://github.com/tianocore/edk2-basetools >

The current plan is shortly after the stable tag is created, a series
of patches will come into edk2 that redirects the build system into the
new python module as well as adds additional documentation. You can see a
sample of this work here: https://github.com/matthewfcarlson/edk2 < https://github.com/matthewfcarlson/edk2
as this has a branch that has the work required to use the basetools pip
module. The patches won't delete the Basetools/Sources/Python folder but
will allow users to select between them. After a certain grace period, the
python folder will be deleted and the pip module will be the de facto way
of using basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should
the grace period be?
2. During the grace period, should basetools commits land in both
places or just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the
one in EDK2 or the pip module)? Currently the approach being considered is
a simple environmental variable? One of the key considerations is
transparency since it won't be apparent what is being used for a
particular build without some sort of mechanism to notify the developer.
With two seperate versions of Basetools, it becomes very easy for the
version of basetools you're using to not be the one you expect.
Matthew,

I’ll throw out some current developer centric ideas.

1) If you `source edksetup.sh` (edksetup.bat) you get the current
behavior, and you add an argument you get the pip flavor? So maybe
`edksetup.bat pip-basetools`?
2) We have similar issues to this with env variables and the build
command dumps them out when it runs. Can we use the current EDK_TOOL_PATH?
Or maybe add an extra print to show that the pip module is being used?

I've skimmed:

- the earlier discussion linked above (rooted at < https://edk2.groups.io/g/rfc/message/270
),
- the even earlier comments in the "Discussion: Basetools a separate repo"
thread on edk2-devel:

https://edk2.groups.io/g/devel/message/57482
http://mid.mail-archive.com/734D49CCEBEEF84792F5B80ED585239D5C5019CE@SHSMSX104.ccr.corp.intel.com


If I still understand / remember correctly, the way at least *I* would use
the new feature is the following:

- set up a new virtual python environment,

- either install the new pip module "permanently" in the virtual
environment, or else install it in "editable mode" from a git checkout
(but *still* in the virtual environment)

- build edk2 with the virtual environment active

That is, for me anyway, the key distinguishing factor would be that I'd be
in a python virtual environment where this particular python module
existed / were installed.

Does this answer question (3)? Because, in my case anyway, I wouldn't have
to be *notified* about using the separate basetools repo vs. using the one
resident in edk2 -- instead, I'd have to *activate* the separate basetools
repo myself, as first step. So if that activation brings some queriable
feature into the environment (sets a new environment variable or makes a
new python package appear in the environment), then I think it's good
enough -- the usual tools that I run then can query these artifacts.

In short (I guess): commands should use the in-tree variant, unless I
activate the virtual environment that has the basetools PIP module
installed.

I think it would be fine to require that, *if* someone intends to activate
such a python virtual environment, *then* they do so *before* running
"edksetup.sh". So "edksetup.sh" could check for the python env having the
external basetools repo / module active. Hopefully that would be "early
enough".


Regarding the grace period -- questions (1) and (2):

- The patches introducing the new feature to edk2 should be posted to the
list. These patches should also add a warning to "edksetup.sh" that urges
the developer to use the out-of-tree basetools repo / PIP module, in case
"edksetup.sh" determines the current choice is the in-tree variant (that
is, the virtual env is inactive, or does not contain the new PIP module)

- While the patches are pending approval, BaseTools development is put on
hold (no fixes, no features).

- For every package (subsystem) listed in Maintainers.txt, in *both* edk2
*and* edk2-platfomrs, at least one "M" person is required to report back
with a Tested-by, meaning that they built said package successfully with
the new PIP module.

- When this feedback is complete, the patch is merged, and the new PIP
module becomes the default build system (see the edksetup.sh warning
described above).

- Optimally, the above (= comprehensive testing feedback, and merging)
would occur *early* in the development cycle (just after the last stable
tag).

- Going forward, bug reports and feature requests are only addressed in
the new (out-of-tree) module. If someone reports that they have to switch
back, *temporarily*, for whatever reason, to the in-tree variant, and the
in-tree variant no longer builds edk2 for them, then such issues can be
resolved on a case-by-case basis, *after* the issue is reported. Point
being, we make the out-of-tree system the new default because of the
comprehensive and strict initial testing requirements (see above); after
which the old system is preserved for a while only as a fallback. If the
fallback proves lacking later on (but still during the grace period), then
the community works to resolve the issue in one of two ways: either help
the issue reporter eliminate their need to use the fallback in the first
place, or backport the subject bugfix/feature to the fallback.

- After the *next* stable release (which still contains both the fallback
and the support for the out-of-tree PIP module), the fallback is removed.

Ultimately this would make the grace period almost one full development
cycle, in which cycle the new system should be tested comprehensively, and
become the default, near the beginning of the period.

This is just my proposal. Some of the other stewards are temporarily away;
I'd suggest waiting for their feedback too.

To finish up, I would like to highlight something from the earlier RFC:

"""
Contribution/Dev Process:
Since this is a separate repo, it will follow a slightly different
contribution and code review process.
1.      Github PR process will be used for contributions and code review
feedback
a.      The yet to be released “Tianocore PR archiver” will be used to
send to a dedicated list for basetools patch review archive
2.      PRs will only be committed if they keep linear history (no merge
commits)
3.      The PR review must be approved by at least 2 members of the
basetools team (not including the author)
4.      The PR must pass all automated checks
a.      Formatting/style
b.      Unit tests
c.      Code coverage (can’t commit change that would decrease overall %)
d.      DCO enforcement - https://probot.github.io/apps/dco/
e.      See other python requirements from the Python coding standard
5.      Github Issues will be used for non-security sensitive
bugs/issues/feature requests
"""

Point (1a) is a pre-requisite for merging the edk2 patches!

We cannot make the new system the default unless its development process
is integrated with the github-to-email gateway (webhook).

Thanks!
Laszlo



Re: [edk2-devel] Basetools as a pip module

Andrew Fish <afish@...>
 

Matthew,

I did not meant to imply we should optimize for the current installed base, I just think it is useful to think about them. I think Lazlo is pointing out what is best for the project and that should have more weight than the installed base, but it is always good to think how things impact different groups.

In terms of the UI I was thinking of pointing to the pip install location vs. your git repo not so much a boolean. I actually don’t quite understand what EDK_TOOL_PATH actually means if BaseTools is a pip module. If that is an obsolete concept then we should remove it, and replace with some kind of statement that the pip installed BaseTools are being used. Another question if I run build from the command line how did my path get set? For example my user account has 5 different versions of edk2 in it how do I configure different versions of BaseTools? How do I pip multiple versions on to my system? I guess my repo could have a yml file that points to the version of the tools that I use, but that seems like a per user, not per repo config? It seems to me we could have developers that want to contribute to edk2 and work on their own code base and that could rehire two different Basetool versions installed on the system, and I think we need a story for that.

For the macOS Xcode compiler you can install as many versions as you want and there is a command line tool to let you set the current version of Xcode, and to show you the currently select versions. Basically the tools in the magic location in your path are just redirectors to the currently selected tools.

Thanks,

Andrew Fish

On Sep 2, 2020, at 12:06 PM, Matthew Carlson <matthewfcarlson@gmail.com> wrote:

Andrew:

I think leveraging the existing edksetup is a great idea. Using the existing EDK_TOOL_PATH variable could work but it seems clunky. Since the pip module wouldn't be a path, it seems strange to put a boolean value in a variable meant to hold a path. I definitely think that the scripts could print whether they're using the pip modules or the in-source tools. Since Lazlo suggested that pip will be the default, we could have the in-source modules notify of the fact that you're using the in-source modules. An additional feature for the pip module could be printing the version that they are (since you can use the pip infrastructure to query the version of a given module within a python script). Another option would be simply trying the pip module first and then falling back to the in-source module. There would be a slight speed penalty (likely around 10ms) but since this would only apply to trim and build, it should have relatively low impact.

Lazlo:
Thank you for the excellent summary of the different pieces of the discussion along with the links. To answer your first point, I think what a developer does with their pip module is largely up to them. They could do a virtual environment, they could just do what the requirements state, or pip install from a checked out basetools.I do think there are some variables that the virtual environment sets up that would be a good signal whether you're in a virtual environment or not. I agree with your approach of basetools development going into the out of edk2 repo and the importance of making sure package maintainers test and validate their areas with the new setup. I would personally try to get this early into the development cycle, (just after this next stable tag this week) to give the community and developers the most amount of time to get used to things. A trial period of one release makes sense.

I also agree that the gateway is important in maintaining cohesion between the new and the old. Hopefully that's nearing completion.

Hopefully other stewards will weigh in but otherwise we'll move ahead with a proposed solution in patches next week.

-Matthew Carlson


On Wed, Sep 2, 2020 at 1:49 AM Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>> wrote:
On 09/02/20 02:49, Andrew Fish via groups.io <http://groups.io/> wrote:


On Sep 1, 2020, at 4:35 PM, Matthew Carlson <matthewfcarlson@gmail.com <mailto:matthewfcarlson@gmail.com>> wrote:

Hello all,

A recent topic on the RFC mailing list went out and the work on moving Basetools/Sources/Python to a separate repo has started. See the RFC conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270

The repo in question is here: https://github.com/tianocore/edk2-basetools

The current plan is shortly after the stable tag is created, a series of patches will come into edk2 that redirects the build system into the new python module as well as adds additional documentation. You can see a sample of this work here: https://github.com/matthewfcarlson/edk2 as this has a branch that has the work required to use the basetools pip module. The patches won't delete the Basetools/Sources/Python folder but will allow users to select between them. After a certain grace period, the python folder will be deleted and the pip module will be the de facto way of using basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should the grace period be?
2. During the grace period, should basetools commits land in both places or just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the one in EDK2 or the pip module)? Currently the approach being considered is a simple environmental variable? One of the key considerations is transparency since it won't be apparent what is being used for a particular build without some sort of mechanism to notify the developer. With two seperate versions of Basetools, it becomes very easy for the version of basetools you're using to not be the one you expect.
Matthew,

I’ll throw out some current developer centric ideas.

1) If you `source edksetup.sh` (edksetup.bat) you get the current behavior, and you add an argument you get the pip flavor? So maybe `edksetup.bat pip-basetools`?
2) We have similar issues to this with env variables and the build command dumps them out when it runs. Can we use the current EDK_TOOL_PATH? Or maybe add an extra print to show that the pip module is being used?
I've skimmed:

- the earlier discussion linked above (rooted at <https://edk2.groups.io/g/rfc/message/270>),

- the even earlier comments in the "Discussion: Basetools a separate repo" thread on edk2-devel:

https://edk2.groups.io/g/devel/message/57482
http://mid.mail-archive.com/734D49CCEBEEF84792F5B80ED585239D5C5019CE@SHSMSX104.ccr.corp.intel.com

If I still understand / remember correctly, the way at least *I* would use the new feature is the following:

- set up a new virtual python environment,

- either install the new pip module "permanently" in the virtual environment, or else install it in "editable mode" from a git checkout (but *still* in the virtual environment)

- build edk2 with the virtual environment active

That is, for me anyway, the key distinguishing factor would be that I'd be in a python virtual environment where this particular python module existed / were installed.

Does this answer question (3)? Because, in my case anyway, I wouldn't have to be *notified* about using the separate basetools repo vs. using the one resident in edk2 -- instead, I'd have to *activate* the separate basetools repo myself, as first step. So if that activation brings some queriable feature into the environment (sets a new environment variable or makes a new python package appear in the environment), then I think it's good enough -- the usual tools that I run then can query these artifacts.

In short (I guess): commands should use the in-tree variant, unless I activate the virtual environment that has the basetools PIP module installed.

I think it would be fine to require that, *if* someone intends to activate such a python virtual environment, *then* they do so *before* running "edksetup.sh". So "edksetup.sh" could check for the python env having the external basetools repo / module active. Hopefully that would be "early enough".


Regarding the grace period -- questions (1) and (2):

- The patches introducing the new feature to edk2 should be posted to the list. These patches should also add a warning to "edksetup.sh" that urges the developer to use the out-of-tree basetools repo / PIP module, in case "edksetup.sh" determines the current choice is the in-tree variant (that is, the virtual env is inactive, or does not contain the new PIP module)

- While the patches are pending approval, BaseTools development is put on hold (no fixes, no features).

- For every package (subsystem) listed in Maintainers.txt, in *both* edk2 *and* edk2-platfomrs, at least one "M" person is required to report back with a Tested-by, meaning that they built said package successfully with the new PIP module.

- When this feedback is complete, the patch is merged, and the new PIP module becomes the default build system (see the edksetup.sh warning described above).

- Optimally, the above (= comprehensive testing feedback, and merging) would occur *early* in the development cycle (just after the last stable tag).

- Going forward, bug reports and feature requests are only addressed in the new (out-of-tree) module. If someone reports that they have to switch back, *temporarily*, for whatever reason, to the in-tree variant, and the in-tree variant no longer builds edk2 for them, then such issues can be resolved on a case-by-case basis, *after* the issue is reported. Point being, we make the out-of-tree system the new default because of the comprehensive and strict initial testing requirements (see above); after which the old system is preserved for a while only as a fallback. If the fallback proves lacking later on (but still during the grace period), then the community works to resolve the issue in one of two ways: either help the issue reporter eliminate their need to use the fallback in the first place, or backport the subject bugfix/feature to the fallback.

- After the *next* stable release (which still contains both the fallback and the support for the out-of-tree PIP module), the fallback is removed.

Ultimately this would make the grace period almost one full development cycle, in which cycle the new system should be tested comprehensively, and become the default, near the beginning of the period.

This is just my proposal. Some of the other stewards are temporarily away; I'd suggest waiting for their feedback too.

To finish up, I would like to highlight something from the earlier RFC:

"""
Contribution/Dev Process:
Since this is a separate repo, it will follow a slightly different contribution and code review process.
1. Github PR process will be used for contributions and code review feedback
a. The yet to be released “Tianocore PR archiver” will be used to send to a dedicated list for basetools patch review archive
2. PRs will only be committed if they keep linear history (no merge commits)
3. The PR review must be approved by at least 2 members of the basetools team (not including the author)
4. The PR must pass all automated checks
a. Formatting/style
b. Unit tests
c. Code coverage (can’t commit change that would decrease overall %)
d. DCO enforcement - https://probot.github.io/apps/dco/
e. See other python requirements from the Python coding standard
5. Github Issues will be used for non-security sensitive bugs/issues/feature requests
"""

Point (1a) is a pre-requisite for merging the edk2 patches!

We cannot make the new system the default unless its development process is integrated with the github-to-email gateway (webhook).

Thanks!
Laszlo


Re: [edk2-devel] Basetools as a pip module

Matthew Carlson <matthewfcarlson@...>
 

Andrew:

I think leveraging the existing edksetup is a great idea. Using the
existing EDK_TOOL_PATH variable could work but it seems clunky. Since the
pip module wouldn't be a path, it seems strange to put a boolean value in a
variable meant to hold a path. I definitely think that the scripts could
print whether they're using the pip modules or the in-source tools. Since
Lazlo suggested that pip will be the default, we could have the in-source
modules notify of the fact that you're using the in-source modules. An
additional feature for the pip module could be printing the version that
they are (since you can use the pip infrastructure to query the version of
a given module within a python script). Another option would be simply
trying the pip module first and then falling back to the in-source module.
There would be a slight speed penalty (likely around 10ms) but since this
would only apply to trim and build, it should have relatively low impact.

Lazlo:
Thank you for the excellent summary of the different pieces of the
discussion along with the links. To answer your first point, I think what a
developer does with their pip module is largely up to them. They could do a
virtual environment, they could just do what the requirements state, or pip
install from a checked out basetools.I do think there are some variables
that the virtual environment sets up that would be a good signal whether
you're in a virtual environment or not. I agree with your approach of
basetools development going into the out of edk2 repo and the importance of
making sure package maintainers test and validate their areas with the new
setup. I would personally try to get this early into the development cycle,
(just after this next stable tag this week) to give the community and
developers the most amount of time to get used to things. A trial period of
one release makes sense.

I also agree that the gateway is important in maintaining cohesion between
the new and the old. Hopefully that's nearing completion.

Hopefully other stewards will weigh in but otherwise we'll move ahead with
a proposed solution in patches next week.

-Matthew Carlson

On Wed, Sep 2, 2020 at 1:49 AM Laszlo Ersek <lersek@redhat.com> wrote:

On 09/02/20 02:49, Andrew Fish via groups.io wrote:


On Sep 1, 2020, at 4:35 PM, Matthew Carlson <matthewfcarlson@gmail.com>
wrote:

Hello all,

A recent topic on the RFC mailing list went out and the work on moving
Basetools/Sources/Python to a separate repo has started. See the RFC
conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270 <
https://edk2.groups.io/g/rfc/topic/74009714#270>

The repo in question is here:
https://github.com/tianocore/edk2-basetools <
https://github.com/tianocore/edk2-basetools>

The current plan is shortly after the stable tag is created, a series
of patches will come into edk2 that redirects the build system into the new
python module as well as adds additional documentation. You can see a
sample of this work here: https://github.com/matthewfcarlson/edk2 <
https://github.com/matthewfcarlson/edk2> as this has a branch that has
the work required to use the basetools pip module. The patches won't delete
the Basetools/Sources/Python folder but will allow users to select between
them. After a certain grace period, the python folder will be deleted and
the pip module will be the de facto way of using basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should
the grace period be?
2. During the grace period, should basetools commits land in both
places or just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the
one in EDK2 or the pip module)? Currently the approach being considered is
a simple environmental variable? One of the key considerations is
transparency since it won't be apparent what is being used for a particular
build without some sort of mechanism to notify the developer. With two
seperate versions of Basetools, it becomes very easy for the version of
basetools you're using to not be the one you expect.
Matthew,

I’ll throw out some current developer centric ideas.

1) If you `source edksetup.sh` (edksetup.bat) you get the current
behavior, and you add an argument you get the pip flavor? So maybe
`edksetup.bat pip-basetools`?
2) We have similar issues to this with env variables and the build
command dumps them out when it runs. Can we use the current EDK_TOOL_PATH?
Or maybe add an extra print to show that the pip module is being used?

I've skimmed:

- the earlier discussion linked above (rooted at <
https://edk2.groups.io/g/rfc/message/270>),

- the even earlier comments in the "Discussion: Basetools a separate repo"
thread on edk2-devel:

https://edk2.groups.io/g/devel/message/57482

http://mid.mail-archive.com/734D49CCEBEEF84792F5B80ED585239D5C5019CE@SHSMSX104.ccr.corp.intel.com

If I still understand / remember correctly, the way at least *I* would use
the new feature is the following:

- set up a new virtual python environment,

- either install the new pip module "permanently" in the virtual
environment, or else install it in "editable mode" from a git checkout (but
*still* in the virtual environment)

- build edk2 with the virtual environment active

That is, for me anyway, the key distinguishing factor would be that I'd be
in a python virtual environment where this particular python module existed
/ were installed.

Does this answer question (3)? Because, in my case anyway, I wouldn't have
to be *notified* about using the separate basetools repo vs. using the one
resident in edk2 -- instead, I'd have to *activate* the separate basetools
repo myself, as first step. So if that activation brings some queriable
feature into the environment (sets a new environment variable or makes a
new python package appear in the environment), then I think it's good
enough -- the usual tools that I run then can query these artifacts.

In short (I guess): commands should use the in-tree variant, unless I
activate the virtual environment that has the basetools PIP module
installed.

I think it would be fine to require that, *if* someone intends to activate
such a python virtual environment, *then* they do so *before* running
"edksetup.sh". So "edksetup.sh" could check for the python env having the
external basetools repo / module active. Hopefully that would be "early
enough".


Regarding the grace period -- questions (1) and (2):

- The patches introducing the new feature to edk2 should be posted to the
list. These patches should also add a warning to "edksetup.sh" that urges
the developer to use the out-of-tree basetools repo / PIP module, in case
"edksetup.sh" determines the current choice is the in-tree variant (that
is, the virtual env is inactive, or does not contain the new PIP module)

- While the patches are pending approval, BaseTools development is put on
hold (no fixes, no features).

- For every package (subsystem) listed in Maintainers.txt, in *both* edk2
*and* edk2-platfomrs, at least one "M" person is required to report back
with a Tested-by, meaning that they built said package successfully with
the new PIP module.

- When this feedback is complete, the patch is merged, and the new PIP
module becomes the default build system (see the edksetup.sh warning
described above).

- Optimally, the above (= comprehensive testing feedback, and merging)
would occur *early* in the development cycle (just after the last stable
tag).

- Going forward, bug reports and feature requests are only addressed in
the new (out-of-tree) module. If someone reports that they have to switch
back, *temporarily*, for whatever reason, to the in-tree variant, and the
in-tree variant no longer builds edk2 for them, then such issues can be
resolved on a case-by-case basis, *after* the issue is reported. Point
being, we make the out-of-tree system the new default because of the
comprehensive and strict initial testing requirements (see above); after
which the old system is preserved for a while only as a fallback. If the
fallback proves lacking later on (but still during the grace period), then
the community works to resolve the issue in one of two ways: either help
the issue reporter eliminate their need to use the fallback in the first
place, or backport the subject bugfix/feature to the fallback.

- After the *next* stable release (which still contains both the fallback
and the support for the out-of-tree PIP module), the fallback is removed.

Ultimately this would make the grace period almost one full development
cycle, in which cycle the new system should be tested comprehensively, and
become the default, near the beginning of the period.

This is just my proposal. Some of the other stewards are temporarily away;
I'd suggest waiting for their feedback too.

To finish up, I would like to highlight something from the earlier RFC:

"""
Contribution/Dev Process:
Since this is a separate repo, it will follow a slightly different
contribution and code review process.
1. Github PR process will be used for contributions and code review
feedback
a. The yet to be released “Tianocore PR archiver” will be used to
send to a dedicated list for basetools patch review archive
2. PRs will only be committed if they keep linear history (no merge
commits)
3. The PR review must be approved by at least 2 members of the
basetools team (not including the author)
4. The PR must pass all automated checks
a. Formatting/style
b. Unit tests
c. Code coverage (can’t commit change that would decrease overall %)
d. DCO enforcement - https://probot.github.io/apps/dco/
e. See other python requirements from the Python coding standard
5. Github Issues will be used for non-security sensitive
bugs/issues/feature requests
"""

Point (1a) is a pre-requisite for merging the edk2 patches!

We cannot make the new system the default unless its development process
is integrated with the github-to-email gateway (webhook).

Thanks!
Laszlo


Re: [edk2-devel] Basetools as a pip module

Laszlo Ersek
 

On 09/02/20 02:49, Andrew Fish via groups.io wrote:


On Sep 1, 2020, at 4:35 PM, Matthew Carlson <matthewfcarlson@gmail.com> wrote:

Hello all,

A recent topic on the RFC mailing list went out and the work on moving Basetools/Sources/Python to a separate repo has started. See the RFC conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270

The repo in question is here: https://github.com/tianocore/edk2-basetools

The current plan is shortly after the stable tag is created, a series of patches will come into edk2 that redirects the build system into the new python module as well as adds additional documentation. You can see a sample of this work here: https://github.com/matthewfcarlson/edk2 as this has a branch that has the work required to use the basetools pip module. The patches won't delete the Basetools/Sources/Python folder but will allow users to select between them. After a certain grace period, the python folder will be deleted and the pip module will be the de facto way of using basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should the grace period be?
2. During the grace period, should basetools commits land in both places or just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the one in EDK2 or the pip module)? Currently the approach being considered is a simple environmental variable? One of the key considerations is transparency since it won't be apparent what is being used for a particular build without some sort of mechanism to notify the developer. With two seperate versions of Basetools, it becomes very easy for the version of basetools you're using to not be the one you expect.
Matthew,

I’ll throw out some current developer centric ideas.

1) If you `source edksetup.sh` (edksetup.bat) you get the current behavior, and you add an argument you get the pip flavor? So maybe `edksetup.bat pip-basetools`?
2) We have similar issues to this with env variables and the build command dumps them out when it runs. Can we use the current EDK_TOOL_PATH? Or maybe add an extra print to show that the pip module is being used?
I've skimmed:

- the earlier discussion linked above (rooted at <https://edk2.groups.io/g/rfc/message/270>),

- the even earlier comments in the "Discussion: Basetools a separate repo" thread on edk2-devel:

https://edk2.groups.io/g/devel/message/57482
http://mid.mail-archive.com/734D49CCEBEEF84792F5B80ED585239D5C5019CE@SHSMSX104.ccr.corp.intel.com

If I still understand / remember correctly, the way at least *I* would use the new feature is the following:

- set up a new virtual python environment,

- either install the new pip module "permanently" in the virtual environment, or else install it in "editable mode" from a git checkout (but *still* in the virtual environment)

- build edk2 with the virtual environment active

That is, for me anyway, the key distinguishing factor would be that I'd be in a python virtual environment where this particular python module existed / were installed.

Does this answer question (3)? Because, in my case anyway, I wouldn't have to be *notified* about using the separate basetools repo vs. using the one resident in edk2 -- instead, I'd have to *activate* the separate basetools repo myself, as first step. So if that activation brings some queriable feature into the environment (sets a new environment variable or makes a new python package appear in the environment), then I think it's good enough -- the usual tools that I run then can query these artifacts.

In short (I guess): commands should use the in-tree variant, unless I activate the virtual environment that has the basetools PIP module installed.

I think it would be fine to require that, *if* someone intends to activate such a python virtual environment, *then* they do so *before* running "edksetup.sh". So "edksetup.sh" could check for the python env having the external basetools repo / module active. Hopefully that would be "early enough".


Regarding the grace period -- questions (1) and (2):

- The patches introducing the new feature to edk2 should be posted to the list. These patches should also add a warning to "edksetup.sh" that urges the developer to use the out-of-tree basetools repo / PIP module, in case "edksetup.sh" determines the current choice is the in-tree variant (that is, the virtual env is inactive, or does not contain the new PIP module)

- While the patches are pending approval, BaseTools development is put on hold (no fixes, no features).

- For every package (subsystem) listed in Maintainers.txt, in *both* edk2 *and* edk2-platfomrs, at least one "M" person is required to report back with a Tested-by, meaning that they built said package successfully with the new PIP module.

- When this feedback is complete, the patch is merged, and the new PIP module becomes the default build system (see the edksetup.sh warning described above).

- Optimally, the above (= comprehensive testing feedback, and merging) would occur *early* in the development cycle (just after the last stable tag).

- Going forward, bug reports and feature requests are only addressed in the new (out-of-tree) module. If someone reports that they have to switch back, *temporarily*, for whatever reason, to the in-tree variant, and the in-tree variant no longer builds edk2 for them, then such issues can be resolved on a case-by-case basis, *after* the issue is reported. Point being, we make the out-of-tree system the new default because of the comprehensive and strict initial testing requirements (see above); after which the old system is preserved for a while only as a fallback. If the fallback proves lacking later on (but still during the grace period), then the community works to resolve the issue in one of two ways: either help the issue reporter eliminate their need to use the fallback in the first place, or backport the subject bugfix/feature to the fallback.

- After the *next* stable release (which still contains both the fallback and the support for the out-of-tree PIP module), the fallback is removed.

Ultimately this would make the grace period almost one full development cycle, in which cycle the new system should be tested comprehensively, and become the default, near the beginning of the period.

This is just my proposal. Some of the other stewards are temporarily away; I'd suggest waiting for their feedback too.

To finish up, I would like to highlight something from the earlier RFC:

"""
Contribution/Dev Process:
Since this is a separate repo, it will follow a slightly different contribution and code review process.
1. Github PR process will be used for contributions and code review feedback
a. The yet to be released “Tianocore PR archiver” will be used to send to a dedicated list for basetools patch review archive
2. PRs will only be committed if they keep linear history (no merge commits)
3. The PR review must be approved by at least 2 members of the basetools team (not including the author)
4. The PR must pass all automated checks
a. Formatting/style
b. Unit tests
c. Code coverage (can’t commit change that would decrease overall %)
d. DCO enforcement - https://probot.github.io/apps/dco/
e. See other python requirements from the Python coding standard
5. Github Issues will be used for non-security sensitive bugs/issues/feature requests
"""

Point (1a) is a pre-requisite for merging the edk2 patches!

We cannot make the new system the default unless its development process is integrated with the github-to-email gateway (webhook).

Thanks!
Laszlo


Re: [edk2-devel] Basetools as a pip module

Andrew Fish <afish@...>
 

On Sep 1, 2020, at 4:35 PM, Matthew Carlson <matthewfcarlson@gmail.com> wrote:

Hello all,

A recent topic on the RFC mailing list went out and the work on moving Basetools/Sources/Python to a separate repo has started. See the RFC conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270

The repo in question is here: https://github.com/tianocore/edk2-basetools

The current plan is shortly after the stable tag is created, a series of patches will come into edk2 that redirects the build system into the new python module as well as adds additional documentation. You can see a sample of this work here: https://github.com/matthewfcarlson/edk2 as this has a branch that has the work required to use the basetools pip module. The patches won't delete the Basetools/Sources/Python folder but will allow users to select between them. After a certain grace period, the python folder will be deleted and the pip module will be the de facto way of using basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should the grace period be?
2. During the grace period, should basetools commits land in both places or just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the one in EDK2 or the pip module)? Currently the approach being considered is a simple environmental variable? One of the key considerations is transparency since it won't be apparent what is being used for a particular build without some sort of mechanism to notify the developer. With two seperate versions of Basetools, it becomes very easy for the version of basetools you're using to not be the one you expect.
Matthew,

I’ll throw out some current developer centric ideas.

1) If you `source edksetup.sh` (edksetup.bat) you get the current behavior, and you add an argument you get the pip flavor? So maybe `edksetup.bat pip-basetools`?
2) We have similar issues to this with env variables and the build command dumps them out when it runs. Can we use the current EDK_TOOL_PATH? Or maybe add an extra print to show that the pip module is being used?

Thanks,

Andrew Fish


Thank you.
-Matthew Carlson


Basetools as a pip module

Matthew Carlson <matthewfcarlson@...>
 

Hello all,

A recent topic on the RFC mailing list went out and the work on moving
Basetools/Sources/Python to a separate repo has started. See the RFC
conversation here: https://edk2.groups.io/g/rfc/topic/74009714#270

The repo in question is here: https://github.com/tianocore/edk2-basetools

The current plan is shortly after the stable tag is created, a series of
patches will come into edk2 that redirects the build system into the new
python module as well as adds additional documentation. You can see a
sample of this work here: https://github.com/matthewfcarlson/edk2 as this
has a branch that has the work required to use the basetools pip module.
The patches won't delete the Basetools/Sources/Python folder but will allow
users to select between them. After a certain grace period, the python
folder will be deleted and the pip module will be the de facto way of using
basetools.

Three questions need to be answered:

1. After the patches that enable the pip module land, how long should the
grace period be?
2. During the grace period, should basetools commits land in both places or
just in the edk2-basetools directory?
3. How should the user be able to select which basetools to use (the one in
EDK2 or the pip module)? Currently the approach being considered is a
simple environmental variable? One of the key considerations is
transparency since it won't be apparent what is being used for a particular
build without some sort of mechanism to notify the developer. With two
seperate versions of Basetools, it becomes very easy for the version of
basetools you're using to not be the one you expect.

Thank you.
-Matthew Carlson


Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Michael D Kinney
 

Hi Samer,

Comments included below.

Mike

-----Original Message-----
From: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com>
Sent: Monday, August 10, 2020 11:37 AM
To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; rfc@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Samer El-Haj-
Mahmoud <Samer.El-Haj-Mahmoud@arm.com>
Subject: RE: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Mike,

Looks good as a starting point!

Acked-by: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com>


I do have a few questions on this sentence: "Specification text changes are held within the affected source repository,
using the GitHub flavor of markdown, in a file (or split across several files) with .md suffix."

- For TianoCore, is the "affected source repository" this sentence is referring to edk2-staging, or edk2?
This will typically be in the edk2-staging repository. We have no plans to put these specification text changes
into the edk2 repo. The idea of additional repositories is for upstream/downstream dependencies or other components
that may be impacted by a proposed changes (i.e. OSes). We may need to determine if we want to archive ECRs after
the spec is published and the branch has been merged.


- If the proposed specification and associated code starts in a branch in edk2-staging respiratory, when does it get
accepted into edk2/edk2-platforms? Is it when the proposed specification change reaches a certain status (such as
"accepted by industry standard forum"), or when the formal specification (with that proposed change) is published by the
UEFI Forum ?
The starting approach will wait for the change to appear in a published specification. Platforms can choose to use
before that point by merging in changes from the edk2-staging branch with the BZXXXX prefixes.


- Any guidance on the specification text md file(s) names (and location) within the repository?
In the root directory of the branch. It would be good if the Readme.md in the root clearly identified it
it is an ECR branch with link to the ECR MD document.


- If the change includes some graphics, is there any guidance on inclusion of the graphics files in the repository?
GitHub Markdown has an easy syntax to include images. We can work on some small examples/templates in the
edk2-staging repo for single MD file, multiple MD files, and adding images. Images are typically put
into an Images directory below the MD file. We may want to consider recommending SVG so they render
well at all resolutions and are small text files instead of binary formats like PNG. ASCII art inline in
the MD file is also an option.


Thanks,
--Samer



-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
D Kinney via groups.io
Sent: Friday, August 7, 2020 9:07 PM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@intel.com>; rfc@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>;
Leif Lindholm <leif@nuviainc.com>
Subject: Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

A version of this Wiki page is also provided here for review:

https://github.com/mdkinney/edk2/wiki/EDK-II-Code-First-Process

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
Michael
D Kinney
Sent: Friday, August 7, 2020 6:05 PM
To: devel@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>;
Leif Lindholm <leif@nuviainc.com>
Subject: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

Based on the following RFC:

https://edk2.groups.io/g/rfc/message/258

Additional updates:
* Add examples of all specifications currently maintained by
the UEFI Forums.
* Added specification change template using a CC-BY-4.0 license.
* Add source code example for an enum value
* Minor grammar updates to change from an RFC proposal to an
active process.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
EDK-II-Code-First-Process.md | 182
+++++++++++++++++++++++++++++++++++
1 file changed, 182 insertions(+)
create mode 100644 EDK-II-Code-First-Process.md

diff --git a/EDK-II-Code-First-Process.md
b/EDK-II-Code-First-Process.md new file mode 100644 index
0000000..d5c938e
--- /dev/null
+++ b/EDK-II-Code-First-Process.md
@@ -0,0 +1,182 @@
+The EDK II Code First Process is a process by which new features can
+be added to UEFI Forum specifications after first having been
+designed and prototyped in the open.
+
+This process lets changes and the development of new features happen
+in the open, without violating the UEFI forum bylaws which prevent
+publication of code for in-draft features/changes.
+
+The process does not in fact change the UEFI bylaws - the change is
+that the development (of both specification and code) happens in the
+open. The resulting specification update is then submitted to the
+appropriate working group as an Engineering Change Request (ECR), and
+voted on. For the UEFI Forum, this is a change in workflow, not a change
in process.
+
+ECRs are tracked in a UEFI Forum Mantis instance, access restricted
+to UEFI Forum Members. TianoCore enables this new process by
+providing areas on [TianoCore
+Bugzilla](https://bugzilla.tianocore.org) to track both specification
+updates and reference implementations and new repositories under
[TianoCore GitHub](https://github.com/tianocore) dedicated to hold "code
first".
+
+# TianoCore Bugzilla
+
+[TianoCore Bugzilla](bugzilla.tianocore.org) has a product categories
+for
+ * ACPI Specification
+ * UEFI Shell Specification
+ * UEFI Platform Initialization Distribution Packaging Specification
+ * UEFI Platform Initialization Specification Specification
+ * UEFI Specification
+
+Each product category has separate components for
+ * Specification
+ * Reference implementation
+
+# TianoCore GitHub
+
+Reference implementations targeting the EDK II open source project
+are held in branches in the
+[edk2-staging](https://github.com/tianocore/edk2-staging)
+repository.
+
+Additional repositories for implementing reference features in
+additional open source projects can be added in the future, as required.
+
+Specification text changes are held within the affected source
+repository, using the GitHub flavor of markdown, in a file (or split
+across several files) with .md suffix. Multiple files are required
+if changes impact multiple specifications or if the specification is
+large and is easier to maintain if the changes are split across multiple
files.
+
+* NOTE: This one may break down where we have a specification change
+affecting
+ multiple specifications, but at that point we can track it with
+multiple
+ TianoCore Bugzilla entries.
+
+## Specification Text Template
+
+The following is a template of specification text changes using the
+GitHub flavor of markdown. The title and complete description of the
+specification changes must be provided in the specification text
+along with the name and version of the specification the change
+applies. The `Status` of the specification change always starts in
+the `Draft` state and is updated based on feedback from the industry
+standard forums. The contents of the specification text are required
+to use the [Creative Commons Attribution 4.0
+International](https://spdx.org/licenses/CC-BY-4.0.html)
+license using a `SPDX-License-Identifier` statement.
+
+```
+# Title: [Must be Filled In]
+
+# Status: [Status]
+
+[Status] must be one of the following:
+* Draft
+* Submitted to industry standard forum
+* Accepted by industry standard forum
+* Accepted by industry standard forum with modifications
+* Rejected by industry standard forum
+
+# Document: [Title and Version]
+
+Here are some examples of [Title and Version]:
+* UEFI Specification Version 2.8
+* ACPI Specification Version 6.3
+* UEFI Shell Specification Version 2.2
+* UEFI Platform Initialization Specification Version 1.7
+* UEFI Platform Initialization Distribution Packaging Specification
+Version 1.1
+
+# License
+
+SPDX-License-Identifier: CC-BY-4.0
+
+# Submitter: [TianoCore Community](https://www.tianocore.org)
+
+# Summary of the change
+
+Required Section
+
+# Benefits of the change
+
+Required Section
+
+# Impact of the change
+
+Required Section
+
+# Detailed description of the change [normative updates]
+
+Required Section
+
+# Special Instructions
+
+Optional Section
+```
+
+# Intended workflow
+
+The entity initiating a specification change enters a Bugzilla in the
+appropriate area of [TianoCore Bugzilla](bugzilla.tianocore.org).
+This entry contains the outline of the change, and the full initial draft
text is attached.
+
+If multiple specification updates are interdependent, especially if
+between different specifications, then multiple Bugzilla entries should be
created.
+These Bugzilla entries *must* be linked together with dependencies.
+
+After the Bugzillas have been created, new branches should be created
+in the relevant repositories for each Bugzilla. The branch names
+must use the following format where #### is the Bugzilla ID and
+<Brief Description> is an optional description of the change.
+
+ BZ####-<Brief Description>
+
+If multiple Bugzilla entries must coexist on a single branch, one of
+them is designated the _top-level_, with dependencies properly
+tracked. That Bugzilla is be the one naming the branch.
+
+# Source Code
+
+In order to ensure draft code does not accidentally leak into
+production use, and to signify when the changeover from draft to
+final happens, *all* new or modified[1] identifiers must be prefixed with
the relevant BZ#### identifiers.
+
+* [1] Modified in a non-backwards-compatible way. If, for example, a
statically
+ sized array is grown - this does not need to be prefixed. But a tag in a
+ comment would be *highly* recommended.
+
+## File names
+
+New public header files require the prefix (i.e.
`Bz1234MyNewProtocol.h`).
+Private header files do not need the prefix.
+
+## Contents
+
+The tagging must follow the coding style used by each affected code
base.
+Examples:
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `FunctionName` | `Bz1234FunctionName` | |
+| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
+
+For data structures or enums, any new or non-backwards-compatible
+structs or fields require a prefix. As above, growing an existing
+array in an existing struct requires no prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2]
|
+| `StructField` | `Bz1234StructField` | In existing struct[3] |
+| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2]
|
+| `EnumValue` | `BzEnumValue` | In existing enum[3] |
+
+* [2] If the struct or enum definition is separate from the typedef in the
public
+ header, the definition does not need the prefix.
+* [3] Individual fields in newly added struct or enum do not need prefix,
the
+ struct or enum already carried the prefix.
+
+Variable prefixes indicating global scope ('g' or 'm') go before the BZ
prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `gSomeGuid` | `gBz1234SomeGuid` | |
+
+Local identifiers, including module-global ones (m-prefixed) do not
+require a BZ prefix.
--
2.21.0.windows.1



IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are
not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use
it for any purpose, or store or copy the information in any medium. Thank you.


Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Andrew Fish <afish@...>
 

On Aug 10, 2020, at 11:36 AM, Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com> wrote:

Mike,

Looks good as a starting point!

Acked-by: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com <mailto:Samer.El-Haj-Mahmoud@arm.com>>


I do have a few questions on this sentence: "Specification text changes are held within the affected source repository, using the GitHub flavor of markdown, in a file (or split across several files) with .md suffix."

- For TianoCore, is the "affected source repository" this sentence is referring to edk2-staging, or edk2?

- If the proposed specification and associated code starts in a branch in edk2-staging respiratory, when does it get accepted into edk2/edk2-platforms? Is it when the proposed specification change reaches a certain status (such as "accepted by industry standard forum"), or when the formal specification (with that proposed change) is published by the UEFI Forum ?
Samer,

While in the “code 1st state” there are BZ# prefixes to types and globals. So when the code ends up in the specification the “code 1st” branch is going to need to get cleaned up for naming, and I assume we could remove the MD files before a patch is submitted to the edk2.

We made extra work for ourselves, but we wanted it to be clear what was work in progress.

Thanks,

Andrew Fish

- Any guidance on the specification text md file(s) names (and location) within the repository?

- If the change includes some graphics, is there any guidance on inclusion of the graphics files in the repository?

Thanks,
--Samer



-----Original Message-----
From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> On Behalf Of Michael
D Kinney via groups.io <http://groups.io/>
Sent: Friday, August 7, 2020 9:07 PM
To: devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Kinney, Michael D
<michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com>>; rfc@edk2.groups.io <mailto:rfc@edk2.groups.io>
Cc: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>>; Andrew Fish <afish@apple.com <mailto:afish@apple.com>>;
Leif Lindholm <leif@nuviainc.com <mailto:leif@nuviainc.com>>
Subject: Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

A version of this Wiki page is also provided here for review:

https://github.com/mdkinney/edk2/wiki/EDK-II-Code-First-Process

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
Michael
D Kinney
Sent: Friday, August 7, 2020 6:05 PM
To: devel@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>;
Leif Lindholm <leif@nuviainc.com>
Subject: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

Based on the following RFC:

https://edk2.groups.io/g/rfc/message/258

Additional updates:
* Add examples of all specifications currently maintained by
the UEFI Forums.
* Added specification change template using a CC-BY-4.0 license.
* Add source code example for an enum value
* Minor grammar updates to change from an RFC proposal to an
active process.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
EDK-II-Code-First-Process.md | 182
+++++++++++++++++++++++++++++++++++
1 file changed, 182 insertions(+)
create mode 100644 EDK-II-Code-First-Process.md

diff --git a/EDK-II-Code-First-Process.md
b/EDK-II-Code-First-Process.md new file mode 100644 index
0000000..d5c938e
--- /dev/null
+++ b/EDK-II-Code-First-Process.md
@@ -0,0 +1,182 @@
+The EDK II Code First Process is a process by which new features can
+be added to UEFI Forum specifications after first having been
+designed and prototyped in the open.
+
+This process lets changes and the development of new features happen
+in the open, without violating the UEFI forum bylaws which prevent
+publication of code for in-draft features/changes.
+
+The process does not in fact change the UEFI bylaws - the change is
+that the development (of both specification and code) happens in the
+open. The resulting specification update is then submitted to the
+appropriate working group as an Engineering Change Request (ECR), and
+voted on. For the UEFI Forum, this is a change in workflow, not a change
in process.
+
+ECRs are tracked in a UEFI Forum Mantis instance, access restricted
+to UEFI Forum Members. TianoCore enables this new process by
+providing areas on [TianoCore
+Bugzilla](https://bugzilla.tianocore.org) to track both specification
+updates and reference implementations and new repositories under
[TianoCore GitHub](https://github.com/tianocore) dedicated to hold "code
first".
+
+# TianoCore Bugzilla
+
+[TianoCore Bugzilla](bugzilla.tianocore.org) has a product categories
+for
+ * ACPI Specification
+ * UEFI Shell Specification
+ * UEFI Platform Initialization Distribution Packaging Specification
+ * UEFI Platform Initialization Specification Specification
+ * UEFI Specification
+
+Each product category has separate components for
+ * Specification
+ * Reference implementation
+
+# TianoCore GitHub
+
+Reference implementations targeting the EDK II open source project
+are held in branches in the
+[edk2-staging](https://github.com/tianocore/edk2-staging)
+repository.
+
+Additional repositories for implementing reference features in
+additional open source projects can be added in the future, as required.
+
+Specification text changes are held within the affected source
+repository, using the GitHub flavor of markdown, in a file (or split
+across several files) with .md suffix. Multiple files are required
+if changes impact multiple specifications or if the specification is
+large and is easier to maintain if the changes are split across multiple
files.
+
+* NOTE: This one may break down where we have a specification change
+affecting
+ multiple specifications, but at that point we can track it with
+multiple
+ TianoCore Bugzilla entries.
+
+## Specification Text Template
+
+The following is a template of specification text changes using the
+GitHub flavor of markdown. The title and complete description of the
+specification changes must be provided in the specification text
+along with the name and version of the specification the change
+applies. The `Status` of the specification change always starts in
+the `Draft` state and is updated based on feedback from the industry
+standard forums. The contents of the specification text are required
+to use the [Creative Commons Attribution 4.0
+International](https://spdx.org/licenses/CC-BY-4.0.html)
+license using a `SPDX-License-Identifier` statement.
+
+```
+# Title: [Must be Filled In]
+
+# Status: [Status]
+
+[Status] must be one of the following:
+* Draft
+* Submitted to industry standard forum
+* Accepted by industry standard forum
+* Accepted by industry standard forum with modifications
+* Rejected by industry standard forum
+
+# Document: [Title and Version]
+
+Here are some examples of [Title and Version]:
+* UEFI Specification Version 2.8
+* ACPI Specification Version 6.3
+* UEFI Shell Specification Version 2.2
+* UEFI Platform Initialization Specification Version 1.7
+* UEFI Platform Initialization Distribution Packaging Specification
+Version 1.1
+
+# License
+
+SPDX-License-Identifier: CC-BY-4.0
+
+# Submitter: [TianoCore Community](https://www.tianocore.org)
+
+# Summary of the change
+
+Required Section
+
+# Benefits of the change
+
+Required Section
+
+# Impact of the change
+
+Required Section
+
+# Detailed description of the change [normative updates]
+
+Required Section
+
+# Special Instructions
+
+Optional Section
+```
+
+# Intended workflow
+
+The entity initiating a specification change enters a Bugzilla in the
+appropriate area of [TianoCore Bugzilla](bugzilla.tianocore.org).
+This entry contains the outline of the change, and the full initial draft
text is attached.
+
+If multiple specification updates are interdependent, especially if
+between different specifications, then multiple Bugzilla entries should be
created.
+These Bugzilla entries *must* be linked together with dependencies.
+
+After the Bugzillas have been created, new branches should be created
+in the relevant repositories for each Bugzilla. The branch names
+must use the following format where #### is the Bugzilla ID and
+<Brief Description> is an optional description of the change.
+
+ BZ####-<Brief Description>
+
+If multiple Bugzilla entries must coexist on a single branch, one of
+them is designated the _top-level_, with dependencies properly
+tracked. That Bugzilla is be the one naming the branch.
+
+# Source Code
+
+In order to ensure draft code does not accidentally leak into
+production use, and to signify when the changeover from draft to
+final happens, *all* new or modified[1] identifiers must be prefixed with
the relevant BZ#### identifiers.
+
+* [1] Modified in a non-backwards-compatible way. If, for example, a
statically
+ sized array is grown - this does not need to be prefixed. But a tag in a
+ comment would be *highly* recommended.
+
+## File names
+
+New public header files require the prefix (i.e.
`Bz1234MyNewProtocol.h`).
+Private header files do not need the prefix.
+
+## Contents
+
+The tagging must follow the coding style used by each affected code
base.
+Examples:
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `FunctionName` | `Bz1234FunctionName` | |
+| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
+
+For data structures or enums, any new or non-backwards-compatible
+structs or fields require a prefix. As above, growing an existing
+array in an existing struct requires no prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2]
|
+| `StructField` | `Bz1234StructField` | In existing struct[3] |
+| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2]
|
+| `EnumValue` | `BzEnumValue` | In existing enum[3] |
+
+* [2] If the struct or enum definition is separate from the typedef in the
public
+ header, the definition does not need the prefix.
+* [3] Individual fields in newly added struct or enum do not need prefix,
the
+ struct or enum already carried the prefix.
+
+Variable prefixes indicating global scope ('g' or 'm') go before the BZ
prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `gSomeGuid` | `gBz1234SomeGuid` | |
+
+Local identifiers, including module-global ones (m-prefixed) do not
+require a BZ prefix.
--
2.21.0.windows.1



IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Samer El-Haj-Mahmoud
 

Mike,

Looks good as a starting point!

Acked-by: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com>


I do have a few questions on this sentence: "Specification text changes are held within the affected source repository, using the GitHub flavor of markdown, in a file (or split across several files) with .md suffix."

- For TianoCore, is the "affected source repository" this sentence is referring to edk2-staging, or edk2?

- If the proposed specification and associated code starts in a branch in edk2-staging respiratory, when does it get accepted into edk2/edk2-platforms? Is it when the proposed specification change reaches a certain status (such as "accepted by industry standard forum"), or when the formal specification (with that proposed change) is published by the UEFI Forum ?

- Any guidance on the specification text md file(s) names (and location) within the repository?

- If the change includes some graphics, is there any guidance on inclusion of the graphics files in the repository?

Thanks,
--Samer

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
D Kinney via groups.io
Sent: Friday, August 7, 2020 9:07 PM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@intel.com>; rfc@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>;
Leif Lindholm <leif@nuviainc.com>
Subject: Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

A version of this Wiki page is also provided here for review:

https://github.com/mdkinney/edk2/wiki/EDK-II-Code-First-Process

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
Michael
D Kinney
Sent: Friday, August 7, 2020 6:05 PM
To: devel@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>;
Leif Lindholm <leif@nuviainc.com>
Subject: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process
Wiki Page

Based on the following RFC:

https://edk2.groups.io/g/rfc/message/258

Additional updates:
* Add examples of all specifications currently maintained by
the UEFI Forums.
* Added specification change template using a CC-BY-4.0 license.
* Add source code example for an enum value
* Minor grammar updates to change from an RFC proposal to an
active process.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
EDK-II-Code-First-Process.md | 182
+++++++++++++++++++++++++++++++++++
1 file changed, 182 insertions(+)
create mode 100644 EDK-II-Code-First-Process.md

diff --git a/EDK-II-Code-First-Process.md
b/EDK-II-Code-First-Process.md new file mode 100644 index
0000000..d5c938e
--- /dev/null
+++ b/EDK-II-Code-First-Process.md
@@ -0,0 +1,182 @@
+The EDK II Code First Process is a process by which new features can
+be added to UEFI Forum specifications after first having been
+designed and prototyped in the open.
+
+This process lets changes and the development of new features happen
+in the open, without violating the UEFI forum bylaws which prevent
+publication of code for in-draft features/changes.
+
+The process does not in fact change the UEFI bylaws - the change is
+that the development (of both specification and code) happens in the
+open. The resulting specification update is then submitted to the
+appropriate working group as an Engineering Change Request (ECR), and
+voted on. For the UEFI Forum, this is a change in workflow, not a change
in process.
+
+ECRs are tracked in a UEFI Forum Mantis instance, access restricted
+to UEFI Forum Members. TianoCore enables this new process by
+providing areas on [TianoCore
+Bugzilla](https://bugzilla.tianocore.org) to track both specification
+updates and reference implementations and new repositories under
[TianoCore GitHub](https://github.com/tianocore) dedicated to hold "code
first".
+
+# TianoCore Bugzilla
+
+[TianoCore Bugzilla](bugzilla.tianocore.org) has a product categories
+for
+ * ACPI Specification
+ * UEFI Shell Specification
+ * UEFI Platform Initialization Distribution Packaging Specification
+ * UEFI Platform Initialization Specification Specification
+ * UEFI Specification
+
+Each product category has separate components for
+ * Specification
+ * Reference implementation
+
+# TianoCore GitHub
+
+Reference implementations targeting the EDK II open source project
+are held in branches in the
+[edk2-staging](https://github.com/tianocore/edk2-staging)
+repository.
+
+Additional repositories for implementing reference features in
+additional open source projects can be added in the future, as required.
+
+Specification text changes are held within the affected source
+repository, using the GitHub flavor of markdown, in a file (or split
+across several files) with .md suffix. Multiple files are required
+if changes impact multiple specifications or if the specification is
+large and is easier to maintain if the changes are split across multiple
files.
+
+* NOTE: This one may break down where we have a specification change
+affecting
+ multiple specifications, but at that point we can track it with
+multiple
+ TianoCore Bugzilla entries.
+
+## Specification Text Template
+
+The following is a template of specification text changes using the
+GitHub flavor of markdown. The title and complete description of the
+specification changes must be provided in the specification text
+along with the name and version of the specification the change
+applies. The `Status` of the specification change always starts in
+the `Draft` state and is updated based on feedback from the industry
+standard forums. The contents of the specification text are required
+to use the [Creative Commons Attribution 4.0
+International](https://spdx.org/licenses/CC-BY-4.0.html)
+license using a `SPDX-License-Identifier` statement.
+
+```
+# Title: [Must be Filled In]
+
+# Status: [Status]
+
+[Status] must be one of the following:
+* Draft
+* Submitted to industry standard forum
+* Accepted by industry standard forum
+* Accepted by industry standard forum with modifications
+* Rejected by industry standard forum
+
+# Document: [Title and Version]
+
+Here are some examples of [Title and Version]:
+* UEFI Specification Version 2.8
+* ACPI Specification Version 6.3
+* UEFI Shell Specification Version 2.2
+* UEFI Platform Initialization Specification Version 1.7
+* UEFI Platform Initialization Distribution Packaging Specification
+Version 1.1
+
+# License
+
+SPDX-License-Identifier: CC-BY-4.0
+
+# Submitter: [TianoCore Community](https://www.tianocore.org)
+
+# Summary of the change
+
+Required Section
+
+# Benefits of the change
+
+Required Section
+
+# Impact of the change
+
+Required Section
+
+# Detailed description of the change [normative updates]
+
+Required Section
+
+# Special Instructions
+
+Optional Section
+```
+
+# Intended workflow
+
+The entity initiating a specification change enters a Bugzilla in the
+appropriate area of [TianoCore Bugzilla](bugzilla.tianocore.org).
+This entry contains the outline of the change, and the full initial draft
text is attached.
+
+If multiple specification updates are interdependent, especially if
+between different specifications, then multiple Bugzilla entries should be
created.
+These Bugzilla entries *must* be linked together with dependencies.
+
+After the Bugzillas have been created, new branches should be created
+in the relevant repositories for each Bugzilla. The branch names
+must use the following format where #### is the Bugzilla ID and
+<Brief Description> is an optional description of the change.
+
+ BZ####-<Brief Description>
+
+If multiple Bugzilla entries must coexist on a single branch, one of
+them is designated the _top-level_, with dependencies properly
+tracked. That Bugzilla is be the one naming the branch.
+
+# Source Code
+
+In order to ensure draft code does not accidentally leak into
+production use, and to signify when the changeover from draft to
+final happens, *all* new or modified[1] identifiers must be prefixed with
the relevant BZ#### identifiers.
+
+* [1] Modified in a non-backwards-compatible way. If, for example, a
statically
+ sized array is grown - this does not need to be prefixed. But a tag in a
+ comment would be *highly* recommended.
+
+## File names
+
+New public header files require the prefix (i.e.
`Bz1234MyNewProtocol.h`).
+Private header files do not need the prefix.
+
+## Contents
+
+The tagging must follow the coding style used by each affected code
base.
+Examples:
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `FunctionName` | `Bz1234FunctionName` | |
+| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
+
+For data structures or enums, any new or non-backwards-compatible
+structs or fields require a prefix. As above, growing an existing
+array in an existing struct requires no prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2]
|
+| `StructField` | `Bz1234StructField` | In existing struct[3] |
+| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2]
|
+| `EnumValue` | `BzEnumValue` | In existing enum[3] |
+
+* [2] If the struct or enum definition is separate from the typedef in the
public
+ header, the definition does not need the prefix.
+* [3] Individual fields in newly added struct or enum do not need prefix,
the
+ struct or enum already carried the prefix.
+
+Variable prefixes indicating global scope ('g' or 'm') go before the BZ
prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `gSomeGuid` | `gBz1234SomeGuid` | |
+
+Local identifiers, including module-global ones (m-prefixed) do not
+require a BZ prefix.
--
2.21.0.windows.1



IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Michael D Kinney
 

A version of this Wiki page is also provided here for review:

https://github.com/mdkinney/edk2/wiki/EDK-II-Code-First-Process

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D Kinney
Sent: Friday, August 7, 2020 6:05 PM
To: devel@edk2.groups.io
Cc: Laszlo Ersek <lersek@redhat.com>; Andrew Fish <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
Subject: [edk2-devel] [Wiki][Patch V2] Add EDK II Code First Process Wiki Page

Based on the following RFC:

https://edk2.groups.io/g/rfc/message/258

Additional updates:
* Add examples of all specifications currently maintained by
the UEFI Forums.
* Added specification change template using a CC-BY-4.0 license.
* Add source code example for an enum value
* Minor grammar updates to change from an RFC proposal to an
active process.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
EDK-II-Code-First-Process.md | 182 +++++++++++++++++++++++++++++++++++
1 file changed, 182 insertions(+)
create mode 100644 EDK-II-Code-First-Process.md

diff --git a/EDK-II-Code-First-Process.md b/EDK-II-Code-First-Process.md
new file mode 100644
index 0000000..d5c938e
--- /dev/null
+++ b/EDK-II-Code-First-Process.md
@@ -0,0 +1,182 @@
+The EDK II Code First Process is a process by which new features can be added
+to UEFI Forum specifications after first having been designed and prototyped
+in the open.
+
+This process lets changes and the development of new features happen in the
+open, without violating the UEFI forum bylaws which prevent publication of
+code for in-draft features/changes.
+
+The process does not in fact change the UEFI bylaws - the change is that the
+development (of both specification and code) happens in the open. The resulting
+specification update is then submitted to the appropriate working group as an
+Engineering Change Request (ECR), and voted on. For the UEFI Forum, this is a
+change in workflow, not a change in process.
+
+ECRs are tracked in a UEFI Forum Mantis instance, access restricted to UEFI
+Forum Members. TianoCore enables this new process by providing areas on
+[TianoCore Bugzilla](https://bugzilla.tianocore.org) to track both specification
+updates and reference implementations and new repositories under
+[TianoCore GitHub](https://github.com/tianocore) dedicated to hold "code first".
+
+# TianoCore Bugzilla
+
+[TianoCore Bugzilla](bugzilla.tianocore.org) has a product categories for
+ * ACPI Specification
+ * UEFI Shell Specification
+ * UEFI Platform Initialization Distribution Packaging Specification
+ * UEFI Platform Initialization Specification Specification
+ * UEFI Specification
+
+Each product category has separate components for
+ * Specification
+ * Reference implementation
+
+# TianoCore GitHub
+
+Reference implementations targeting the EDK II open source project are held
+in branches in the [edk2-staging](https://github.com/tianocore/edk2-staging)
+repository.
+
+Additional repositories for implementing reference features in additional open
+source projects can be added in the future, as required.
+
+Specification text changes are held within the affected source repository,
+using the GitHub flavor of markdown, in a file (or split across several files)
+with .md suffix. Multiple files are required if changes impact multiple
+specifications or if the specification is large and is easier to maintain
+if the changes are split across multiple files.
+
+* NOTE: This one may break down where we have a specification change affecting
+ multiple specifications, but at that point we can track it with multiple
+ TianoCore Bugzilla entries.
+
+## Specification Text Template
+
+The following is a template of specification text changes using the GitHub
+flavor of markdown. The title and complete description of the specification
+changes must be provided in the specification text along with the name and
+version of the specification the change applies. The `Status` of the
+specification change always starts in the `Draft` state and is updated based
+on feedback from the industry standard forums. The contents of the specification
+text are required to use the
+[Creative Commons Attribution 4.0 International](https://spdx.org/licenses/CC-BY-4.0.html)
+license using a `SPDX-License-Identifier` statement.
+
+```
+# Title: [Must be Filled In]
+
+# Status: [Status]
+
+[Status] must be one of the following:
+* Draft
+* Submitted to industry standard forum
+* Accepted by industry standard forum
+* Accepted by industry standard forum with modifications
+* Rejected by industry standard forum
+
+# Document: [Title and Version]
+
+Here are some examples of [Title and Version]:
+* UEFI Specification Version 2.8
+* ACPI Specification Version 6.3
+* UEFI Shell Specification Version 2.2
+* UEFI Platform Initialization Specification Version 1.7
+* UEFI Platform Initialization Distribution Packaging Specification Version 1.1
+
+# License
+
+SPDX-License-Identifier: CC-BY-4.0
+
+# Submitter: [TianoCore Community](https://www.tianocore.org)
+
+# Summary of the change
+
+Required Section
+
+# Benefits of the change
+
+Required Section
+
+# Impact of the change
+
+Required Section
+
+# Detailed description of the change [normative updates]
+
+Required Section
+
+# Special Instructions
+
+Optional Section
+```
+
+# Intended workflow
+
+The entity initiating a specification change enters a Bugzilla in the appropriate
+area of [TianoCore Bugzilla](bugzilla.tianocore.org). This entry contains the
+outline of the change, and the full initial draft text is attached.
+
+If multiple specification updates are interdependent, especially if between
+different specifications, then multiple Bugzilla entries should be created.
+These Bugzilla entries *must* be linked together with dependencies.
+
+After the Bugzillas have been created, new branches should be created in the
+relevant repositories for each Bugzilla. The branch names must use the following
+format where #### is the Bugzilla ID and <Brief Description> is an optional
+description of the change.
+
+ BZ####-<Brief Description>
+
+If multiple Bugzilla entries must coexist on a single branch, one of them is
+designated the _top-level_, with dependencies properly tracked. That Bugzilla
+is be the one naming the branch.
+
+# Source Code
+
+In order to ensure draft code does not accidentally leak into production use,
+and to signify when the changeover from draft to final happens, *all* new or
+modified[1] identifiers must be prefixed with the relevant BZ#### identifiers.
+
+* [1] Modified in a non-backwards-compatible way. If, for example, a statically
+ sized array is grown - this does not need to be prefixed. But a tag in a
+ comment would be *highly* recommended.
+
+## File names
+
+New public header files require the prefix (i.e. `Bz1234MyNewProtocol.h`).
+Private header files do not need the prefix.
+
+## Contents
+
+The tagging must follow the coding style used by each affected code base.
+Examples:
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `FunctionName` | `Bz1234FunctionName` | |
+| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
+
+For data structures or enums, any new or non-backwards-compatible structs or
+fields require a prefix. As above, growing an existing array in an existing
+struct requires no prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2] |
+| `StructField` | `Bz1234StructField` | In existing struct[3] |
+| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2] |
+| `EnumValue` | `BzEnumValue` | In existing enum[3] |
+
+* [2] If the struct or enum definition is separate from the typedef in the public
+ header, the definition does not need the prefix.
+* [3] Individual fields in newly added struct or enum do not need prefix, the
+ struct or enum already carried the prefix.
+
+Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.
+
+| Released in spec | Draft version in tree | Comment |
+| --- | --- | --- |
+| `gSomeGuid` | `gBz1234SomeGuid` | |
+
+Local identifiers, including module-global ones (m-prefixed) do not require a
+BZ prefix.
--
2.21.0.windows.1



Re: Git commit message RFC

Laszlo Ersek
 

On 08/06/20 22:05, Artem Shchygel wrote:
Hello Mike,

Thanks for your feedback.

There are several levels of intent behind our proposal.
The one that you mentioned (search and identify commits by certain tag attribute) is important, but not the only one. Moreover one of your examples where you can have both "fix" and "feat" tag in the same commit is exactly the thing we're trying to avoid.
I find that regrettable, because in some (rare!) cases, a commit would
really deserve both tags.

And I also want to emphasize the notion that short tag in subject line must be mandatory.

To me personally adding short tag in the subject line coupled with removing "Package/Module" and CVE references from it serves the following purposes:
1. Mandatory tag forces committer to explicitly state what commit is about. This should prevent mixed commits (like bug fix and improvement together)
I agree this is a good general principle; it's always good to make a
patch author think explicitly about the nature of their patch, and to
make them capture the outcome of that thought process.

But this does not need to be encoded in the subject line, plus see my
point above about the (infrequent) case when a patch is both a fix and a
feature.

2. Removing "Package/Module" reference allows commits across packages boundaries where it makes sense. It also allows reviewer to see changes in their logical context instead of package-boundary context. Now I'm not advocating for huge commits, just commit partitioning better be on logical level than package boundary level.
We've managed very well to split up patches *both* at logical boundaries
*and* at package boundaries, at the same time. What you advocate for in
this point is 100% correct for a self-contained open source project, one
that explicitly disregards forks (such as the Linux kernel); I don't
think it's a good fit for edk2 though.

3. Removing CVE reference as well as "Package/Module" reference frees up the space in the subject line for more meaningful messages. Moreover your example shows why having CVE refs in subject line is bad - what if one commit fixes several of them.
This is a good point.

I'm in favor of including CVEs in subject lines (they are extremely
important, and also exceptional). But including multiple CVE numbers in
the subject is indeed a tough nut.

4. Placing short tag in the subject line allows for commits without long description, that can be created from the command line without the need of firing up the editor, as opposite to adding "Package/Module" ref and named tag inside the commit body. For mandatory things ergonomic matters
As a reviewer, I reject patches with empty or meaningless message
bodies, as a norm (for the packages that I co-maintain / co-review), and
as a community member, I protest against such patches for other packages
too, if I notice them on the list.

A commit message that consists of a subject line and a Signed-off-by,
plus (perhaps) some mechanic boilerplate included from a template, is
entirely useless. The message on a commit is nearly as important as the
code in the commit; sometimes *much more* important. It's entirely
normal to write a full page of rationale for a one-liner code change.

I've been on a crusade for years to train community members to write
more expressive commit messages. Your point#4 is a huge red flag to me,
personally.


All in all we understand that following our proposal to the letter will create disruptions for maintainers/reviewers workflow while simplify things for committers/users. But we still believe this is change for the better.
Unfortunately, in any open source project that professes "no patch
merged without review", the bottleneck is *always* on the
reviewer/maintainer side, and not on the contributor / consumer side.

Any change that makes things harder for reviewers / maintainers will
decrease code quality and long-term coherence (regressions!), and/or
worsen review feedback (such as response time), and/or erode commitment
to the above principle ("no patch merged without review").

Thanks
Laszlo


Re: Git commit message RFC

Laszlo Ersek
 

On 08/06/20 20:43, Kinney, Michael D wrote:

The main requirement is to make sure the Name: Value tag line for the nature of the
commit is unique enough so a grep regular expression will not falsely match a different
commit. I like the **<type name>** syntax to help with uniqueness. We can even
define a Name: Value tag syntax that supports a list of types if a single commit is
doing more than one type of change

TYPE: **fix** | **feat** | **imp** | **doc** | **style** | **chore**
I'd like to suggest "Type: " rather than "TYPE: ".

Furthermore, assuming a consistent use of the "Type: " name (at the
beginning of the line), do we really need the '**' value prefix/suffix?

On one hand, I find '**value**' a bit too "loud"; on the other, the
asterisk is a regexp special character, so it requires escaping with
backslash in git-log's "--grep" option argument, which makes it
uncomfortable. I think the lines:

Type: fix
Type: feature
Type: refactoring
Type: optimization
Type: documentation
Type: style
Type: chore

would read very nice.

(I used the "refactoring" and "optimization" values as two (more
specific) replacements for the "**imp**" value -- "improvement of the
existing code logic without changing functionality".

Another one I can think of is "cleanup", but I think it's included under
"refactoring".)

At this time,

$ git log --oneline --grep='^Type: ' master

returns zero hits, so I think the '^Type: ' prefix makes these tags
unique, without the "**" value prefix/suffix.

Also, I think we should not combine multiple values on a single "Type: "
line. In practice, we've applied this "single-value" rule to all other
"Name: "s. (You have showed that "--all-match" matches multiple lines in
a commit message.)

Finally, I'd like to mention that "Type: fix" and "Type: feature" are
not always mutually exclusive. Sometimes, there are issues that cannot
be surgically / incrementally fixed and that call for a whole-sale
replacement (rewrite) of a function. I wouldn't go as far as
recommending "Type: rewrite" for this, I'm just saying that in some
(rare!) cases, "Type: fix" and "Type: feature" may both show up in the
same commit message. Like, we have a long-standing design issue that
emerges over time, and it gets replaced with new (better designed)
functionality. That's both a fix and a feature.

Thanks!
Laszlo


Re: Git commit message RFC

Artem Shchygel
 

Hello Mike,

Thanks for your feedback.

There are several levels of intent behind our proposal. The one that you mentioned (search and identify commits by certain tag attribute) is important, but not the only one. Moreover one of your examples where you can have both "fix" and "feat" tag in the same commit is exactly the thing we're trying to avoid. And I also want to emphasize the notion that short tag in subject line must be mandatory.

To me personally adding short tag in the subject line coupled with removing "Package/Module" and CVE references from it serves the following purposes: 1. Mandatory tag forces committer to explicitly state what commit is about. This should prevent mixed commits (like bug fix and improvement together) 2. Removing "Package/Module" reference allows commits across packages boundaries where it makes sense. It also allows reviewer to see changes in their logical context instead of package-boundary context. Now I'm not advocating for huge commits, just commit partitioning better be on logical level than package boundary level. 3. Removing CVE reference as well as "Package/Module" reference frees up the space in the subject line for more meaningful messages. Moreover your example shows why having CVE refs in subject line is bad - what if one commit fixes several of them. 4. Placing short tag in the subject line allows for commits without long description, that can be created from the command line without the need of firing up the editor, as opposite to adding "Package/Module" ref and named tag inside the commit body. For mandatory things ergonomic matters

All in all we understand that following our proposal to the letter will create disruptions for maintainers/reviewers workflow while simplify things for committers/users. But we still believe this is change for the better.

Regards, Artem


Re: Git commit message RFC

Michael D Kinney
 

Hi Artem,

We discussed this topic briefly in the EDK II Community meeting this morning With Felix.

For the tags topic, I believe the feature being asked for here is a simple way to
identify and search for commits that have one or more of the tag attributes.

These tags do not have to be in the subject line to find them easily. Laszlo's suggestion
to use a Name: Value pair in the commit body can support this type of search using
standard git commands.

For example, the git log command supports the --grep flag to search for contents of the
commit messages that match a specific pattern. For example, if you want to find all the
commits that resolve a CVE issue from 2019, I can use the following command:

c:\work\GitHub\tianocore\edk2>git log --oneline --grep=CVE-2019
ffde22468e SecurityPkg/TcgPei: Use Migrated FV Info Hob for calculating hash (CVE-2019-11098)
d7c9de51d2 UefiCpuPkg/CpuMpPei: Enable paging and set NP flag to avoid TOCTOU (CVE-2019-11098)
012809cdca SecurityPkg/Tcg2Pei: Use Migrated FV Info Hob for calculating hash (CVE-2019-11098)
4b68cef04c MdeModulePkg/Core: Create Migrated FV Info Hob for calculating hash (CVE-2019-11098)
479613bd06 UefiCpuPkg/SecMigrationPei: Add initial PEIM (CVE-2019-11098)
60b12e69fb UefiCpuPkg/CpuMpPei: Add GDT migration support (CVE-2019-11098)
9bedaec05b MdeModulePkg/PeiCore: Enable T-RAM evacuation in PeiCore (CVE-2019-11098)
1facb8fdef MdeModulePkg: Add new PCD to control the evacuate temporary memory feature (CVE-2019-11098)
1d3215fd24 NetworkPkg/ArpDxe: Recycle invalid ARP packets (CVE-2019-14559)
c230c002ac SecurityPkg/DxeImageVerificationLib: change IsCertHashFoundInDatabase name (CVE-2019-14575)
b1c1147059 SecurityPkg/DxeImageVerificationLib: Differentiate error/search result (2) (CVE-2019-14575)
cb30c8f251 SecurityPkg/DxeImageVerificationLib: plug Data leak in IsForbiddenByDbx() (CVE-2019-14575)
5cd8be6079 SecurityPkg/DxeImageVerificationLib: tighten default result (CVE-2019-14575)
a83dbf008c SecurityPkg/DxeImageVerificationLib: Differentiate error/search result (1) (CVE-2019-14575)
adc6898366 SecurityPkg/DxeImageVerificationLib: refactor db/dbx fetching code (CVE-2019-14575)
929d1a24d1 SecurityPkg/DxeImageVerificationLib: avoid bypass in fetching dbx (CVE-2019-14575)
9e56970090 SecurityPkg/DxeImageVerificationLib: fix wrong fetch dbx in IsAllowedByDb (CVE-2019-14575)
c13742b180 SecurityPkg/DxeImageVerificationLib: reject CertStack.CertNumber==0 per DBX (CVE-2019-14575)
fbb9607223 SecurityPkg/DxeImageVerificationLib: Fix memory leaks (CVE-2019-14575)
578bcdc260 NetworkPkg/Ip4Dxe: Check the received package length (CVE-2019-14559).
e36d5ac7d1 MdeModulePkg/SdMmcPciHcDxe: Fix double PciIo Unmap in TRB creation (CVE-2019-14587)
f1d78c489a MdeModulePkg/DisplayEngine: Zero memory before free (CVE-2019-14558)
764e8ba138 MdeModulePkg/String.c: Zero memory before free (CVE-2019-14558)
c32be82e99 MdeModulePkg/HiiDB: Remove configuration table when it's freed (CVE-2019-14586)
322ac05f8b MdeModulePkg/PiDxeS3BootScriptLib: Fix potential numeric truncation (CVE-2019-14563)
e2fc508128 NetworkPkg/HttpDxe: Set the HostName for the verification (CVE-2019-14553)
703e7ab21f NetworkPkg/TlsDxe: Add the support of host validation to TlsDxe driver (CVE-2019-14553)
1e72b1fb2e CryptoPkg/TlsLib: TlsSetVerifyHost: parse IP address literals as such (CVE-2019-14553)
8d16ef8269 CryptoPkg/Crt: import "inet_pton.c" (CVE-2019-14553)
2ac41c12c0 CryptoPkg/Crt: satisfy "inet_pton.c" dependencies (CVE-2019-14553)
eb520d94db CryptoPkg/Crt: turn strchr() into a function (CVE-2019-14553)
2ca74e1a17 CryptoPkg/TlsLib: Add the new API "TlsSetVerifyHost" (CVE-2019-14553)
31efec8279 MdePkg/Include/Protocol/Tls.h: Add the data type of EfiTlsVerifyHost (CVE-2019-14553)
b26691c471 MdeModulePkg RegularExpressionDxe: Update Oniguruma from v6.9.0 to v6.9.3

Notice that the last one does not have CVE in the subject line. It is in the body and the
git log command found that one too:

c:\work\GitHub\tianocore\edk2>git show b26691c471
commit b26691c47188ce255b8a4d920bf07ddf1431e2cd
Author: Liming Gao <liming.gao@intel.com>
Date: Thu Aug 8 19:53:03 2019 +0800

MdeModulePkg RegularExpressionDxe: Update Oniguruma from v6.9.0 to v6.9.3

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2066
Update Oniguruma to the latest version v6.9.3.
Oniguruma https://github.com/kkos/oniguruma
This release is the security fix release. It includes the changes:
Fixed CVE-2019-13224
Fixed CVE-2019-13225
Fixed many problems (found by libfuzzer programs)

Verify VS2015, GCC5 build.
Verify RegularExpressionProtocol GetInfo() and Match() function.

Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Cinnamon Shia <cinnamon.shia@hpe.com>
Signed-off-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Hao A Wu <hao.a.wu@intel.com>

The main requirement is to make sure the Name: Value tag line for the nature of the
commit is unique enough so a grep regular expression will not falsely match a different
commit. I like the **<type name>** syntax to help with uniqueness. We can even
define a Name: Value tag syntax that supports a list of types if a single commit is
doing more than one type of change

TYPE: **fix** | **feat** | **imp** | **doc** | **style** | **chore**

To search for all commits that begin the line with TYPE: and have **fix** in the same
line, you can use the following command:

git log --oneline --extended-regexp --grep=^TYPE:.*\*\*fix\*\*

To search for all commits that begin the line with TYPE: and have either **fix** OR
**feat** in the same line, you can use the following command:

git log --oneline --extended-regexp --grep=^TYPE:.*\*\*fix\*\* --grep=^TYPE:.*\*\*feat\*\*

To search for all commits that begin the line with TYPE: and have **fix** AND
**feat** in the same line or across multiple TYPE: lines, you can use the following command:

git log --oneline --extended-regexp --all-match --grep=^TYPE:.*\*\*fix\*\* --grep=^TYPE:.*\*\*feat\*\*

As a double check to avoid false positives, I did a search of the entire git log
history for all commits that contain **<any word>**. It hit on 2 commits that
contained ***To*** and ***On***.

c:\work\GitHub\tianocore\edk2>git log --extended-regexp --grep=\*\*[a-zA-Z]+\*\*
commit 186bc15553cb1a2dcfcc2e60091b82d4870a657b
Author: qwang12 <qwang12@6f19259b-4bc3-4df7-8a09-765794883524>
Date: Wed Jan 21 09:48:54 2009 +0000

Rename module name from ***To*** to ***On***. AAAOnBBB means this module produce AAA Protocol/PPI based on BBB. This change improves the readability.

git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@7329 6f19259b-4bc3-4df7-8a09-765794883524

commit 4c9c0f719df00f211790eadf04d21a93a0cdf76c
Author: qwang12 <qwang12@6f19259b-4bc3-4df7-8a09-765794883524>
Date: Wed Jan 21 09:47:43 2009 +0000

Rename module name from ***To*** to ***On***. AAAOnBBB means this module produce AAA Protocol/PPI based on BBB. This change improves the readability.

git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@7328 6f19259b-4bc3-4df7-8a09-765794883524

Best regards,

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Artem Shchygel
Sent: Monday, August 3, 2020 1:24 PM
To: Laszlo Ersek <lersek@redhat.com>; rfc@edk2.groups.io
Subject: Re: [edk2-rfc] Git commit message RFC

On Sat, Aug 1, 2020 at 03:47 PM, Laszlo Ersek wrote:


On 08/01/20 00:19, Artem Shchygel wrote:
Hi, All

In this RFC we're proposing slight changes to Git commit message format to
make it more informative and more friendly for automation tools.

Here is the list of changes proposed

1. Start subject line with the tag that describes nature of the commit. List
of tags is as follows:
- **fix:** For commits that are fixing the bug
- **feat:** For commits that introduce a new feature
- **imp:** For commits that introduce improvement of the existing code
logic without changing functionality
- **doc:** For commits that don't change code logic but deal with
documentation (including adding comments, fix spelling, etc.)
- **style:** For commits that don't change code logic but deal with coding
style (indents, whitespaces, etc.)
- **chore:** For commits that don't affect output target (like changes to
CI scripts)
Having tagged commits will help closed source maintainers to decide which
commits to cherry-pick for projects that are in "maintenance mode" as opposite
to "active development mode"

Hints are very welcome, but not in subject lines. Please use

Name: value
Name: value # comment

style tags near the end of the commit messages instead.
Here are the reasons I believe subject line tags should be in subject line:
1. It is expected for those tags to be mandatory. This requirement will force committer to think one more time about what
commit changes are about. This may (hopefully) lead to more structured commits, where fixes, improvements, features and
style changes are not mixed together. But when you try to make something mandatory it's better to minimize the
friction/cognitive load for actually doing the thing. From this point of view short tag in subject line is easier to add
than separate line with "Name: value" in the commit body, which for trivial changes may even not be there.
2. I believe these tags, while certainly be machine-friendly still have some human-readable value. For example we can
easily say that commits marked as doc/style/chore have no regression impact even if the commit itself is pretty large. We
can also assume that "fix" commits will be smaller and easier to merge/backport than "feat" or "imp" commits.

Subject lines are primarily for human readers. And because edk2 has long
filenames, and we're supposed to include package names and component
names in subject lines, hardly any space is left for actual meaningful
subjects. CVE identifiers are the only exceptions (they *should* be
mentioned in subjects lines; and PatchCheck.py already permits more
characters in subjects that contain CVE identifiers).

2. Remove "Package/Module" reference from subject line. Since subject line
length is limited it's better to be allocated for commit description which is
more important

I'm opposed to this. "Package/Module" is the absolute key by which I
orient myself for determining impact / maintainership relevance /
regression risk. I regularly browse the git history for new commits, and
I entirely orient myself after the Package/Module prefixes.
As you've said above adding "Package/Module" description leave almost no space for meaningful subjects, practically
rendering them useless. Add to that the fact that this information is redundant in the sense, that it's already included
in changed files paths. Now I'm obviously don't know actual workflow maintainers use to perform their tasks, so it's hard
to come up with replacements. But filtering commits changing files in folders of the particular interest can be easily
done with git log command if you work with actual repo. If you look for particular folders in email inbox I think there is
the way to have email subject include "Package/Module" reference, while not having it in the commit subject line

3. Remove "CVE" number from subject line for the same reason. CVE number (if
present) should be placed on separate line after long description (see example
below)

I agree that mentioning CVE IDs in subject lines is an exceptions. I
could let go of that, if contributors were *very* disciplined about
stating CVE IDs properly in commit messages.
I think including CVE reference (which you probably can't easily type right off your head, but have to look it up) takes
the same amount of discipline regardless of whether it's included in the subject line or on the separate line in commit's
body
4. Add optional tag to the long description. List of tags is as follows:
- **[BREAKING CHANGE]:** For commits that break backward compatibility
- **[SECURITY FIX]:** For commits that fix known security vulnerability
Explicitly stating such traits is very welcome (even if not with this
specific format, perhaps) in commit message bodies.
The format of optional tag is flexible, any suggestions are welcome

5. Move bugzilla reference to separate line after long description and
(possible) CVE line (see example below). This move follows the logic of
evaluation of commit nature (read subject, if unclear read long description,
if still unclear see bugzilla)

Bugzilla numbers should already be stated as:

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

or

Link: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

or

Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

tags.
We're not proposing format changes for bugzilla references, just their placement in the commit body. Instead of being in
first lines we propose to move them to the last lines


An example would be:
```
imp: Short one line description of change

Several lines of
description for the
change.

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=1000

Signed-off-by: Contributor Name <contributor@email.server>
Reviewed-by: Reviewer Name <reviewer@reviewer-email.server>
```

A CVE example would be:
```
fix: Short one line description of change

[SECURITY FIX]: Several lines of
description for the
change.

CVE-2018-12180

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2000

Signed-off-by: Contributor Name <contributor@email.server>
Reviewed-by: Reviewer Name <reviewer@reviewer-email.server>
```
I entirely support including as much information as possible in commit
messages; if we can do that in machine readable format, that's even
better. But subject lines are not the place; the commit message bodies are.

I recommend the "Name: value" and "Name: value # comment" formats, but
don't insist on those.

What really matters to me is that we don't litter subject lines with
machine-readable artifacts.

Obviously: this is just my personal opinion.

Thanks
Laszlo
Thank you for taking your time to review our proposal. Your comments allowed me to express our intentions more clearly

Artem


Re: Git commit message RFC

Artem Shchygel
 

On Sat, Aug 1, 2020 at 03:47 PM, Laszlo Ersek wrote:

On 08/01/20 00:19, Artem Shchygel wrote: > Hi, All > > In this RFC we're proposing slight changes to Git commit message format to make it more informative and more friendly for automation tools. > > Here is the list of changes proposed > > 1. Start subject line with the tag that describes nature of the commit. List of tags is as follows: > - fix: For commits that are fixing the bug > - feat: For commits that introduce a new feature > - imp: For commits that introduce improvement of the existing code logic without changing functionality > - doc: For commits that don't change code logic but deal with documentation (including adding comments, fix spelling, etc.) > - style: For commits that don't change code logic but deal with coding style (indents, whitespaces, etc.) > - chore: For commits that don't affect output target (like changes to CI scripts) > Having tagged commits will help closed source maintainers to decide which commits to cherry-pick for projects that are in "maintenance mode" as opposite to "active development mode"

Hints are very welcome, but not in subject lines. Please use

Name: value Name: value # comment

style tags near the end of the commit messages instead.

Here are the reasons I believe subject line tags should be in subject line: 1. It is expected for those tags to be mandatory. This requirement will force committer to think one more time about what commit changes are about. This may (hopefully) lead to more structured commits, where fixes, improvements, features and style changes are not mixed together. But when you try to make something mandatory it's better to minimize the friction/cognitive load for actually doing the thing. From this point of view short tag in subject line is easier to add than separate line with "Name: value" in the commit body, which for trivial changes may even not be there. 2. I believe these tags, while certainly be machine-friendly still have some human-readable value. For example we can easily say that commits marked as doc/style/chore have no regression impact even if the commit itself is pretty large. We can also assume that "fix" commits will be smaller and easier to merge/backport than "feat" or "imp" commits. > > Subject lines are primarily for human readers. And because edk2 has long > filenames, and we're supposed to include package names and component > names in subject lines, hardly any space is left for actual meaningful > subjects. CVE identifiers are the only exceptions (they should be > mentioned in subjects lines; and PatchCheck.py already permits more > characters in subjects that contain CVE identifiers). > > > 2. Remove "Package/Module" reference from subject line. Since subject line > length is limited it's better to be allocated for commit description which is > more important > > I'm opposed to this. "Package/Module" is the absolute key by which I > orient myself for determining impact / maintainership relevance / > regression risk. I regularly browse the git history for new commits, and > I entirely orient myself after the Package/Module prefixes.

As you've said above adding "Package/Module" description leave almost no space for meaningful subjects, practically rendering them useless. Add to that the fact that this information is redundant in the sense, that it's already included in changed files paths. Now I'm obviously don't know actual workflow maintainers use to perform their tasks, so it's hard to come up with replacements. But filtering commits changing files in folders of the particular interest can be easily done with git log command if you work with actual repo. If you look for particular folders in email inbox I think there is the way to have email subject include "Package/Module" reference, while not having it in the commit subject line > > > 3. Remove "CVE" number from subject line for the same reason. CVE number (if > present) should be placed on separate line after long description (see example > below) > > I agree that mentioning CVE IDs in subject lines is an exceptions. I > could let go of that, if contributors were very disciplined about > stating CVE IDs properly in commit messages. >

I think including CVE reference (which you probably can't easily type right off your head, but have to look it up) takes the same amount of discipline regardless of whether it's included in the subject line or on the separate line in commit's body > > 4. Add optional tag to the long description. List of tags is as follows: > > - [BREAKING CHANGE]: For commits that break backward compatibility > > - [SECURITY FIX]: For commits that fix known security vulnerability > > Explicitly stating such traits is very welcome (even if not with this > specific format, perhaps) in commit message bodies.

The format of optional tag is flexible, any suggestions are welcome > > > > 5. Move bugzilla reference to separate line after long description and > (possible) CVE line (see example below). This move follows the logic of > evaluation of commit nature (read subject, if unclear read long description, > if still unclear see bugzilla) > > Bugzilla numbers should already be stated as: > > Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2129 > > or > > Link: https://bugzilla.tianocore.org/show_bug.cgi?id=2129 > > or > > Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=2129 > > tags. >

We're not proposing format changes for bugzilla references, just their placement in the commit body. Instead of being in first lines we propose to move them to the last lines > > > > > An example would be: > > > > imp: Short one line description of change > > > > Several lines of > > description for the > > change. > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=1000 > > > > Signed-off-by: Contributor Name <contributor@...> > > Reviewed-by: Reviewer Name <reviewer@...> > > > > > > A CVE example would be: > > > > fix: Short one line description of change > > > > [SECURITY FIX]: Several lines of > > description for the > > change. > > > > CVE-2018-12180 > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2000 > > > > Signed-off-by: Contributor Name <contributor@...> > > Reviewed-by: Reviewer Name <reviewer@...> > > > > I entirely support including as much information as possible in commit > messages; if we can do that in machine readable format, that's even > better. But subject lines are not the place; the commit message bodies are. > > I recommend the "Name: value" and "Name: value # comment" formats, but > don't insist on those. > > What really matters to me is that we don't litter subject lines with > machine-readable artifacts. > > Obviously: this is just my personal opinion. > > Thanks > Laszlo

Thank you for taking your time to review our proposal. Your comments allowed me to express our intentions more clearly

Artem > >


Re: Git commit message RFC

Laszlo Ersek
 

On 08/01/20 00:19, Artem Shchygel wrote:
Hi, All

In this RFC we're proposing slight changes to Git commit message format to make it more informative and more friendly for automation tools.

Here is the list of changes proposed

1. Start subject line with the tag that describes nature of the commit. List of tags is as follows:
- **fix:** For commits that are fixing the bug
- **feat:** For commits that introduce a new feature
- **imp:** For commits that introduce improvement of the existing code logic without changing functionality
- **doc:** For commits that don't change code logic but deal with documentation (including adding comments, fix spelling, etc.)
- **style:** For commits that don't change code logic but deal with coding style (indents, whitespaces, etc.)
- **chore:** For commits that don't affect output target (like changes to CI scripts)
Having tagged commits will help closed source maintainers to decide which commits to cherry-pick for projects that are in "maintenance mode" as opposite to "active development mode"
Hints are very welcome, but not in subject lines. Please use

Name: value
Name: value # comment

style tags near the end of the commit messages instead.

Subject lines are primarily for human readers. And because edk2 has long
filenames, and we're supposed to include package names and component
names in subject lines, hardly any space is left for actual meaningful
subjects. CVE identifiers are the only exceptions (they *should* be
mentioned in subjects lines; and PatchCheck.py already permits more
characters in subjects that contain CVE identifiers).

2. Remove "Package/Module" reference from subject line. Since subject line length is limited it's better to be allocated for commit description which is more important
I'm opposed to this. "Package/Module" is the absolute key by which I
orient myself for determining impact / maintainership relevance /
regression risk. I regularly browse the git history for new commits, and
I entirely orient myself after the Package/Module prefixes.

3. Remove "CVE" number from subject line for the same reason. CVE number (if present) should be placed on separate line after long description (see example below)
I agree that mentioning CVE IDs in subject lines is an exceptions. I
could let go of that, if contributors were *very* disciplined about
stating CVE IDs properly in commit messages.

4. Add optional tag to the long description. List of tags is as follows:
- **[BREAKING CHANGE]:** For commits that break backward compatibility
- **[SECURITY FIX]:** For commits that fix known security vulnerability
Explicitly stating such traits is very welcome (even if not with this
specific format, perhaps) in commit message bodies.

5. Move bugzilla reference to separate line after long description and (possible) CVE line (see example below). This move follows the logic of evaluation of commit nature (read subject, if unclear read long description, if still unclear see bugzilla)
Bugzilla numbers should already be stated as:

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

or

Link: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

or

Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=2129

tags.



An example would be:
```
imp: Short one line description of change

Several lines of
description for the
change.

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=1000

Signed-off-by: Contributor Name <contributor@email.server>
Reviewed-by: Reviewer Name <reviewer@reviewer-email.server>
```

A CVE example would be:
```
fix: Short one line description of change

[SECURITY FIX]: Several lines of
description for the
change.

CVE-2018-12180

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2000

Signed-off-by: Contributor Name <contributor@email.server>
Reviewed-by: Reviewer Name <reviewer@reviewer-email.server>
```
I entirely support including as much information as possible in commit
messages; if we can do that in machine readable format, that's even
better. But subject lines are not the place; the commit message bodies are.

I recommend the "Name: value" and "Name: value # comment" formats, but
don't insist on those.

What really matters to me is that we don't litter subject lines with
machine-readable artifacts.

Obviously: this is just my personal opinion.

Thanks
Laszlo

361 - 380 of 753