Date   

Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

Agreed. Responsibility to verify the commit message when squashing is always something to be aware of.

With Github, the one who presses the “Close and Merge” (or whatever it’s called) button has the final say on the title and message. We can play with approaches to how the squash merge would work.

I would far prefer the approach of individual PRs for commits to allow for the squash flexibility (and is the strategy I think I would pursue with my PRs). For example, the VarPol PR would be broken up into 9 PRs for each final commit, and we can get them in one by one. Ideally, each one would be a small back and forth and then in. If it had been done that way to begin with, it would be over in a week and a half or so, rather than the multiple months that we’re now verging on.

- Bret

From: Michael D Kinney via groups.io<mailto:michael.d.kinney@...>
Sent: Thursday, May 14, 2020 6:19 PM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@...<mailto:lersek@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of
the EDK II commit message requirements. In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
Of Bret Barkelew via groups.io
Sent: Thursday, May 14, 2020 2:27 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; lersek@...;
rfc@edk2.groups.io
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

I feel like this process is a good compromise. It’s not
perfect (frankly, I’m a fan of enforced squash merges,
which can maintain bisectability if managed well), but
it allows for rapid iteration, ease of contribution,
and approaches the workflow that many who have never
used email to maintain a project would be familiar
with.

It’s code management for the Instagram generation, and
I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael
D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>;
devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
lersek@...<mailto:lersek@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch
series submitted as a pull request. Let the community
do a complete review of the content. The submitter can
add patches to the end of the pull request addressing
feedback and can even add patches that make changes to
previous patches until all feedback/conversations are
resolved. This keeps the conversations complete and
the conversations will also be archived to the email
archive.

At this point, the developer can reformulate the patch
series and do forced push of V2. Reviewers can review
the cleaned up patch series and repeat the process if
there is more feedback, or move to final approval.

By doing all the work on a single pull request, we
minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull
request for each new version of the series. This would
preserve the GitHub conversations for each version of
the pull request. All the earlier ones would be
closed/abandoned, and only the final one would be
closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney,
Michael D <michael.d.kinney@...>;
rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force
push on every tweak, we lose the “thread” of discussion
on what caused the change, what changed as a result,
and the easy hook for the original change requester to
reply directly to the change as is.

- Bret

From: Laszlo Ersek via
groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
Kinney, Michael D<mailto:michael.d.kinney@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlo%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611242055&amp;sdata=Gr4Mw1Yz36xH3riEM7yicL5DIKv1%2FVuXM%2FObxAIWbJo%3D&amp;reserved=0
ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
qQxtLpwH0%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Development-
Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
re.github.io%2Fwiki%2FEDK-II-Development-
Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
contributors-and-
maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outl%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2FfG2%2BrXhM02OXgg%2Fa1G8tBYx0mHIH8lTm%2FF7bKSh10M%3D&amp;reserved=0
ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
for-edk2-contributors-and-
maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
d=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Message-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Message-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Signature-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Signature-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

*
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
j%2FPCToc%3D&reserved=0>)
* [edk2-
platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
)
* [edk2-non-
osi](https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
ved=0<https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-
test](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
ed=0<https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-
libc](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-
staging](https://nam06.safelinks.protection.outlook.com/
/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
eserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.co%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=bR1nvNPVi%2BQ1NrpSJQlwdLTZB3SigLFQ3KV63kUTUzk%3D&amp;reserved=0
m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
started-with-github%2Ffork-a-
repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
ng-started-with-github%2Ffork-a-
repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Fcreating-a-pull-
request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
aborating-with-issues-and-pull-requests%2Fcreating-a-
pull-
request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki
article at some point.)


* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all

s/make add/add/

changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.
Do I understand correctly that this recommends the
contributor first
push incremental patches on top of the series, then do
a rebase
(squashing updates as necessary) and finally do a
force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to
locally comparing
the v(n) patch set to v(n+1) with git-range-diff,
and/or with some
personal scripts. I wouldn't encourage incremental
patches appended --
even temporarily -- to the branch, because (a) it's
extra review work
(it requires me to review something that has zero
chance to get into the
git history as-is), and (b) it superficially resembles
the
github.com-specific bad practice called "squash on
merge", and (c) it
runs the risk that the maintainer responsible for
ultimately merging the
series ends up actually merging the incremental (=
"fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set?
Labels?

Not sure about the best tooling. My recommendation
would be to require
reviewers to start providing their feedback within one
week.

One thing that I find important is that a maintainer
can signal "I got
your work in my queue, but I may need more time". And a
special case of
that are automated out-of-office responses. I think
they are very
helpful (when a contributor feels they are bottlenecked
on review), but
I'm not sure how one can configure that via github. I
certainly would
not share my out-of-office times with github. (I set
the start/end dates
in my email infrastructure, at the moment, but the out-
of-office
messages it sends do not contain the dates either, on
purpose.)


## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root

s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails

s/sends/send/

are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email

s/maintainer ship/maintainership/

notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.ou%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=rMEZi8aziILY8pK5e%2BvnY3TgsvV4EqzQ2qXf8QvaqIc%3D&amp;reserved=0
tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
hub%2Fmanaging-subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
0>

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>

Important: as the name says ("-poc"), this is a Proof
of Concept list,
for now. Once we're ready to switch over, I'll file an
internal ticket
at RH to either rename the list, or (which is probably
better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the
webhook development
/ bugfixing (if any) could perhaps occur in parallel to
the normal edk2
workflow.


* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Freviewing-changes-in-
pull-
requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
ollaborating-with-issues-and-pull-requests%2Freviewing-
changes-in-pull-
requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as
above)


* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney
<michael.d.kinney@...<mailto:michael.d.kinney@int
el.com>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
archive-
webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-
poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.o%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ybz14MP2cfsQUGM2JxpsZNq2q%2BZU0e835ZNpKpV9M9Y%3D&amp;reserved=0
utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
2Fprivate%2Ftianocore-code-review-
poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
served=0>

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ug4axzMxZLKuGz8Eps6xepyEjv1er8atkiOZDpFca2c%3D&amp;reserved=0
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
z14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
b9A%3D&reserved=0>

## Example patch series with < 10 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
A%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
pXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
igs%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
zAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
eserved=0>

## Example patch series with > 80 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=oLtR%2Br3OOn3hqOQ6glrp56aXidbNCpd%2F8w06mY8j7dA%3D&amp;reserved=0.
outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
erved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=m34ZxHM9zr3VFz25TIiYpmxnqR0hPT96UIJppWgU4os%3D&amp;reserved=0
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please
consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.

Hmmm, good idea in general, but there have been valid
exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the
webhook)!
Laszlo







Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of
the EDK II commit message requirements. In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
Of Bret Barkelew via groups.io
Sent: Thursday, May 14, 2020 2:27 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; lersek@...;
rfc@edk2.groups.io
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

I feel like this process is a good compromise. It’s not
perfect (frankly, I’m a fan of enforced squash merges,
which can maintain bisectability if managed well), but
it allows for rapid iteration, ease of contribution,
and approaches the workflow that many who have never
used email to maintain a project would be familiar
with.

It’s code management for the Instagram generation, and
I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael
D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>;
devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
lersek@...<mailto:lersek@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch
series submitted as a pull request. Let the community
do a complete review of the content. The submitter can
add patches to the end of the pull request addressing
feedback and can even add patches that make changes to
previous patches until all feedback/conversations are
resolved. This keeps the conversations complete and
the conversations will also be archived to the email
archive.

At this point, the developer can reformulate the patch
series and do forced push of V2. Reviewers can review
the cleaned up patch series and repeat the process if
there is more feedback, or move to final approval.

By doing all the work on a single pull request, we
minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull
request for each new version of the series. This would
preserve the GitHub conversations for each version of
the pull request. All the earlier ones would be
closed/abandoned, and only the final one would be
closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney,
Michael D <michael.d.kinney@...>;
rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force
push on every tweak, we lose the “thread” of discussion
on what caused the change, what changed as a result,
and the easy hook for the original change requester to
reply directly to the change as is.

- Bret

From: Laszlo Ersek via
groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
Kinney, Michael D<mailto:michael.d.kinney@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
amp;reserved=0<https://nam06.safelinks.protection.outlo
ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
qQxtLpwH0%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Development-
Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
re.github.io%2Fwiki%2FEDK-II-Development-
Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
contributors-and-
maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
&amp;reserved=0<https://nam06.safelinks.protection.outl
ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
for-edk2-contributors-and-
maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
d=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Message-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Message-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Signature-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Signature-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

*
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
j%2FPCToc%3D&reserved=0>)
* [edk2-
platforms](https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
)
* [edk2-non-
osi](https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
ved=0<https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-
test](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
ed=0<https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-
libc](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-
staging](https://nam06.safelinks.protection.outlook.com
/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
eserved=0<https://nam06.safelinks.protection.outlook.co
m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
started-with-github%2Ffork-a-
repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
ng-started-with-github%2Ffork-a-
repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Fcreating-a-pull-
request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
aborating-with-issues-and-pull-requests%2Fcreating-a-
pull-
request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki
article at some point.)


* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all

s/make add/add/

changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.
Do I understand correctly that this recommends the
contributor first
push incremental patches on top of the series, then do
a rebase
(squashing updates as necessary) and finally do a
force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to
locally comparing
the v(n) patch set to v(n+1) with git-range-diff,
and/or with some
personal scripts. I wouldn't encourage incremental
patches appended --
even temporarily -- to the branch, because (a) it's
extra review work
(it requires me to review something that has zero
chance to get into the
git history as-is), and (b) it superficially resembles
the
github.com-specific bad practice called "squash on
merge", and (c) it
runs the risk that the maintainer responsible for
ultimately merging the
series ends up actually merging the incremental (=
"fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set?
Labels?

Not sure about the best tooling. My recommendation
would be to require
reviewers to start providing their feedback within one
week.

One thing that I find important is that a maintainer
can signal "I got
your work in my queue, but I may need more time". And a
special case of
that are automated out-of-office responses. I think
they are very
helpful (when a contributor feels they are bottlenecked
on review), but
I'm not sure how one can configure that via github. I
certainly would
not share my out-of-office times with github. (I set
the start/end dates
in my email infrastructure, at the moment, but the out-
of-office
messages it sends do not contain the dates either, on
purpose.)


## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root

s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails

s/sends/send/

are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email

s/maintainer ship/maintainership/

notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
3D&amp;reserved=0<https://nam06.safelinks.protection.ou
tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
hub%2Fmanaging-subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
0>

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>

Important: as the name says ("-poc"), this is a Proof
of Concept list,
for now. Once we're ready to switch over, I'll file an
internal ticket
at RH to either rename the list, or (which is probably
better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the
webhook development
/ bugfixing (if any) could perhaps occur in parallel to
the normal edk2
workflow.


* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Freviewing-changes-in-
pull-
requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
ollaborating-with-issues-and-pull-requests%2Freviewing-
changes-in-pull-
requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as
above)


* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney
<michael.d.kinney@...<mailto:michael.d.kinney@int
el.com>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
archive-
webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-
poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
%3D&amp;reserved=0<https://nam06.safelinks.protection.o
utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
2Fprivate%2Ftianocore-code-review-
poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
served=0>

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.prote
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
z14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.p
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
b9A%3D&reserved=0>

## Example patch series with < 10 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.p
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
A%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
pXA%3D&amp;reserved=0<https://nam06.safelinks.protectio
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
igs%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protect
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
zAE%3D&amp;reserved=0<https://nam06.safelinks.protectio
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
eserved=0>

## Example patch series with > 80 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
4%3D&amp;reserved=0<https://nam06.safelinks.protection.
outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
erved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protect
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.prote
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please
consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.

Hmmm, good idea in general, but there have been valid
exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the
webhook)!
Laszlo







Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

I feel like this process is a good compromise. It’s not perfect (frankly, I’m a fan of enforced squash merges, which can maintain bisectability if managed well), but it allows for rapid iteration, ease of contribution, and approaches the workflow that many who have never used email to maintain a project would be familiar with.

It’s code management for the Instagram generation, and I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@...<mailto:lersek@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request. Let the community do a complete review of the content. The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved. This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2. Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series. This would preserve the GitHub conversations for each version of the pull request. All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney, Michael D <michael.d.kinney@...>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6qQxtLpwH0%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjBSMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJj%2FPCToc%3D&reserved=0>)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8NsNTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6MgfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjzDwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90ZNlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeRye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=FJwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=0>

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3KoiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...<mailto:michael.d.kinney@...>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcwfqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSfEobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&reserved=0>

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pwz14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUkb9A%3D&reserved=0>

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6hA%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiAigs%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&reserved=0>

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&sdata=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: [RFCv2] code-first process for UEFI-forum specifications

Samer El-Haj-Mahmoud
 

Leif, Ray,

I have not seen any discussion on this thread since March(!)...

Please see my comments below.

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Ni, Ray via
Groups.Io
Sent: Wednesday, March 25, 2020 1:15 AM
To: rfc@edk2.groups.io; leif@...; devel@edk2.groups.io
Cc: Felixp@...; Doran, Mark <mark.doran@...>; Andrew Fish
<afish@...>; Laszlo Ersek <lersek@...>; Kinney, Michael D
<michael.d.kinney@...>
Subject: Re: [edk2-rfc] [RFCv2] code-first process for UEFI-forum
specifications


## Github
New repositories will be added for holding the text changes and the source
code.

Specification text changes will be held within the affected source
repository, in the Github flavour of markdown, in a file (or split
across several files) with .md suffix.
What's the case when multiple .MD files are needed?

(This one may break down where we have a specification change
affecting multiple specifications, but at that point we can track it
with multiple BZ entries)


## Source code
In order to ensure draft code does not accidentally leak into
production use, and to signify when the changeover from draft to final
happens, *all* new or modified[1] identifiers need to be prefixed with the
relevant BZ####.

[1] Modified in a non-backwards-compatible way. If, for example, a
statically
sized array is grown - this does not need to be prefixed. But a tag in a
comment would be *highly* recommended.
If a protocol is enhanced to provide more interfaces with increased revision
number, would you like the protocol name to be prefixed with BZ####?
Or just the new interfaces added to the protocol are prefixed the BZ####?
I think just prefixing the new interfaces can meet the purpose.
I think pre-fixing the new interfaces is sufficient. Otherwise, you need to modify all code using the existing interfaces (for build verification)


But the protocol definition is changed, it also needs to be prefixed according
to this flow.
Can you clarify a bit more?
A changed protocol definition is not backwards compatible, and typically results in a new protocol GUID. In that case, it really becomes a new definition and need to be pre-fixed per this rule. Right?


### File names
New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
Private header files do not need the prefix.

### Contents

The tagging must follow the coding style used by each affected codebase.
Examples:

| Released in spec | Draft version in tree | Comment |
| --- | --- | --- |
| `FunctionName` | `Bz1234FunctionName` | |
| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
If FunctionName or HEADER_MACRO is defined in non-public header files, I
don't think they require the prefix. Do you agree?

For data structures or enums, any new or non-backwards-compatible
structs or fields require a prefix. As above, growing an existing
array in an existing struct requires no prefix.

| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only
[2] |
| `StructField` | `Bz1234StructField` | In existing struct[3] |
| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only
[2] |

[2] If the struct or enum definition is separate from the typedef in the
public
header, the definition does not need the prefix.
What does "separate" mean?
Does it mean "struct or enum in the public header BzXXX.h don't need the
prefix"?
If yes, then I think macros defined in BzXXX.h also don't need the prefix.

[3] Individual fields in newly added typedefd struct do not need prefix, the
struct already carried the prefix.

Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.

| `gSomeGuid` | `gBz1234SomeGuid` | |

Local identifiers, including module-global ones (m-prefixed) do not
require a BZ prefix.
I think only the names (struct type name, enum type name, interface name,
protocol/ppi name) defined in public header files need the BZ prefix when the
public header doesn't have prefix.
Right?
The way I read it, *all* new (and non-backward modified) identifiers (typedef struct, typedef enum, and new structfield in existing struct) need to be pre-fixed, regardless if the filename is prefixed or not.
Correct?


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.


Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Laszlo Ersek
 

On 04/29/20 02:33, Matthew Carlson via groups.io wrote:

Versioning and Dependencies:
To minimize the dependency challenges and "bisectability" I would suggest we leverage the versioning capabilities within pip and repo tagging. With versioning you have lots of options as you can lock to a specific version which requires an update each time or you can use some sort of floating version within the tuple of version (xx.yy.zz). These two tools can make this pretty flexible.

In a scenario of DEC or INF syntax change, the suggested workflow would be:
1. Create the issue for basetools
2. Update basetools python
3. Write the unit test that shows it works as expected
4. Check in and make a release
5. Update edk2 pip-requirements.txt to require at least this new version. This gives you the tracking necessary to align with the tools.
6. Use this new feature in the edk2 fw code.
Here's an example why the above procedure (i.e., strict & lock-step
versioning) is important:

https://bugzilla.tianocore.org/show_bug.cgi?id=2719

When looking at a particular commit in edk2, for example for backporting
purposes, or else when looking at the whole edk2 tree *at* a particular
commit, it must be clear to the reader what basetools *state* was able
to build that commit / that tree, at the time. Because then the reader
will also be able to either backport the necessary basetools patches
too, or else (perhaps more simply) upgrade their separate basetools
component to that particular version.

So, I agree with the suggested workflow, I just wanted to emphasize how
important it is.

Thanks
Laszlo


Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Laszlo Ersek
 

Hi Matthew,

On 05/13/20 00:40, Matthew Carlson via groups.io wrote:
Since we haven't had any feedback and the deadline is quickly approaching. We are going to move ahead by creating a new repo inside of TianoCore and creating patches post-stable tag and submit them to the mailing list as soon as the stable tag is made.

If any other comments of feedback, feel free to chime in. If anyone has any basetools python code changes, please coordinate with us as the patch will be removing Basetools inside of the EDK2 repo. We want to make sure no changes are lost or misplaced.
I didn't provide any feedback in this specific thread because I thought
our discussion earlier was sufficient feedback from me.

(just commenting on the particular "we haven't had any feedback" bit)

Thanks,
Laszlo


Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Matthew Carlson
 

Since we haven't had any feedback and the deadline is quickly approaching. We are going to move ahead by creating a new repo inside of TianoCore and creating patches post-stable tag and submit them to the mailing list as soon as the stable tag is made.

If any other comments of feedback, feel free to chime in. If anyone has any basetools python code changes, please coordinate with us as the patch will be removing Basetools inside of the EDK2 repo. We want to make sure no changes are lost or misplaced.


Re: [RFCv2] code-first process for UEFI-forum specifications

Leif Lindholm
 

On Wed, Mar 25, 2020 at 05:14:59 +0000, Ni, Ray wrote:

## Github
New repositories will be added for holding the text changes and the source code.

Specification text changes will be held within the affected source repository,
in the Github flavour of markdown, in a file (or split across several files)
with .md suffix.
What's the case when multiple .MD files are needed?
For example if a branch covers changes to multiple specifications, as
described elsewhere. Or if it simply makes sense due to content size.
It is possible, now we've migrated to .rst for edk2, that we should
change the format recommentded in this proposal too.

(This one may break down where we have a specification change affecting multiple
specifications, but at that point we can track it with multiple BZ entries)


## Source code
In order to ensure draft code does not accidentally leak into production use,
and to signify when the changeover from draft to final happens, *all* new or
modified[1] identifiers need to be prefixed with the relevant BZ####.

[1] Modified in a non-backwards-compatible way. If, for example, a statically
sized array is grown - this does not need to be prefixed. But a tag in a
comment would be *highly* recommended.
If a protocol is enhanced to provide more interfaces with increased revision number,
would you like the protocol name to be prefixed with BZ####?
Or just the new interfaces added to the protocol are prefixed the BZ####?
I think just prefixing the new interfaces can meet the purpose.
Adding new interfaces to a protocol does not affect its
backwards-compatibility, which was what I was trying to cover above.
If you can think of a better way of describing it. I am very open to
suggestions.

But the protocol definition is changed, it also needs to be prefixed according to this flow.
Can you clarify a bit more?
In that instance, only the new interfaces would need the prefix.


### File names
New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
Private header files do not need the prefix.

### Contents

The tagging must follow the coding style used by each affected codebase.
Examples:

| Released in spec | Draft version in tree | Comment |
| --- | --- | --- |
| `FunctionName` | `Bz1234FunctionName` | |
| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
If FunctionName or HEADER_MACRO is defined in non-public header files, I don't
think they require the prefix. Do you agree?
Only public interfaces need prefix. This also means that non-public
interfaces should be STATIC where possible.

For data structures or enums, any new or non-backwards-compatible structs or
fields require a prefix. As above, growing an existing array in an existing
struct requires no prefix.

| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2] |
| `StructField` | `Bz1234StructField` | In existing struct[3] |
| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2] |

[2] If the struct or enum definition is separate from the typedef in the public
header, the definition does not need the prefix.
What does "separate" mean?
Does it mean "struct or enum in the public header BzXXX.h don't need the prefix"?
If yes, then I think macros defined in BzXXX.h also don't need the prefix.
Struct or enum definitions in the public header BzXXX.h don't need the
prefix *when they have a public-facing typedef with the prefix*.
Everything new or not-backwards-compatible needs to be referred to via
the prefixed names in external modules.

I.e. we can have
struct _SomeNewThing {
};

typedef struct _SomeNewThing BzXXX_PROTOCOL;

(This is meant simply as shorthand, reducing the amount of changes
required for the published version.)

[3] Individual fields in newly added typedefd struct do not need prefix, the
struct already carried the prefix.

Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.

| `gSomeGuid` | `gBz1234SomeGuid` | |

Local identifiers, including module-global ones (m-prefixed) do not require a
BZ prefix.
I think only the names (struct type name, enum type name, interface name, protocol/ppi name)
defined in public header files need the BZ prefix when the public header doesn't have prefix.
Right?
That is one way we *could* do it. It is not one I am proposing.
My idea is that it should be very clear from looking at code whether
it includes non-ratified proposal code.

/
Leif


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/11/20 22:09, Bret Barkelew wrote:
As a counterpoint: if we force a new branch or force push on every tweak, we lose the “threadâ€&#65533; of discussion on what caused the change,
This is a github.com limitation.

And the email archive mitigates it.

In the current process, when I review v2 of a 10-part series, I have one
Thunderbird window open with the v1 thread, containing both the v1
patches and my (and others') review comments given for them.

(I open the new window by right-clicking the v1 blurb, and then
selecting "open message in new window". Then I navigate between the
messages of the v1 thread with the "f" and "b" hotkeys. The "scope" of
the new window is set to the v1 thread, recursively, when I open the new
window like that, and so "f" and "b" just do the right thing.)

In another window, to the right side, I run "git-range-diff", to
interdiff the v1 patches (patch by patch) with the v2 patches. (An
interdiff is a diff of diffs.) Importantly, the interdiff also
highlights commit message differences.

I verify that all the feedback comments from the v1 thread have been
addressed (per patch), and also that any otherwise "uncalled-for"
changes in v2 are in fact justified. (The contributor may have
justifiedly implemented further changes than what I requested under v1.)
This is also the reason why I meticulously number my feedback comments,
as I'm going to require a complete (one by one) coverage in the next
version of the patch set. (Except for those comments of course that the
contributor successfully refutes.)

When the v2 series has different structure from v1, then git-range-diff
is not as helpful -- in that case, I compare only a subset of the
patches like described above, and the entirely new patches in v2 I have
to review from zero.

The entire process depends on having unfettered access to comments given
for *any* earlier version of the patch set (it's not uncommon that I
refer back to v(n-3) or v(n-2) when reviewing v(n)), with those comments
being tightly bound (for display and for re-reading) to their subject
patches.

The github webui destroys (at least visually) the comments given before
a force-push. I can't fathom how incremental reviews can work on
github.com *at all*, in other projects. Hence my earlier suggestion to
use new pull requests rather than force-pushes.

But the mailing list archive generated by the webhook will solve this
completely -- I will use that list as a primary review support tool (for
v2, v3, ...), not only as an archive.


... After all, I guess I could reformulate like this: it's not my intent
to prevent people from pushing incremental fixups *temporarily*; I'm
only saying I will ignore those patches, and I will review only the next
full version of the branch.


My concern that does persist is this: "it runs the risk that the
maintainer responsible for ultimately merging the series ends up
actually merging the incremental (= "fixup") patches in isolation
(without squashing them)".

The git history should neither be littered with fixup patches, nor
contain huge squashes. The structure of a patch series is a first class
trait; it is an aspect to iterate upon, when a branch is being
contributed. The tooling should support that. (And the list traffic
generated by the webhook does.)

For instance, the last time I've given feedback regarding patch series
structure was just an hour ago, under the series "[PATCH V4 00/27]
Disabling safe string constraint assertions". I requested moving a hunk
from patch#1 to patch#26. Having the hunk in patch#1 does not break
bisection, and it's irrelevant for the end-state after the whole series
is applied (the end-state is the same). But the hunk still doesn't
*belong* in patch#1 -- wherever we add a new bit to a bitmask PCD
(patch#26), the UNI file (= documentation) udpate belongs in the exact
same patch.

what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.
No matter what I say about an incremental/fixup patch in isolation,
things can easily go wrong when the contributor squashes the fixup into
the more substantial patch that needs the fixup. Not to mention any
commit message updates on the more substantial patch, as necessitated by
the fixup. So I'll have to review the next full version of the topic
branch anyway, with git-range-diff, and compare the interdiff against my
earlier feedback.

Thanks!
Laszlo



- Bret

From: Laszlo Ersek via groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo





Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request. Let the community do a complete review of the content. The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved. This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2. Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series. This would preserve the GitHub conversations for each version of the pull request. All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney, Michael D <michael.d.kinney@...>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...<mailto:michael.d.kinney@...>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/11/20 03:37, Michael D Kinney wrote:

There was feedback from Laszlo related to rebase for
pull requests using the current CI process.
To clarify, I don't think we should allow any github-side automatism to
auto-rebase pull requests. I think such rebases need to occur on
personal developer machines, under human oversight, and then resubmitted
(likely: force-pushed). My request is that the build costs (time,
energy) associated with such force-pushes be reduced somehow.

For example, on a local machine, the following sequence:

$ git checkout master
$ git pull
$ git rebase -i master my_topic_branch
$ build ...

would trigger an incremental build. *.c files not touched by either
operation would not have to be re-built (assuming their dependencies
didn't change either, such as lib class headers, protocol headers, ...)

Thanks,
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/10/20 23:43, Rebecca Cran wrote:
Mike,

On 5/10/20 3:29 PM, Michael D Kinney wrote:

There is no difference between CI checks run during code review
and the final CI checks before merge.  I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.
I'd suggest following what other Github projects do, which I think is to
run the CI checks automatically on every change that's made in a pull
request - I don't know if it might also be necessary to run them during
the merge, if master has changed in the meantime. That gives the
_submitter_ feedback about any changes they need to make, instead of
having to wait until the maintainer tells them their change has broken
something: it speeds up the development process.
Build-testing at every stage through a patch series is important for
ensuring bisectability.

But there's a critical ingredient to that: based on the assumption that
our build system / build rules are good, the builds mentioned above
should be *incremental*.

That is, if we have a patch set with 10 patches, then then the first
patch in the series should trigger a complete build, and the 9 later
patches should trigger only incremental builds.

(During a bisection, the same commits wouldn't be visited in that same
order of course, but that's where the sanity of the build system / build
rules comes in! Basically, if your builds succeed with a linear
progression through the series, then the build system / build rules
ought to *guarantee* that the same "tree states" will build
incrementally just fine when visited in any particular order. "git
checkout" updates the relevant files, and the build system should be
able to derive the minimum set of necessary actions.

Anyway, digression ends.)

The incremental nature of builds is important for saving energy, and
also for saving developer time. The above 10-part example series should
not take 10 times as long to build as 10 independent patches, submitted
in isolation. Patches#2 through #10 should only rebuild a few modules
each (unless lib class headers, protocol headers and such are modified).



Mergify is more flexible.  We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing.  GitHub merge button by
default squashes all commits into a single commit.
Wouldn't disabling all but "Allow rebase merging" do the same thing
without the additional potential failure point? Though it sounds like
we've resolved the problems with Mergify, so it's not important.

https://help.github.com/en/github/administering-a-repository/configuring-commit-squashing-for-pull-requests
mergify has been pretty stable for me!

Thanks,
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/10/20 23:29, Michael D Kinney wrote:
Rebecca,

There is no difference between CI checks run during code review
and the final CI checks before merge. I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.

Mergify is more flexible. We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing. GitHub merge button by
default squashes all commits into a single commit.
(

Wow, "squash-on-merge" is even the *default* now? That's terrible.
Unfortunately, github.com sets a very bad example with this, which is
made worse by github's popularity.

How can we expect developers to think about bisectability and patch
series structuring as first class traits of their contributions if
github.com actively educates them to ignore those aspects? Shaking my head.

)

Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/09/20 06:22, Ni, Ray wrote:
Mike,
It's a huge improvement to me as an Outlook user if pull-request-based review is enabled!

Please help me to understand: The pull-request-based review has been enabled naturally when edk2
was migrated to Github. People don't use it because it's not accepted by community. Your process
tries to meet community's needs of achieving all review comments in mails so pull-request-based
review can be accepted by community. Right?

I just subscribed at https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc with
empty password.
I received the confirmation mail and clicked the link in the mail to confirm.
But I waited for ~15 minutes and didn't receive the additional mail containing the auto-generated password.
That's because the proof-of-concept list is currently subscriber-only,
and subscription requests have to be manually approved -- by Phil, or by
me. The PoC list contains a bunch of webhook test messages, and while
they are not secret, they are not useful to the grand public (and
arguably shouldn't be indexed by web search engines either).

Once we go live, the intent is that production list be publicly visible.
(Of course spam could become a problem; we'll see.)

Importantly, I totally don't "insist" that the email archive be hosted
on redhat.com (in fact it's extra moderation work for me, which I don't
necessarily welcome); I just offered because Red Hat associates can
request such public-facing mailing lists if they support relevant open
source development efforts.

The traffic should be federated to multiple lists, preferably, and the
redhat.com-hosted list need not be the primary archive address. Wherever
the primary list will live, we can subscribe the mail-archive.com daemon
to it, too.

I went to https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the mail achieve doesn't need password?
The password protection should remain in place for now, I think. I've
approved your subscription request; sorry about the delay. (I avoid
reading work email on the weekend.)

Thanks!
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://github.com/tianocore/edk2/blob/master/ReadMe.rst
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
* https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hi Ray,

Comments below.

Mike

-----Original Message-----
From: Ni, Ray <ray.ni@...>
Sent: Friday, May 8, 2020 9:23 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; devel@edk2.groups.io
Subject: RE: [edk2-rfc] GitHub Pull Request based Code
Review Process

Mike,
It's a huge improvement to me as an Outlook user if
pull-request-based review is enabled!

Please help me to understand: The pull-request-based
review has been enabled naturally when edk2
was migrated to Github. People don't use it because
it's not accepted by community. Your process
tries to meet community's needs of achieving all review
comments in mails so pull-request-based
review can be accepted by community. Right?
Yes.


I just subscribed at
https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc with
empty password.
I received the confirmation mail and clicked the link
in the mail to confirm.
But I waited for ~15 minutes and didn't receive the
additional mail containing the auto-generated password.
I went to
https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the
mail achieve doesn't need password?
Laszlo has graciously volunteered to help setup this email
subscription service to help evaluate the POC. I will let
him comment on the settings available. He should also be
able to approve your subscription so you can see the archive
and receive any emails generated by the webhook.


Please advise me what else I can try. I am happy to try
as many steps as I can before all the process are
enabled.

Thanks,
Ray


-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On
Behalf Of Michael D Kinney
Sent: Saturday, May 9, 2020 11:00 AM
To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@...>
Subject: [edk2-rfc] GitHub Pull Request based Code
Review Process

Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://github.com/tianocore/edk2/blob/master/ReadMe.rs
t
*
https://github.com/tianocore/tianocore.github.io/wiki/E
DK-II-Development-Process
*
https://github.com/tianocore/tianocore.github.io/wiki/L
aszlo's-unkempt-git-guide-for-edk2-contributors-and-
maintainers
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Message-Format
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Signature-Format

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-
platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-
non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-
staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-
started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all
changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.

**OPEN**: How should minimum review period be set?
Labels?

## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root
of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails
are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email
notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://help.github.com/en/github/managing-
subscriptions-and-notifications-on-github/configuring-
notifications

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc

* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/reviewing-changes-in-
pull-requests

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...>
[mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-
webhook

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-
code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/index.html

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike




Re: GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hello,

I have added the following repository to TianoCore to
support the evaluation of the GitHub pull request based
code review process and the email archive webbook. This
is a copy of tianocore/edk2 repo as of May 10, 2020.

https://github.com/tianocore/edk2-codereview

I have updated Maintainers.txt in this repo to add
GitHub IDs for the maintainers and reviewers. Please
review these updates to make sure they are correct.

https://github.com/tianocore/edk2-codereview/blob/master/Maintainers.txt

There are a few maintainers and reviewers that I need
GitHub IDs. Please send me your GitHub IDs and I will
complete the update of Maintainers.txt.

M: Chao Zhang <chao.b.zhang@...>
R: Julien Grall <julien@...>
R: Marc-André Lureau <marcandre.lureau@...>
R: Stefan Berger <stefanb@...>
R: Liran Alon <liran.alon@...>
R: Nikita Leshenko <nikita.leshchenko@...>

Thanks,

Mike

-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Friday, May 8, 2020 8:00 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@...>
Subject: [edk2-rfc] GitHub Pull Request based Code
Review Process

Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for all
repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or Readme.rst
at the root of
repositories along with a few Wiki pages:

*
https://github.com/tianocore/edk2/blob/master/ReadMe.rs
t
*
https://github.com/tianocore/tianocore.github.io/wiki/E
DK-II-Development-Process
*
https://github.com/tianocore/tianocore.github.io/wiki/L
aszlo's-unkempt-git-guide-for-edk2-contributors-and-
maintainers
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Message-Format
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Signature-Format

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements. It
does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only. You
will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the proposed
GitHub pull request
based code review process, details on the email archive
service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review and
retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today
to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks pass.
With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set of
CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like
to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-
platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-
osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-
staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-
with-github/fork-a-repo

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements. The
only change with this
RFC is that the Cc: lines to maintainers/reviewers
should **not** be added.
The Cc: lines are still supported, but they should
only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all
changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced push
to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.

**OPEN**: How should minimum review period be set?
Labels?

## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root
of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual commit
assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails
are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows all
review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is merged
or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch'
the repositories that
you have maintainer ship or review responsibilities
and that email
notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://help.github.com/en/github/managing-
subscriptions-and-notifications-on-github/configuring-
notifications

* Subscribe to the email archive associated with the
TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-
code-review-poc

* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/reviewing-changes-in-
pull-requests

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit message
tags. If there are not
issues, set the `push` label to run final set of CI
checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the
end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...>
[mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat
email subscription
service. Please subscribe to receive the emails and to
be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/index.html

The following sections show some example pull requests
and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview
repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch spans
more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Rebecca,

I agree that the first version should rerun CI checks
on every time commits are added to a PR or there is a
forced push to the PR.

Perhaps we should use Draft Pull Requests as a way
to indicate the content is not ready for code review
or CI checks yet.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests#draft-pull-requests

We also want emails added to the email archive when
the pull request is either abandoned or merged.
merify can add comments to a PR that are picked up
by the webhook.

I agree with reducing the number of services required.
There was feedback from Laszlo related to rebase for
pull requests using the current CI process. I will
do more investigations of GitHub features, webhook
features, and Mergify features to see if there is
simpler overall solution.

Mike

-----Original Message-----
From: Rebecca Cran <rebecca@...>
Sent: Sunday, May 10, 2020 2:44 PM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; rfc@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process

Mike,

On 5/10/20 3:29 PM, Michael D Kinney wrote:

There is no difference between CI checks run during
code review
and the final CI checks before merge. I think it is
an interesting
conversation to decide how many times those CI checks
should be
run and if they should run automatically on every
change during
review or on demand.
I'd suggest following what other Github projects do,
which I think is to
run the CI checks automatically on every change that's
made in a pull
request - I don't know if it might also be necessary to
run them during
the merge, if master has changed in the meantime. That
gives the
_submitter_ feedback about any changes they need to
make, instead of
having to wait until the maintainer tells them their
change has broken
something: it speeds up the development process.

Mergify is more flexible. We want to make sure the
git history
is linear with not git merges and supports both
single patches
and patch series without squashing. GitHub merge
button by
default squashes all commits into a single commit.
Wouldn't disabling all but "Allow rebase merging" do
the same thing
without the additional potential failure point? Though
it sounds like
we've resolved the problems with Mergify, so it's not
important.

https://help.github.com/en/github/administering-a-
repository/configuring-commit-squashing-for-pull-
requests


--
Rebecca Cran


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Rebecca,

There is no difference between CI checks run during code review
and the final CI checks before merge. I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.

Mergify is more flexible. We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing. GitHub merge button by
default squashes all commits into a single commit.

Thanks,

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On
Behalf Of Rebecca Cran
Sent: Saturday, May 9, 2020 11:25 AM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>; rfc@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process

On 5/8/20 8:59 PM, Michael D Kinney wrote:

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.
What's the difference between the CI that runs when a
user submits the
Pull Request, and the final CI checks that run before
the request is merged?

Also, I'm wondering why Mergify is being used instead
of the maintainer
hitting the "Merge Pull Request" button, or however
it's worded?


--
Rebecca Cran



481 - 500 of 772