Date   

Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

I’ll pour another cup of tea to that.

- Bret

________________________________
From: Desimone, Nathaniel L <nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 2:02:49 PM
To: rfc@edk2.groups.io <rfc@edk2.groups.io>; Bret Barkelew <Bret.Barkelew@...>; devel@edk2.groups.io <devel@edk2.groups.io>; spbrogan@... <spbrogan@...>; lersek@... <lersek@...>; Kinney, Michael D <michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

To be completely fair, I think we are splitting hairs on details here. I think both of us are in 90% agreement, and we are both passionate enough about our work to argue that last 10% to the grave.

I totally understand the desire for bisectability by the way. TianoCore is a huge codebase, the core modules have several extremely large functions, and very little in the way of explicit documentation. It has taken me years to learn how this beast works. I think it is possible to not squash every patch series and still maintain bisectability.

For example, your VariablePolicy patch series; we definitely want the patch that adds VariablePolicyLib to MdeModulePkg merged before the patch that adds it to OvmfPkg. But if the patch series is done carefully it can still be bisectable. In fact, bisectability will only be maintained iff we merge the entire series in the order that you/Michael sent it; if OvmfPkg gets merged first, then OvmfPkg will fail to build until the MdeModulePkg patch is merged. I don't think it would be the right thing to squash the OvmfPkg & MdeModulePkg patches together, as they really are distinct steps that you took on your journey towards making the VariablePolicy sausage.

Of course, there may be other patch series that would be logical to squash, especially if the author has not been careful to maintain bisectability. For example, I think of some patch series went a little overboard and could have been done in maybe 1-2 patches instead of 8-10. I would be happy to compromise with you and say that squashes can be done in circumstances where both the maintainer and the author agree to it.

Finally, I believe I can speak for everyone here that we all welcome your contributions. I think Mike and the rest of the community are trying to adjust the process to make contributing viable for a larger set of people. But at the same time, you must realize that TianoCore isn't just going to do everything exactly the same way that Microsoft does. You and Sean are expected to compromise with the rest of the community.

Thanks,
Nate

On 5/19/20, 1:11 PM, "rfc@edk2.groups.io on behalf of Bret Barkelew via groups.io" <rfc@edk2.groups.io on behalf of bret.barkelew@...> wrote:

I will honor Mike Kinney’s efforts with my vote of confidence.
I think we’re headed in the right direction, even with some of the things that I disagree with.

In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

- Bret

From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 12:59 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew@...>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@..." <bret.barkelew@...>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@...>, "spbrogan@..." <spbrogan@...>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@..." <lersek@...>, "Kinney, Michael D" <michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=xYwOvgWRR2emIUFhy3CG%2Frxs774JyHIhlA0%2BrzV8dlU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=xYwOvgWRR2emIUFhy3CG%2Frxs774JyHIhlA0%2BrzV8dlU%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829127641&amp;sdata=Fvqm3EWSpquv3QSxvh8uhAK1tSxlz%2Fwd7EeeyBSMQis%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=vi503StyynvzgVF1JO6HeL0enBF0gpne%2FmZFa5nyx9Q%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=YXgZgfwFGRiHzlN92j7jaf8hPaA58iD21A483yCesB8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=YXgZgfwFGRiHzlN92j7jaf8hPaA58iD21A483yCesB8%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=Cad7XxJOMK%2FEsczlQVu3DcITjVQbC1j797Q11DbAISU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=Cad7XxJOMK%2FEsczlQVu3DcITjVQbC1j797Q11DbAISU%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=3ToBlN2v4o6ip13o7isAMg29pMcmh9SBSvMIDojvl8o%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=3ToBlN2v4o6ip13o7isAMg29pMcmh9SBSvMIDojvl8o%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=jic1pWXehzaiYdq3ihpR7uXZ9R0T0XdsUsc%2FHeAgpUo%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829137632&amp;sdata=jic1pWXehzaiYdq3ihpR7uXZ9R0T0XdsUsc%2FHeAgpUo%3D&amp;reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829147624&amp;sdata=LtOhDuYjnpe1OspUg33%2BEhSxnd0fG9COnCJjSrXJM9E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C2edef9b2c759477da79708d7fc38039d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255189829147624&amp;sdata=LtOhDuYjnpe1OspUg33%2BEhSxnd0fG9COnCJjSrXJM9E%3D&amp;reserved=0>

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@...>; lersek@...;
> bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@...>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
>
> Nate/Laszlo,
>
> Regarding a squash merge workflow. I agree it can be abused and we all
> have seen terrible examples. But a patch series that contains 500+ file
> changes isn't really much better. Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
>
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes. It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression. Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
>
> So, the ask is that we don't exclude squash merge workflows. Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted. Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
>
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process. This workflow is also are much easier for
> casual or first time contributors.
>
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description. First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
>
> Thanks
> Sean
>
>
>
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@...]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@...;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> > Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> > Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> > designated MdeModulePkg review), on the day of posting;
> > no
> >
> > other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> > (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate DeSimone
 

Hi Bret,

To be completely fair, I think we are splitting hairs on details here. I think both of us are in 90% agreement, and we are both passionate enough about our work to argue that last 10% to the grave.

I totally understand the desire for bisectability by the way. TianoCore is a huge codebase, the core modules have several extremely large functions, and very little in the way of explicit documentation. It has taken me years to learn how this beast works. I think it is possible to not squash every patch series and still maintain bisectability.

For example, your VariablePolicy patch series; we definitely want the patch that adds VariablePolicyLib to MdeModulePkg merged before the patch that adds it to OvmfPkg. But if the patch series is done carefully it can still be bisectable. In fact, bisectability will only be maintained iff we merge the entire series in the order that you/Michael sent it; if OvmfPkg gets merged first, then OvmfPkg will fail to build until the MdeModulePkg patch is merged. I don't think it would be the right thing to squash the OvmfPkg & MdeModulePkg patches together, as they really are distinct steps that you took on your journey towards making the VariablePolicy sausage.

Of course, there may be other patch series that would be logical to squash, especially if the author has not been careful to maintain bisectability. For example, I think of some patch series went a little overboard and could have been done in maybe 1-2 patches instead of 8-10. I would be happy to compromise with you and say that squashes can be done in circumstances where both the maintainer and the author agree to it.

Finally, I believe I can speak for everyone here that we all welcome your contributions. I think Mike and the rest of the community are trying to adjust the process to make contributing viable for a larger set of people. But at the same time, you must realize that TianoCore isn't just going to do everything exactly the same way that Microsoft does. You and Sean are expected to compromise with the rest of the community.

Thanks,
Nate

On 5/19/20, 1:11 PM, "rfc@edk2.groups.io on behalf of Bret Barkelew via groups.io" <rfc@edk2.groups.io on behalf of bret.barkelew@...> wrote:

I will honor Mike Kinney’s efforts with my vote of confidence.
I think we’re headed in the right direction, even with some of the things that I disagree with.

In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

- Bret

From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 12:59 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew@...>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@..." <bret.barkelew@...>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@...>, "spbrogan@..." <spbrogan@...>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@..." <lersek@...>, "Kinney, Michael D" <michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662145479&sdata=m66IViN3G%2BbJpBwolRXf8d3BpWQeRXs495WYxnsD85M%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662155477&sdata=VuYPqwcmYTrbiYdo%2B8K5irX8k6rgMgEoC2fY8eAocWA%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=lMrhfWKBWKGkjNnXJJy7%2BisrugTi0J%2FUkmtnj7Vxb7Q%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=rltVYSQcSLG2sGP4d2awDIuWV11nYQcdxvqyPxDM4XE%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662175469&sdata=b0J%2BL4nddt3g%2FmidZO61tlkgVqrRsufOUHGjRqh1CJM%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=zhHUUmckRdY45oYIPodqP9r3Sh4Q4t%2FZYRLULOiAERs%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=MNUfWJhjb1UfPFciUJ2dcu9uWQwUNkS3PJO%2BeMkFVxA%3D&reserved=0>

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
> Sent: Tuesday, May 19, 2020 9:54 AM
> To: rfc@edk2.groups.io; Desimone, Nathaniel L
> <nathaniel.l.desimone@...>; lersek@...;
> bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@...>
> Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
> Process
>
> Nate/Laszlo,
>
> Regarding a squash merge workflow. I agree it can be abused and we all
> have seen terrible examples. But a patch series that contains 500+ file
> changes isn't really much better. Just because it is broken into multiple
> commits doesn't mean it is the right set of commits.
>
> Anyway a squash merge workflow works amazingly well with and is
> optimized for a web based review and PR processes. It allows a user to
> respond to changes, fix issues, learn thru the PR process, all while keeping
> complete track of the progression. Then once all "status"
> checks and reviews are complete, it is squashed into a neat commit for
> mainline, containing only the relevant data in the message.
>
> So, the ask is that we don't exclude squash merge workflows. Those
> reviewing the PR can decide what is appropriate for the PR content
> submitted. Just as you would request changes to the contents (or
> ordering) of a commit in a series, if the reviewers don't agree that the PR
> contents should be in a single commit then obviously it shouldn't be
> squashed to one.
>
> Contributions like spelling fixes, typos, minor bug fixes, documentation
> additions/fixes, etc all are great examples of PRs that can easily leverage
> squash merges and this workflow significantly lowers the burden of the
> contribution and review process. This workflow is also are much easier for
> casual or first time contributors.
>
> I don't exactly know how we would enable this but I assume we could
> leverage tags or make it clear in the PR description. First step is to get
> alignment that a squash merge workflow, while not appropriate for all
> contributions, is not something to be excluded.
>
> Thanks
> Sean
>
>
>
> On 5/19/2020 12:21 AM, Nate DeSimone wrote:
> > Hi All,
> >
> >
> >
> > I tend to agree with most of Laszlo's points. Specifically, that moving to pull
> requests will not fix the fact that maintainers are usually busy people and
> don't always give feedback in a punctual manner. Like Laszlo, I would also
> prefer that we do not squash patch series. My biggest reason for not
> squashing patch series is because when you put everything into a single
> commit, I have had to review commits with 500+ files changed. Opening git
> difftool on a commit like that is awful.
> >
> >
> >
> > However, I would like to register my general endorsement for pull requests
> or some other web based system of code review… and I don’t have an
> Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
> coreboot and other projects. But since we are using Github for hosting, pull
> requests are an easy switch and a logical choice. My main reason for being
> excited about pull requests mostly has to do with the amount of manual
> effort required to be a TianoCore maintainer right now. I have set up my
> email filter so that the mailing list is categorized like so:
> >
> >
> >
> > [cid:image001.png@...]
> >
> >
> >
> > Implementing the logic to parse the contents of emails to categorize them
> like this required me to define no less than 12 email filter rules in Microsoft
> Outlook, and I have to change my filtering logic every time I am
> added/removed from a Maintainers.txt file. I’m sure every other maintainer
> has spent a time separately implementing filtering logic like I have. This helps,
> but still for every thread, I have to go and check if one of the other
> maintainers has already reviewed/pushed that patch series yet, and if not
> review/push it. If I have ] feedback on a patch series, I have to categorize it
> as awaiting response from author and check up on it from time to time,
> sometimes I ping the author directly and remind them to send a new patch
> series. Implementing this state machine is a lot of manual work and it kind of
> feels like I’m a telephone operator in the 1950s. I greatly welcome
> automation here as I am sure it will increase the number of patch series I am
> able to review per hour.
> >
> >
> >
> > Thanks,
> >
> > Nate
> >
> >
> >
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
> > Ersek
> > Sent: Friday, May 15, 2020 2:08 AM
> > To: rfc@edk2.groups.io; bret.barkelew@...;
> > devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
> > Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
> > Request based Code Review Process
> >
> >
> >
> > On 05/15/20 06:49, Bret Barkelew via groups.io wrote:
> >
> >
> >
> >> I would far prefer the approach of individual PRs for commits to
> >> allow
> >
> >> for the squash flexibility (and is the strategy I think I would
> >> pursue
> >
> >> with my PRs). For example, the VarPol PR would be broken up into 9
> >> PRs
> >
> >> for each final commit, and we can get them in one by one.
> >
> >> Ideally, each one would be a small back and forth and then in. If it
> >
> >> had been done that way to begin with, it would be over in a week and
> >> a
> >
> >> half or so, rather than the multiple months that we’re now verging
> >
> >> on.
> >
> >
> >
> > This differs extremely from how we've been working on edk2-devel (or
> from how any git-based project works that I've ever been involved with).
> >
> > And I think the above workflow is out of scope, for migrating the edk2
> process to github.
> >
> >
> >
> > Again, the structuring of a patch series is a primary trait. Iterating only on
> individual patches does not allow for the reordering / restructuring of the
> patch series (dropping patches, reordering patches, inserting patches,
> moving hunks between patches).
> >
> >
> >
> > It's common that the necessity to revise an earlier patch emerges while
> reworking a later patch. For instance, the git-rebase(1) manual dedicates a
> separate section to "splitting commits".
> >
> >
> >
> > In the initial evaluation of "web forges", Phabricator was one of the
> "contestants". Phabricator didn't support the "patch series" concept at all, it
> only supported review requests for individual patches, and it supported
> setting up dependencies between them. So, for example, a 27-patch series
> would require 27 submissions and 26 dependencies.
> >
> >
> >
> > Lacking support for the patch series concept was an immediate deal
> breaker with Phabricator.
> >
> >
> >
> > The longest patch series I've ever submitted to edk2-devel had 58 patches.
> It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
> and the patch count varied significantly:
> >
> >
> >
> > v1: 58 patches (25 Jul 2015)
> >
> > v2: 41 patches ( 9 Oct 2015)
> >
> > v3: 52 patches (15 Oct 2015)
> >
> > v4: 41 patches ( 3 Nov 2015)
> >
> > v5: 33 patches (27 Nov 2015)
> >
> >
> >
> > (The significant drop in the patch count was due to Mike Kinney open
> > sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
> > huge work in its own right), allowing me to drop the Quark-originated
> > 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
> >
> >
> >
> > The contribution process should make difficult things possible, even if that
> complicates simple things somewhat. A process that makes simple things
> simple and difficult things impossible is useless. This is what the Instagram
> generation seems to be missing.
> >
> >
> >
> >
> >
> > I don't know why the VariablePolicy work took months. I can see the
> following threads on the list:
> >
> >
> >
> > * [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
> >
> > Fri, 10 Apr 2020 11:36:01 -0700
> >
> >
> >
> > * [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
> >
> > Mon, 11 May 2020 23:46:23 -0700
> >
> >
> >
> > I have two sets of comments:
> >
> >
> >
> > (1) It's difficult to tell in retrospect (because the series seem to have been
> posted with somewhat problematic threading), but the delay apparently
> came from multiple sources.
> >
> >
> >
> > (1a) Review was slow and spotty.
> >
> >
> >
> > The v1 blurb received some comments in the first week after it was posted.
> But the rest of the v1 series (the actual patches) received feedback like this:
> >
> >
> >
> > - v1 1/9: no feedback
> >
> > - v1 2/9: 12 days after posting
> >
> > - v1 3/9: 16 days after posting
> >
> > - v1 4/9: no feedback
> >
> > - v1 5/9: no feedback
> >
> > - v1 6/9: no feedback
> >
> > - v1 7/9: no feedback
> >
> > - v1 8/9: no feedback
> >
> > - v1 9/9: no feedback
> >
> >
> >
> > (1b) There was also quite some time between the last response in the v1
> thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
> 11th).
> >
> >
> >
> > (1c) The v2 blurb got almost immediate, and numerous feedback (on the
> day of posting, and the day after). Regarding the individual patches, they
> didn't fare too well:
> >
> >
> >
> > - v2 01/12: superficial comment on the day of posting from me (not a
> >
> > designated MdeModulePkg review), on the day of posting;
> > no
> >
> > other feedback thus far
> >
> > - v2 02/12: ditto
> >
> > - v2 03/12: no feedback
> >
> > - v2 04/12: superficial (coding style) comments on the day of posting
> >
> > - v2 05/12: no feedback
> >
> > - v2 06/12: no feedback
> >
> > - v2 07/12: no feedback
> >
> > - v2 08/12: no feedback
> >
> > - v2 09/12: no feedback
> >
> > - v2 10/12: no feedback
> >
> > - v2 11/12: reasonably in-depth review from responsible co-maintainer
> >
> > (yours truly), on the day of posting
> >
> > - v2 12/12: no feedback
> >
> >
> >
> > In total, I don't think the current process takes the blame for the delay. If
> reviewers don't care (or have no time) now, that problem will not change
> with the transition to github.com.
> >
> >
> >
> >
> >
> > (2) The VariablePolicy series is actually a good example that patch series
> restructuring is important.
> >
> >
> >
> > (2a) The patch count went from 9 (in v1) to 12 (in v2).
> >
> >
> >
> > (2b) And under v2, Liming still pointed out: "To keep each commit build
> pass, the patch set should first add new library instance, then add the library
> instance into each platform DSC, last update Variable driver to consume new
> library instance."
> >
> >
> >
> > Furthermore, I requested enabling the feature in ArmVirtPkg too, and
> maybe (based on owner feedback) UefiPayloadPkg.
> >
> >
> >
> > Thus, the v2->v3 update will most likely bring about both patch order
> changes, and an increased patch count.
> >
> >
> >
> > Thanks
> >
> > Laszlo
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

(+Leif, +Andrew)

Sean,

On 05/19/20 18:54, Sean Brogan wrote:
Nate/Laszlo,

Regarding a squash merge workflow.  I agree it can be abused and we all
have seen terrible examples.  But a patch series that contains 500+ file
changes isn't really much better.  Just because it is broken into
multiple commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is
optimized for a web based review and PR processes.  It allows a user to
respond to changes, fix issues, learn thru the PR process, all while
keeping complete track of the progression.  Then once all "status"
checks and reviews are complete, it is squashed into a neat commit for
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows.  Those
reviewing the PR can decide what is appropriate for the PR content
submitted.  Just as you would request changes to the contents (or
ordering) of a commit in a series, if the reviewers don't agree that the
PR contents should be in a single commit then obviously it shouldn't be
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation
additions/fixes, etc all are great examples of PRs that can easily
leverage squash merges and this workflow significantly lowers the burden
of the contribution and review process.  This workflow is also are much
easier for casual or first time contributors.

I don't exactly know how we would enable this but I assume we could
leverage tags or make it clear in the PR description.  First step is to
get alignment that a squash merge workflow, while not appropriate for
all contributions, is not something to be excluded.
the scope for migrating the contribution & review workflows off the
mailing list and to github.com was set many months ago. That scope does
not include institutionalized changes to patch set structuring criteria.
The "git forge" evaluations that we had spent weeks/months on also
focused on how candidate systems would honor a patch series' structure;
i.e., how faithful the system would remain to the contributors' and
reviewers' shared intent, with a specific patch set.

Your proposal to "don't exclude squash merge workflows" is a trap. If we
tolerate that option -- which is obviously the sloppy, and hence more
convenient, option for some maintainers and some contributors, to the
detriment of the git history --, then almost every core maintainer will
use it as frequently as they can. In the long term, that will hurt many
consumers of the core code. It will limit the ability of people not
regularly dealing with a particular core module to file a fine-grained
bug report for that module, maybe even propose a fix. From the
regression analyst's side, if the bug report starts with "I have a
bisection log", that's already a good day. And your proposal would
destroy that option, because maintainers and people in general are
irrepairably lazy and undisciplined. We cannot post a community member
shoulder-by-shoulder with every core package reviewer/maintainer to
prevent the latter from approving a squash-on-merge, out of pure
laziness. I'm 100% sure the "option" to squash-on-merge would
*immediately* be abused for a lot more than just "typo fixes". There
isn't enough manpower to watch the watchers, so "no squash-on-merge"
needs to be a general rule.

I'm very sad that you're trying to wiggle such a crucial and intrusive
workflow change into the scope of this transition. Every time
squash-on-merge has come up over the years (regardless of this
transition), we've labeled it as one thing never to do, because it
destroys information (and/or even encourages not *creating* that
historical information in the first place, which is of course important
in reality).

Well, anyway, here's my feedback: if squash-on-merge is permitted in
edk2 or in basetools (or in any other external repository that's a hard
requirement for building edk2), that's a deal breaker for me, and I'll
hand in my resignation as a steward.

Maybe you'd consider that a win, I don't know -- but I couldn't remain a
steward with a straight face after failing to protect what I consider
one of the core values of open source / distributed development.

Thanks,
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

I will honor Mike Kinney’s efforts with my vote of confidence.
I think we’re headed in the right direction, even with some of the things that I disagree with.

In my history with TianoCore, I have learned to not be so quick to say “this is fucking stupid”. Every time I’ve done that, I’ve later discovered the reasons behind it, and even come to the conclusion that the designers were quite clever.

That said, I want to contribute. And I won’t with the current system. I hope to be able to with the future system.

- Bret

From: Desimone, Nathaniel L<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 12:59 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew@...>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@..." <bret.barkelew@...>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@...>, "spbrogan@..." <spbrogan@...>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@..." <lersek@...>, "Kinney, Michael D" <michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662145479&sdata=m66IViN3G%2BbJpBwolRXf8d3BpWQeRXs495WYxnsD85M%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662155477&sdata=VuYPqwcmYTrbiYdo%2B8K5irX8k6rgMgEoC2fY8eAocWA%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=lMrhfWKBWKGkjNnXJJy7%2BisrugTi0J%2FUkmtnj7Vxb7Q%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662165472&sdata=rltVYSQcSLG2sGP4d2awDIuWV11nYQcdxvqyPxDM4XE%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662175469&sdata=b0J%2BL4nddt3g%2FmidZO61tlkgVqrRsufOUHGjRqh1CJM%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=zhHUUmckRdY45oYIPodqP9r3Sh4Q4t%2FZYRLULOiAERs%3D&reserved=0>
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&data=02%7C01%7Cbret.barkelew%40microsoft.com%7Cbf6d5aed67374c95ca3508d7fc2f20ec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255151662185463&sdata=MNUfWJhjb1UfPFciUJ2dcu9uWQwUNkS3PJO%2BeMkFVxA%3D&reserved=0>

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
Sent: Tuesday, May 19, 2020 9:54 AM
To: rfc@edk2.groups.io; Desimone, Nathaniel L
<nathaniel.l.desimone@...>; lersek@...;
bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
Process

Nate/Laszlo,

Regarding a squash merge workflow. I agree it can be abused and we all
have seen terrible examples. But a patch series that contains 500+ file
changes isn't really much better. Just because it is broken into multiple
commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is
optimized for a web based review and PR processes. It allows a user to
respond to changes, fix issues, learn thru the PR process, all while keeping
complete track of the progression. Then once all "status"
checks and reviews are complete, it is squashed into a neat commit for
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows. Those
reviewing the PR can decide what is appropriate for the PR content
submitted. Just as you would request changes to the contents (or
ordering) of a commit in a series, if the reviewers don't agree that the PR
contents should be in a single commit then obviously it shouldn't be
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation
additions/fixes, etc all are great examples of PRs that can easily leverage
squash merges and this workflow significantly lowers the burden of the
contribution and review process. This workflow is also are much easier for
casual or first time contributors.

I don't exactly know how we would enable this but I assume we could
leverage tags or make it clear in the PR description. First step is to get
alignment that a squash merge workflow, while not appropriate for all
contributions, is not something to be excluded.

Thanks
Sean



On 5/19/2020 12:21 AM, Nate DeSimone wrote:
Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull
requests will not fix the fact that maintainers are usually busy people and
don't always give feedback in a punctual manner. Like Laszlo, I would also
prefer that we do not squash patch series. My biggest reason for not
squashing patch series is because when you put everything into a single
commit, I have had to review commits with 500+ files changed. Opening git
difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests
or some other web based system of code review… and I don’t have an
Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
coreboot and other projects. But since we are using Github for hosting, pull
requests are an easy switch and a logical choice. My main reason for being
excited about pull requests mostly has to do with the amount of manual
effort required to be a TianoCore maintainer right now. I have set up my
email filter so that the mailing list is categorized like so:



[cid:image001.png@...]



Implementing the logic to parse the contents of emails to categorize them
like this required me to define no less than 12 email filter rules in Microsoft
Outlook, and I have to change my filtering logic every time I am
added/removed from a Maintainers.txt file. I’m sure every other maintainer
has spent a time separately implementing filtering logic like I have. This helps,
but still for every thread, I have to go and check if one of the other
maintainers has already reviewed/pushed that patch series yet, and if not
review/push it. If I have ] feedback on a patch series, I have to categorize it
as awaiting response from author and check up on it from time to time,
sometimes I ping the author directly and remind them to send a new patch
series. Implementing this state machine is a lot of manual work and it kind of
feels like I’m a telephone operator in the 1950s. I greatly welcome
automation here as I am sure it will increase the number of patch series I am
able to review per hour.



Thanks,

Nate



-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...;
devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



I would far prefer the approach of individual PRs for commits to
allow
for the squash flexibility (and is the strategy I think I would
pursue
with my PRs). For example, the VarPol PR would be broken up into 9
PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a
half or so, rather than the multiple months that we’re now verging
on.


This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2
process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on
individual patches does not allow for the reordering / restructuring of the
patch series (dropping patches, reordering patches, inserting patches,
moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual dedicates a
separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at all, it
only supported review requests for individual patches, and it supported
setting up dependencies between them. So, for example, a 27-patch series
would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches.
It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that
complicates simple things somewhat. A process that makes simple things
simple and difficult things impossible is useless. This is what the Instagram
generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the
following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been
posted with somewhat problematic threading), but the delay apparently
came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted.
But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches, they
didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

designated MdeModulePkg review), on the day of posting;
no

other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

(yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If
reviewers don't care (or have no time) now, that problem will not change
with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series
restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the library
instance into each platform DSC, last update Variable driver to consume new
library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.



Thanks

Laszlo











Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate DeSimone
 

Hi Bret,

I believe you missed my point. I don’t want my patch series to be merged piece by piece; I want it merged all at once, in the order that I specified.

I tend to agree with Laszlo that you are choosing not to learn how to use Git properly. Commit early, commit often, perfect later, publish once is the Git best practice. You should not hide the sausage making, which is exactly what you are proposing. I find it unfortunate that you consider refusing to learn GIt best practices a mark of prestige.

Thanks,
Nate

From: <devel@edk2.groups.io> on behalf of "Bret Barkelew via groups.io" <bret.barkelew@...>
Reply-To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "bret.barkelew@..." <bret.barkelew@...>
Date: Tuesday, May 19, 2020 at 12:35 PM
To: "devel@edk2.groups.io" <devel@edk2.groups.io>, "Desimone, Nathaniel L" <nathaniel.l.desimone@...>, "spbrogan@..." <spbrogan@...>, "rfc@edk2.groups.io" <rfc@edk2.groups.io>, "lersek@..." <lersek@...>, "Kinney, Michael D" <michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
Sent: Tuesday, May 19, 2020 9:54 AM
To: rfc@edk2.groups.io; Desimone, Nathaniel L
<nathaniel.l.desimone@...>; lersek@...;
bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
Process

Nate/Laszlo,

Regarding a squash merge workflow. I agree it can be abused and we all
have seen terrible examples. But a patch series that contains 500+ file
changes isn't really much better. Just because it is broken into multiple
commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is
optimized for a web based review and PR processes. It allows a user to
respond to changes, fix issues, learn thru the PR process, all while keeping
complete track of the progression. Then once all "status"
checks and reviews are complete, it is squashed into a neat commit for
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows. Those
reviewing the PR can decide what is appropriate for the PR content
submitted. Just as you would request changes to the contents (or
ordering) of a commit in a series, if the reviewers don't agree that the PR
contents should be in a single commit then obviously it shouldn't be
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation
additions/fixes, etc all are great examples of PRs that can easily leverage
squash merges and this workflow significantly lowers the burden of the
contribution and review process. This workflow is also are much easier for
casual or first time contributors.

I don't exactly know how we would enable this but I assume we could
leverage tags or make it clear in the PR description. First step is to get
alignment that a squash merge workflow, while not appropriate for all
contributions, is not something to be excluded.

Thanks
Sean



On 5/19/2020 12:21 AM, Nate DeSimone wrote:
Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull
requests will not fix the fact that maintainers are usually busy people and
don't always give feedback in a punctual manner. Like Laszlo, I would also
prefer that we do not squash patch series. My biggest reason for not
squashing patch series is because when you put everything into a single
commit, I have had to review commits with 500+ files changed. Opening git
difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests
or some other web based system of code review… and I don’t have an
Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
coreboot and other projects. But since we are using Github for hosting, pull
requests are an easy switch and a logical choice. My main reason for being
excited about pull requests mostly has to do with the amount of manual
effort required to be a TianoCore maintainer right now. I have set up my
email filter so that the mailing list is categorized like so:



[cid:image001.png@...]



Implementing the logic to parse the contents of emails to categorize them
like this required me to define no less than 12 email filter rules in Microsoft
Outlook, and I have to change my filtering logic every time I am
added/removed from a Maintainers.txt file. I’m sure every other maintainer
has spent a time separately implementing filtering logic like I have. This helps,
but still for every thread, I have to go and check if one of the other
maintainers has already reviewed/pushed that patch series yet, and if not
review/push it. If I have ] feedback on a patch series, I have to categorize it
as awaiting response from author and check up on it from time to time,
sometimes I ping the author directly and remind them to send a new patch
series. Implementing this state machine is a lot of manual work and it kind of
feels like I’m a telephone operator in the 1950s. I greatly welcome
automation here as I am sure it will increase the number of patch series I am
able to review per hour.



Thanks,

Nate



-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...;
devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



I would far prefer the approach of individual PRs for commits to
allow
for the squash flexibility (and is the strategy I think I would
pursue
with my PRs). For example, the VarPol PR would be broken up into 9
PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a
half or so, rather than the multiple months that we’re now verging
on.


This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2
process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on
individual patches does not allow for the reordering / restructuring of the
patch series (dropping patches, reordering patches, inserting patches,
moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual dedicates a
separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at all, it
only supported review requests for individual patches, and it supported
setting up dependencies between them. So, for example, a 27-patch series
would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches.
It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that
complicates simple things somewhat. A process that makes simple things
simple and difficult things impossible is useless. This is what the Instagram
generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the
following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been
posted with somewhat problematic threading), but the delay apparently
came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted.
But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches, they
didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

designated MdeModulePkg review), on the day of posting;
no

other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

(yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If
reviewers don't care (or have no time) now, that problem will not change
with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series
restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the library
instance into each platform DSC, last update Variable driver to consume new
library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.



Thanks

Laszlo











Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

Nate, I believe you missed Sean’s point.

Each one of those packages should have been a separate PR.
Ergo, no information would have been lost in the squash.

Also, it’s not so much that we *can’t* learn. It’s that we choose not to. Around here, it’s a mark of prestige to not open doors with your face if it seems like there’s a better way. Makes it easier to focus on the work.

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone@...>
Sent: Tuesday, May 19, 2020 11:02 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; spbrogan@...<mailto:spbrogan@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; lersek@...<mailto:lersek@...>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59779&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=fVz16E37%2BwW5pSgRxI45K7nWPDlIoS0HuI8UCGmEwjY%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59780&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=4q0lC1BSlSoQ3p0HGWwlph09HTjgJRo4nTO2Qx59%2Fjc%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59781&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=XQVSwPMXdpDJXj9nkuvq2fenwhNt6HGGZXsJwH5Bu8E%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59782&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=kCULGBc6%2Bifcn3cnPTV1odHI1ZUxuWQePN3POKKS3SM%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59783&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=SCOhUMdNXHIymGLaw9z3JTh%2Fe2BfaJaAyEC99EkG%2Fvg%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59784&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=epET6Wk30bIHQCvEDFLkeHEfmm9tzlxRrJ%2FQAuEfQFs%3D&amp;reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fedk2.groups.io%2Fg%2Fdevel%2Fmessage%2F59785&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C6e47f5e045f740536a0708d7fc1ed290%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637255081625996312&amp;sdata=N8T7HjerJVvyGg94yMWjLm%2Fw7WDdXOdby1JpOYlPeVc%3D&amp;reserved=0

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
Sent: Tuesday, May 19, 2020 9:54 AM
To: rfc@edk2.groups.io; Desimone, Nathaniel L
<nathaniel.l.desimone@...>; lersek@...;
bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
Process

Nate/Laszlo,

Regarding a squash merge workflow. I agree it can be abused and we all
have seen terrible examples. But a patch series that contains 500+ file
changes isn't really much better. Just because it is broken into multiple
commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is
optimized for a web based review and PR processes. It allows a user to
respond to changes, fix issues, learn thru the PR process, all while keeping
complete track of the progression. Then once all "status"
checks and reviews are complete, it is squashed into a neat commit for
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows. Those
reviewing the PR can decide what is appropriate for the PR content
submitted. Just as you would request changes to the contents (or
ordering) of a commit in a series, if the reviewers don't agree that the PR
contents should be in a single commit then obviously it shouldn't be
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation
additions/fixes, etc all are great examples of PRs that can easily leverage
squash merges and this workflow significantly lowers the burden of the
contribution and review process. This workflow is also are much easier for
casual or first time contributors.

I don't exactly know how we would enable this but I assume we could
leverage tags or make it clear in the PR description. First step is to get
alignment that a squash merge workflow, while not appropriate for all
contributions, is not something to be excluded.

Thanks
Sean



On 5/19/2020 12:21 AM, Nate DeSimone wrote:
Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull
requests will not fix the fact that maintainers are usually busy people and
don't always give feedback in a punctual manner. Like Laszlo, I would also
prefer that we do not squash patch series. My biggest reason for not
squashing patch series is because when you put everything into a single
commit, I have had to review commits with 500+ files changed. Opening git
difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests
or some other web based system of code review… and I don’t have an
Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
coreboot and other projects. But since we are using Github for hosting, pull
requests are an easy switch and a logical choice. My main reason for being
excited about pull requests mostly has to do with the amount of manual
effort required to be a TianoCore maintainer right now. I have set up my
email filter so that the mailing list is categorized like so:



[cid:image001.png@...]



Implementing the logic to parse the contents of emails to categorize them
like this required me to define no less than 12 email filter rules in Microsoft
Outlook, and I have to change my filtering logic every time I am
added/removed from a Maintainers.txt file. I’m sure every other maintainer
has spent a time separately implementing filtering logic like I have. This helps,
but still for every thread, I have to go and check if one of the other
maintainers has already reviewed/pushed that patch series yet, and if not
review/push it. If I have ] feedback on a patch series, I have to categorize it
as awaiting response from author and check up on it from time to time,
sometimes I ping the author directly and remind them to send a new patch
series. Implementing this state machine is a lot of manual work and it kind of
feels like I’m a telephone operator in the 1950s. I greatly welcome
automation here as I am sure it will increase the number of patch series I am
able to review per hour.



Thanks,

Nate



-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...;
devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



I would far prefer the approach of individual PRs for commits to
allow
for the squash flexibility (and is the strategy I think I would
pursue
with my PRs). For example, the VarPol PR would be broken up into 9
PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a
half or so, rather than the multiple months that we’re now verging
on.


This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2
process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on
individual patches does not allow for the reordering / restructuring of the
patch series (dropping patches, reordering patches, inserting patches,
moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual dedicates a
separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at all, it
only supported review requests for individual patches, and it supported
setting up dependencies between them. So, for example, a 27-patch series
would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches.
It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that
complicates simple things somewhat. A process that makes simple things
simple and difficult things impossible is useless. This is what the Instagram
generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the
following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been
posted with somewhat problematic threading), but the delay apparently
came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted.
But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches, they
didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

designated MdeModulePkg review), on the day of posting;
no

other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

(yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If
reviewers don't care (or have no time) now, that problem will not change
with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series
restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the library
instance into each platform DSC, last update Variable driver to consume new
library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.



Thanks

Laszlo











Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate DeSimone
 

Hi Sean,

My recent spelling fix patch series is a good example of why this is a bad idea actually:

https://edk2.groups.io/g/devel/message/59779
https://edk2.groups.io/g/devel/message/59780
https://edk2.groups.io/g/devel/message/59781
https://edk2.groups.io/g/devel/message/59782
https://edk2.groups.io/g/devel/message/59783
https://edk2.groups.io/g/devel/message/59784
https://edk2.groups.io/g/devel/message/59785

Notice that I split along package boundaries, because the maintainers for each package is a different set of people. If my patch series was squashed at merge time... how do I know who reviewed what? If the commit set is not correct.. I tend to say so in my feedback :). The only sane way to squash this series would be to have a human re-write all the commit messages, which I am against.

Generally those that prefer an easily bisectable history have such preference mostly due to the usage of validators that immediately resort bisecting as a method to root cause an issue since they tend to not understand the code very well. Edk2 already has 12 years of non-bisectable history, so this method is going to be ineffective anyway.

With regard to sending squashed commits, I understand that those who are new may have difficulty sending a properly formatted patch series, but frankly attempting to shield them from having to learn I am strongly against. I suggest that Microsoft invest in its human capital similar to how Intel does. If you cannot figure out how to send a properly formatted patch series... then do your work on the internal codebase (or perhaps MU.) Within the Intel, having the skillset to contribute to TianoCore is considered a mark of prestige, and thus needs to be earned.

TLDR, I will reject squashed commits on any packages that I maintain.

Thanks,
Nate

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Sean
Sent: Tuesday, May 19, 2020 9:54 AM
To: rfc@edk2.groups.io; Desimone, Nathaniel L
<nathaniel.l.desimone@...>; lersek@...;
bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@...>
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review
Process

Nate/Laszlo,

Regarding a squash merge workflow. I agree it can be abused and we all
have seen terrible examples. But a patch series that contains 500+ file
changes isn't really much better. Just because it is broken into multiple
commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is
optimized for a web based review and PR processes. It allows a user to
respond to changes, fix issues, learn thru the PR process, all while keeping
complete track of the progression. Then once all "status"
checks and reviews are complete, it is squashed into a neat commit for
mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows. Those
reviewing the PR can decide what is appropriate for the PR content
submitted. Just as you would request changes to the contents (or
ordering) of a commit in a series, if the reviewers don't agree that the PR
contents should be in a single commit then obviously it shouldn't be
squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation
additions/fixes, etc all are great examples of PRs that can easily leverage
squash merges and this workflow significantly lowers the burden of the
contribution and review process. This workflow is also are much easier for
casual or first time contributors.

I don't exactly know how we would enable this but I assume we could
leverage tags or make it clear in the PR description. First step is to get
alignment that a squash merge workflow, while not appropriate for all
contributions, is not something to be excluded.

Thanks
Sean



On 5/19/2020 12:21 AM, Nate DeSimone wrote:
Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull
requests will not fix the fact that maintainers are usually busy people and
don't always give feedback in a punctual manner. Like Laszlo, I would also
prefer that we do not squash patch series. My biggest reason for not
squashing patch series is because when you put everything into a single
commit, I have had to review commits with 500+ files changed. Opening git
difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests
or some other web based system of code review… and I don’t have an
Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with
coreboot and other projects. But since we are using Github for hosting, pull
requests are an easy switch and a logical choice. My main reason for being
excited about pull requests mostly has to do with the amount of manual
effort required to be a TianoCore maintainer right now. I have set up my
email filter so that the mailing list is categorized like so:



[cid:image001.png@...]



Implementing the logic to parse the contents of emails to categorize them
like this required me to define no less than 12 email filter rules in Microsoft
Outlook, and I have to change my filtering logic every time I am
added/removed from a Maintainers.txt file. I’m sure every other maintainer
has spent a time separately implementing filtering logic like I have. This helps,
but still for every thread, I have to go and check if one of the other
maintainers has already reviewed/pushed that patch series yet, and if not
review/push it. If I have ] feedback on a patch series, I have to categorize it
as awaiting response from author and check up on it from time to time,
sometimes I ping the author directly and remind them to send a new patch
series. Implementing this state machine is a lot of manual work and it kind of
feels like I’m a telephone operator in the 1950s. I greatly welcome
automation here as I am sure it will increase the number of patch series I am
able to review per hour.



Thanks,

Nate



-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo
Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...;
devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



I would far prefer the approach of individual PRs for commits to
allow
for the squash flexibility (and is the strategy I think I would
pursue
with my PRs). For example, the VarPol PR would be broken up into 9
PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a
half or so, rather than the multiple months that we’re now verging
on.


This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2
process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on
individual patches does not allow for the reordering / restructuring of the
patch series (dropping patches, reordering patches, inserting patches,
moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual dedicates a
separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at all, it
only supported review requests for individual patches, and it supported
setting up dependencies between them. So, for example, a 27-patch series
would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches.
It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged),
and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that
complicates simple things somewhat. A process that makes simple things
simple and difficult things impossible is useless. This is what the Instagram
generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the
following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been
posted with somewhat problematic threading), but the delay apparently
came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted.
But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series (May
11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches, they
didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

designated MdeModulePkg review), on the day of posting;
no

other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

(yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If
reviewers don't care (or have no time) now, that problem will not change
with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series
restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the library
instance into each platform DSC, last update Variable driver to consume new
library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.



Thanks

Laszlo











Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate DeSimone
 

On 5/19/20 01:40, Laszlo Ersek wrote:

That seems strange. I have one rule per edk2-* list, for moving such incoming
email into the appropriate list folder. That's all.

While I read all the subject lines (skim all the threads) on edk2-devel,
generally, if you share reviewer or maintainer responsibilities for some
subsystem, then people posting patches for that subsystem are supposed to
CC you explicitly, in addition to messaging the list.
I tend to make the assumption that people do not CC me on the patches that they are supposed to CC me on. So I set up my filtering rules to do a deep inspection of the message contents to see if it touches a package that I maintain.

Checking whether others have commented is near trivial if your MUA
supports a threaded view.

Checking whether a co-maintainer of yours has pushed a given series is also
simple if they diligently report the fact of merging on the list (in the subject
patch threads).
Yes, checking for comments is trivial. However, my fellow co-maintainers are not very diligent on sending push notifications. So when I see comments from one of my fellow co-maintainers I immediately ask myself the question: "Did they already push this, and does it make sense for me to spend time reviewing this patch series?" Answering that question involves a git pull and a review of history in gitk to see what has been done already.

I think this is not your job, as a reviewer/maintainer. Once your review is
complete, or blocked on a question you need an answer to, the ball is back in
the contributor's court. They can answer, or post the next version, whenever
they see fit. Until then, the most they can expect of you is answering any
further questions they might have for understanding your previous feedback
better. You need not push contributors to complete their contributions.
I think my experience is colored somewhat here. I'd say more than half the time, the contributor is another Intel employee. Often times, they are contributing code changes that I asked them to implement. :)

"State machine" is a very good analogy! Personally, I don't find it tiresome.
Yes, it's important to recognize the events (= new emails) that trigger
transitions between states. (For example: when I complete a review, when I
get a new version of a series or a brand new series, when I get asked a
question.) Once I recognize those events correctly, I just diligently massage
said tags ("stars").

And I keep iterating over my set of "starred" messages; I do actual work
(e.g., reviews) in "bottom halves"; detached from new emails.

I don't find this a burden as I have to manage my "real life" with task lists
anyway. Without them, my real life would collapse in a week; so it's nothing
unusual for me. (And no, I don't allow shady cloud-based automatisms to
manage my life for me; I value my privacy way above my
comfort.)
Agreed that I also keep my personal task lists in a paper notebook and manage my real life list manually. However, my real life list is much smaller (since I have most of the context in my head already)... and its private. Everything I do on this mailing list is public anyway, so having some centralized service keep track of state transitions doesn't bother me. The "bottom half" of that state transition is going to generate a public email from my address, so it's not like the current state of the state machine that I'm running in my head is private.

Thanks,
Nate

Thanks!
Laszlo



Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Sean
 

Nate/Laszlo,

Regarding a squash merge workflow. I agree it can be abused and we all have seen terrible examples. But a patch series that contains 500+ file changes isn't really much better. Just because it is broken into multiple commits doesn't mean it is the right set of commits.

Anyway a squash merge workflow works amazingly well with and is optimized for a web based review and PR processes. It allows a user to respond to changes, fix issues, learn thru the PR process, all while keeping complete track of the progression. Then once all "status" checks and reviews are complete, it is squashed into a neat commit for mainline, containing only the relevant data in the message.

So, the ask is that we don't exclude squash merge workflows. Those reviewing the PR can decide what is appropriate for the PR content submitted. Just as you would request changes to the contents (or ordering) of a commit in a series, if the reviewers don't agree that the PR contents should be in a single commit then obviously it shouldn't be squashed to one.

Contributions like spelling fixes, typos, minor bug fixes, documentation additions/fixes, etc all are great examples of PRs that can easily leverage squash merges and this workflow significantly lowers the burden of the contribution and review process. This workflow is also are much easier for casual or first time contributors.

I don't exactly know how we would enable this but I assume we could leverage tags or make it clear in the PR description. First step is to get alignment that a squash merge workflow, while not appropriate for all contributions, is not something to be excluded.

Thanks
Sean

On 5/19/2020 12:21 AM, Nate DeSimone wrote:
Hi All,
I tend to agree with most of Laszlo's points. Specifically, that moving to pull requests will not fix the fact that maintainers are usually busy people and don't always give feedback in a punctual manner. Like Laszlo, I would also prefer that we do not squash patch series. My biggest reason for not squashing patch series is because when you put everything into a single commit, I have had to review commits with 500+ files changed. Opening git difftool on a commit like that is awful.
However, I would like to register my general endorsement for pull requests or some other web based system of code review… and I don’t have an Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with coreboot and other projects. But since we are using Github for hosting, pull requests are an easy switch and a logical choice. My main reason for being excited about pull requests mostly has to do with the amount of manual effort required to be a TianoCore maintainer right now. I have set up my email filter so that the mailing list is categorized like so:
[cid:image001.png@...]
Implementing the logic to parse the contents of emails to categorize them like this required me to define no less than 12 email filter rules in Microsoft Outlook, and I have to change my filtering logic every time I am added/removed from a Maintainers.txt file. I’m sure every other maintainer has spent a time separately implementing filtering logic like I have. This helps, but still for every thread, I have to go and check if one of the other maintainers has already reviewed/pushed that patch series yet, and if not review/push it. If I have ] feedback on a patch series, I have to categorize it as awaiting response from author and check up on it from time to time, sometimes I ping the author directly and remind them to send a new patch series. Implementing this state machine is a lot of manual work and it kind of feels like I’m a telephone operator in the 1950s. I greatly welcome automation here as I am sure it will increase the number of patch series I am able to review per hour.
Thanks,
Nate
-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process
On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

I would far prefer the approach of individual PRs for commits to allow
for the squash flexibility (and is the strategy I think I would pursue
with my PRs). For example, the VarPol PR would be broken up into 9 PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and a
half or so, rather than the multiple months that we’re now verging
on.
This differs extremely from how we've been working on edk2-devel (or from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2 process to github.
Again, the structuring of a patch series is a primary trait. Iterating only on individual patches does not allow for the reordering / restructuring of the patch series (dropping patches, reordering patches, inserting patches, moving hunks between patches).
It's common that the necessity to revise an earlier patch emerges while reworking a later patch. For instance, the git-rebase(1) manual dedicates a separate section to "splitting commits".
In the initial evaluation of "web forges", Phabricator was one of the "contestants". Phabricator didn't support the "patch series" concept at all, it only supported review requests for individual patches, and it supported setting up dependencies between them. So, for example, a 27-patch series would require 27 submissions and 26 dependencies.
Lacking support for the patch series concept was an immediate deal breaker with Phabricator.
The longest patch series I've ever submitted to edk2-devel had 58 patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged), and the patch count varied significantly:
v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)
(The significant drop in the patch count was due to Mike Kinney open sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was huge work in its own right), allowing me to drop the Quark-originated 32-bit-only PiSmmCpuDxeSmm variant, from my series.)
The contribution process should make difficult things possible, even if that complicates simple things somewhat. A process that makes simple things simple and difficult things impossible is useless. This is what the Instagram generation seems to be missing.
I don't know why the VariablePolicy work took months. I can see the following threads on the list:
* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
Fri, 10 Apr 2020 11:36:01 -0700
* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
Mon, 11 May 2020 23:46:23 -0700
I have two sets of comments:
(1) It's difficult to tell in retrospect (because the series seem to have been posted with somewhat problematic threading), but the delay apparently came from multiple sources.
(1a) Review was slow and spotty.
The v1 blurb received some comments in the first week after it was posted. But the rest of the v1 series (the actual patches) received feedback like this:
- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback
(1b) There was also quite some time between the last response in the v1 thread (Apr 26th, as far as I can see), and the posting of the v2 series (May 11th).
(1c) The v2 blurb got almost immediate, and numerous feedback (on the day of posting, and the day after). Regarding the individual patches, they didn't fare too well:
- v2 01/12: superficial comment on the day of posting from me (not a
designated MdeModulePkg review), on the day of posting; no
other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
(yours truly), on the day of posting
- v2 12/12: no feedback
In total, I don't think the current process takes the blame for the delay. If reviewers don't care (or have no time) now, that problem will not change with the transition to github.com.
(2) The VariablePolicy series is actually a good example that patch series restructuring is important.
(2a) The patch count went from 9 (in v1) to 12 (in v2).
(2b) And under v2, Liming still pointed out: "To keep each commit build pass, the patch set should first add new library instance, then add the library instance into each platform DSC, last update Variable driver to consume new library instance."
Furthermore, I requested enabling the feature in ArmVirtPkg too, and maybe (based on owner feedback) UefiPayloadPkg.
Thus, the v2->v3 update will most likely bring about both patch order changes, and an increased patch count.
Thanks
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/19/20 09:21, Desimone, Nathaniel L wrote:

However, I would like to register my general endorsement for pull
requests or some other web based system of code review... and I don't
have an Instagram account by the way :) Personally, I prefer Gerrit as
I use it a lot with coreboot and other projects. But since we are
using Github for hosting, pull requests are an easy switch and a
logical choice. My main reason for being excited about pull requests
mostly has to do with the amount of manual effort required to be a
TianoCore maintainer right now.
My understanding is that, at this point, we're inevitably going to
migrate the contribution/review workflow to GitHub. I believe the switch
is going to happen once the email webhook has been deemed functional and
stable enough by the community.

Digression starts:

Implementing the logic to parse the contents of emails to categorize
them like this required me to define no less than 12 email filter
rules in Microsoft Outlook, and I have to change my filtering logic
every time I am added/removed from a Maintainers.txt file.
That seems strange. I have one rule per edk2-* list, for moving such
incoming email into the appropriate list folder. That's all.

While I read all the subject lines (skim all the threads) on edk2-devel,
generally, if you share reviewer or maintainer responsibilities for some
subsystem, then people posting patches for that subsystem are supposed
to CC you explicitly, in addition to messaging the list.

How you handle messages from then on may be a personal matter of course.
I simply tag ("star") such messages (patches / series pending my
review), and I revisit my "set of starred messages" every day (sometimes
multiple times per day).

I'm sure every other maintainer has spent a time separately
implementing filtering logic like I have. This helps, but still for
every thread, I have to go and check if one of the other maintainers
has already reviewed/pushed that patch series yet, and if not
review/push it.
Checking whether others have commented is near trivial if your MUA
supports a threaded view.

Checking whether a co-maintainer of yours has pushed a given series is
also simple if they diligently report the fact of merging on the list
(in the subject patch threads).

If I have feedback on a patch series, I have to categorize it as
awaiting response from author and check up on it from time to time,
sometimes I ping the author directly and remind them to send a new
patch series.
I think this is not your job, as a reviewer/maintainer. Once your review
is complete, or blocked on a question you need an answer to, the ball is
back in the contributor's court. They can answer, or post the next
version, whenever they see fit. Until then, the most they can expect of
you is answering any further questions they might have for understanding
your previous feedback better. You need not push contributors to
complete their contributions.

Implementing this state machine is a lot of manual work and it kind of
feels like I'm a telephone operator in the 1950s. I greatly welcome
automation here as I am sure it will increase the number of patch
series I am able to review per hour.
"State machine" is a very good analogy! Personally, I don't find it
tiresome. Yes, it's important to recognize the events (= new emails)
that trigger transitions between states. (For example: when I complete a
review, when I get a new version of a series or a brand new series, when
I get asked a question.) Once I recognize those events correctly, I just
diligently massage said tags ("stars").

And I keep iterating over my set of "starred" messages; I do actual work
(e.g., reviews) in "bottom halves"; detached from new emails.

I don't find this a burden as I have to manage my "real life" with task
lists anyway. Without them, my real life would collapse in a week; so
it's nothing unusual for me. (And no, I don't allow shady cloud-based
automatisms to manage my life for me; I value my privacy way above my
comfort.)

Thanks!
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Nate DeSimone
 

Hi All,



I tend to agree with most of Laszlo's points. Specifically, that moving to pull requests will not fix the fact that maintainers are usually busy people and don't always give feedback in a punctual manner. Like Laszlo, I would also prefer that we do not squash patch series. My biggest reason for not squashing patch series is because when you put everything into a single commit, I have had to review commits with 500+ files changed. Opening git difftool on a commit like that is awful.



However, I would like to register my general endorsement for pull requests or some other web based system of code review… and I don’t have an Instagram account by the way :) Personally, I prefer Gerrit as I use it a lot with coreboot and other projects. But since we are using Github for hosting, pull requests are an easy switch and a logical choice. My main reason for being excited about pull requests mostly has to do with the amount of manual effort required to be a TianoCore maintainer right now. I have set up my email filter so that the mailing list is categorized like so:



[cid:image001.png@...]



Implementing the logic to parse the contents of emails to categorize them like this required me to define no less than 12 email filter rules in Microsoft Outlook, and I have to change my filtering logic every time I am added/removed from a Maintainers.txt file. I’m sure every other maintainer has spent a time separately implementing filtering logic like I have. This helps, but still for every thread, I have to go and check if one of the other maintainers has already reviewed/pushed that patch series yet, and if not review/push it. If I have ] feedback on a patch series, I have to categorize it as awaiting response from author and check up on it from time to time, sometimes I ping the author directly and remind them to send a new patch series. Implementing this state machine is a lot of manual work and it kind of feels like I’m a telephone operator in the 1950s. I greatly welcome automation here as I am sure it will increase the number of patch series I am able to review per hour.



Thanks,

Nate

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Laszlo Ersek
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io; bret.barkelew@...; devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process



On 05/15/20 06:49, Bret Barkelew via groups.io wrote:



I would far prefer the approach of individual PRs for commits to allow
for the squash flexibility (and is the strategy I think I would pursue
with my PRs). For example, the VarPol PR would be broken up into 9 PRs
for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and a
half or so, rather than the multiple months that we’re now verging
on.


This differs extremely from how we've been working on edk2-devel (or from how any git-based project works that I've ever been involved with).

And I think the above workflow is out of scope, for migrating the edk2 process to github.



Again, the structuring of a patch series is a primary trait. Iterating only on individual patches does not allow for the reordering / restructuring of the patch series (dropping patches, reordering patches, inserting patches, moving hunks between patches).



It's common that the necessity to revise an earlier patch emerges while reworking a later patch. For instance, the git-rebase(1) manual dedicates a separate section to "splitting commits".



In the initial evaluation of "web forges", Phabricator was one of the "contestants". Phabricator didn't support the "patch series" concept at all, it only supported review requests for individual patches, and it supported setting up dependencies between them. So, for example, a 27-patch series would require 27 submissions and 26 dependencies.



Lacking support for the patch series concept was an immediate deal breaker with Phabricator.



The longest patch series I've ever submitted to edk2-devel had 58 patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was merged), and the patch count varied significantly:



v1: 58 patches (25 Jul 2015)

v2: 41 patches ( 9 Oct 2015)

v3: 52 patches (15 Oct 2015)

v4: 41 patches ( 3 Nov 2015)

v5: 33 patches (27 Nov 2015)



(The significant drop in the patch count was due to Mike Kinney open sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was huge work in its own right), allowing me to drop the Quark-originated 32-bit-only PiSmmCpuDxeSmm variant, from my series.)



The contribution process should make difficult things possible, even if that complicates simple things somewhat. A process that makes simple things simple and difficult things impossible is useless. This is what the Instagram generation seems to be missing.





I don't know why the VariablePolicy work took months. I can see the following threads on the list:



* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature

Fri, 10 Apr 2020 11:36:01 -0700



* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature

Mon, 11 May 2020 23:46:23 -0700



I have two sets of comments:



(1) It's difficult to tell in retrospect (because the series seem to have been posted with somewhat problematic threading), but the delay apparently came from multiple sources.



(1a) Review was slow and spotty.



The v1 blurb received some comments in the first week after it was posted. But the rest of the v1 series (the actual patches) received feedback like this:



- v1 1/9: no feedback

- v1 2/9: 12 days after posting

- v1 3/9: 16 days after posting

- v1 4/9: no feedback

- v1 5/9: no feedback

- v1 6/9: no feedback

- v1 7/9: no feedback

- v1 8/9: no feedback

- v1 9/9: no feedback



(1b) There was also quite some time between the last response in the v1 thread (Apr 26th, as far as I can see), and the posting of the v2 series (May 11th).



(1c) The v2 blurb got almost immediate, and numerous feedback (on the day of posting, and the day after). Regarding the individual patches, they didn't fare too well:



- v2 01/12: superficial comment on the day of posting from me (not a

designated MdeModulePkg review), on the day of posting; no

other feedback thus far

- v2 02/12: ditto

- v2 03/12: no feedback

- v2 04/12: superficial (coding style) comments on the day of posting

- v2 05/12: no feedback

- v2 06/12: no feedback

- v2 07/12: no feedback

- v2 08/12: no feedback

- v2 09/12: no feedback

- v2 10/12: no feedback

- v2 11/12: reasonably in-depth review from responsible co-maintainer

(yours truly), on the day of posting

- v2 12/12: no feedback



In total, I don't think the current process takes the blame for the delay. If reviewers don't care (or have no time) now, that problem will not change with the transition to github.com.





(2) The VariablePolicy series is actually a good example that patch series restructuring is important.



(2a) The patch count went from 9 (in v1) to 12 (in v2).



(2b) And under v2, Liming still pointed out: "To keep each commit build pass, the patch set should first add new library instance, then add the library instance into each platform DSC, last update Variable driver to consume new library instance."



Furthermore, I requested enabling the feature in ArmVirtPkg too, and maybe (based on owner feedback) UefiPayloadPkg.



Thus, the v2->v3 update will most likely bring about both patch order changes, and an increased patch count.



Thanks

Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Philippe Mathieu-Daudé <philmd@...>
 

Hi Bret,

On 5/15/20 5:43 PM, Bret Barkelew via groups.io wrote:
I agree with some of your points, but I don’t believe that this calls for dependencies at all.
Which points are you disagreeing?

If a PR can pass CI with the changes, it’s functionally unordered.
And if a PR can’t, it has to wait until the PRs that can are in.
This also allows the group to focus on getting one thing done at a time.
I use rebase all the time and agree that it’s very good at precise history management. If a given PR requires that level of control, those tools will always be there.
But just as you say that the simple should not preclude the difficult, the difficult 5% should not needlessly complicated the simple 95%.
For what it’s worth, this is all posturing on my part. I intend – and, indeed, am eager to – follow the process that we’ve been helping Mike to set up.
- Bret
From: Laszlo Ersek<mailto:lersek@...>
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process
On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

I would far prefer the approach of individual PRs for commits to
allow for the squash flexibility (and is the strategy I think I would
pursue with my PRs). For example, the VarPol PR would be broken up
into 9 PRs for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a half or so, rather than the multiple months that we’re now verging
on.
This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2
process to github.
Again, the structuring of a patch series is a primary trait. Iterating
only on individual patches does not allow for the reordering /
restructuring of the patch series (dropping patches, reordering patches,
inserting patches, moving hunks between patches).
It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual
dedicates a separate section to "splitting commits".
In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at
all, it only supported review requests for individual patches, and it
supported setting up dependencies between them. So, for example, a
27-patch series would require 27 submissions and 26 dependencies.
Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.
The longest patch series I've ever submitted to edk2-devel had 58
patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
merged), and the patch count varied significantly:
v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)
(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)
The contribution process should make difficult things possible, even if
that complicates simple things somewhat. A process that makes simple
things simple and difficult things impossible is useless. This is what
the Instagram generation seems to be missing.
I don't know why the VariablePolicy work took months. I can see the
following threads on the list:
* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
Fri, 10 Apr 2020 11:36:01 -0700
* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
Mon, 11 May 2020 23:46:23 -0700
I have two sets of comments:
(1) It's difficult to tell in retrospect (because the series seem to
have been posted with somewhat problematic threading), but the delay
apparently came from multiple sources.
(1a) Review was slow and spotty.
IIUC it is easier for the "Instagram generation" to write a GitHub
plugin which ping an unmerged pullrequest for them, rather than tracking
their WiP and send a "ping" via an email client.

That reminds me of the Prophet tool:

Prophet: The first generate-and-validate tool that uses machine learning
techniques to learn useful knowledge from past human patches to
recognize correct patches. It is evaluated on the same benchmark as
GenProg and generate correct patches (i.e., equivalent to human patches)
for 18 out of 69 cases.

https://groups.csail.mit.edu/pac/patchgen/
https://en.wikipedia.org/wiki/Automatic_bug_fixing#C [8]

Use it as source, combined with a fuzzer that open GH pull-requests, and see if a patch get merged... /s

The v1 blurb received some comments in the first week after it was
posted. But the rest of the v1 series (the actual patches) received
feedback like this:
- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback
(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series
(May 11th).
(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches,
they didn't fare too well:
- v2 01/12: superficial comment on the day of posting from me (not a
designated MdeModulePkg review), on the day of posting; no
other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
(yours truly), on the day of posting
- v2 12/12: no feedback
In total, I don't think the current process takes the blame for the
delay. If reviewers don't care (or have no time) now, that problem will
not change with the transition to github.com.
(2) The VariablePolicy series is actually a good example that patch
series restructuring is important.
(2a) The patch count went from 9 (in v1) to 12 (in v2).
(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the
library instance into each platform DSC, last update Variable driver to
consume new library instance."
Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.
Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.
Thanks
Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

I agree with some of your points, but I don’t believe that this calls for dependencies at all.
If a PR can pass CI with the changes, it’s functionally unordered.
And if a PR can’t, it has to wait until the PRs that can are in.

This also allows the group to focus on getting one thing done at a time.

I use rebase all the time and agree that it’s very good at precise history management. If a given PR requires that level of control, those tools will always be there.

But just as you say that the simple should not preclude the difficult, the difficult 5% should not needlessly complicated the simple 95%.

For what it’s worth, this is all posturing on my part. I intend – and, indeed, am eager to – follow the process that we’ve been helping Mike to set up.

- Bret

From: Laszlo Ersek<mailto:lersek@...>
Sent: Friday, May 15, 2020 2:08 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

I would far prefer the approach of individual PRs for commits to
allow for the squash flexibility (and is the strategy I think I would
pursue with my PRs). For example, the VarPol PR would be broken up
into 9 PRs for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a half or so, rather than the multiple months that we’re now verging
on.
This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2
process to github.

Again, the structuring of a patch series is a primary trait. Iterating
only on individual patches does not allow for the reordering /
restructuring of the patch series (dropping patches, reordering patches,
inserting patches, moving hunks between patches).

It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual
dedicates a separate section to "splitting commits".

In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at
all, it only supported review requests for individual patches, and it
supported setting up dependencies between them. So, for example, a
27-patch series would require 27 submissions and 26 dependencies.

Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.

The longest patch series I've ever submitted to edk2-devel had 58
patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
merged), and the patch count varied significantly:

v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)

(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)

The contribution process should make difficult things possible, even if
that complicates simple things somewhat. A process that makes simple
things simple and difficult things impossible is useless. This is what
the Instagram generation seems to be missing.


I don't know why the VariablePolicy work took months. I can see the
following threads on the list:

* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
Fri, 10 Apr 2020 11:36:01 -0700

* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
Mon, 11 May 2020 23:46:23 -0700

I have two sets of comments:

(1) It's difficult to tell in retrospect (because the series seem to
have been posted with somewhat problematic threading), but the delay
apparently came from multiple sources.

(1a) Review was slow and spotty.

The v1 blurb received some comments in the first week after it was
posted. But the rest of the v1 series (the actual patches) received
feedback like this:

- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback

(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series
(May 11th).

(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches,
they didn't fare too well:

- v2 01/12: superficial comment on the day of posting from me (not a
designated MdeModulePkg review), on the day of posting; no
other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
(yours truly), on the day of posting
- v2 12/12: no feedback

In total, I don't think the current process takes the blame for the
delay. If reviewers don't care (or have no time) now, that problem will
not change with the transition to github.com.


(2) The VariablePolicy series is actually a good example that patch
series restructuring is important.

(2a) The patch count went from 9 (in v1) to 12 (in v2).

(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the
library instance into each platform DSC, last update Variable driver to
consume new library instance."

Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.

Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.

Thanks
Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

“… boundless and bare, The lone and level sands stretch far away.”

- Bret

From: Laszlo Ersek<mailto:lersek@...>
Sent: Friday, May 15, 2020 12:34 AM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; Kinney, Michael D<mailto:michael.d.kinney@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/14/20 23:26, Bret Barkelew via groups.io wrote:

It’s code management for the Instagram generation
I find this an extremely good characterization!

And, I find the fact soul-destroying.

Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/15/20 06:49, Bret Barkelew via groups.io wrote:

I would far prefer the approach of individual PRs for commits to
allow for the squash flexibility (and is the strategy I think I would
pursue with my PRs). For example, the VarPol PR would be broken up
into 9 PRs for each final commit, and we can get them in one by one.
Ideally, each one would be a small back and forth and then in. If it
had been done that way to begin with, it would be over in a week and
a half or so, rather than the multiple months that we’re now verging
on.
This differs extremely from how we've been working on edk2-devel (or
from how any git-based project works that I've ever been involved with).
And I think the above workflow is out of scope, for migrating the edk2
process to github.

Again, the structuring of a patch series is a primary trait. Iterating
only on individual patches does not allow for the reordering /
restructuring of the patch series (dropping patches, reordering patches,
inserting patches, moving hunks between patches).

It's common that the necessity to revise an earlier patch emerges while
reworking a later patch. For instance, the git-rebase(1) manual
dedicates a separate section to "splitting commits".

In the initial evaluation of "web forges", Phabricator was one of the
"contestants". Phabricator didn't support the "patch series" concept at
all, it only supported review requests for individual patches, and it
supported setting up dependencies between them. So, for example, a
27-patch series would require 27 submissions and 26 dependencies.

Lacking support for the patch series concept was an immediate deal
breaker with Phabricator.

The longest patch series I've ever submitted to edk2-devel had 58
patches. It was SMM enablement for OVMF. It went from v1 to v5 (v5 was
merged), and the patch count varied significantly:

v1: 58 patches (25 Jul 2015)
v2: 41 patches ( 9 Oct 2015)
v3: 52 patches (15 Oct 2015)
v4: 41 patches ( 3 Nov 2015)
v5: 33 patches (27 Nov 2015)

(The significant drop in the patch count was due to Mike Kinney open
sourcing and upstreaming the *real* PiSmmCpuDxeSmm driver (which was
huge work in its own right), allowing me to drop the Quark-originated
32-bit-only PiSmmCpuDxeSmm variant, from my series.)

The contribution process should make difficult things possible, even if
that complicates simple things somewhat. A process that makes simple
things simple and difficult things impossible is useless. This is what
the Instagram generation seems to be missing.


I don't know why the VariablePolicy work took months. I can see the
following threads on the list:

* [edk2-devel] [PATCH v1 0/9] Add the VariablePolicy feature
Fri, 10 Apr 2020 11:36:01 -0700

* [edk2-devel] [PATCH v2 00/12] Add the VariablePolicy feature
Mon, 11 May 2020 23:46:23 -0700

I have two sets of comments:

(1) It's difficult to tell in retrospect (because the series seem to
have been posted with somewhat problematic threading), but the delay
apparently came from multiple sources.

(1a) Review was slow and spotty.

The v1 blurb received some comments in the first week after it was
posted. But the rest of the v1 series (the actual patches) received
feedback like this:

- v1 1/9: no feedback
- v1 2/9: 12 days after posting
- v1 3/9: 16 days after posting
- v1 4/9: no feedback
- v1 5/9: no feedback
- v1 6/9: no feedback
- v1 7/9: no feedback
- v1 8/9: no feedback
- v1 9/9: no feedback

(1b) There was also quite some time between the last response in the v1
thread (Apr 26th, as far as I can see), and the posting of the v2 series
(May 11th).

(1c) The v2 blurb got almost immediate, and numerous feedback (on the
day of posting, and the day after). Regarding the individual patches,
they didn't fare too well:

- v2 01/12: superficial comment on the day of posting from me (not a
designated MdeModulePkg review), on the day of posting; no
other feedback thus far
- v2 02/12: ditto
- v2 03/12: no feedback
- v2 04/12: superficial (coding style) comments on the day of posting
- v2 05/12: no feedback
- v2 06/12: no feedback
- v2 07/12: no feedback
- v2 08/12: no feedback
- v2 09/12: no feedback
- v2 10/12: no feedback
- v2 11/12: reasonably in-depth review from responsible co-maintainer
(yours truly), on the day of posting
- v2 12/12: no feedback

In total, I don't think the current process takes the blame for the
delay. If reviewers don't care (or have no time) now, that problem will
not change with the transition to github.com.


(2) The VariablePolicy series is actually a good example that patch
series restructuring is important.

(2a) The patch count went from 9 (in v1) to 12 (in v2).

(2b) And under v2, Liming still pointed out: "To keep each commit build
pass, the patch set should first add new library instance, then add the
library instance into each platform DSC, last update Variable driver to
consume new library instance."

Furthermore, I requested enabling the feature in ArmVirtPkg too, and
maybe (based on owner feedback) UefiPayloadPkg.

Thus, the v2->v3 update will most likely bring about both patch order
changes, and an increased patch count.

Thanks
Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/14/20 23:26, Bret Barkelew via groups.io wrote:

It’s code management for the Instagram generation
I find this an extremely good characterization!

And, I find the fact soul-destroying.

Laszlo


Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Matthew Carlson
 

Sorry- I wasn't more clear with my initial message. We meant with this forum here. The discussion in the earlier forum brought up several great points and outlined a pretty decent list of must haves in terms of workflow and process. I've been brainstorming some ways to get the lock-step versioning you're describing and I agree that it's quite important.


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

Agreed. Responsibility to verify the commit message when squashing is always something to be aware of.

With Github, the one who presses the “Close and Merge” (or whatever it’s called) button has the final say on the title and message. We can play with approaches to how the squash merge would work.

I would far prefer the approach of individual PRs for commits to allow for the squash flexibility (and is the strategy I think I would pursue with my PRs). For example, the VarPol PR would be broken up into 9 PRs for each final commit, and we can get them in one by one. Ideally, each one would be a small back and forth and then in. If it had been done that way to begin with, it would be over in a week and a half or so, rather than the multiple months that we’re now verging on.

- Bret

From: Michael D Kinney via groups.io<mailto:michael.d.kinney@...>
Sent: Thursday, May 14, 2020 6:19 PM
To: rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@...<mailto:lersek@...>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of
the EDK II commit message requirements. In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
Of Bret Barkelew via groups.io
Sent: Thursday, May 14, 2020 2:27 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; lersek@...;
rfc@edk2.groups.io
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

I feel like this process is a good compromise. It’s not
perfect (frankly, I’m a fan of enforced squash merges,
which can maintain bisectability if managed well), but
it allows for rapid iteration, ease of contribution,
and approaches the workflow that many who have never
used email to maintain a project would be familiar
with.

It’s code management for the Instagram generation, and
I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael
D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>;
devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
lersek@...<mailto:lersek@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch
series submitted as a pull request. Let the community
do a complete review of the content. The submitter can
add patches to the end of the pull request addressing
feedback and can even add patches that make changes to
previous patches until all feedback/conversations are
resolved. This keeps the conversations complete and
the conversations will also be archived to the email
archive.

At this point, the developer can reformulate the patch
series and do forced push of V2. Reviewers can review
the cleaned up patch series and repeat the process if
there is more feedback, or move to final approval.

By doing all the work on a single pull request, we
minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull
request for each new version of the series. This would
preserve the GitHub conversations for each version of
the pull request. All the earlier ones would be
closed/abandoned, and only the final one would be
closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney,
Michael D <michael.d.kinney@...>;
rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force
push on every tweak, we lose the “thread” of discussion
on what caused the change, what changed as a result,
and the easy hook for the original change requester to
reply directly to the change as is.

- Bret

From: Laszlo Ersek via
groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
Kinney, Michael D<mailto:michael.d.kinney@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlo%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611242055&amp;sdata=Gr4Mw1Yz36xH3riEM7yicL5DIKv1%2FVuXM%2FObxAIWbJo%3D&amp;reserved=0
ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
qQxtLpwH0%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Development-
Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
re.github.io%2Fwiki%2FEDK-II-Development-
Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
contributors-and-
maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outl%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2FfG2%2BrXhM02OXgg%2Fa1G8tBYx0mHIH8lTm%2FF7bKSh10M%3D&amp;reserved=0
ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
for-edk2-contributors-and-
maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
d=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Message-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Message-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Signature-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Signature-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

*
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
j%2FPCToc%3D&reserved=0>)
* [edk2-
platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=B0BrCucxkG8t6JzA0b113MLW5PocHmW54lhtOAQdR%2F4%3D&amp;reserved=0.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
)
* [edk2-non-
osi](https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
ved=0<https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-
test](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
ed=0<https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-
libc](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-
staging](https://nam06.safelinks.protection.outlook.com/
/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
eserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.co%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=bR1nvNPVi%2BQ1NrpSJQlwdLTZB3SigLFQ3KV63kUTUzk%3D&amp;reserved=0
m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
started-with-github%2Ffork-a-
repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
ng-started-with-github%2Ffork-a-
repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Fcreating-a-pull-
request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
aborating-with-issues-and-pull-requests%2Fcreating-a-
pull-
request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki
article at some point.)


* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all

s/make add/add/

changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.
Do I understand correctly that this recommends the
contributor first
push incremental patches on top of the series, then do
a rebase
(squashing updates as necessary) and finally do a
force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to
locally comparing
the v(n) patch set to v(n+1) with git-range-diff,
and/or with some
personal scripts. I wouldn't encourage incremental
patches appended --
even temporarily -- to the branch, because (a) it's
extra review work
(it requires me to review something that has zero
chance to get into the
git history as-is), and (b) it superficially resembles
the
github.com-specific bad practice called "squash on
merge", and (c) it
runs the risk that the maintainer responsible for
ultimately merging the
series ends up actually merging the incremental (=
"fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set?
Labels?

Not sure about the best tooling. My recommendation
would be to require
reviewers to start providing their feedback within one
week.

One thing that I find important is that a maintainer
can signal "I got
your work in my queue, but I may need more time". And a
special case of
that are automated out-of-office responses. I think
they are very
helpful (when a contributor feels they are bottlenecked
on review), but
I'm not sure how one can configure that via github. I
certainly would
not share my out-of-office times with github. (I set
the start/end dates
in my email infrastructure, at the moment, but the out-
of-office
messages it sends do not contain the dates either, on
purpose.)


## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root

s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails

s/sends/send/

are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email

s/maintainer ship/maintainership/

notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.ou%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=rMEZi8aziILY8pK5e%2BvnY3TgsvV4EqzQ2qXf8QvaqIc%3D&amp;reserved=0
tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
hub%2Fmanaging-subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
0>

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>

Important: as the name says ("-poc"), this is a Proof
of Concept list,
for now. Once we're ready to switch over, I'll file an
internal ticket
at RH to either rename the list, or (which is probably
better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the
webhook development
/ bugfixing (if any) could perhaps occur in parallel to
the normal edk2
workflow.


* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Freviewing-changes-in-
pull-
requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.c%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=%2BfY0MvXLLyf%2Bt7w2VFkWgfnEGFjgqdJo7WsZmQN8yLE%3D&amp;reserved=0
om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
ollaborating-with-issues-and-pull-requests%2Freviewing-
changes-in-pull-
requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as
above)


* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney
<michael.d.kinney@...<mailto:michael.d.kinney@int
el.com>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
archive-
webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-
poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.o%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ybz14MP2cfsQUGM2JxpsZNq2q%2BZU0e835ZNpKpV9M9Y%3D&amp;reserved=0
utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
2Fprivate%2Ftianocore-code-review-
poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
served=0>

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611252057&amp;sdata=ug4axzMxZLKuGz8Eps6xepyEjv1er8atkiOZDpFca2c%3D&amp;reserved=0
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
z14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
b9A%3D&reserved=0>

## Example patch series with < 10 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.p%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=7pNJG%2BcGgplnp0gOQmPx8ly3IgxLk7DBPW0liL%2BQ97c%3D&amp;reserved=0
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
A%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
pXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
igs%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
zAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protectio%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=g37erq7Um78njCs8L%2BZ75MBqyGEoZrIEk1epGOJ5va0%3D&amp;reserved=0
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
eserved=0>

## Example patch series with > 80 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=oLtR%2Br3OOn3hqOQ6glrp56aXidbNCpd%2F8w06mY8j7dA%3D&amp;reserved=0.
outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
erved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protect%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=GcYd84S79b4L09WB1b40cfbx5y%2FREQnWZsljxNeAm2U%3D&amp;reserved=0
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.prote%2F&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C4235599e115f484ba25c08d7f86dfde9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637251023611262052&amp;sdata=m34ZxHM9zr3VFz25TIiYpmxnqR0hPT96UIJppWgU4os%3D&amp;reserved=0
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please
consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.

Hmmm, good idea in general, but there have been valid
exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the
webhook)!
Laszlo







Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Bret,

If the original submission is a single patch, and the code review
generates changes that are added as additional patches for review,
but the intent in the end is still a single patch, then squashing
them all at the end is correct.

Using the GitHub feature to squash them is a challenge because of
the EDK II commit message requirements. In order to make sure the
final commit message for the one commit is correct, the developer
should squash and edit the commit message.

There may be opportunities to automate these operations, but the
developer still needs to do the final review and have the ability
to do further edits of the commit message.

Mike

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf
Of Bret Barkelew via groups.io
Sent: Thursday, May 14, 2020 2:27 PM
To: Kinney, Michael D <michael.d.kinney@...>;
devel@edk2.groups.io; lersek@...;
rfc@edk2.groups.io
Subject: Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

I feel like this process is a good compromise. It’s not
perfect (frankly, I’m a fan of enforced squash merges,
which can maintain bisectability if managed well), but
it allows for rapid iteration, ease of contribution,
and approaches the workflow that many who have never
used email to maintain a project would be familiar
with.

It’s code management for the Instagram generation, and
I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael
D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>;
devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
lersek@...<mailto:lersek@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney,
Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch
series submitted as a pull request. Let the community
do a complete review of the content. The submitter can
add patches to the end of the pull request addressing
feedback and can even add patches that make changes to
previous patches until all feedback/conversations are
resolved. This keeps the conversations complete and
the conversations will also be archived to the email
archive.

At this point, the developer can reformulate the patch
series and do forced push of V2. Reviewers can review
the cleaned up patch series and repeat the process if
there is more feedback, or move to final approval.

By doing all the work on a single pull request, we
minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull
request for each new version of the series. This would
preserve the GitHub conversations for each version of
the pull request. All the earlier ones would be
closed/abandoned, and only the final one would be
closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney,
Michael D <michael.d.kinney@...>;
rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc]
GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force
push on every tweak, we lose the “thread” of discussion
on what caused the change, what changed as a result,
and the easy hook for the original change requester to
reply directly to the change as is.

- Bret

From: Laszlo Ersek via
groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
Kinney, Michael D<mailto:michael.d.kinney@...>;
rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub
Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaste
r%2FReadMe.rst&amp;data=02%7C01%7Cbret.barkelew%40micro
soft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf8
6f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&am
p;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&
amp;reserved=0<https://nam06.safelinks.protection.outlo
ok.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2
%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barke
lew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482659
51667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6
qQxtLpwH0%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FEDK-II-Development-
Process&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianoco
re.github.io%2Fwiki%2FEDK-II-Development-
Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0
JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-
contributors-and-
maintainers&amp;data=02%7C01%7Cbret.barkelew%40microsof
t.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f1
41af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;s
data=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D
&amp;reserved=0<https://nam06.safelinks.protection.outl
ook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftia
nocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-
for-edk2-contributors-and-
maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.co
m%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2
qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserve
d=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Message-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Message-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjB
SMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io
%2Fwiki%2FCommit-Signature-
Format&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com
%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af9
1ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=
Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocor
e.github.io%2Fwiki%2FCommit-Signature-
Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb
61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2
d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8
wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

*
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJ
j%2FPCToc%3D&reserved=0>)
* [edk2-
platforms](https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&amp;data=02%7C01%7Cbret.barkelew%40microsoft.
com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sda
ta=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp
;reserved=0<https://nam06.safelinks.protection.outlook.
com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%
7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91
ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8Ns
NTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>
)
* [edk2-non-
osi](https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lr
EsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reser
ved=0<https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-
osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ
2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-
test](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v
205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserv
ed=0<https://nam06.safelinks.protection.outlook.com/?ur
l=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6M
gfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-
libc](https://nam06.safelinks.protection.outlook.com/?u
rl=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tz
t293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjz
DwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-
staging](https://nam06.safelinks.protection.outlook.com
/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata
=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;r
eserved=0<https://nam06.safelinks.protection.outlook.co
m/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-
staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40
XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&amp;dat
a=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b
544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db4
7%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5
Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https:/
/nam06.safelinks.protection.outlook.com/?url=https%3A%2
F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.B
arkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebe
b5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248
265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9J
zgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-
started-with-github%2Ffork-a-
repo&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7
C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=um
I3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;rese
rved=0<https://nam06.safelinks.protection.outlook.com/?
url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetti
ng-started-with-github%2Ffork-a-
repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61
ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7
cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90Z
NlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Fcreating-a-pull-
request&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcoll
aborating-with-issues-and-pull-requests%2Fcreating-a-
pull-
request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeR
ye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

s/decryption/description/

(Because I'm assuming this will turn into a wiki
article at some point.)


* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all

s/make add/add/

changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.
Do I understand correctly that this recommends the
contributor first
push incremental patches on top of the series, then do
a rebase
(squashing updates as necessary) and finally do a
force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to
locally comparing
the v(n) patch set to v(n+1) with git-range-diff,
and/or with some
personal scripts. I wouldn't encourage incremental
patches appended --
even temporarily -- to the branch, because (a) it's
extra review work
(it requires me to review something that has zero
chance to get into the
git history as-is), and (b) it superficially resembles
the
github.com-specific bad practice called "squash on
merge", and (c) it
runs the risk that the maintainer responsible for
ultimately merging the
series ends up actually merging the incremental (=
"fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set?
Labels?

Not sure about the best tooling. My recommendation
would be to require
reviewers to start providing their feedback within one
week.

One thing that I find important is that a maintainer
can signal "I got
your work in my queue, but I may need more time". And a
special case of
that are automated out-of-office responses. I think
they are very
helpful (when a contributor feels they are bottlenecked
on review), but
I'm not sure how one can configure that via github. I
certainly would
not share my out-of-office times with github. (I set
the start/end dates
in my email infrastructure, at the moment, but the out-
of-office
messages it sends do not contain the dates either, on
purpose.)


## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root

s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails

s/sends/send/

are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email

s/maintainer ship/maintainership/

notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-
subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&amp;data=02%7C01%7Cbret.barkelew%40micros
oft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86
f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp
;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%
3D&amp;reserved=0<https://nam06.safelinks.protection.ou
tlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgit
hub%2Fmanaging-subscriptions-and-notifications-on-
github%2Fconfiguring-
notifications&data=02%7C01%7CBret.Barkelew%40microsoft.
com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141
af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=F
JwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=
0>

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3K
oiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>

Important: as the name says ("-poc"), this is a Proof
of Concept list,
for now. Once we're ready to switch over, I'll file an
internal ticket
at RH to either rename the list, or (which is probably
better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the
webhook development
/ bugfixing (if any) could perhaps occur in parallel to
the normal edk2
workflow.


* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaboratin
g-with-issues-and-pull-requests%2Freviewing-changes-in-
pull-
requests&amp;data=02%7C01%7Cbret.barkelew%40microsoft.c
om%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141a
f91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdat
a=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;
reserved=0<https://nam06.safelinks.protection.outlook.c
om/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fc
ollaborating-with-issues-and-pull-requests%2Freviewing-
changes-in-pull-
requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7
Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91a
b2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2
fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as
above)


* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney
<michael.d.kinney@...<mailto:michael.d.kinney@int
el.com>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-
webhook&amp;data=02%7C01%7Cbret.barkelew%40microsoft.co
m%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af
91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata
=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;res
erved=0<https://nam06.safelinks.protection.outlook.com/
?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-
archive-
webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7C
b61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcw
fqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianoc
ore-code-review-
poc&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C
1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab
2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0I
uvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserve
d=0<https://nam06.safelinks.protection.outlook.com/?url
=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fti
anocore-code-review-
poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61c
e42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7c
d011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSf
EobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-
poc%2Findex.html&amp;data=02%7C01%7Cbret.barkelew%40mic
rosoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988b
f86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&
amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M
%3D&amp;reserved=0<https://nam06.safelinks.protection.o
utlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%
2Fprivate%2Ftianocore-code-review-
poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microso
ft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f
141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdat
a=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&re
served=0>

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300289&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzh
kpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.prote
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pw
z14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fthread.html%2300340&amp;data=02%7C01%7Cbret.barke
lew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%
7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275
86678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtj
dFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.p
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%
40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72
f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63724826595177
7024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUk
b9A%3D&reserved=0>

## Example patch series with < 10 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00289.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4
ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.p
rotection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com
%2Fmailman%2Fprivate%2Ftianocore-code-review-
poc%2F2020-
May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6h
A%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00030.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gF
pXA%3D&amp;reserved=0<https://nam06.safelinks.protectio
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&s
data=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiA
igs%3D&reserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00018.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9
lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protect
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00008.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNT
zAE%3D&amp;reserved=0<https://nam06.safelinks.protectio
n.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailm
an%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&s
data=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&r
eserved=0>

## Example patch series with > 80 patches

*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00198.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d
4%3D&amp;reserved=0<https://nam06.safelinks.protection.
outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman
%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&res
erved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00116.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV
7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protect
ion.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmai
lman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&s
data=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&r
eserved=0>
*
https://nam06.safelinks.protection.outlook.com/?url=htt
ps%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianoco
re-code-review-poc%2F2020-
May%2Fmsg00035.html&amp;data=02%7C01%7Cbret.barkelew%40
microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f9
88bf86f141af91ab2d7cd011db47%7C1%7C0%7C6372482275866786
40&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%
2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.prote
ction.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fm
ailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-
May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40micr
osoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf
86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&s
data=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D
&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please
consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.

Hmmm, good idea in general, but there have been valid
exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the
webhook)!
Laszlo







Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

I feel like this process is a good compromise. It’s not perfect (frankly, I’m a fan of enforced squash merges, which can maintain bisectability if managed well), but it allows for rapid iteration, ease of contribution, and approaches the workflow that many who have never used email to maintain a project would be familiar with.

It’s code management for the Instagram generation, and I for one welcome our new insect overlords.

- Bret

From: Kinney, Michael D<mailto:michael.d.kinney@...>
Sent: Monday, May 11, 2020 1:43 PM
To: Bret Barkelew<mailto:Bret.Barkelew@...>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; lersek@...<mailto:lersek@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request. Let the community do a complete review of the content. The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved. This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2. Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series. This would preserve the GitHub conversations for each version of the pull request. All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@...>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@...; Kinney, Michael D <michael.d.kinney@...>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek@...>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@...>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=S%2Fu9iHwOFHKtYL7jeqIVGZLwDRbG%2F8%2BUm6qQxtLpwH0%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951667090&sdata=l4uZzb0JliBkFXCQ7YfNBXs3Aoky0RYQn5gVT34AlH4%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=Wo2qJFt7cHi5zZS96kCml7MZI%2B32v%2FiRqPdICvpTw5c%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951677089&sdata=RjfxykjBSMEU%2BqsYkAmDPl%2FIgvBTPx%2BCvSIOPexpcc8%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951687191&sdata=jPadwqi8wSOKaVOKU3o2JOegzvTqdz8o7bSLyT%2B0El8%3D&reserved=0>

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=YhFQ9Fxt8Y0kFdOTgY2v9vML9uCOpS9j5cKJj%2FPCToc%3D&reserved=0>)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951697080&sdata=dG8NsNTDjSX05wQWXargPnJydEGQyvZCia%2BSOIUyG6o%3D&reserved=0>)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=Mkz1CUW26UJ2X6wIEgO0UGz0ZJqWhkXMFwc5v5420tU%3D&reserved=0>)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951707067&sdata=fzDqTRet6MgfwmVIq8mb5%2BgB3rAiVteCQklombfBMaU%3D&reserved=0>)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=wpaAwogDjzDwRzpoB5vInxeqtMkNoawyPFg1j3m9omw%3D&reserved=0>)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951717077&sdata=5ehzG40XuHbF56QwzFHyY8krO1NwGDWnFVMZdZ9OUvM%3D&reserved=0>)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951727054&sdata=IrlV0ptekGi4Sz%2FxBYhIa5LiRuI3jlKE9JzgwbKGP6o%3D&reserved=0>)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=pC%2F9W90ZNlmGQFNy97ee1xQMJ93XC%2Bdo5lmIdZhnRk0%3D&reserved=0>

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951737053&sdata=sZjbyeRye35une4C2y5RQ83ah%2B01o4XhFNXsvIXT1l0%3D&reserved=0>

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=FJwHjpFL7UFtZ9N12pSvQSXci3IqCJJVJXI9dDzlHHU%3D&reserved=0>

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951747044&sdata=zbDuJd9lW3KoiuzGo6%2BU14Mnude7lZnbCeXQHJnR%2BCg%3D&reserved=0>
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=Iav2d2fP2vw9mS9uA76tN3uQcIbl52D6BT963cD7ZC4%3D&reserved=0>

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@...<mailto:michael.d.kinney@...>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951757040&sdata=FihVwcwfqewohsmqUECWG36TR1iCQVhVHq02iMZkEL8%3D&reserved=0>

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=482jeizloSfEobAk0F8%2BvP9FZptdF6oC90A%2BYSQZcds%3D&reserved=0>

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951767033&sdata=78HRihY2z%2Fll0qoC05RMh8I%2Bwri9rwIwblxNBYFPecw%3D&reserved=0>

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=%2BU8aFUi4kPBeJdlyW%2FERNtp5Fum5gZrWkgCTI8pwz14%3D&reserved=0>

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951777024&sdata=gXn%2B9z8MIdnJyUkk2xR1mK%2FL710jUDEeNhK74EUkb9A%3D&reserved=0>

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=yvE8%2F%2FKOJUj%2BD4ZmtT3fZPAb2KBsGcAqxt%2FKwRKa6hA%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951787022&sdata=zz5BgISidxyQVI%2BQihn%2Fq%2Fv9%2B5%2Fn2vpaeyvFbIiAigs%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=yN0WkrPdk1EEvyPwlqCG%2B6HTOpa3iT85w2QhXQJ2xYE%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951797016&sdata=qF2vw8XdQfY0ECclzhbpd60vIgTj2Bzl5RjC%2F4EdbA8%3D&reserved=0>

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=uXAPx9iTDbQbiv40j3EiqIP51nlFLQ43ghvAsjt4dhs%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951807013&sdata=4ANIbH2YDPjN7H4gHGtbgNkAbM9EeVFOf%2FoWqTiXtiA%3D&reserved=0>
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&data=02%7C01%7CBret.Barkelew%40microsoft.com%7Cb61ce42027c0428ab31408d7f5ebeb5b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248265951817004&sdata=1jX%2FuPSMlRvzN9eZcv5U7k4JmwfJJwaVhDJ%2BrA6mPYc%3D&reserved=0>

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo

481 - 500 of 789