Date   

Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Laszlo Ersek
 

Hi Matthew,

On 05/13/20 00:40, Matthew Carlson via groups.io wrote:
Since we haven't had any feedback and the deadline is quickly approaching. We are going to move ahead by creating a new repo inside of TianoCore and creating patches post-stable tag and submit them to the mailing list as soon as the stable tag is made.

If any other comments of feedback, feel free to chime in. If anyone has any basetools python code changes, please coordinate with us as the patch will be removing Basetools inside of the EDK2 repo. We want to make sure no changes are lost or misplaced.
I didn't provide any feedback in this specific thread because I thought
our discussion earlier was sufficient feedback from me.

(just commenting on the particular "we haven't had any feedback" bit)

Thanks,
Laszlo


Re: [RFC] BaseTools/Source/Python as a standalone python package in independent repo

Matthew Carlson
 

Since we haven't had any feedback and the deadline is quickly approaching. We are going to move ahead by creating a new repo inside of TianoCore and creating patches post-stable tag and submit them to the mailing list as soon as the stable tag is made.

If any other comments of feedback, feel free to chime in. If anyone has any basetools python code changes, please coordinate with us as the patch will be removing Basetools inside of the EDK2 repo. We want to make sure no changes are lost or misplaced.


Re: [RFCv2] code-first process for UEFI-forum specifications

Leif Lindholm
 

On Wed, Mar 25, 2020 at 05:14:59 +0000, Ni, Ray wrote:

## Github
New repositories will be added for holding the text changes and the source code.

Specification text changes will be held within the affected source repository,
in the Github flavour of markdown, in a file (or split across several files)
with .md suffix.
What's the case when multiple .MD files are needed?
For example if a branch covers changes to multiple specifications, as
described elsewhere. Or if it simply makes sense due to content size.
It is possible, now we've migrated to .rst for edk2, that we should
change the format recommentded in this proposal too.

(This one may break down where we have a specification change affecting multiple
specifications, but at that point we can track it with multiple BZ entries)


## Source code
In order to ensure draft code does not accidentally leak into production use,
and to signify when the changeover from draft to final happens, *all* new or
modified[1] identifiers need to be prefixed with the relevant BZ####.

[1] Modified in a non-backwards-compatible way. If, for example, a statically
sized array is grown - this does not need to be prefixed. But a tag in a
comment would be *highly* recommended.
If a protocol is enhanced to provide more interfaces with increased revision number,
would you like the protocol name to be prefixed with BZ####?
Or just the new interfaces added to the protocol are prefixed the BZ####?
I think just prefixing the new interfaces can meet the purpose.
Adding new interfaces to a protocol does not affect its
backwards-compatibility, which was what I was trying to cover above.
If you can think of a better way of describing it. I am very open to
suggestions.

But the protocol definition is changed, it also needs to be prefixed according to this flow.
Can you clarify a bit more?
In that instance, only the new interfaces would need the prefix.


### File names
New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
Private header files do not need the prefix.

### Contents

The tagging must follow the coding style used by each affected codebase.
Examples:

| Released in spec | Draft version in tree | Comment |
| --- | --- | --- |
| `FunctionName` | `Bz1234FunctionName` | |
| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |
If FunctionName or HEADER_MACRO is defined in non-public header files, I don't
think they require the prefix. Do you agree?
Only public interfaces need prefix. This also means that non-public
interfaces should be STATIC where possible.

For data structures or enums, any new or non-backwards-compatible structs or
fields require a prefix. As above, growing an existing array in an existing
struct requires no prefix.

| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2] |
| `StructField` | `Bz1234StructField` | In existing struct[3] |
| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2] |

[2] If the struct or enum definition is separate from the typedef in the public
header, the definition does not need the prefix.
What does "separate" mean?
Does it mean "struct or enum in the public header BzXXX.h don't need the prefix"?
If yes, then I think macros defined in BzXXX.h also don't need the prefix.
Struct or enum definitions in the public header BzXXX.h don't need the
prefix *when they have a public-facing typedef with the prefix*.
Everything new or not-backwards-compatible needs to be referred to via
the prefixed names in external modules.

I.e. we can have
struct _SomeNewThing {
};

typedef struct _SomeNewThing BzXXX_PROTOCOL;

(This is meant simply as shorthand, reducing the amount of changes
required for the published version.)

[3] Individual fields in newly added typedefd struct do not need prefix, the
struct already carried the prefix.

Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.

| `gSomeGuid` | `gBz1234SomeGuid` | |

Local identifiers, including module-global ones (m-prefixed) do not require a
BZ prefix.
I think only the names (struct type name, enum type name, interface name, protocol/ppi name)
defined in public header files need the BZ prefix when the public header doesn't have prefix.
Right?
That is one way we *could* do it. It is not one I am proposing.
My idea is that it should be very clear from looking at code whether
it includes non-ratified proposal code.

/
Leif


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/11/20 22:09, Bret Barkelew wrote:
As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread� of discussion on what caused the change,
This is a github.com limitation.

And the email archive mitigates it.

In the current process, when I review v2 of a 10-part series, I have one
Thunderbird window open with the v1 thread, containing both the v1
patches and my (and others') review comments given for them.

(I open the new window by right-clicking the v1 blurb, and then
selecting "open message in new window". Then I navigate between the
messages of the v1 thread with the "f" and "b" hotkeys. The "scope" of
the new window is set to the v1 thread, recursively, when I open the new
window like that, and so "f" and "b" just do the right thing.)

In another window, to the right side, I run "git-range-diff", to
interdiff the v1 patches (patch by patch) with the v2 patches. (An
interdiff is a diff of diffs.) Importantly, the interdiff also
highlights commit message differences.

I verify that all the feedback comments from the v1 thread have been
addressed (per patch), and also that any otherwise "uncalled-for"
changes in v2 are in fact justified. (The contributor may have
justifiedly implemented further changes than what I requested under v1.)
This is also the reason why I meticulously number my feedback comments,
as I'm going to require a complete (one by one) coverage in the next
version of the patch set. (Except for those comments of course that the
contributor successfully refutes.)

When the v2 series has different structure from v1, then git-range-diff
is not as helpful -- in that case, I compare only a subset of the
patches like described above, and the entirely new patches in v2 I have
to review from zero.

The entire process depends on having unfettered access to comments given
for *any* earlier version of the patch set (it's not uncommon that I
refer back to v(n-3) or v(n-2) when reviewing v(n)), with those comments
being tightly bound (for display and for re-reading) to their subject
patches.

The github webui destroys (at least visually) the comments given before
a force-push. I can't fathom how incremental reviews can work on
github.com *at all*, in other projects. Hence my earlier suggestion to
use new pull requests rather than force-pushes.

But the mailing list archive generated by the webhook will solve this
completely -- I will use that list as a primary review support tool (for
v2, v3, ...), not only as an archive.


... After all, I guess I could reformulate like this: it's not my intent
to prevent people from pushing incremental fixups *temporarily*; I'm
only saying I will ignore those patches, and I will review only the next
full version of the branch.


My concern that does persist is this: "it runs the risk that the
maintainer responsible for ultimately merging the series ends up
actually merging the incremental (= "fixup") patches in isolation
(without squashing them)".

The git history should neither be littered with fixup patches, nor
contain huge squashes. The structure of a patch series is a first class
trait; it is an aspect to iterate upon, when a branch is being
contributed. The tooling should support that. (And the list traffic
generated by the webhook does.)

For instance, the last time I've given feedback regarding patch series
structure was just an hour ago, under the series "[PATCH V4 00/27]
Disabling safe string constraint assertions". I requested moving a hunk
from patch#1 to patch#26. Having the hunk in patch#1 does not break
bisection, and it's irrelevant for the end-state after the whole series
is applied (the end-state is the same). But the hunk still doesn't
*belong* in patch#1 -- wherever we add a new bit to a bitmask PCD
(patch#26), the UNI file (= documentation) udpate belongs in the exact
same patch.

what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.
No matter what I say about an incremental/fixup patch in isolation,
things can easily go wrong when the contributor squashes the fixup into
the more substantial patch that needs the fixup. Not to mention any
commit message updates on the more substantial patch, as necessitated by
the fixup. So I'll have to review the next full version of the topic
branch anyway, with git-range-diff, and compare the interdiff against my
earlier feedback.

Thanks!
Laszlo



- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo





Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hi Bret,

This is a good point.

What I am proposing is the first version of the patch series submitted as a pull request. Let the community do a complete review of the content. The submitter can add patches to the end of the pull request addressing feedback and can even add patches that make changes to previous patches until all feedback/conversations are resolved. This keeps the conversations complete and the conversations will also be archived to the email archive.

At this point, the developer can reformulate the patch series and do forced push of V2. Reviewers can review the cleaned up patch series and repeat the process if there is more feedback, or move to final approval.

By doing all the work on a single pull request, we minimize the total number of pull requests in the repo.

An alternative approach would be to open a new pull request for each new version of the series. This would preserve the GitHub conversations for each version of the pull request. All the earlier ones would be closed/abandoned, and only the final one would be closed/merged.

Best regards,

Mike

From: Bret Barkelew <Bret.Barkelew@microsoft.com>
Sent: Monday, May 11, 2020 1:10 PM
To: devel@edk2.groups.io; lersek@redhat.com; Kinney, Michael D <michael.d.kinney@intel.com>; rfc@edk2.groups.io
Subject: RE: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Bret Barkelew <bret.barkelew@...>
 

As a counterpoint: if we force a new branch or force push on every tweak, we lose the “thread” of discussion on what caused the change, what changed as a result, and the easy hook for the original change requester to reply directly to the change as is.

- Bret

From: Laszlo Ersek via groups.io<mailto:lersek=redhat.com@groups.io>
Sent: Monday, May 11, 2020 12:39 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; rfc@edk2.groups.io<mailto:rfc@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FReadMe.rst&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=lVjWRLsBC3xJpyRFeDrGjFhMOzAgi2V3vsAPxj7lIDw%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Development-Process&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=sgAhQxCpyjmzC%2FW%2BFiLLwaF2M8wscBz3k93ne25qUXs%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=eHP9fcPMw6yjqTU%2B%2BUZ3FZkq8jZeM1LTU6dGTzmFp4Q%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Message-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=uq8G6nGyLpa7m%2F0fD2pwrcM9uixbKs6SLTge8e77M%2FY%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FCommit-Signature-Format&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Mz8dUn2L8dFwJdlo4LbaIKt2JrWE%2Fn4tBtVWenK%2F8Ck%3D&amp;reserved=0

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)
* [edk2-platforms](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-platforms&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=g8mgGL6B%2FRsvm3935OpZMctOTKUoeHGi8jPuCVKQjbI%3D&amp;reserved=0)
* [edk2-non-osi](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-non-osi&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=9lrEsZWOpc3wqylKs7yF%2FzxYwZsUUamP3oUrWDWcHCc%3D&amp;reserved=0)
* [edk2-test](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-test&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=8v205MD3HTYg3yLmGJS3SIDA5um9sVJfOa5CXViZjyU%3D&amp;reserved=0)
* [edk2-libc](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-libc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Tzt293HJzFnGSkh1mUBew8dAsaZ4axWq2ml8UhQ%2FSTI%3D&amp;reserved=0)
* [edk2-staging](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2-staging&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=bcNbt7Y7KoBrcnW4fAc4jbGgJL%2B4lYUkVLhYNo37OiM%3D&amp;reserved=0)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586668645&amp;sdata=Jvbl8ypdXIi7U5Jnr3s0TOx6hD54N55mdsbXi9sCznM%3D&amp;reserved=0)

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fgetting-started-with-github%2Ffork-a-repo&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=umI3eqOh03qmt9YlPo33ujypu90YwImAvuxh5SlrM%2Bw%3D&amp;reserved=0

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Fcreating-a-pull-request&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=2GVrQy0FGwd4CCeGveh99HL3zS1ekRfAAaKhhRiOMpU%3D&amp;reserved=0

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fmanaging-subscriptions-and-notifications-on-github%2Fconfiguring-notifications&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=OlkiyymcQi39P8%2FOJZG4yjh4h%2FHerkHBe5bCSQQFLOU%3D&amp;reserved=0

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.github.com%2Fen%2Fgithub%2Fcollaborating-with-issues-and-pull-requests%2Freviewing-changes-in-pull-requests&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=51Ljm3wUbBTWT8hcaBD1ZQznSROvAQqnoTzQmD6K%2FLY%3D&amp;reserved=0

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2-email-archive-webhook&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=7CJNJMEXrxoynjavmEwjzUyRbfNUIZ3FEG4kDRXvhI4%3D&amp;reserved=0

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Ftianocore-code-review-poc&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=q0IuvS318pEkJU2td9xX87oIm0LbSlEvOvhpyOOFrE8%3D&amp;reserved=0

The email archives are at this link:

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2Findex.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=nedUfkmMmI5T6GtAxQCW4q6xt38%2FezeDYmfq6cpRD0M%3D&amp;reserved=0

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300289&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=GtrEudehfXiSU6ZwH2zKO35CPPPVk0ctZIzhkpI6DkE%3D&amp;reserved=0

## Example patch series with 1 patch

https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fthread.html%2300340&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=ZGpI8%2BzIA9OMFm3pSCc2DQ4F5ZxtDSAXtjdFjD%2BY3NA%3D&amp;reserved=0

## Example patch series with < 10 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00289.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=JyaUyvYfZD7b%2F2wN%2BpS%2B68b%2BwyKoZ3Rba4ol%2FyahQVU%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00030.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=bQHIJIQq4Pri8iK3vPxMDMWz%2BKtXcyuPdhr8y7gFpXA%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00018.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=uMIRGOq%2BVCOSwDzXkG4yueYS4ZJ7BWfsp3Z4%2B9lh6hE%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00008.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=3CBkdqDxRt4IxtECpWQdKJL%2Bf4HFqqHCXo4loxNTzAE%3D&amp;reserved=0

## Example patch series with > 80 patches

* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00198.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=fDfQnifOMzyzLMdP4xH8koKCiSj7ZiuYyrrSZXTf3d4%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00116.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=lcxA3tTna%2BdmTpcNMmPlS%2B47llMAcIEjhCEqxV7TDOc%3D&amp;reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Fprivate%2Ftianocore-code-review-poc%2F2020-May%2Fmsg00035.html&;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C1dcf1f8c03b544e5095408d7f5e2fd56%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637248227586678640&amp;sdata=CgvZ8e%2B7L4nacvRE35KqEyC%2F1CjDYP6wI10qn%2BoX39Y%3D&amp;reserved=0

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/11/20 03:37, Michael D Kinney wrote:

There was feedback from Laszlo related to rebase for
pull requests using the current CI process.
To clarify, I don't think we should allow any github-side automatism to
auto-rebase pull requests. I think such rebases need to occur on
personal developer machines, under human oversight, and then resubmitted
(likely: force-pushed). My request is that the build costs (time,
energy) associated with such force-pushes be reduced somehow.

For example, on a local machine, the following sequence:

$ git checkout master
$ git pull
$ git rebase -i master my_topic_branch
$ build ...

would trigger an incremental build. *.c files not touched by either
operation would not have to be re-built (assuming their dependencies
didn't change either, such as lib class headers, protocol headers, ...)

Thanks,
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/10/20 23:43, Rebecca Cran wrote:
Mike,

On 5/10/20 3:29 PM, Michael D Kinney wrote:

There is no difference between CI checks run during code review
and the final CI checks before merge.  I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.
I'd suggest following what other Github projects do, which I think is to
run the CI checks automatically on every change that's made in a pull
request - I don't know if it might also be necessary to run them during
the merge, if master has changed in the meantime. That gives the
_submitter_ feedback about any changes they need to make, instead of
having to wait until the maintainer tells them their change has broken
something: it speeds up the development process.
Build-testing at every stage through a patch series is important for
ensuring bisectability.

But there's a critical ingredient to that: based on the assumption that
our build system / build rules are good, the builds mentioned above
should be *incremental*.

That is, if we have a patch set with 10 patches, then then the first
patch in the series should trigger a complete build, and the 9 later
patches should trigger only incremental builds.

(During a bisection, the same commits wouldn't be visited in that same
order of course, but that's where the sanity of the build system / build
rules comes in! Basically, if your builds succeed with a linear
progression through the series, then the build system / build rules
ought to *guarantee* that the same "tree states" will build
incrementally just fine when visited in any particular order. "git
checkout" updates the relevant files, and the build system should be
able to derive the minimum set of necessary actions.

Anyway, digression ends.)

The incremental nature of builds is important for saving energy, and
also for saving developer time. The above 10-part example series should
not take 10 times as long to build as 10 independent patches, submitted
in isolation. Patches#2 through #10 should only rebuild a few modules
each (unless lib class headers, protocol headers and such are modified).



Mergify is more flexible.  We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing.  GitHub merge button by
default squashes all commits into a single commit.
Wouldn't disabling all but "Allow rebase merging" do the same thing
without the additional potential failure point? Though it sounds like
we've resolved the problems with Mergify, so it's not important.

https://help.github.com/en/github/administering-a-repository/configuring-commit-squashing-for-pull-requests
mergify has been pretty stable for me!

Thanks,
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/10/20 23:29, Michael D Kinney wrote:
Rebecca,

There is no difference between CI checks run during code review
and the final CI checks before merge. I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.

Mergify is more flexible. We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing. GitHub merge button by
default squashes all commits into a single commit.
(

Wow, "squash-on-merge" is even the *default* now? That's terrible.
Unfortunately, github.com sets a very bad example with this, which is
made worse by github's popularity.

How can we expect developers to think about bisectability and patch
series structuring as first class traits of their contributions if
github.com actively educates them to ignore those aspects? Shaking my head.

)

Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/09/20 06:22, Ni, Ray wrote:
Mike,
It's a huge improvement to me as an Outlook user if pull-request-based review is enabled!

Please help me to understand: The pull-request-based review has been enabled naturally when edk2
was migrated to Github. People don't use it because it's not accepted by community. Your process
tries to meet community's needs of achieving all review comments in mails so pull-request-based
review can be accepted by community. Right?

I just subscribed at https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc with
empty password.
I received the confirmation mail and clicked the link in the mail to confirm.
But I waited for ~15 minutes and didn't receive the additional mail containing the auto-generated password.
That's because the proof-of-concept list is currently subscriber-only,
and subscription requests have to be manually approved -- by Phil, or by
me. The PoC list contains a bunch of webhook test messages, and while
they are not secret, they are not useful to the grand public (and
arguably shouldn't be indexed by web search engines either).

Once we go live, the intent is that production list be publicly visible.
(Of course spam could become a problem; we'll see.)

Importantly, I totally don't "insist" that the email archive be hosted
on redhat.com (in fact it's extra moderation work for me, which I don't
necessarily welcome); I just offered because Red Hat associates can
request such public-facing mailing lists if they support relevant open
source development efforts.

The traffic should be federated to multiple lists, preferably, and the
redhat.com-hosted list need not be the primary archive address. Wherever
the primary list will live, we can subscribe the mail-archive.com daemon
to it, too.

I went to https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the mail achieve doesn't need password?
The password protection should remain in place for now, I think. I've
approved your subscription request; sorry about the delay. (I avoid
reading work email on the weekend.)

Thanks!
Laszlo


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Laszlo Ersek
 

On 05/09/20 04:59, Michael D Kinney wrote:
Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://github.com/tianocore/edk2/blob/master/ReadMe.rst
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
* https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.
s/decryption/description/

(Because I'm assuming this will turn into a wiki article at some point.)


* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
s/make add/add/

changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.
Do I understand correctly that this recommends the contributor first
push incremental patches on top of the series, then do a rebase
(squashing updates as necessary) and finally do a force-push, for the
next round of review?

To me as a reviewer, that's extra work. I'm used to locally comparing
the v(n) patch set to v(n+1) with git-range-diff, and/or with some
personal scripts. I wouldn't encourage incremental patches appended --
even temporarily -- to the branch, because (a) it's extra review work
(it requires me to review something that has zero chance to get into the
git history as-is), and (b) it superficially resembles the
github.com-specific bad practice called "squash on merge", and (c) it
runs the risk that the maintainer responsible for ultimately merging the
series ends up actually merging the incremental (= "fixup") patches in
isolation (without squashing them).


**OPEN**: How should minimum review period be set? Labels?
Not sure about the best tooling. My recommendation would be to require
reviewers to start providing their feedback within one week.

One thing that I find important is that a maintainer can signal "I got
your work in my queue, but I may need more time". And a special case of
that are automated out-of-office responses. I think they are very
helpful (when a contributor feels they are bottlenecked on review), but
I'm not sure how one can configure that via github. I certainly would
not share my out-of-office times with github. (I set the start/end dates
in my email infrastructure, at the moment, but the out-of-office
messages it sends do not contain the dates either, on purpose.)


## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
s/cross references/cross reference them/ ?

of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
s/sends/send/

are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
s/maintainer ship/maintainership/

notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc
Important: as the name says ("-poc"), this is a Proof of Concept list,
for now. Once we're ready to switch over, I'll file an internal ticket
at RH to either rename the list, or (which is probably better) to create
a new list (no "-poc" suffix).

The second option seems more useful because then the webhook development
/ bugfixing (if any) could perhaps occur in parallel to the normal edk2
workflow.


* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.
(same question about the incremental fixup patches as above)


* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.
s/generates/generate/

(or s/Assign/Assigns/)


https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
The logging sounds very useful, thank you.

Whenever a log message relates to an email, please consider logging the
message-id of that email, if possible.

* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
Hmmm, good idea in general, but there have been valid exceptions to this
rule.

* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.
Thank you for writing this up (and for implementing the webhook)!
Laszlo


Re: GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hi Ray,

Comments below.

Mike

-----Original Message-----
From: Ni, Ray <ray.ni@intel.com>
Sent: Friday, May 8, 2020 9:23 PM
To: rfc@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@intel.com>; devel@edk2.groups.io
Subject: RE: [edk2-rfc] GitHub Pull Request based Code
Review Process

Mike,
It's a huge improvement to me as an Outlook user if
pull-request-based review is enabled!

Please help me to understand: The pull-request-based
review has been enabled naturally when edk2
was migrated to Github. People don't use it because
it's not accepted by community. Your process
tries to meet community's needs of achieving all review
comments in mails so pull-request-based
review can be accepted by community. Right?
Yes.


I just subscribed at
https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc with
empty password.
I received the confirmation mail and clicked the link
in the mail to confirm.
But I waited for ~15 minutes and didn't receive the
additional mail containing the auto-generated password.
I went to
https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the
mail achieve doesn't need password?
Laszlo has graciously volunteered to help setup this email
subscription service to help evaluate the POC. I will let
him comment on the settings available. He should also be
able to approve your subscription so you can see the archive
and receive any emails generated by the webhook.


Please advise me what else I can try. I am happy to try
as many steps as I can before all the process are
enabled.

Thanks,
Ray


-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On
Behalf Of Michael D Kinney
Sent: Saturday, May 9, 2020 11:00 AM
To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@intel.com>
Subject: [edk2-rfc] GitHub Pull Request based Code
Review Process

Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for
all repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or
Readme.rst at the root of
repositories along with a few Wiki pages:

*
https://github.com/tianocore/edk2/blob/master/ReadMe.rs
t
*
https://github.com/tianocore/tianocore.github.io/wiki/E
DK-II-Development-Process
*
https://github.com/tianocore/tianocore.github.io/wiki/L
aszlo's-unkempt-git-guide-for-edk2-contributors-and-
maintainers
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Message-Format
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Signature-Format

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements.
It does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only.
You will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the
proposed GitHub pull request
based code review process, details on the email
archive service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review
and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests
today to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks
pass. With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set
of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would
like to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-
platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-
non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-
staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-
started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements.
The only change with this
RFC is that the Cc: lines to
maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they
should only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore
edk2/master

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all
changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced
push to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.

**OPEN**: How should minimum review period be set?
Labels?

## TianoCore GitHub Email Archive Webhook Service
Steps
* Receive an event that a new pull request was
opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root
of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual
commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails
are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows
all review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is
merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to
'Watch' the repositories that
you have maintainer ship or review
responsibilities and that email
notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://help.github.com/en/github/managing-
subscriptions-and-notifications-on-github/configuring-
notifications

* Subscribe to the email archive associated with
the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc

* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/reviewing-changes-in-
pull-requests

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at
the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com>
[mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-
webhook

# Email Archive Subscription Service

The emails are being delivered to the following
RedHat email subscription
service. Please subscribe to receive the emails and
to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-
code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/index.html

The following sections show some example pull
requests and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-
codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch
spans more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike




Re: GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hello,

I have added the following repository to TianoCore to
support the evaluation of the GitHub pull request based
code review process and the email archive webbook. This
is a copy of tianocore/edk2 repo as of May 10, 2020.

https://github.com/tianocore/edk2-codereview

I have updated Maintainers.txt in this repo to add
GitHub IDs for the maintainers and reviewers. Please
review these updates to make sure they are correct.

https://github.com/tianocore/edk2-codereview/blob/master/Maintainers.txt

There are a few maintainers and reviewers that I need
GitHub IDs. Please send me your GitHub IDs and I will
complete the update of Maintainers.txt.

M: Chao Zhang <chao.b.zhang@intel.com>
R: Julien Grall <julien@xen.org>
R: Marc-André Lureau <marcandre.lureau@redhat.com>
R: Stefan Berger <stefanb@linux.ibm.com>
R: Liran Alon <liran.alon@oracle.com>
R: Nikita Leshenko <nikita.leshchenko@oracle.com>

Thanks,

Mike

-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@intel.com>
Sent: Friday, May 8, 2020 8:00 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney,
Michael D <michael.d.kinney@intel.com>
Subject: [edk2-rfc] GitHub Pull Request based Code
Review Process

Hello,

This is a proposal to change from the current email-
based code review process to
a GitHub pull request-based code review process for all
repositories maintained
in TianoCore. The current email-based code review
process and commit message
requirements are documented in Readme.md or Readme.rst
at the root of
repositories along with a few Wiki pages:

*
https://github.com/tianocore/edk2/blob/master/ReadMe.rs
t
*
https://github.com/tianocore/tianocore.github.io/wiki/E
DK-II-Development-Process
*
https://github.com/tianocore/tianocore.github.io/wiki/L
aszlo's-unkempt-git-guide-for-edk2-contributors-and-
maintainers
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Message-Format
*
https://github.com/tianocore/tianocore.github.io/wiki/C
ommit-Signature-Format

The goal is to post changes by opening a GitHub pull
request and perform all
code review activity using the GitHub web interface.
This proposal does not
change any licenses or commit message requirements. It
does require all
developers, maintainers, and reviewers to have GitHub
accounts.

One requirement that was collected from previous
discussions on this topic is
the need for an email archive of all patches and code
review activities. The
existing GitHub features to produce an email archive
were deemed insufficient.
A proof of concept of a GitHub webhook has been
implemented to provide the email
archive service. This email archive is read-only. You
will not be able to send
emails to this archive or reply to emails in the
archive.

The sections below provide more details on the proposed
GitHub pull request
based code review process, details on the email archive
service, and a set of
remaining tasks make the email archive service
production quality. It does not
make sense to support both the existing email-based
code review and the GitHub
pull request-based code review at the same time.
Instead, this proposal is to
switch to the GitHub pull request-based code review and
retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today
to run automated
CI checks on the code changes and allows a maintainer
to set the `push` label to
request the changes to be merged if all CI checks pass.
With this proposal,
once the code review is complete and the commit
messages have been updated, the
same pull request can be used to perform a final set of
CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and
the email archive service
over the next two weeks with close of comments on
Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like
to see the community
agree to make this change as soon as all remaining
tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-
platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-
osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-
staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use
[edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of
[edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-
with-github/fork-a-repo

* Create a new branch from edk2/master in personal
fork of edk2 repository.

* Add set of commits for new feature or bug fix to
new branch. Make sure to
follow the commit message format requirements. The
only change with this
RFC is that the Cc: lines to maintainers/reviewers
should **not** be added.
The Cc: lines are still supported, but they should
only be used to add
reviewers that do not have GitHub IDs or are not
members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill
in the pull request title
and decryption information for Patch #0. Do not
leave defaults.

* Do not assign reviewers. The webhook assigns
maintainers and reviewers to
the pull request and each commit in the pull
request.

* If maintainers/reviewers provide feedback that
requires changes, then make
add commits to the current branch with the
requested changes. Once all
changes are accepted on the current branch,
reformulate the patch series and
commit comments as needed for perform a forced push
to the branch in the
personal fork of the edk2 repository. This step
may be repeated if multiple
versions of the patch series are required to
address all code review
feedback.

**OPEN**: How should minimum review period be set?
Labels?

## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull
request and each commit in
the pull request and cross references against
`Maintainters.txt` in the root
of the repository to assign maintainers/reviewers
to the pull request and
each commit in the pull request. Individual commit
assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the
email archive. Emails
are also sent to any Cc: tags in the commit
messages.

* If the author/developer performs a forced push to
the branch in their
personal fork of the edk2 repository, then a new
set of patch review emails
with patch series Vx is sent to the email archive
and any Cc: tags in commit
messages.

* Receive events associated with all code review
activities and generate
and send emails to the email archive that shows all
review comments and
all responses closely matching the email contents
seen in the current email
based code review process.

* Generate and send email when pull request is merged
or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch'
the repositories that
you have maintainer ship or review responsibilities
and that email
notifications from GitHub are enabled. This
enables email notifications
when a maintainer/reviewer is assigned to a pull
request and individual
commits.

https://help.github.com/en/github/managing-
subscriptions-and-notifications-on-github/configuring-
notifications

* Subscribe to the email archive associated with the
TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-
code-review-poc

* Review pull requests and commits assigned by the
TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI
to provide all review
feedback.

https://help.github.com/en/github/collaborating-
with-issues-and-pull-requests/reviewing-changes-in-
pull-requests

* Wait for Author/Developer to respond to all
feedback and add commits with
code changes as needed to resolve all feedback.
This step may be repeated
if the developer/author need to produce multiple
versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by,
Acked-by, and Tested-by
responses on individual commits. Or add Series-
reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull
request.

* Wait for Developer/Author to add tags to commit
messages in the pull request.

* Perform final review of patches and commit message
tags. If there are not
issues, set the `push` label to run final set of CI
checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the
end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com>
[mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request
based on assignments
documented in Maintainers.txt and generates an email
archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat
email subscription
service. Please subscribe to receive the emails and to
be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-
review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/index.html

The following sections show some example pull requests
and code reviews to
help review the generated emails, their contents, and
threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-
review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-
code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of
new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview
repository
* Update BaseTools/Scripts/GetMaintainer.py to be
compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests
and emails
* Determine if webhook requests must be serialized?
Current POC is serialized.
* Make sure webhook has error handling for all
unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7
support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning
maintainers/reviewers.
* Add a simple check that fails if a single patch spans
more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-
by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned
maintainers/reviewers.
Once all commits have required tags, auto update
commit messages in the
branch and wait for maintainer to set the `Push`
label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Rebecca,

I agree that the first version should rerun CI checks
on every time commits are added to a PR or there is a
forced push to the PR.

Perhaps we should use Draft Pull Requests as a way
to indicate the content is not ready for code review
or CI checks yet.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests#draft-pull-requests

We also want emails added to the email archive when
the pull request is either abandoned or merged.
merify can add comments to a PR that are picked up
by the webhook.

I agree with reducing the number of services required.
There was feedback from Laszlo related to rebase for
pull requests using the current CI process. I will
do more investigations of GitHub features, webhook
features, and Mergify features to see if there is
simpler overall solution.

Mike

-----Original Message-----
From: Rebecca Cran <rebecca@bsdio.com>
Sent: Sunday, May 10, 2020 2:44 PM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@intel.com>; rfc@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process

Mike,

On 5/10/20 3:29 PM, Michael D Kinney wrote:

There is no difference between CI checks run during
code review
and the final CI checks before merge. I think it is
an interesting
conversation to decide how many times those CI checks
should be
run and if they should run automatically on every
change during
review or on demand.
I'd suggest following what other Github projects do,
which I think is to
run the CI checks automatically on every change that's
made in a pull
request - I don't know if it might also be necessary to
run them during
the merge, if master has changed in the meantime. That
gives the
_submitter_ feedback about any changes they need to
make, instead of
having to wait until the maintainer tells them their
change has broken
something: it speeds up the development process.

Mergify is more flexible. We want to make sure the
git history
is linear with not git merges and supports both
single patches
and patch series without squashing. GitHub merge
button by
default squashes all commits into a single commit.
Wouldn't disabling all but "Allow rebase merging" do
the same thing
without the additional potential failure point? Though
it sounds like
we've resolved the problems with Mergify, so it's not
important.

https://help.github.com/en/github/administering-a-
repository/configuring-commit-squashing-for-pull-
requests


--
Rebecca Cran


Re: [edk2-devel] [edk2-rfc] GitHub Pull Request based Code Review Process

Michael D Kinney
 

Rebecca,

There is no difference between CI checks run during code review
and the final CI checks before merge. I think it is an interesting
conversation to decide how many times those CI checks should be
run and if they should run automatically on every change during
review or on demand.

Mergify is more flexible. We want to make sure the git history
is linear with not git merges and supports both single patches
and patch series without squashing. GitHub merge button by
default squashes all commits into a single commit.

Thanks,

Mike

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On
Behalf Of Rebecca Cran
Sent: Saturday, May 9, 2020 11:25 AM
To: devel@edk2.groups.io; Kinney, Michael D
<michael.d.kinney@intel.com>; rfc@edk2.groups.io
Subject: Re: [edk2-devel] [edk2-rfc] GitHub Pull
Request based Code Review Process

On 5/8/20 8:59 PM, Michael D Kinney wrote:

* Perform final review of patches and commit
message tags. If there are not
issues, set the `push` label to run final set of
CI checks and auto merge
the pull request into master.
What's the difference between the CI that runs when a
user submits the
Pull Request, and the final CI checks that run before
the request is merged?

Also, I'm wondering why Mergify is being used instead
of the maintainer
hitting the "Merge Pull Request" button, or however
it's worded?


--
Rebecca Cran




Re: GitHub Pull Request based Code Review Process

Ni, Ray
 

Mike,
It's a huge improvement to me as an Outlook user if pull-request-based review is enabled!

Please help me to understand: The pull-request-based review has been enabled naturally when edk2
was migrated to Github. People don't use it because it's not accepted by community. Your process
tries to meet community's needs of achieving all review comments in mails so pull-request-based
review can be accepted by community. Right?

I just subscribed at https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc with
empty password.
I received the confirmation mail and clicked the link in the mail to confirm.
But I waited for ~15 minutes and didn't receive the additional mail containing the auto-generated password.
I went to https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289.
However, the page requires me to enter password.
Can you please change the setting so that viewing the mail achieve doesn't need password?

Please advise me what else I can try. I am happy to try as many steps as I can before all the process are enabled.

Thanks,
Ray

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Michael D Kinney
Sent: Saturday, May 9, 2020 11:00 AM
To: devel@edk2.groups.io; rfc@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: [edk2-rfc] GitHub Pull Request based Code Review Process

Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://github.com/tianocore/edk2/blob/master/ReadMe.rst
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
* https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.

* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.

**OPEN**: How should minimum review period be set? Labels?

## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike




GitHub Pull Request based Code Review Process

Michael D Kinney
 

Hello,

This is a proposal to change from the current email-based code review process to
a GitHub pull request-based code review process for all repositories maintained
in TianoCore. The current email-based code review process and commit message
requirements are documented in Readme.md or Readme.rst at the root of
repositories along with a few Wiki pages:

* https://github.com/tianocore/edk2/blob/master/ReadMe.rst
* https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Development-Process
* https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
* https://github.com/tianocore/tianocore.github.io/wiki/Commit-Signature-Format

The goal is to post changes by opening a GitHub pull request and perform all
code review activity using the GitHub web interface. This proposal does not
change any licenses or commit message requirements. It does require all
developers, maintainers, and reviewers to have GitHub accounts.

One requirement that was collected from previous discussions on this topic is
the need for an email archive of all patches and code review activities. The
existing GitHub features to produce an email archive were deemed insufficient.
A proof of concept of a GitHub webhook has been implemented to provide the email
archive service. This email archive is read-only. You will not be able to send
emails to this archive or reply to emails in the archive.

The sections below provide more details on the proposed GitHub pull request
based code review process, details on the email archive service, and a set of
remaining tasks make the email archive service production quality. It does not
make sense to support both the existing email-based code review and the GitHub
pull request-based code review at the same time. Instead, this proposal is to
switch to the GitHub pull request-based code review and retire the email based
code review process on the same date.

The edk2 repository is using GitHub pull requests today to run automated
CI checks on the code changes and allows a maintainer to set the `push` label to
request the changes to be merged if all CI checks pass. With this proposal,
once the code review is complete and the commit messages have been updated, the
same pull request can be used to perform a final set of CI checks and merge the
changes into the master branch.

I would like to collect feedback on this proposal and the email archive service
over the next two weeks with close of comments on Friday May 22, 2020. If all
issues and concerns can be addressed, then I would like to see the community
agree to make this change as soon as all remaining tasks are completed.

# TianoCore Repositories to enable

* [edk2](https://github.com/tianocore/edk2)
* [edk2-platforms](https://github.com/tianocore/edk2-platforms)
* [edk2-non-osi](https://github.com/tianocore/edk2-non-osi)
* [edk2-test](https://github.com/tianocore/edk2-test)
* [edk2-libc](https://github.com/tianocore/edk2-libc)
* [edk2-staging](https://github.com/tianocore/edk2-staging)

# GitHub Pull Request Code Review Process

**NOTE**: All steps below use [edk2](https://github.com/tianocore/edk2) as an
example. Several repositories are supported.

## Author/Developer Steps
* Create a personal fork of [edk2](https://github.com/tianocore/edk2)

https://help.github.com/en/github/getting-started-with-github/fork-a-repo

* Create a new branch from edk2/master in personal fork of edk2 repository.

* Add set of commits for new feature or bug fix to new branch. Make sure to
follow the commit message format requirements. The only change with this
RFC is that the Cc: lines to maintainers/reviewers should **not** be added.
The Cc: lines are still supported, but they should only be used to add
reviewers that do not have GitHub IDs or are not members of TianoCore.

* Push branch with new commits to personal fork
* Create a pull request against TianoCore edk2/master

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

* If pull request has more than 1 commit, then fill in the pull request title
and decryption information for Patch #0. Do not leave defaults.

* Do not assign reviewers. The webhook assigns maintainers and reviewers to
the pull request and each commit in the pull request.

* If maintainers/reviewers provide feedback that requires changes, then make
add commits to the current branch with the requested changes. Once all
changes are accepted on the current branch, reformulate the patch series and
commit comments as needed for perform a forced push to the branch in the
personal fork of the edk2 repository. This step may be repeated if multiple
versions of the patch series are required to address all code review
feedback.

**OPEN**: How should minimum review period be set? Labels?

## TianoCore GitHub Email Archive Webhook Service Steps
* Receive an event that a new pull request was opened
* Evaluate the files modified by the entire pull request and each commit in
the pull request and cross references against `Maintainters.txt` in the root
of the repository to assign maintainers/reviewers to the pull request and
each commit in the pull request. Individual commit assignments are performed
by adding a commit comment of the following form:

[CodeReview] Review-request @mdkinney

* Generate and sends git patch review emails to the email archive. Emails
are also sent to any Cc: tags in the commit messages.

* If the author/developer performs a forced push to the branch in their
personal fork of the edk2 repository, then a new set of patch review emails
with patch series Vx is sent to the email archive and any Cc: tags in commit
messages.

* Receive events associated with all code review activities and generate
and send emails to the email archive that shows all review comments and
all responses closely matching the email contents seen in the current email
based code review process.

* Generate and send email when pull request is merged or closed.

## Maintainer/Reviewer Steps

* Make sure GitHub configuration is setup to 'Watch' the repositories that
you have maintainer ship or review responsibilities and that email
notifications from GitHub are enabled. This enables email notifications
when a maintainer/reviewer is assigned to a pull request and individual
commits.

https://help.github.com/en/github/managing-subscriptions-and-notifications-on-github/configuring-notifications

* Subscribe to the email archive associated with the TianoCore GitHub Email
Archive Webhook Service.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

* Review pull requests and commits assigned by the TianoCore GitHub Email
Archive Webhook Service and use the GitHub web UI to provide all review
feedback.

https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/reviewing-changes-in-pull-requests

* Wait for Author/Developer to respond to all feedback and add commits with
code changes as needed to resolve all feedback. This step may be repeated
if the developer/author need to produce multiple versions of the patch
series to address all feedback.

* Once all feedback is addressed, add Reviewed-by, Acked-by, and Tested-by
responses on individual commits. Or add Series-reviewed-by, Series-acked-by,
or Series-Tested-by responses to the entire pull request.

* Wait for Developer/Author to add tags to commit messages in the pull request.

* Perform final review of patches and commit message tags. If there are not
issues, set the `push` label to run final set of CI checks and auto merge
the pull request into master.

# Maintainers.txt Format Changes

Add GitHub IDs of all maintainers and reviewers at the end of M: and R: lines
in []. For example:

M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]

# TianoCore GitHub Email Archive Webhook Service

Assign reviewers to commits in a GitHub pull request based on assignments
documented in Maintainers.txt and generates an email archive of all pull request
and code review activities.

https://github.com/mdkinney/edk2-email-archive-webhook

# Email Archive Subscription Service

The emails are being delivered to the following RedHat email subscription
service. Please subscribe to receive the emails and to be able to view the
email archives.

https://www.redhat.com/mailman/listinfo/tianocore-code-review-poc

The email archives are at this link:

https://www.redhat.com/mailman/private/tianocore-code-review-poc/index.html

The following sections show some example pull requests and code reviews to
help review the generated emails, their contents, and threading.

## Email Achieve Thread View

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00289

## Example patch series with 1 patch

https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/thread.html#00340

## Example patch series with < 10 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00289.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00030.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00018.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00008.html

## Example patch series with > 80 patches

* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00198.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00116.html
* https://www.redhat.com/mailman/private/tianocore-code-review-poc/2020-May/msg00035.html

# Tasks to Complete

* Create edk2-codereview repository for evaluation of new code review process.
* Add GitHub IDs to Maintainers.txt in edk2-codereview repository
* Update BaseTools/Scripts/GetMaintainer.py to be compatible with GitHub IDs at
the end of M: and R: statements
* Update webhook to use Rabbit MQ to manage requests and emails
* Determine if webhook requests must be serialized? Current POC is serialized.
* Make sure webhook has error handling for all unexpected events/states.
* Add logging of all events and emails to webhook
* Add admin interface to webhook
* Deploy webhook on a production server with 24/7 support

# Ideas for Future Enhancements

* Run PatchCheck.py before assigning maintainers/reviewers.
* Add a simple check that fails if a single patch spans more than one package.
* Monitor comments for Reviewed-by, Acked-by, Tested-by, Series-Reviewed-by,
Series-Acked-by, Series-Tested-by made by assigned maintainers/reviewers.
Once all commits have required tags, auto update commit messages in the
branch and wait for maintainer to set the `Push` label to run CI and auto
merge if all CI checks pass.

Best regards,

Mike


PKCS7 Authenticated Variable Enrollment

Wadhawan, Divneil R <divneil.r.wadhawan@...>
 

Hi,

I am trying to test if the enrollment of Authenticated Variables is possible with verification of signature with different key sizes.
File: AuthService.c :: Function: VerifyTimeBasedPayload() -> Pkcs7Verify()
So, to achieve that, I did the following:

a. Generate the KEK.auth
uuidgen --random > GUID.txt

openssl req -newkey rsa:3072 -nodes -keyout PK.key -new -x509 -sha384 -days 3650 -subj "/CN=my Platform Key/" -out PK.crt
openssl x509 -outform DER -in PK.crt -out PK.cer

openssl req -newkey rsa:3072 -nodes -keyout KEK.key -new -x509 -sha384 -days 3650 -subj "/CN=my Key Exchange Key/" -out KEK.crt
openssl x509 -outform DER -in KEK.crt -out KEK.cer
cert-to-efi-sig-list -g "$(< GUID.txt)" KEK.crt KEK.esl
sign-efi-sig-list -g "$(< GUID.txt)" -k PK.key -c PK.crt KEK KEK.esl KEK.auth

b. Enroll PK.cer from BIOS menu in custom mode

c. Hack the KEK enrollment connected to BIOS Menu (HII interface) and disable custom mode and then gRT->SetVariable() on input data
File: SecureBootConfigImpl.c :: Function: EnrollKeyExchangeKey()

I found that the KEK.auth format is rejected by VerifyTimeBasedPayload() in
(CompareMem (SigData + 13, &mSha256OidValue, sizeof (mSha256OidValue)) != 0)) {

So, I did a hexdump of KEK.auth and could see that data is not as expected. Here's the dump I have of KEK.auth

//EFI_Time (16 bytes)
0000000 e5 07 03 10 0e 39 00 00 00 00 00 00 00 00 00 00

// WIN_CERTIFICATE_UEFI_GUID (16bytes + 8bytes)
// SigData starts at value 0x30 post WIN_CERTIFICATE_UEFI_GUID. At offset of 13 from 0x30 the code expects sha256 OID
0000020 0e 07 00 00 00 02 f1 0e 9d d2 af 4a df 68 ee 49
0000040 8a a9 34 7d 37 56 65 a7 30 82 06 f2 06 09 2a 86

0000060 48 86 f7 0d 01 07 02 a0 82 06 e3 30 82 06 df 02

// sha256 oid starts from here: 60 86 48 01 65 03 04 02 (last 8 bytes + 1byte in next line)
0000100 01 01 31 0f 30 0d 06 09 60 86 48 01 65 03 04 02
0000120 01 05 00 30 0b 06 09 2a 86 48 86 f7 0d 01 07 01

So, the sha256 oid is present but is at a far away offset. Can you please help in identifying the problem component.
I also wanted to talk about the oid value of sha256 present, where as sha384 is used.
The above data dump is same in terms of offset for RSA2048/SHA256 KEK.auth
I have based this method on the understanding that, if we have some OS interface to set the variables, it will pass the KEK.auth directly into gRT->SetVariable().

Regards,
Divneil


[RFC] BaseTools/Source/Python as a standalone python package in independent repo

Matthew Carlson
 

Hello Tianocore Community,

I’m submitting an RFC, proposing the movement of the basetools folder in edk2 to a separate repo and treated as a separate python project.

We talked about it during the April 16th design meeting and on devel (https://edk2.groups.io/g/devel/topic/73069134#58048), feel free to look over the discussion.

Here’s a basic overview of the what and the why behind this proposal:

Why a separate repo?
The recent efforts in expanding the role of CI in the platform and core code of EDK2 will pay big dividends in the future, leading to higher quality code and easier integrations for everyone. Having basetools as it’s own repo would simplify adding a similar CI/linting process and unit-tests to the basetools python code, leading to higher quality code.

A second major benefit is it would allow others that write tools for UEFI and Edk2 to leverage this vast resource of python code using standard python package inclusion. It would allow those tools to be decoupled from edk2 source and provide a consistent and managed user experience. The python project would be published as a Pip module for those that want to leverage the basetools modules the same way they leverage the existing python ecosystem. Packing basetools as a pip module, would reach the most developers and provide the most flexibility and versatility. There are numerous way this could be used; Pip is just one method suggested here. Other ways to leverage this are described below.

Why a pip module?
The investment into basetools is sizable and it has some amazing functionality that’s difficult to reproduce. For example, the DSC, FDF, INF, and DEC parsers handle an incredible amount of edge cases. If I wanted to write a tool that could do a CI validation check or build a UEFI capsule, currently I would need to clone all of EDK2 to get basetools. If it was in a separate repo and available as a wheel, as a developer, I could include it via pip and have that dependency managed with virtual environment or just in the global cache. In addition, other tools that currently are difficult to build would become possible with access to the Basetools functionality.

However, there have been some concerns expressed about having a global basetools and the impact this has on developers with multiple workspaces (potentially at different versions). There are several tools and strategies to consider for managing this dependency. Some outlined below.

How will this change your workflow?
If this moved there would have to be a change for all platforms using edk2 and we have been evaluating options. These could become requirements a developer must do before building edk2 or with minimal effort could be added to the edksetup or build scripts. These can also be more easily isolated if python virtual environments are used.

For those just consuming released versions of basetools python code:
Option A: leverage Python package management and install a released version from Pypi into a per project virtual environment.
Option B: leverage pip to install from the git repo at known tag/release (pip_requirements)

For those wanting to do active development of basetools within an edk2 project:
Option C: Clone the python package source and install the package locally (pip install -e ./). All changes made in the local package are reflected into your python site packages.

Option D: An offline server that builds edk2. You would need to download the wheel, tar, or git repo before hand. Pip supports installing from a variety of sources: https://packaging.python.org/tutorials/installing-packages/ such as a tar.gz, a local folder, or a wheel.

PIP vs Submodules:
The issue with submodule and not using python packages (pip is really just a super convenient helper to install that consistently) is that it requires the paradigm of file system python path management. Think of this as edk1 vs edk2 public includes (if you remember back that far), basically you just figure out the path from yourself to the file you want and add that. It gives you no ability to apply consistent ways to access modules and in the end means you just have a folder with python scripts/modules in it. It causes major pain for downstream consumers when/if there is ever refactoring (and we know there needs to be refactoring). It leads to others not being able to extend or leverage these modules without introducing significant fragility.

What Pip does is gives you a way to consistently install and access the python modules. You can pip install from a local copy of the git repo or you can pip install a "release" using something like pypi. Either way it doesn't change how consuming code accesses the python modules. This is the value of pip in this case.

If there is a strong desire from the developer community to have a workflow that avoids pip I believe a little documentation could handle that. Pip is just a helper. Python packages can be built from the source and installed. I see no reason to recommend this as it requires numerous more commands but if avoiding pip is your goal, it is possible.

More value for python package management (PIP):
As we start looking at refactoring and potentially moving tools from C to python we can take advantage of package management to lower our maintenance burden. For example Brotli has a pypi release and thus we wouldn't need to carry that code. https://pypi.org/project/Brotli/

Versioning and Dependencies:
To minimize the dependency challenges and "bisectability" I would suggest we leverage the versioning capabilities within pip and repo tagging. With versioning you have lots of options as you can lock to a specific version which requires an update each time or you can use some sort of floating version within the tuple of version (xx.yy.zz). These two tools can make this pretty flexible.

In a scenario of DEC or INF syntax change, the suggested workflow would be:
1. Create the issue for basetools
2. Update basetools python
3. Write the unit test that shows it works as expected
4. Check in and make a release
5. Update edk2 pip-requirements.txt to require at least this new version. This gives you the tracking necessary to align with the tools.
6. Use this new feature in the edk2 fw code.

In the scenario of a change in BaseTools that causes a break for a downstream project (like EDK2 or a closed source platform), it is easy to simply checkout the previous pip_requirements and pip install the previous release or to install locally (using option C), checking out the commit in Basetools and inspecting the git history to debug the issue. It’s easy for a developer to modify source locally and test against local code. This is covered by workflow option C. It is very easy to manage and is one of the reasons we are proposing this change.

Demo:
We have a demo of what this would look like: https://github.com/matthewfcarlson/edk2-pytool-base/
And the EDK2 that leverages it https://github.com/matthewfcarlson/edk2/tree/feature/pip-basetools

Contribution/Dev Process:
Since this is a separate repo, it will follow a slightly different contribution and code review process.
1. Github PR process will be used for contributions and code review feedback
a. The yet to be released “Tianocore PR archiver” will be used to send to a dedicated list for basetools patch review archive
2. PRs will only be committed if they keep linear history (no merge commits)
3. The PR review must be approved by at least 2 members of the basetools team (not including the author)
4. The PR must pass all automated checks
a. Formatting/style
b. Unit tests
c. Code coverage (can’t commit change that would decrease overall %)
d. DCO enforcement - https://probot.github.io/apps/dco/
e. See other python requirements from the Python coding standard
5. Github Issues will be used for non-security sensitive bugs/issues/feature requests

Releases:
1. Version will follow Semantic Versioning (https://semver.org/) xx.yy.zz
a. X is major version and will update at incompatible change for Core basetools API
b. Y is minor version. Will be updated when new functionality is added in compatible manner
c. Z is patch version. Updated with each change
d. Target 1.0.0 for 2020 Q3 stable tag timeframe
2. Git tags will be created for each version
3. Github “release” will be created for each version
4. Git branches will not be created proactively. If servicing is determined to be needed then a branch can be created from the tag.
5. Pypi release will be made for each version
6. Github milestones will be used for tracking content in every minor version
What happens next?
Right now, we’re gathering feedback and seeing if anyone has an concerns or platforms that this would not work for. We’d love to hear what you have to say. Baring any serious concerns, we’d move forward with:
1. Create new GitHub repo on tianocore for the basetools project
2. Develop the testing, PR, and release process
3. Release the initial version to pypi
4. Delete the source folder in edk2 repo and replace with readme and method to get pip version installed (this patch will be post Q2 stable tag to give developers time to adjust).
5. Continually improve basetools and add more testing
This RFC will close May 11th-12th, please respond with comments and questions before that date.

What’s the long-term plan?
The current tentative long term plan is to merge some or all of basetools in with the existing edk2-pytool-library repo. This would likely involve the conversion of C based basetools to python based ones. This is still an active conversation, and we’d like to hear your thoughts.


Matthew Carlson
Core UEFI
Microsoft


Re: [RFCv2] code-first process for UEFI-forum specifications

Samer El-Haj-Mahmoud
 

Acked-by: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com>

-----Original Message-----
From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Leif Lindholm
via Groups.Io
Sent: Monday, March 23, 2020 3:06 PM
To: devel@edk2.groups.io; rfc@edk2.groups.io
Cc: Felixp@ami.com; Mark Doran <mark.doran@intel.com>; Andrew Fish
<afish@apple.com>; Laszlo Ersek <lersek@redhat.com>; Michael D Kinney
<michael.d.kinney@intel.com>
Subject: [edk2-rfc] [RFCv2] code-first process for UEFI-forum specifications

Changes to v2 of this proposal:
- Feedback from Laszlo[a] and Mike[b] incorporated.
- I opted to view Mike's responses to Laszlo's questions as
accepted, as no follow-up was made.

Feedback from Felix[c] *not* incorporated, as while I agree with all of it, I am
not convinced that information should go here, but should instead reside with
the UEFI Forum. (This text documents the public part of the process - it would
cause me slight impedance mismatch to have it also document the non-public
part.)

[a] https://edk2.groups.io/g/devel/message/53422
[b] https://edk2.groups.io/g/devel/message/53738
[c] https://edk2.groups.io/g/devel/message/54453

/
Leif

---

This is a proposal for a process by which new features can be added to UEFI
forum specifications after first having been designed and prototyped in the
open.

This process lets changes and the development of new features happen in the
open, without violating the UEFI forum bylaws which prevent publication of
code for in-draft features/changes.

The process does not in fact change the UEFI bylaws - the change is that the
development (of both specification and code) happens in the open. The
resulting specification update is then submitted to the appropriate working
goup as an Engineering Change Request (ECR), and voted on. For the UEFI
Forum, this is a change in workflow, not a change in process.

ECRs are tracked in a UEFI Forum Mantis instance, access restricted to UEFI
Forum Members. TianoCore will enable this new process by providing areas
on https://bugzilla.tianocore.org/ to track both specification updates and
reference implementations and new repositories under
https://github.com/tianocore/ dedicated to hold "code first".


## Bugzilla

bugzilla.tianocore.org will have a product category each for
* ACPI Specification
* PI Specification
* UEFI Specification

Each product category will have a separate components for
* Specification
* Reference implementation


## Github
New repositories will be added for holding the text changes and the source
code.

Specification text changes will be held within the affected source repository,
in the Github flavour of markdown, in a file (or split across several files) with
.md suffix.
(This one may break down where we have a specification change affecting
multiple specifications, but at that point we can track it with multiple BZ
entries)

Reference implementations targeting EDK2 will be held in branches on edk2-
staging.
Additional repositories for implementing reference features in additional
open source projects can be added in the future, as required.


## Intended workflow
The entity initiating a specifiation update raises a Bugzilla in the appropriate
area in bugzilla.tianocore.org. This entry contains the outline of the change,
and the full initial draft text is attached.

If multiple specification updates are interdependent, especially if between
different specifications, then multiple bugzilla entries should be created.
These bugzilla entries *must* be linked together with dependencies.

After the BZs have been created, new branches should be created in the
relevant repositories for each bugzilla - the branch names should be BZ####,
where #### describes the bugzilla ID assigned, optionally followed by a '-' and
something more descriptive. If multiple bugzilla entries must coexist on a
single branch, one of them is designated the 'top-level', with dependencies
properly tracked.
That BZ will be the one naming the branch.


## Source code
In order to ensure draft code does not accidentally leak into production use,
and to signify when the changeover from draft to final happens, *all* new or
modified[1] identifiers need to be prefixed with the relevant BZ####.

[1] Modified in a non-backwards-compatible way. If, for example, a statically
sized array is grown - this does not need to be prefixed. But a tag in a
comment would be *highly* recommended.

### File names
New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
Private header files do not need the prefix.

### Contents

The tagging must follow the coding style used by each affected codebase.
Examples:

| Released in spec | Draft version in tree | Comment |
| --- | --- | --- |
| `FunctionName` | `Bz1234FunctionName` | |
| `HEADER_MACRO` | `BZ1234_HEADER_MACRO` | |

For data structures or enums, any new or non-backwards-compatible structs
or fields require a prefix. As above, growing an existing array in an existing
struct requires no prefix.

| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT` | Typedef only [2]
|
| `StructField` | `Bz1234StructField` | In existing struct[3] |
| `typedef SOME_ENUM` | `BZ1234_SOME_ENUM` | Typedef only [2]
|

[2] If the struct or enum definition is separate from the typedef in the public
header, the definition does not need the prefix.
[3] Individual fields in newly added typedefd struct do not need prefix, the
struct already carried the prefix.

Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.

| `gSomeGuid` | `gBz1234SomeGuid` | |

Local identifiers, including module-global ones (m-prefixed) do not require a
BZ prefix.

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

461 - 480 of 747