TianoCore Community Design Meeting Minutes - Sep 19


Ni, Ray
 

(Kondal, Sean, Mike, could you please reply to correct the minutes if there is something wrong or not captured?)
1. Python Development Process (Intel)
Presenter: Kondal Purma
Slides: https://edk2.groups.io/g/devel/files/Designs/2019/0919/PythonDevelopmentProcess-OpenSource.pdf

* Problem
Page #2: Gap in predefined consistent code style and toolset for python development flow.

* Proposal: Python development process flow chart
Page #4
Code style: PEP 8
Docstring: PEP 257
Code checker: flake8
Page #6, #7 for detailed flake8 config
Docstring checker: flake8-docstrings
unittest: python unittest library
code complexity measure:
https://github.com/PyCQA/mccabe
https://en.wikipedia.org/wiki/Cyclomatic_complexity

* Opens
@Sean: Why use unittest library, not pytest?
@Kondal: Agree to use pytest.

@Sean: Suggest using strict python types to avoid potential bugs in code

@Sean: Will the python coding style checking included in CI server?
@Mike: Finalize the requirement of python coding style first so developers have the chance to run it locally before sending code review. Agree to have the server to run check automatically.

@Sean: Base tool python code is not package organized, difficult to run the unit test or reuse the code.

@Mike: There will be 10-20 pages of gitbook in markdown format to document the requirement of python code.

@Sean: File/folder name/case don't follow PEP8. flake8 doesn't test that, need the standard of file/folder name.

@Liming: What will be included in the final document?
@Mike: Generally, about how to write good python code.

@Sean: Can leverage pytools

@Mike: Will have small group discussion and come out with the proposal draft

Thanks,
Ray


rebecca@...
 

On 2019-09-20 12:48, Ni, Ray wrote:
* Proposal: Python development process flow chart
Page #4
Code style: PEP 8
Docstring: PEP 257
Code checker: flake8
Page #6, #7 for detailed flake8 config
Docstring checker: flake8-docstrings
unittest: python unittest library
code complexity measure:
https://github.com/PyCQA/mccabe
https://en.wikipedia.org/wiki/Cyclomatic_complexity
Unfortunately I missed the meeting, but as I've said in the past on the
devel mailing list, I'd suggest using pylama: https://github.com/klen/pylama

Code audit tool for Python and JavaScript. Pylama wraps these tools:

* pycodestyle <https://github.com/PyCQA/pycodestyle> (formerly pep8) ©
2012-2013, Florent Xicluna;
* pydocstyle <https://github.com/PyCQA/pydocstyle/> (formerly pep257
by Vladimir Keleshev) © 2014, Amir Rachum;
* PyFlakes <https://github.com/pyflakes/pyflakes> © 2005-2013, Kevin
Watters;
* Mccabe
<http://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html>
© Ned Batchelder;
* Pylint <http://pylint.org> © 2013, Logilab (should be installed
'pylama_pylint' module);
* Radon <https://github.com/rubik/radon> © Michele Lacchia
* gjslint <https://developers.google.com/closure/utilities> © The
Closure Linter Authors (should be installed 'pylama_gjslint' module);
* eradicate <https://github.com/myint/eradicate> © Steven Myint;
* Mypy <https://github.com/python/mypy> © Jukka Lehtosalo and
contributors;


--
Rebecca Cran


Bob Feng
 

Ray,

I remember we also discuss the type hints in Python, adding the data type for function's parameter and return value.

I think mypy is most common tool for doing type checking.

Thanks,
Bob

-----Original Message-----
From: announce@edk2.groups.io <announce@edk2.groups.io> On Behalf Of Ni, Ray
Sent: Saturday, September 21, 2019 2:49 AM
To: announce@edk2.groups.io
Cc: Purma, Kondal R <kondal.r.purma@...>; Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>
Subject: [edk2-announce] TianoCore Community Design Meeting Minutes - Sep 19

(Kondal, Sean, Mike, could you please reply to correct the minutes if there is something wrong or not captured?) 1. Python Development Process (Intel)
Presenter: Kondal Purma
Slides: https://edk2.groups.io/g/devel/files/Designs/2019/0919/PythonDevelopmentProcess-OpenSource.pdf

* Problem
Page #2: Gap in predefined consistent code style and toolset for python development flow.

* Proposal: Python development process flow chart Page #4 Code style: PEP 8
Docstring: PEP 257
Code checker: flake8
Page #6, #7 for detailed flake8 config Docstring checker: flake8-docstrings
unittest: python unittest library
code complexity measure:
https://github.com/PyCQA/mccabe
https://en.wikipedia.org/wiki/Cyclomatic_complexity

* Opens
@Sean: Why use unittest library, not pytest?
@Kondal: Agree to use pytest.

@Sean: Suggest using strict python types to avoid potential bugs in code

@Sean: Will the python coding style checking included in CI server?
@Mike: Finalize the requirement of python coding style first so developers have the chance to run it locally before sending code review. Agree to have the server to run check automatically.

@Sean: Base tool python code is not package organized, difficult to run the unit test or reuse the code.

@Mike: There will be 10-20 pages of gitbook in markdown format to document the requirement of python code.

@Sean: File/folder name/case don't follow PEP8. flake8 doesn't test that, need the standard of file/folder name.

@Liming: What will be included in the final document?
@Mike: Generally, about how to write good python code.

@Sean: Can leverage pytools

@Mike: Will have small group discussion and come out with the proposal draft

Thanks,
Ray


Ni, Ray
 

Bob, Rebecca,
Thanks for the reply. Please be involved in the group discussion with Mike, Sean, Kondal for a better python development process.

Thanks,
Ray

-----Original Message-----
From: Feng, Bob C <bob.c.feng@...>
Sent: Friday, September 20, 2019 10:19 PM
To: Ni, Ray <ray.ni@...>; announce@edk2.groups.io
Cc: Purma, Kondal R <kondal.r.purma@...>; Sean Brogan <sean.brogan@...>; Kinney, Michael D
<michael.d.kinney@...>
Subject: RE: TianoCore Community Design Meeting Minutes - Sep 19

Ray,

I remember we also discuss the type hints in Python, adding the data type for function's parameter and return value.

I think mypy is most common tool for doing type checking.

Thanks,
Bob

-----Original Message-----
From: announce@edk2.groups.io <announce@edk2.groups.io> On Behalf Of Ni, Ray
Sent: Saturday, September 21, 2019 2:49 AM
To: announce@edk2.groups.io
Cc: Purma, Kondal R <kondal.r.purma@...>; Sean Brogan <sean.brogan@...>; Kinney, Michael D
<michael.d.kinney@...>
Subject: [edk2-announce] TianoCore Community Design Meeting Minutes - Sep 19

(Kondal, Sean, Mike, could you please reply to correct the minutes if there is something wrong or not captured?) 1. Python
Development Process (Intel)
Presenter: Kondal Purma
Slides: https://edk2.groups.io/g/devel/files/Designs/2019/0919/PythonDevelopmentProcess-OpenSource.pdf

* Problem
Page #2: Gap in predefined consistent code style and toolset for python development flow.

* Proposal: Python development process flow chart Page #4 Code style: PEP 8
Docstring: PEP 257
Code checker: flake8
Page #6, #7 for detailed flake8 config Docstring checker: flake8-docstrings
unittest: python unittest library
code complexity measure:
https://github.com/PyCQA/mccabe
https://en.wikipedia.org/wiki/Cyclomatic_complexity

* Opens
@Sean: Why use unittest library, not pytest?
@Kondal: Agree to use pytest.

@Sean: Suggest using strict python types to avoid potential bugs in code

@Sean: Will the python coding style checking included in CI server?
@Mike: Finalize the requirement of python coding style first so developers have the chance to run it locally before sending
code review. Agree to have the server to run check automatically.

@Sean: Base tool python code is not package organized, difficult to run the unit test or reuse the code.

@Mike: There will be 10-20 pages of gitbook in markdown format to document the requirement of python code.

@Sean: File/folder name/case don't follow PEP8. flake8 doesn't test that, need the standard of file/folder name.

@Liming: What will be included in the final document?
@Mike: Generally, about how to write good python code.

@Sean: Can leverage pytools

@Mike: Will have small group discussion and come out with the proposal draft

Thanks,
Ray


rebecca@...
 

I mentioned pylama because in addition to flake8, the meeting notes also mention mccabe and pep8 tools, which pylama wraps.

Rebecca

On Sep 24, 2019, at 12:41 AM, Purma, Kondal R <kondal.r.purma@...> wrote:


Rebecca,

Pylama definitely a good implementation available in python world and additionally it also does java script checking(which is not applicable to EDK2), however I feel flake8 might be suitable enough for EDKII or build related tools environment.

I feel falke8 is right choice because we can develop plugin for typical use cases only applicable to firmware development .

We already using flake8 for “edk2-pytool-extensions”, so a common set of tools may be make more sense.

Appreciate your thoughts on this further .

Thanks,
Kondal.

From: Rebecca Cran <rebecca@...>
Sent: Friday, September 20, 2019 5:32 PM
To: Ni, Ray <ray.ni@...>; announce@edk2.groups.io
Cc: Purma, Kondal R <kondal.r.purma@...>; Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [edk2-announce] TianoCore Community Design Meeting Minutes - Sep 19

On 2019-09-20 12:48, Ni, Ray wrote:

* Proposal: Python development process flow chart
Page #4
Code style: PEP 8
Docstring: PEP 257
Code checker: flake8
Page #6, #7 for detailed flake8 config
Docstring checker: flake8-docstrings
unittest: python unittest library
code complexity measure:
https://github.com/PyCQA/mccabe
https://en.wikipedia.org/wiki/Cyclomatic_complexity
Unfortunately I missed the meeting, but as I've said in the past on the devel mailing list, I'd suggest using pylama: https://github.com/klen/pylama

Code audit tool for Python and JavaScript. Pylama wraps these tools:

pycodestyle (formerly pep8) © 2012-2013, Florent Xicluna;
pydocstyle (formerly pep257 by Vladimir Keleshev) © 2014, Amir Rachum;
PyFlakes © 2005-2013, Kevin Watters;
Mccabe © Ned Batchelder;
Pylint © 2013, Logilab (should be installed 'pylama_pylint' module);
Radon © Michele Lacchia
gjslint © The Closure Linter Authors (should be installed 'pylama_gjslint' module);
eradicate © Steven Myint;
Mypy © Jukka Lehtosalo and contributors;


--
Rebecca Cran


Purma, Kondal R
 

Oh ok I understand the confusion. Let me provide more details.

In presentation slides we gave the initial background on topics to cover python code static analysis, so added PEP8, PEP257 as reference standards.
Then recommended flake8 which wraps the following and cover all our requirements:

* PyFlakes -- A simple program which checks Python source files for errors
* Pycodestyle -- pycodestyle is a tool to check your Python code against some of the style conventions in PEP 8<http://www.python.org/dev/peps/pep-0008/>
* Ned Batchelder's McCabe script: Ned's script to check McCabe complexity
Thanks,
Kondal.
From: Rebecca Cran <rebecca@...>
Sent: Tuesday, September 24, 2019 8:05 AM
To: Purma, Kondal R <kondal.r.purma@...>
Cc: Ni, Ray <ray.ni@...>; announce@edk2.groups.io; Sean Brogan <sean.brogan@...>; Kinney, Michael D <michael.d.kinney@...>
Subject: Re: [edk2-announce] TianoCore Community Design Meeting Minutes - Sep 19

I mentioned pylama because in addition to flake8, the meeting notes also mention mccabe and pep8 tools, which pylama wraps.

Rebecca
On Sep 24, 2019, at 12:41 AM, Purma, Kondal R <kondal.r.purma@...<mailto:kondal.r.purma@...>> wrote:

Rebecca,

Pylama definitely a good implementation available in python world and additionally it also does java script checking(which is not applicable to EDK2), however I feel flake8 might be suitable enough for EDKII or build related tools environment.

I feel falke8 is right choice because we can develop plugin for typical use cases only applicable to firmware development .

We already using flake8 for “edk2-pytool-extensions”, so a common set of tools may be make more sense.

Appreciate your thoughts on this further .

Thanks,
Kondal.

From: Rebecca Cran <rebecca@...<mailto:rebecca@...>>
Sent: Friday, September 20, 2019 5:32 PM
To: Ni, Ray <ray.ni@...<mailto:ray.ni@...>>; announce@edk2.groups.io<mailto:announce@edk2.groups.io>
Cc: Purma, Kondal R <kondal.r.purma@...<mailto:kondal.r.purma@...>>; Sean Brogan <sean.brogan@...<mailto:sean.brogan@...>>; Kinney, Michael D <michael.d.kinney@...<mailto:michael.d.kinney@...>>
Subject: Re: [edk2-announce] TianoCore Community Design Meeting Minutes - Sep 19

On 2019-09-20 12:48, Ni, Ray wrote:



* Proposal: Python development process flow chart

Page #4

Code style: PEP 8

Docstring: PEP 257

Code checker: flake8

Page #6, #7 for detailed flake8 config

Docstring checker: flake8-docstrings

unittest: python unittest library

code complexity measure:

https://github.com/PyCQA/mccabe

https://en.wikipedia.org/wiki/Cyclomatic_complexity

Unfortunately I missed the meeting, but as I've said in the past on the devel mailing list, I'd suggest using pylama: https://github.com/klen/pylama

Code audit tool for Python and JavaScript. Pylama wraps these tools:

* pycodestyle<https://github.com/PyCQA/pycodestyle> (formerly pep8) © 2012-2013, Florent Xicluna;
* pydocstyle<https://github.com/PyCQA/pydocstyle/> (formerly pep257 by Vladimir Keleshev) © 2014, Amir Rachum;
* PyFlakes<https://github.com/pyflakes/pyflakes> © 2005-2013, Kevin Watters;
* Mccabe<http://nedbatchelder.com/blog/200803/python_code_complexity_microtool.html> © Ned Batchelder;
* Pylint<http://pylint.org> © 2013, Logilab (should be installed 'pylama_pylint' module);
* Radon<https://github.com/rubik/radon> © Michele Lacchia
* gjslint<https://developers.google.com/closure/utilities> © The Closure Linter Authors (should be installed 'pylama_gjslint' module);
* eradicate<https://github.com/myint/eradicate> © Steven Myint;
* Mypy<https://github.com/python/mypy> © Jukka Lehtosalo and contributors;



--
Rebecca Cran


Purma, Kondal R
 

Hi ,

I modified and incorporated all the discussions points and feedback . PDF version of presentation attached .
We are working on creating a document to maintain on version control system.

Thanks,
Kondal.