Re: Google Summer of Code Interested Student


Nate DeSimone
 

I've created a new wiki page for this task with all the information I have gathered thus far. I've done some more experimentation and found that there are several newer terminal emulators that don't support DEC Special Graphics so I've reduced the number of modes where DEC Special Graphics should be preferred. Laszlo, if you could take a look at the terminal type matrix I created that would be very helpful.

https://github.com/tianocore/tianocore.github.io/wiki/Tasks-Terminal-driver-improvements

Thanks,
Nate

-----Original Message-----
From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of Nate
DeSimone
Sent: Thursday, March 11, 2021 9:46 PM
To: cadenkline9@gmail.com
Cc: Laszlo Ersek <lersek@redhat.com>; discuss@edk2.groups.io;
devel@edk2.groups.io
Subject: Re: [edk2-discuss] Google Summer of Code Interested Student

Hi Caden,

Great to meet you and welcome to the TianoCore project! Glad you hear you
are interested! Sorry it has taken me a while to get back to you, researching
the topics you are interested in ended up being somewhat involved 😊.

I went back and did some investigation of current state of the terminal
driver, and some of the work has already been done. However, there are
some things missing and some odd bugs that need attention. To give you a
little more detail, the Terminal driver is located at
https://github.com/tianocore/edk2/tree/master/MdeModulePkg/Universal
/Console/TerminalDxe

The most prevalent use case for the terminal driver is to display the BIOS
setup menu on headless server systems using a PC style serial port
connected to a laptop via null modem. This allows administrators to adjust
BIOS settings on rack mounted systems without needing to connect a
monitor and keyboard.

Historically, the BIOS setup menu would be rendered using the IBM PC VGA
text mode, which encoded text using code page 437 (CP437). This was
important for box-drawing characters, such as β”Œ , ─ , and ┐ , which VGA text
mode encodes as 0xDA, 0xC4, and 0xBF respectively. However, most
terminal emulators assume text to be encoded in UTF-8. Unicode defines
these box drawing characters as 0x250C, 0x2500, and 0x2510. In UTF-8
encoding, these characters translate into 3 byte sequences of (0xE2, 0x94,
0x8C), (0xE2, 0x94, 0x80), and (0xE2, 0x94, 0x90) respectively. The VGA
encodings of these box characters will end up generating errors if one
attempts to decode them as strict UTF-8, though most terminals assume that
the intended characters to be drawn are Ú, Γ„, and ΒΏ, which have the Unicode
character codes 0xDA, 0xC4, and 0xBF. The end result is the BIOS setup menu
looks like this:

ΓšΓ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿
Β³ Device Manager Β³
ÀÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ

Devices List List all the Driver
> Driver Health Manager Health instances to
> RAM Disk Configuration manage
> OVMF Platform Configuration
> iSCSI Configuration
> Network Device List


Press ESC to exit.


ΓšΓ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„Γ„
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿
Β³ Β³
Β³ ^v=Move Highlight <Enter>=Select Entry Esc=Exit Β³
ÀÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ

Instead of like this:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
─────────────────────┐
β”‚ Device Manager β”‚
└─────────────────────────────────────────────────────────
β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Devices List List all the Driver
> Driver Health Manager Health instances to
> RAM Disk Configuration manage
> OVMF Platform Configuration
> iSCSI Configuration
> Network Device List


Press ESC to exit.


β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
─────────────────────┐
β”‚ β”‚
β”‚ ^v=Move Highlight <Enter>=Select Entry Esc=Exit β”‚
└─────────────────────────────────────────────────────────
β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

The terminal driver has fully supported both the legacy CP437 encoding and
the UTF-8 encoding for more than 10 years now. Which mode to use is part
of the configuration settings given to the terminal driver at start up.
However, those configuration settings need to come from somewhere. For
example, OVMF has the following page in its setup menu for configuring the
serial terminal:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
─────────────────────┐
β”‚ Set COM Attributes β”‚
└─────────────────────────────────────────────────────────
β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Set COM Baud Rate <115200> Set COM Baud Rate
Set COM Data Bits <8>
Set COM Parity <None>
Set COM Stop Bits <One>
Set COM Terminal Type <PC_ANSI>
Set COM Flow Control <None>

Commit Changes and Exit
Discard Changes and Exit


β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
─────────────────────┐
β”‚ F9=Reset to Defaults F10=Save β”‚
β”‚ ^v=Move Highlight <Enter>=Select Entry Esc=Exit β”‚
└─────────────────────────────────────────────────────────
β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

The default terminal type is PC_ANSI, which uses CP437. In this day and age
that is probably not the right default anymore, though one could argue
whether PC_ANSI being the default is a "bug". Here is the list of supported
terminal types:

- PC_ANSI
- VT_100
- VT_100_PLUS
- VT_UTF8
- TTY_TERM
- LINUX
- XTERM_R6
- VT_400
- SCO

Now, here is the first unquestionable bug. All of these terminal types except
for VT_UTF8 output the box drawing characters using CP437. The VT-100 did
not use CP437 for box drawing characters. On the VT-100, the terminal driver
should output the escape sequence "ESC ( 0" to switch the character set
from ASCII to "DEC Special Graphics". Then, while in DEC Special Graphics
mode, β”Œ , ─ , and ┐ , are encoded as 0x6C, 0x71, and 0x6B respectively. After
outputting the box drawing characters, the terminal driver should switch
back to ASCII using the escape sequence "ESC ( B". Implementing this will
introduce the interesting problem of optimizing display performance by
limiting the number of character mode switches.

For all the modes listed above, the VT-100 method should be used for
drawing box characters (and other characters in the DEC special graphics
character set) with the exception of PC_ANSI and VT_UTF8, which should
use CP437 and UTF-8 respectively. In general, DEC special graphics has been
around for such a long time that all terminal emulators support it, so it should
be the preferred method of outputting characters outside the initial 0-127
basic ASCII codes, with UTF-8 as a fallback if the character can't be encoded
by either basic ASCII or DEC special graphics. The difference between
VT_UTF8 and the other modes is that DEC special graphics should never be
used. PC_ANSI mode should never use DEC special graphics either.

Now, here is the second bug. That BIOS setup menu page that OVMF has for
configuring the serial port has a field for setting the terminal type. But,
changing the value in that field doesn't actually change the configuration data
that is sent to the terminal driver. So the terminal driver always ends up using
PC_ANSI mode even if the user changes that setting. This isn’t a bug in the
terminal driver really, it’s a bug in OVMF's setup menu implementation. But it
does create the appearance of a problem in the terminal driver and should
be fixed as part of this GSoC project. This should be fixed in both he OVMF
implementation and the MinPlatform implementation.

I'm not sure if the terminal driver improvements would absorb the entire 10
week coding window. If you had time left, you could consider spending it
writing unit tests.

Looking at your experience, it seems like you have more experience with
Python coding than with C coding. Both the terminal driver improvements
and unit tests would be written in C. Another option you could consider is we
have a lot of Python code in TianoCore as well. The two major pieces of
Python code are BaseTools and EdkRepo. BaseTools provides the build
system for TianoCore and implements the logic necessary to compile a BIOS
ROM file from source. EdkRepo is the multi-repository tool for EDK II.
EdkRepo automates common developer workflows for projects that use
more than one git repository (many TianoCore projects do). We would gladly
accept project proposals for either BaseTools or EdkRepo. If either of those
interest you I can point you to some places where they can be improved and
give you some project ideas.

The most important outcome from GSoC in our view is that our students
learn something, get some exposure, and have a great experience that they
will remember fondly for years to come. We want you to be successful.
Gauge your comfort level and pick a project that you feel you can achieve in
the 10 week period. Sorry for the long email, but I hope it helps. Finally I'd
like to reiterate... Welcome to the project!

With Best Regards,
Nate

-----Original Message-----
From: Laszlo Ersek <lersek@redhat.com>
Sent: Wednesday, March 10, 2021 8:17 AM
To: discuss@edk2.groups.io
Cc: cadenkline9@gmail.com; Desimone, Nathaniel L
<nathaniel.l.desimone@intel.com>
Subject: Re: [edk2-discuss] Google Summer of Code Interested Student

adding Nate

On 03/10/21 03:10, mailto:cadenkline9@gmail.com wrote:
Hello, My name is Caden Kline. I am a freshmen Computer Science major in
the US. I intend to specialize in Systems or Security or both. The main two
tasks I am hoping to apply for are "Terminal driver improvements" and
"Writing Unit Tests". However, I am primarily interested in any system level
work and willing to work on anything. I am concerned about the difficulty in
completing these tasks so I'm going to list my experience.

My relevant experience for C programming language is a one semester
introduction to C and Unix class I am currently taking. Outside of formal
experience, I have primarily interacted with C and assembly with capture the
flag/wargame binary exploitation challenges, and unfinished projects such as
a chip8 emulator. My primary programming experience is Java and Python
thanks to my high school and college classes. I have participated in several
past google code-ins. My github profile is https://github.com/Pokemod97 .

Is there anything I can do to improve my chances to be selected or any
other feedback? Thank you for taking the time to read this message.







Join discuss@edk2.groups.io to automatically receive all group messages.