Date
1 - 1 of 1
(Unit) testing device drivers from test app running under OS
Albecki, Mateusz
Hello,
In the past few years there has been some effort to add unit tests to EDK2 components. Some unit tests that execute under OS and can be run as a part of CI were added for some simple modules such as SafeIntLib, ResetSystemLib etc however, to my knowledge, no tests were added for modules which directly communicate with hardware. I've tried investigating what it would take to implement a test code, that compiles to the OS app, for a device driver(I've chosen SdMmcPciHcDxe driver since I am familiar with it). Since the app needs to run under OS there needs to be some kind of device simulation in place. I've come up with 3 ideas on how to simulate the device:
1. Implement device model in the test code
2. Connect to the virtual machine
3. Connect to the RTL simulation
I've tried to describe results of my investigation in this gist(it includes links to the specific sections of the code): https://gist.github.com/malbecki/154f7e436e9c8d67dd87dd9234e85773
Code: https://github.com/matalbec/edk2/tree/qemu_unit_tests
A few general comments that I didn't put in the gist:
1. Even though I am using the EDK2 unit test framework some might consider the kind of tests I've written as integration tests. I execute large portions of the behavior(whole block io) and verify end result without worrying too much about intermittent states. I believe those tests to be most valuable.
2. In my opinion by far the most promising method is to use QEMU as a device model provider. Tests written in this way have all of the advantages of running in the OS while avoiding the slow simulation times of RTL and labour costs of writing your own device model.
3. The added advantage of testing in this way is the improved quality of life when developing EDK2 drivers. During my work I was able to build and deploy the driver in less than a minute and in case of any errors I was able to use gdb(without any weird problems that typically arise with remote debugger connection) to debug. Compared to building the whole bios image, loading it on the platform, booting the platform and then hoping that the debugger will connect this time working on the OS app is extremely simple and fast. Even when comparing building OVMF and running custom OVMF on QEMU vs building the test driver with QMP and running that - test driver with QMP is much simpler to work with.
I would like to get feedback to following questions:
1. Is anyone interested in modeling the device in the EDK2 test code itself and if yes does my LocalRegisterSpaceLib seems useful? As I have mentioned in the gist I think it helped me to write SD host controller model but that might be just me.
2. Provided that there is interest in the libraries I have developed for local model where should they be placed. Right now they live in UnitTestFrameworkPkg but maybe separate pkg would be better.
3. For testing with virtual machines would there be interest in pursuing Qemu based testing for EDK2 CI?
4. Any other feedback
Thanks,
Mateusz
---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.
Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
In the past few years there has been some effort to add unit tests to EDK2 components. Some unit tests that execute under OS and can be run as a part of CI were added for some simple modules such as SafeIntLib, ResetSystemLib etc however, to my knowledge, no tests were added for modules which directly communicate with hardware. I've tried investigating what it would take to implement a test code, that compiles to the OS app, for a device driver(I've chosen SdMmcPciHcDxe driver since I am familiar with it). Since the app needs to run under OS there needs to be some kind of device simulation in place. I've come up with 3 ideas on how to simulate the device:
1. Implement device model in the test code
2. Connect to the virtual machine
3. Connect to the RTL simulation
I've tried to describe results of my investigation in this gist(it includes links to the specific sections of the code): https://gist.github.com/malbecki/154f7e436e9c8d67dd87dd9234e85773
Code: https://github.com/matalbec/edk2/tree/qemu_unit_tests
A few general comments that I didn't put in the gist:
1. Even though I am using the EDK2 unit test framework some might consider the kind of tests I've written as integration tests. I execute large portions of the behavior(whole block io) and verify end result without worrying too much about intermittent states. I believe those tests to be most valuable.
2. In my opinion by far the most promising method is to use QEMU as a device model provider. Tests written in this way have all of the advantages of running in the OS while avoiding the slow simulation times of RTL and labour costs of writing your own device model.
3. The added advantage of testing in this way is the improved quality of life when developing EDK2 drivers. During my work I was able to build and deploy the driver in less than a minute and in case of any errors I was able to use gdb(without any weird problems that typically arise with remote debugger connection) to debug. Compared to building the whole bios image, loading it on the platform, booting the platform and then hoping that the debugger will connect this time working on the OS app is extremely simple and fast. Even when comparing building OVMF and running custom OVMF on QEMU vs building the test driver with QMP and running that - test driver with QMP is much simpler to work with.
I would like to get feedback to following questions:
1. Is anyone interested in modeling the device in the EDK2 test code itself and if yes does my LocalRegisterSpaceLib seems useful? As I have mentioned in the gist I think it helped me to write SD host controller model but that might be just me.
2. Provided that there is interest in the libraries I have developed for local model where should they be placed. Right now they live in UnitTestFrameworkPkg but maybe separate pkg would be better.
3. For testing with virtual machines would there be interest in pursuing Qemu based testing for EDK2 CI?
4. Any other feedback
Thanks,
Mateusz
---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.
Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.