Guidance about CI


Boeuf, Sebastien
 

Hi folks,

I was wondering if you would be okay with me adding Cloud Hypervisor to the EDK2 CI.
The idea would be to run a quick/simple test that Cloud Hypervisor can properly boot
with the OVMF binary built from source on every pull request.

And if you think that makes sense, any guidance on how to achieve that the right way?

Thanks,
Sebastien
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Sean
 

can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do similar things. They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean

On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
ight way?


Boeuf, Sebastien
 

Hi Sean,

Cloud Hypervisor can boot on Microsoft Azure VMs as this is what our project
relies on to validate every PR. And it should be able to boot on any other CI
providers.

but I don't understand where the QEMU binary is built (I will need to do the same
for Cloud Hypervisor so that it's available on the machine), and I can't find the actual
tests that are being run there.

Thanks,
Sebastien

From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Sean <spbrogan@...>
Sent: Wednesday, January 5, 2022 12:40 AM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Boeuf, Sebastien <sebastien.boeuf@...>; Yao, Jiewen <jiewen.yao@...>; kraxel@... <kraxel@...>
Subject: Re: [edk2-devel] Guidance about CI
 
can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do
similar things.  They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to
the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean


On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
> ight way?





---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Boeuf, Sebastien
 

Ah nevermind I found out QEMU was installed from packaging.

We don't have packages for Cloud Hypervisor, but we can download
a static binary from a specific release, do you think that would be
acceptable?

Thanks,
Sebastien

From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Boeuf, Sebastien <sebastien.boeuf@...>
Sent: Wednesday, January 5, 2022 2:32 PM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Yao, Jiewen <jiewen.yao@...>; kraxel@... <kraxel@...>; spbrogan@... <spbrogan@...>
Subject: Re: [edk2-devel] Guidance about CI
 
Hi Sean,

Cloud Hypervisor can boot on Microsoft Azure VMs as this is what our project
relies on to validate every PR. And it should be able to boot on any other CI
providers.

but I don't understand where the QEMU binary is built (I will need to do the same
for Cloud Hypervisor so that it's available on the machine), and I can't find the actual
tests that are being run there.

Thanks,
Sebastien

From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Sean <spbrogan@...>
Sent: Wednesday, January 5, 2022 12:40 AM
To: devel@edk2.groups.io <devel@edk2.groups.io>; Boeuf, Sebastien <sebastien.boeuf@...>; Yao, Jiewen <jiewen.yao@...>; kraxel@... <kraxel@...>
Subject: Re: [edk2-devel] Guidance about CI
 
can cloud hypervisor boot on any of the free CI providers?
If you look at ArmVirt, Ovmf, and even the emulatorpkg those all do
similar things.  They are able to run on azure dev-ops cloud agents.

If that is the case following a pattern from defined here would lead to
the easiest and most consistent path.

https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/EmulatorPkg/PlatformCI
https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI

Thanks
Sean


On 1/4/2022 3:37 PM, Boeuf, Sebastien wrote:
> ight way?





---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number:  302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Gerd Hoffmann
 

On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
Ah nevermind I found out QEMU was installed from packaging.
On ubuntu.

We don't have packages for Cloud Hypervisor, but we can download
a static binary from a specific release, do you think that would be
acceptable?
As far I know the same happens for qemu on windows,
so that should be fine.

take care,
Gerd


Boeuf, Sebastien
 

On Wed, 2022-01-05 at 17:55 +0100, kraxel@... wrote:
On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
Ah nevermind I found out QEMU was installed from packaging.
On ubuntu.

We don't have packages for Cloud Hypervisor, but we can download
a static binary from a specific release, do you think that would be
acceptable?
As far I know the same happens for qemu on windows,
so that should be fine.
Cool!

BTW, about microvm, I saw that you're skipping QEMU, so does that mean
you're not *really* testing that OVMF works with microvm, or am I
missing something?

Thanks,
Sebastien


take care,
  Gerd
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Gerd Hoffmann
 

On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
On Wed, 2022-01-05 at 17:55 +0100, kraxel@... wrote:
On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
Ah nevermind I found out QEMU was installed from packaging.
On ubuntu.

We don't have packages for Cloud Hypervisor, but we can download
a static binary from a specific release, do you think that would be
acceptable?
As far I know the same happens for qemu on windows,
so that should be fine.
Cool!

BTW, about microvm, I saw that you're skipping QEMU, so does that mean
you're not *really* testing that OVMF works with microvm, or am I
missing something?
Correct, not tested right now. The code needs some improvements, it's
not flexible enough, havn't found the time to do that yet. Also have
to check whenever the qemu version shipped by ubuntu is new enough to
actually have microvm support ...

take care,
Gerd


Boeuf, Sebastien
 

On Fri, 2022-01-07 at 13:17 +0100, kraxel@... wrote:
On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
On Wed, 2022-01-05 at 17:55 +0100, kraxel@... wrote:
On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
Ah nevermind I found out QEMU was installed from packaging.
On ubuntu.

We don't have packages for Cloud Hypervisor, but we can
download
a static binary from a specific release, do you think that
would be
acceptable?
As far I know the same happens for qemu on windows,
so that should be fine.
Cool!

BTW, about microvm, I saw that you're skipping QEMU, so does that
mean
you're not *really* testing that OVMF works with microvm, or am I
missing something?
Correct, not tested right now.  The code needs some improvements,
it's
not flexible enough, havn't found the time to do that yet.  Also have
to check whenever the qemu version shipped by ubuntu is new enough to
actually have microvm support ...
Okay thanks for your confirmation, that makes sense :)


take care,
  Gerd
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.


Boeuf, Sebastien
 

Is the CI only relying on QEMU using TCG? I have some patches to run
the CI with Cloud Hypervisor but I got an error about /dev/kvm not
being available :/

Is there a certain type of machine used by the AzurePipelines that
would support nested virtualization?

Thanks,
Sebastien

On Fri, 2022-01-07 at 13:17 +0100, kraxel@... wrote:
On Thu, Jan 06, 2022 at 02:21:59PM +0000, Boeuf, Sebastien wrote:
On Wed, 2022-01-05 at 17:55 +0100, kraxel@... wrote:
On Wed, Jan 05, 2022 at 01:44:01PM +0000, Boeuf, Sebastien wrote:
Ah nevermind I found out QEMU was installed from packaging.
On ubuntu.

We don't have packages for Cloud Hypervisor, but we can
download
a static binary from a specific release, do you think that
would be
acceptable?
As far I know the same happens for qemu on windows,
so that should be fine.
Cool!

BTW, about microvm, I saw that you're skipping QEMU, so does that
mean
you're not *really* testing that OVMF works with microvm, or am I
missing something?
Correct, not tested right now.  The code needs some improvements,
it's
not flexible enough, havn't found the time to do that yet.  Also have
to check whenever the qemu version shipped by ubuntu is new enough to
actually have microvm support ...

take care,
  Gerd
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company)
Registered headquarters: "Les Montalets"- 2, rue de Paris,
92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 4,572,000 Euros

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.