Linux Kernel soft hangs if 8259 PIC is not configured by BIOS: was (RE: [edk2-devel] [RFC][PATCH v1 0/1] PcAtChipsetPkg: Remove framework modules)


Kirkendall, Garrett
 

Since this change will be removing the 8259 PIC driver, I'll mention a quirk I ran into in the Linux Kernel up through at least 5.0 (and I am sure later, I just didn't verify):

Does anyone have pull in the Linux Kernel world to get them to take this seriously? Hopefully someday soon end users will let legacy devices, like the PIC, die a long overdue and graceful death.

I sent the following to the responsible Linux Kernel developers and quickly got back a "working as designed" response:

*************************
I am trying to boot a UEFI BIOS with minimal legacy hardware support. The Linux kernel soft hangs when the PIC is not configured by the BIOS because it is using IOAPIC. Hopefully, this provides enough information.

Observed under Ubuntu Server Linux 18.04 LTS with kernel 4.15.0, and with kernel compiled from source tag v5.0

Where it hangs:
Soft hang occurs in calibrate_APIC_clock(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L805
specific location of soft hang waiting for interrupts: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L854


How it gets to the hang:
If 8259A PIC is not configured before kernel is launched, HPET IRQ 0 registration fails because probe_8259A returns PIC as not available and therefore interrupt descriptors 0-15 are not allocated. This happens when BIOS does not configure 8259A PIC because it uses IOAPIC.

This sequence prevents allocating interrupts 0-15 unless PIC is configured before kernel starts. legacy_pic.init = init_8259A is not called before early_irq_init():
early_irq_init(): https://github.com/torvalds/linux/blob/v5.0/init/main.c#L642
initcnt = arch_probe_nr_irqs(): https://github.com/torvalds/linux/blob/v5.0/kernel/irq/irqdesc.c#L512
return legacy_pic->probe(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/vector.c#L656
default_legacy_pic.probe: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/i8259.c#L418
probe_8259A(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/i8259.c#L301
interrupt 0-15 descriptors not allocated and prevents IOAPIC interrupts 0-15: https://github.com/torvalds/linux/blob/v5.0/kernel/irq/irqdesc.c#L525

In this call is where init_8259A is called and PIC is initialized. However, interrupt descriptors for 0-15 were not allocated in early_irq_init() sequence, so descriptors are not available later.
init_IRQ() : https://github.com/torvalds/linux/blob/v5.0/init/main.c#L643

This sequence tries to register the HPET to irq0, but irq0 descriptor is not allocated by early_irq_init() sequence:
late_time_init(): https://github.com/torvalds/linux/blob/v5.0/init/main.c#L703
late_time_init = x86_late_time_init: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L107
x86_init.timers.timer_init(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L92
x86_init.timers.timer_init = hpet_time_init: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/x86_init.c#L75
setup_default_timer_irq(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L83
if (setup_irq(0, &irq0)): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L78

This gets called at some point after the above sequences, I couldn't track it all the way back to main.c easily:
x86_init.timers.setup_percpu_clockev = setup_boot_APIC_clock: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/x86_init.c#L74
https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L961
*** soft hang in calibrate_APIC_clock(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L854

*************************

GARRETT KIRKENDALL
SMTS Firmware Engineer | CTE
7171 Southwest Parkway, Austin, TX 78735 USA
AMD   facebook  |  amd.com

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Wu, Hao A via Groups.Io
Sent: Monday, May 27, 2019 1:37 AM
To: devel@edk2.groups.io
Cc: Hao A Wu <hao.a.wu@...>; Ray Ni <ray.ni@...>; Andrew Fish <afish@...>; Laszlo Ersek <lersek@...>; Leif Lindholm <leif.lindholm@...>; Michael D Kinney <michael.d.kinney@...>
Subject: [edk2-devel] [RFC][PATCH v1 0/1] PcAtChipsetPkg: Remove framework modules

[CAUTION: External Email]

'''
Please note that this patch will be hold until all the below requirements are met:

A. edk2-stable201905 is created;
B. OvmfPkg has drop its usage of the legacy ISA stack (which includes
the IsaAcpiDxe driver);
C. UefiPayloadPkg has been updated to drop its consume to the
8259InterruptControllerDxe driver.
'''

This patch itself will not be sent to the mailing list and is only available at:
https://github.com/hwu25/edk2/tree/delete_framework
(https://github.com/hwu25/edk2/commit/84e58f89f6d03f9cc3399cced9d5d5529e06a416)


Below modules will be removed from PcAtChipsetPkg:
* PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
* PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
* PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf

They are considered legacy framework components and will no longer be used after the removal of IntelFramework[Module]Pkg.

Also, the unused (after the modules being removed) PCDs will be deleted in package level DEC/UNI files.

Cc: Ray Ni <ray.ni@...>
Cc: Andrew Fish <afish@...>
Cc: Laszlo Ersek <lersek@...>
Cc: Leif Lindholm <leif.lindholm@...>
Cc: Michael D Kinney <michael.d.kinney@...>


Hao A Wu (1):
PcAtChipsetPkg: Remove framework modules

PcAtChipsetPkg/PcAtChipsetPkg.dec | 58 --
PcAtChipsetPkg/PcAtChipsetPkg.dsc | 5 +-
PcAtChipsetPkg/8254TimerDxe/8254Timer.inf | 42 --
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf | 46 --
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf | 50 --
PcAtChipsetPkg/8254TimerDxe/Timer.h | 185 ------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h | 220 -------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h | 269 ---------
PcAtChipsetPkg/8254TimerDxe/Timer.c | 401 -------------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c | 622 --------------------
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c | 301 ----------
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c | 353 -----------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c | 386 ------------
PcAtChipsetPkg/8254TimerDxe/Timer.uni | 16 -
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni | 14 -
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.uni | 16 -
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Extra.uni | 14 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni | 16 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni | 14 -
PcAtChipsetPkg/PcAtChipsetPkg.uni | 52 +-
20 files changed, 2 insertions(+), 3078 deletions(-) delete mode 100644 PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.h
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.c
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.uni
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.uni
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Extra.uni
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni

--
2.12.0.windows.1


Michael D Kinney
 

Hi Garrett,

Two thoughts on this issue:

1) Just because 8259 is not used by UEFI FW does not mean the 8259 HW should
not be initialized into an OS compatible state. This init code could be
part of the chipset/si init code without requiring the 8259 driver/protocol.
2) The HPET driver in PcAtChipsetPkg can be configured to use IOAPIC or MSI
delivery for interrupts. Is there a way for Linux to also use MSI
delivery instead of IOAPIC delivery?

Thanks,

Mike

-----Original Message-----
From: Kirkendall, Garrett
[mailto:Garrett.Kirkendall@...]
Sent: Tuesday, May 28, 2019 9:36 AM
To: devel@edk2.groups.io; Wu, Hao A
<hao.a.wu@...>
Cc: Ni, Ray <ray.ni@...>; Andrew Fish
<afish@...>; Laszlo Ersek <lersek@...>;
Leif Lindholm <leif.lindholm@...>; Kinney,
Michael D <michael.d.kinney@...>
Subject: Linux Kernel soft hangs if 8259 PIC is not
configured by BIOS: was (RE: [edk2-devel] [RFC][PATCH
v1 0/1] PcAtChipsetPkg: Remove framework modules)

Since this change will be removing the 8259 PIC driver,
I'll mention a quirk I ran into in the Linux Kernel up
through at least 5.0 (and I am sure later, I just
didn't verify):

Does anyone have pull in the Linux Kernel world to get
them to take this seriously? Hopefully someday soon
end users will let legacy devices, like the PIC, die a
long overdue and graceful death.

I sent the following to the responsible Linux Kernel
developers and quickly got back a "working as designed"
response:

*************************
I am trying to boot a UEFI BIOS with minimal legacy
hardware support. The Linux kernel soft hangs when the
PIC is not configured by the BIOS because it is using
IOAPIC. Hopefully, this provides enough information.

Observed under Ubuntu Server Linux 18.04 LTS with
kernel 4.15.0, and with kernel compiled from source tag
v5.0

Where it hangs:
Soft hang occurs in calibrate_APIC_clock():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L805
specific location of soft hang waiting for interrupts:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L854


How it gets to the hang:
If 8259A PIC is not configured before kernel is
launched, HPET IRQ 0 registration fails because
probe_8259A returns PIC as not available and therefore
interrupt descriptors 0-15 are not allocated. This
happens when BIOS does not configure 8259A PIC because
it uses IOAPIC.

This sequence prevents allocating interrupts 0-15
unless PIC is configured before kernel starts.
legacy_pic.init = init_8259A is not called before
early_irq_init():
early_irq_init():
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L642
initcnt = arch_probe_nr_irqs():
https://github.com/torvalds/linux/blob/v5.0/kernel/irq/
irqdesc.c#L512
return legacy_pic->probe():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/vector.c#L656
default_legacy_pic.probe:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/i8259.c#L418
probe_8259A():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/i8259.c#L301
interrupt 0-15 descriptors not allocated and
prevents IOAPIC interrupts 0-15:
https://github.com/torvalds/linux/blob/v5.0/kernel/irq/
irqdesc.c#L525

In this call is where init_8259A is called and PIC is
initialized. However, interrupt descriptors for 0-15
were not allocated in early_irq_init() sequence, so
descriptors are not available later.
init_IRQ() :
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L643

This sequence tries to register the HPET to irq0, but
irq0 descriptor is not allocated by early_irq_init()
sequence:
late_time_init():
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L703
late_time_init = x86_late_time_init:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L107
x86_init.timers.timer_init():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L92
x86_init.timers.timer_init = hpet_time_init:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/x86_init.c#L75
setup_default_timer_irq():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L83
if (setup_irq(0, &irq0)):
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L78

This gets called at some point after the above
sequences, I couldn't track it all the way back to
main.c easily:
x86_init.timers.setup_percpu_clockev =
setup_boot_APIC_clock:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/x86_init.c#L74

https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L961
*** soft hang in calibrate_APIC_clock():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L854

*************************

GARRETT KIRKENDALL
SMTS Firmware Engineer | CTE
7171 Southwest Parkway, Austin, TX 78735 USA
AMD   facebook  |  amd.com

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On
Behalf Of Wu, Hao A via Groups.Io
Sent: Monday, May 27, 2019 1:37 AM
To: devel@edk2.groups.io
Cc: Hao A Wu <hao.a.wu@...>; Ray Ni
<ray.ni@...>; Andrew Fish <afish@...>;
Laszlo Ersek <lersek@...>; Leif Lindholm
<leif.lindholm@...>; Michael D Kinney
<michael.d.kinney@...>
Subject: [edk2-devel] [RFC][PATCH v1 0/1]
PcAtChipsetPkg: Remove framework modules

[CAUTION: External Email]

'''
Please note that this patch will be hold until all the
below requirements are met:

A. edk2-stable201905 is created;
B. OvmfPkg has drop its usage of the legacy ISA stack
(which includes
the IsaAcpiDxe driver);
C. UefiPayloadPkg has been updated to drop its consume
to the
8259InterruptControllerDxe driver.
'''

This patch itself will not be sent to the mailing list
and is only available at:
https://github.com/hwu25/edk2/tree/delete_framework
(https://github.com/hwu25/edk2/commit/84e58f89f6d03f9cc
3399cced9d5d5529e06a416)


Below modules will be removed from PcAtChipsetPkg:
* PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
* PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
* PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf

They are considered legacy framework components and
will no longer be used after the removal of
IntelFramework[Module]Pkg.

Also, the unused (after the modules being removed) PCDs
will be deleted in package level DEC/UNI files.

Cc: Ray Ni <ray.ni@...>
Cc: Andrew Fish <afish@...>
Cc: Laszlo Ersek <lersek@...>
Cc: Leif Lindholm <leif.lindholm@...>
Cc: Michael D Kinney <michael.d.kinney@...>


Hao A Wu (1):
PcAtChipsetPkg: Remove framework modules

PcAtChipsetPkg/PcAtChipsetPkg.dec
| 58 --
PcAtChipsetPkg/PcAtChipsetPkg.dsc
| 5 +-
PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
| 42 --
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
| 46 --
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
| 50 --
PcAtChipsetPkg/8254TimerDxe/Timer.h
| 185 ------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
| 220 -------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
| 269 ---------
PcAtChipsetPkg/8254TimerDxe/Timer.c
| 401 -------------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
| 622 --------------------
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
| 301 ----------
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
| 353 -----------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
| 386 ------------
PcAtChipsetPkg/8254TimerDxe/Timer.uni
| 16 -
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
| 14 -

PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.un
i | 16 -

PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Ext
ra.uni | 14 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
| 16 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni
| 14 -
PcAtChipsetPkg/PcAtChipsetPkg.uni
| 52 +-
20 files changed, 2 insertions(+), 3078 deletions(-)
delete mode 100644
PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.h
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.c
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
delete mode 100644
PcAtChipsetPkg/8254TimerDxe/Timer.uni
delete mode 100644
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.un
i
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Ext
ra.uni
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni

--
2.12.0.windows.1



Leif Lindholm
 

Hi Garrett,

Presumably you mean https://lkml.org/lkml/2019/5/9/304?

I'm not familiar enough with the PC platform to stick my oar in, but
Gleixner's counterquestion is an honest one - he does not see an
obvious solution to the problem. And unless someone finds the problem
interesting, this is very much a "patches welcome" sort of scenario.

If Mike's suggestions don't help bring this forward, I would recommend
you loop in all of the X86 maintainers, not just tglx.

It may also be worth asking one of your Linux devs to have a look -
Suravee Suthikulpanit is still at AMD, right?

Best Regards,

Leif

On Tue, May 28, 2019 at 04:36:06PM +0000, Kirkendall, Garrett wrote:
Since this change will be removing the 8259 PIC driver, I'll mention
a quirk I ran into in the Linux Kernel up through at least 5.0 (and
I am sure later, I just didn't verify):

Does anyone have pull in the Linux Kernel world to get them to take
this seriously? Hopefully someday soon end users will let legacy
devices, like the PIC, die a long overdue and graceful death.

I sent the following to the responsible Linux Kernel developers and
quickly got back a "working as designed" response:

*************************
I am trying to boot a UEFI BIOS with minimal legacy hardware
support. The Linux kernel soft hangs when the PIC is not configured
by the BIOS because it is using IOAPIC. Hopefully, this provides
enough information.

Observed under Ubuntu Server Linux 18.04 LTS with kernel 4.15.0, and with kernel compiled from source tag v5.0

Where it hangs:
Soft hang occurs in calibrate_APIC_clock(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L805
specific location of soft hang waiting for interrupts: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L854


How it gets to the hang:
If 8259A PIC is not configured before kernel is launched, HPET IRQ 0 registration fails because probe_8259A returns PIC as not available and therefore interrupt descriptors 0-15 are not allocated. This happens when BIOS does not configure 8259A PIC because it uses IOAPIC.

This sequence prevents allocating interrupts 0-15 unless PIC is configured before kernel starts. legacy_pic.init = init_8259A is not called before early_irq_init():
early_irq_init(): https://github.com/torvalds/linux/blob/v5.0/init/main.c#L642
initcnt = arch_probe_nr_irqs(): https://github.com/torvalds/linux/blob/v5.0/kernel/irq/irqdesc.c#L512
return legacy_pic->probe(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/vector.c#L656
default_legacy_pic.probe: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/i8259.c#L418
probe_8259A(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/i8259.c#L301
interrupt 0-15 descriptors not allocated and prevents IOAPIC interrupts 0-15: https://github.com/torvalds/linux/blob/v5.0/kernel/irq/irqdesc.c#L525

In this call is where init_8259A is called and PIC is initialized. However, interrupt descriptors for 0-15 were not allocated in early_irq_init() sequence, so descriptors are not available later.
init_IRQ() : https://github.com/torvalds/linux/blob/v5.0/init/main.c#L643

This sequence tries to register the HPET to irq0, but irq0 descriptor is not allocated by early_irq_init() sequence:
late_time_init(): https://github.com/torvalds/linux/blob/v5.0/init/main.c#L703
late_time_init = x86_late_time_init: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L107
x86_init.timers.timer_init(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L92
x86_init.timers.timer_init = hpet_time_init: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/x86_init.c#L75
setup_default_timer_irq(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L83
if (setup_irq(0, &irq0)): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/time.c#L78

This gets called at some point after the above sequences, I couldn't track it all the way back to main.c easily:
x86_init.timers.setup_percpu_clockev = setup_boot_APIC_clock: https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/x86_init.c#L74
https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L961
*** soft hang in calibrate_APIC_clock(): https://github.com/torvalds/linux/blob/v5.0/arch/x86/kernel/apic/apic.c#L854

*************************

GARRETT KIRKENDALL
SMTS Firmware Engineer | CTE
7171 Southwest Parkway, Austin, TX 78735 USA
AMD   facebook  |  amd.com

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Wu, Hao A via Groups.Io
Sent: Monday, May 27, 2019 1:37 AM
To: devel@edk2.groups.io
Cc: Hao A Wu <hao.a.wu@...>; Ray Ni <ray.ni@...>; Andrew Fish <afish@...>; Laszlo Ersek <lersek@...>; Leif Lindholm <leif.lindholm@...>; Michael D Kinney <michael.d.kinney@...>
Subject: [edk2-devel] [RFC][PATCH v1 0/1] PcAtChipsetPkg: Remove framework modules

[CAUTION: External Email]

'''
Please note that this patch will be hold until all the below requirements are met:

A. edk2-stable201905 is created;
B. OvmfPkg has drop its usage of the legacy ISA stack (which includes
the IsaAcpiDxe driver);
C. UefiPayloadPkg has been updated to drop its consume to the
8259InterruptControllerDxe driver.
'''

This patch itself will not be sent to the mailing list and is only available at:
https://github.com/hwu25/edk2/tree/delete_framework
(https://github.com/hwu25/edk2/commit/84e58f89f6d03f9cc3399cced9d5d5529e06a416)


Below modules will be removed from PcAtChipsetPkg:
* PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
* PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
* PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf

They are considered legacy framework components and will no longer be used after the removal of IntelFramework[Module]Pkg.

Also, the unused (after the modules being removed) PCDs will be deleted in package level DEC/UNI files.

Cc: Ray Ni <ray.ni@...>
Cc: Andrew Fish <afish@...>
Cc: Laszlo Ersek <lersek@...>
Cc: Leif Lindholm <leif.lindholm@...>
Cc: Michael D Kinney <michael.d.kinney@...>


Hao A Wu (1):
PcAtChipsetPkg: Remove framework modules

PcAtChipsetPkg/PcAtChipsetPkg.dec | 58 --
PcAtChipsetPkg/PcAtChipsetPkg.dsc | 5 +-
PcAtChipsetPkg/8254TimerDxe/8254Timer.inf | 42 --
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf | 46 --
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf | 50 --
PcAtChipsetPkg/8254TimerDxe/Timer.h | 185 ------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h | 220 -------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h | 269 ---------
PcAtChipsetPkg/8254TimerDxe/Timer.c | 401 -------------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c | 622 --------------------
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c | 301 ----------
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c | 353 -----------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c | 386 ------------
PcAtChipsetPkg/8254TimerDxe/Timer.uni | 16 -
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni | 14 -
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.uni | 16 -
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Extra.uni | 14 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni | 16 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni | 14 -
PcAtChipsetPkg/PcAtChipsetPkg.uni | 52 +-
20 files changed, 2 insertions(+), 3078 deletions(-) delete mode 100644 PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.h
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.c
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.uni
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.uni
delete mode 100644 PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Extra.uni
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni

--
2.12.0.windows.1




Kirkendall, Garrett
 

See below

-----Original Message-----
From: Kinney, Michael D <michael.d.kinney@...>
Sent: Tuesday, May 28, 2019 12:07 PM
To: Kirkendall, Garrett <Garrett.Kirkendall@...>; devel@edk2.groups.io; Wu, Hao A <hao.a.wu@...>; Kinney, Michael D <michael.d.kinney@...>
Cc: Ni, Ray <ray.ni@...>; Andrew Fish <afish@...>; Laszlo Ersek <lersek@...>; Leif Lindholm <leif.lindholm@...>
Subject: RE: Linux Kernel soft hangs if 8259 PIC is not configured by BIOS: was (RE: [edk2-devel] [RFC][PATCH v1 0/1] PcAtChipsetPkg: Remove framework modules)

[CAUTION: External Email]

Hi Garrett,

Two thoughts on this issue:

1) Just because 8259 is not used by UEFI FW does not mean the 8259 HW should
not be initialized into an OS compatible state. This init code could be
part of the chipset/si init code without requiring the 8259 driver/protocol.

[Garrett:] The Linux Kernel only allocates "descriptors"/structures for Interrupts 0-15, PIC or IOAPIC mode, if a legacy PIC is available and responds the expected way. The Kernel will initialize the PIC, but not until after this initial probe of the PIC. If the PIC simply did not exist in the hardware, these interrupts would not be allowed/allocated even for an IOAPIC. I was hoping to not need to do anything with the legacy PIC since I had no plans to use it for this UEFI implementation.

2) The HPET driver in PcAtChipsetPkg can be configured to use IOAPIC or MSI
delivery for interrupts. Is there a way for Linux to also use MSI
delivery instead of IOAPIC delivery?

[Garrett:] This is a very early timer initialization sequence and the PIC/IOAPIC will be fully initialized later. Since the PIC didn't respond correctly, I don't think interrupts 0-15 would be allowed later. I'm not sure if the kernel allows MSI for the HPET when everything is configured, I didn't go slogging through to figure out.

Thanks,

Mike

-----Original Message-----
From: Kirkendall, Garrett
[mailto:Garrett.Kirkendall@...]
Sent: Tuesday, May 28, 2019 9:36 AM
To: devel@edk2.groups.io; Wu, Hao A
<hao.a.wu@...>
Cc: Ni, Ray <ray.ni@...>; Andrew Fish <afish@...>; Laszlo
Ersek <lersek@...>; Leif Lindholm <leif.lindholm@...>;
Kinney, Michael D <michael.d.kinney@...>
Subject: Linux Kernel soft hangs if 8259 PIC is not configured by
BIOS: was (RE: [edk2-devel] [RFC][PATCH
v1 0/1] PcAtChipsetPkg: Remove framework modules)

Since this change will be removing the 8259 PIC driver, I'll mention a
quirk I ran into in the Linux Kernel up through at least 5.0 (and I am
sure later, I just didn't verify):

Does anyone have pull in the Linux Kernel world to get them to take
this seriously? Hopefully someday soon end users will let legacy
devices, like the PIC, die a long overdue and graceful death.

I sent the following to the responsible Linux Kernel developers and
quickly got back a "working as designed"
response:

*************************
I am trying to boot a UEFI BIOS with minimal legacy hardware support.
The Linux kernel soft hangs when the PIC is not configured by the BIOS
because it is using IOAPIC. Hopefully, this provides enough
information.

Observed under Ubuntu Server Linux 18.04 LTS with kernel 4.15.0, and
with kernel compiled from source tag
v5.0

Where it hangs:
Soft hang occurs in calibrate_APIC_clock():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L805
specific location of soft hang waiting for interrupts:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L854


How it gets to the hang:
If 8259A PIC is not configured before kernel is launched, HPET IRQ 0
registration fails because probe_8259A returns PIC as not available
and therefore interrupt descriptors 0-15 are not allocated. This
happens when BIOS does not configure 8259A PIC because it uses IOAPIC.

This sequence prevents allocating interrupts 0-15 unless PIC is
configured before kernel starts.
legacy_pic.init = init_8259A is not called before
early_irq_init():
early_irq_init():
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L642
initcnt = arch_probe_nr_irqs():
https://github.com/torvalds/linux/blob/v5.0/kernel/irq/
irqdesc.c#L512
return legacy_pic->probe():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/vector.c#L656
default_legacy_pic.probe:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/i8259.c#L418
probe_8259A():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/i8259.c#L301
interrupt 0-15 descriptors not allocated and prevents IOAPIC
interrupts 0-15:
https://github.com/torvalds/linux/blob/v5.0/kernel/irq/
irqdesc.c#L525

In this call is where init_8259A is called and PIC is initialized.
However, interrupt descriptors for 0-15 were not allocated in
early_irq_init() sequence, so descriptors are not available later.
init_IRQ() :
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L643

This sequence tries to register the HPET to irq0, but
irq0 descriptor is not allocated by early_irq_init()
sequence:
late_time_init():
https://github.com/torvalds/linux/blob/v5.0/init/main.c
#L703
late_time_init = x86_late_time_init:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L107
x86_init.timers.timer_init():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L92
x86_init.timers.timer_init = hpet_time_init:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/x86_init.c#L75
setup_default_timer_irq():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L83
if (setup_irq(0, &irq0)):
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/time.c#L78

This gets called at some point after the above sequences, I couldn't
track it all the way back to main.c easily:
x86_init.timers.setup_percpu_clockev =
setup_boot_APIC_clock:
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/x86_init.c#L74

https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L961
*** soft hang in calibrate_APIC_clock():
https://github.com/torvalds/linux/blob/v5.0/arch/x86/ke
rnel/apic/apic.c#L854

*************************

GARRETT KIRKENDALL
SMTS Firmware Engineer | CTE
7171 Southwest Parkway, Austin, TX 78735 USA
AMD facebook |  amd.com

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Wu, Hao
A via Groups.Io
Sent: Monday, May 27, 2019 1:37 AM
To: devel@edk2.groups.io
Cc: Hao A Wu <hao.a.wu@...>; Ray Ni <ray.ni@...>; Andrew
Fish <afish@...>; Laszlo Ersek <lersek@...>; Leif
Lindholm <leif.lindholm@...>; Michael D Kinney
<michael.d.kinney@...>
Subject: [edk2-devel] [RFC][PATCH v1 0/1]
PcAtChipsetPkg: Remove framework modules

[CAUTION: External Email]

'''
Please note that this patch will be hold until all the below
requirements are met:

A. edk2-stable201905 is created;
B. OvmfPkg has drop its usage of the legacy ISA stack (which includes
the IsaAcpiDxe driver);
C. UefiPayloadPkg has been updated to drop its consume to the
8259InterruptControllerDxe driver.
'''

This patch itself will not be sent to the mailing list and is only
available at:
https://github.com/hwu25/edk2/tree/delete_framework
(https://github.com/hwu25/edk2/commit/84e58f89f6d03f9cc
3399cced9d5d5529e06a416)


Below modules will be removed from PcAtChipsetPkg:
* PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
* PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
* PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf

They are considered legacy framework components and will no longer be
used after the removal of IntelFramework[Module]Pkg.

Also, the unused (after the modules being removed) PCDs will be
deleted in package level DEC/UNI files.

Cc: Ray Ni <ray.ni@...>
Cc: Andrew Fish <afish@...>
Cc: Laszlo Ersek <lersek@...>
Cc: Leif Lindholm <leif.lindholm@...>
Cc: Michael D Kinney <michael.d.kinney@...>


Hao A Wu (1):
PcAtChipsetPkg: Remove framework modules

PcAtChipsetPkg/PcAtChipsetPkg.dec
| 58 --
PcAtChipsetPkg/PcAtChipsetPkg.dsc
| 5 +-
PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
| 42 --
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
| 46 --
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
| 50 --
PcAtChipsetPkg/8254TimerDxe/Timer.h
| 185 ------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
| 220 -------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
| 269 ---------
PcAtChipsetPkg/8254TimerDxe/Timer.c
| 401 -------------
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
| 622 --------------------
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
| 301 ----------
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
| 353 -----------
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
| 386 ------------
PcAtChipsetPkg/8254TimerDxe/Timer.uni
| 16 -
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
| 14 -

PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.un
i | 16 -

PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Ext
ra.uni | 14 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
| 16 -
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni
| 14 -
PcAtChipsetPkg/PcAtChipsetPkg.uni
| 52 +-
20 files changed, 2 insertions(+), 3078 deletions(-) delete mode
100644 PcAtChipsetPkg/8254TimerDxe/8254Timer.inf
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.inf
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.inf
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.h
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.h
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h
delete mode 100644 PcAtChipsetPkg/8254TimerDxe/Timer.c
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/8259.c
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/ComponentName.c
delete mode 100644 PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.c
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
delete mode 100644
PcAtChipsetPkg/8254TimerDxe/Timer.uni
delete mode 100644
PcAtChipsetPkg/8254TimerDxe/TimerExtra.uni
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259.un
i
delete mode 100644
PcAtChipsetPkg/8259InterruptControllerDxe/Legacy8259Ext
ra.uni
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpi.uni
delete mode 100644
PcAtChipsetPkg/IsaAcpiDxe/IsaAcpiExtra.uni

--
2.12.0.windows.1