Activity log for bug #1937295

Date Who What changed Old value New value Message
2021-07-22 17:16:15 Jeff Lane  bug added bug
2021-07-22 21:34:29 Michael Hudson-Doyle affects syslinux (Ubuntu) subiquity
2021-08-23 08:02:51 shangsong attachment added latest 20.10 sosreport https://bugs.launchpad.net/subiquity/+bug/1937295/+attachment/5519932/+files/sosreport-ubuntu-server-1937295-2021-08-23-zjuwwzn.tar.xz
2021-08-25 10:33:22 shangsong attachment added sosreport-ubuntu-server-raid10-2021-08-25-nbgxkng.tar.xz https://bugs.launchpad.net/subiquity/+bug/1937295/+attachment/5520451/+files/sosreport-ubuntu-server-raid10-2021-08-25-nbgxkng.tar.xz
2021-08-25 10:34:31 shangsong bug added subscriber shangsong
2022-01-18 13:47:56 shangsong attachment added sosreport for 18.04.6 LTS https://bugs.launchpad.net/subiquity/+bug/1937295/+attachment/5555170/+files/sosreport-ubuntu-server-1937295-2022-01-18-tpjuxzk.tar.xz
2022-01-18 17:12:26 Jeff Lane  subiquity: status New Incomplete
2022-02-15 23:17:36 Jeff Lane  bug task added linux (Ubuntu)
2022-02-15 23:17:53 Jeff Lane  nominated for series Ubuntu Jammy
2022-02-15 23:17:53 Jeff Lane  bug task added linux (Ubuntu Jammy)
2022-02-15 23:17:53 Jeff Lane  nominated for series Ubuntu Focal
2022-02-15 23:17:53 Jeff Lane  bug task added linux (Ubuntu Focal)
2022-02-15 23:17:53 Jeff Lane  nominated for series Ubuntu Impish
2022-02-15 23:17:53 Jeff Lane  bug task added linux (Ubuntu Impish)
2022-02-15 23:17:53 Jeff Lane  nominated for series Ubuntu Bionic
2022-02-15 23:17:53 Jeff Lane  bug task added linux (Ubuntu Bionic)
2022-02-15 23:30:09 Ubuntu Kernel Bot linux (Ubuntu): status New Incomplete
2022-02-15 23:30:11 Ubuntu Kernel Bot linux (Ubuntu Bionic): status New Incomplete
2022-02-15 23:30:13 Ubuntu Kernel Bot linux (Ubuntu Focal): status New Incomplete
2022-02-15 23:30:16 Ubuntu Kernel Bot linux (Ubuntu Impish): status New Incomplete
2022-02-16 04:44:36 Jeff Lane  summary Installation hangs on VROC systems during Bionic installs Installation hangs on VROC systems during Bionic installsPCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2022-02-16 04:44:48 Jeff Lane  summary Installation hangs on VROC systems during Bionic installsPCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2022-02-16 04:53:26 Jeff Lane  linux (Ubuntu Bionic): status Incomplete Invalid
2022-02-16 04:53:30 Jeff Lane  linux (Ubuntu Focal): status Incomplete In Progress
2022-02-16 04:53:32 Jeff Lane  linux (Ubuntu Impish): status Incomplete In Progress
2022-02-16 04:53:34 Jeff Lane  linux (Ubuntu Jammy): status Incomplete In Progress
2022-02-16 04:53:37 Jeff Lane  linux (Ubuntu Focal): assignee Jeff Lane (bladernr)
2022-02-16 04:53:39 Jeff Lane  linux (Ubuntu Impish): assignee Jeff Lane (bladernr)
2022-02-16 04:53:41 Jeff Lane  linux (Ubuntu Jammy): assignee Jeff Lane (bladernr)
2022-02-16 04:53:44 Jeff Lane  linux (Ubuntu Jammy): importance Undecided Medium
2022-02-16 04:53:45 Jeff Lane  linux (Ubuntu Impish): importance Undecided Medium
2022-02-16 04:53:47 Jeff Lane  linux (Ubuntu Focal): importance Undecided Medium
2022-02-16 04:54:47 Jeff Lane  description A hardware partner has been testing 18.04 subiquity images on their servers with VROC enabled and configured in a RAID 10 setup. In their own words: Steps to reproduce: 1. Boot into VRoC controller in uEFI Setup and create a raid10 disk. 2. Install Ubuntu 18.04.5 on the RAID10. 3. The system hangs at "Partitions formatting 33%". After looking at the launchpad (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1896578/), the fix was included in the updated kernel. [Quotes from the launchpad] The released kernels are: Hirsute: 5.11.0-22-generic Groovy: 5.8.0-59-generic Focal: 5.4.0-77-generic Bionic: 4.15.0-147-generic I've asked them to also confirm this on 20.04.2, and check that 20.04.3 dailies fix the issue. It is at least a very reasonable hypothesis that this will also break on all current ISO installs as none of them are respun once released to include updated SRUs in the installation media. This currently affects 20.04.2 but that will be resolved shortly when 20.04.3 releases as the GA and HWE kernels in that image should have the SRU that fixes this issue. However, 18.04 has no further releases, and even the 18.04.5 daily-live and daily images on cdimages.ubuntu.com are not built after 18.04.5 was released. [Impact] A hardware partner discovered they were unable to install Ubuntu on some servers using VROC setups. They point to this issue involving DMAR that is blocking discovery of the VROC RAID devices: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2565e5b69c44b4e42469afea3cc5a97e74d1ed45 `git bisect` points to this offending commit ee81ee84f873 ("PCI: vmd: Disable MSI-X remapping when possible"), which disables VMD MSI remapping. The IOMMU hardware blocks the compatibility format interrupt request because Interrupt Remapping Enable Status (IRES) and Extended Interrupt Mode Enable (EIME) are enabled. Please refer to section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d spec. To fix the issue, VMD driver still enables the interrupt remapping irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem enables the interrupt remapping. [Fix] 2565e5b69c44 PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU [Test Plan]     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install affected Ubuntu release on the RAID10.     3. The system hangs at "Partitions formatting 33%". [Where problems could occur] The fix itself is a very small change to drivers/pci/controller/vmd.c and problems should not occur, the patch has been tested and accepted upstream as of 5.16. That said, I doubt this will fix 18.04.6 as it would require a respin to get the patched kernel onto the ISO. 20.04 should pick it up in ISO in 20.04.5, so there could still be the initial issue since those ISOs would be lacking the patched kernel. [Other Info] As noted, this would need to not only land in the kernel but land in the kernel in the ISO to resolve the issue in the installation process. I'll bring this back as far as Focal with the expectation that while 20.04.4 is too late, it will be present using the GA kernel in 20.04.5 later on. ************************************************************************* Original Bug Summary: A hardware partner has been testing 18.04 subiquity images on their servers with VROC enabled and configured in a RAID 10 setup. In their own words: Steps to reproduce:     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install Ubuntu 18.04.5 on the RAID10.     3. The system hangs at "Partitions formatting 33%". After looking at the launchpad (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1896578/), the fix was included in the updated kernel.    [Quotes from the launchpad]       The released kernels are:               Hirsute: 5.11.0-22-generic               Groovy: 5.8.0-59-generic               Focal: 5.4.0-77-generic               Bionic: 4.15.0-147-generic I've asked them to also confirm this on 20.04.2, and check that 20.04.3 dailies fix the issue. It is at least a very reasonable hypothesis that this will also break on all current ISO installs as none of them are respun once released to include updated SRUs in the installation media. This currently affects 20.04.2 but that will be resolved shortly when 20.04.3 releases as the GA and HWE kernels in that image should have the SRU that fixes this issue. However, 18.04 has no further releases, and even the 18.04.5 daily-live and daily images on cdimages.ubuntu.com are not built after 18.04.5 was released.
2022-02-16 04:57:18 Jeff Lane  description [Impact] A hardware partner discovered they were unable to install Ubuntu on some servers using VROC setups. They point to this issue involving DMAR that is blocking discovery of the VROC RAID devices: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2565e5b69c44b4e42469afea3cc5a97e74d1ed45 `git bisect` points to this offending commit ee81ee84f873 ("PCI: vmd: Disable MSI-X remapping when possible"), which disables VMD MSI remapping. The IOMMU hardware blocks the compatibility format interrupt request because Interrupt Remapping Enable Status (IRES) and Extended Interrupt Mode Enable (EIME) are enabled. Please refer to section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d spec. To fix the issue, VMD driver still enables the interrupt remapping irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem enables the interrupt remapping. [Fix] 2565e5b69c44 PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU [Test Plan]     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install affected Ubuntu release on the RAID10.     3. The system hangs at "Partitions formatting 33%". [Where problems could occur] The fix itself is a very small change to drivers/pci/controller/vmd.c and problems should not occur, the patch has been tested and accepted upstream as of 5.16. That said, I doubt this will fix 18.04.6 as it would require a respin to get the patched kernel onto the ISO. 20.04 should pick it up in ISO in 20.04.5, so there could still be the initial issue since those ISOs would be lacking the patched kernel. [Other Info] As noted, this would need to not only land in the kernel but land in the kernel in the ISO to resolve the issue in the installation process. I'll bring this back as far as Focal with the expectation that while 20.04.4 is too late, it will be present using the GA kernel in 20.04.5 later on. ************************************************************************* Original Bug Summary: A hardware partner has been testing 18.04 subiquity images on their servers with VROC enabled and configured in a RAID 10 setup. In their own words: Steps to reproduce:     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install Ubuntu 18.04.5 on the RAID10.     3. The system hangs at "Partitions formatting 33%". After looking at the launchpad (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1896578/), the fix was included in the updated kernel.    [Quotes from the launchpad]       The released kernels are:               Hirsute: 5.11.0-22-generic               Groovy: 5.8.0-59-generic               Focal: 5.4.0-77-generic               Bionic: 4.15.0-147-generic I've asked them to also confirm this on 20.04.2, and check that 20.04.3 dailies fix the issue. It is at least a very reasonable hypothesis that this will also break on all current ISO installs as none of them are respun once released to include updated SRUs in the installation media. This currently affects 20.04.2 but that will be resolved shortly when 20.04.3 releases as the GA and HWE kernels in that image should have the SRU that fixes this issue. However, 18.04 has no further releases, and even the 18.04.5 daily-live and daily images on cdimages.ubuntu.com are not built after 18.04.5 was released. [Impact] A hardware partner discovered they were unable to install Ubuntu on some servers using VROC setups. They point to this issue involving DMAR that is blocking discovery of the VROC RAID devices: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2565e5b69c44b4e42469afea3cc5a97e74d1ed45 `git bisect` points to this offending commit ee81ee84f873 ("PCI: vmd: Disable MSI-X remapping when possible"), which disables VMD MSI remapping. The IOMMU hardware blocks the compatibility format interrupt request because Interrupt Remapping Enable Status (IRES) and Extended Interrupt Mode Enable (EIME) are enabled. Please refer to section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d spec. To fix the issue, VMD driver still enables the interrupt remapping irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem enables the interrupt remapping. [Fix] 2565e5b69c44 PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU [Test Plan]     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install affected Ubuntu release on the RAID10.     3. The system hangs at "Partitions formatting 33%". [Where problems could occur] The fix itself is a very small change to drivers/pci/controller/vmd.c and problems should not occur. The root cause was discovered by the hardware partner's engineers, who tested and submitted it upstream where it was accepted and landed in 5.16. That said, I doubt this will fix 18.04.6 as it would require a respin to get the patched kernel onto the ISO. 20.04 should pick it up in ISO in 20.04.5, so there could still be the initial issue since those ISOs would be lacking the patched kernel. [Other Info] As noted, this would need to not only land in the kernel but land in the kernel in the ISO to resolve the issue in the installation process. I'll bring this back as far as Focal with the expectation that while 20.04.4 is too late, it will be present using the GA kernel in 20.04.5 later on. ************************************************************************* Original Bug Summary: A hardware partner has been testing 18.04 subiquity images on their servers with VROC enabled and configured in a RAID 10 setup. In their own words: Steps to reproduce:     1. Boot into VRoC controller in uEFI Setup and create a raid10 disk.     2. Install Ubuntu 18.04.5 on the RAID10.     3. The system hangs at "Partitions formatting 33%". After looking at the launchpad (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1896578/), the fix was included in the updated kernel.    [Quotes from the launchpad]       The released kernels are:               Hirsute: 5.11.0-22-generic               Groovy: 5.8.0-59-generic               Focal: 5.4.0-77-generic               Bionic: 4.15.0-147-generic I've asked them to also confirm this on 20.04.2, and check that 20.04.3 dailies fix the issue. It is at least a very reasonable hypothesis that this will also break on all current ISO installs as none of them are respun once released to include updated SRUs in the installation media. This currently affects 20.04.2 but that will be resolved shortly when 20.04.3 releases as the GA and HWE kernels in that image should have the SRU that fixes this issue. However, 18.04 has no further releases, and even the 18.04.5 daily-live and daily images on cdimages.ubuntu.com are not built after 18.04.5 was released.
2022-02-16 15:30:07 Jeff Lane  summary PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU [SRU]PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU
2022-02-16 15:42:39 Jeff Lane  subiquity: status Incomplete Invalid
2022-02-17 12:50:02 Kleber Sacilotto de Souza linux (Ubuntu Impish): status In Progress Fix Committed
2022-02-24 11:02:47 Ubuntu Kernel Bot tags verification-needed-impish
2022-03-02 11:29:47 shangsong attachment added dmesg and OS info https://bugs.launchpad.net/ubuntu/jammy/+source/linux/+bug/1937295/+attachment/5564971/+files/ubuntu2110.txt
2022-03-17 09:37:36 Kleber Sacilotto de Souza tags verification-needed-impish verification-done-impish
2022-03-17 14:02:38 Ubuntu Kernel Bot tags verification-done-impish verification-done-impish verification-needed-focal
2022-03-18 15:27:27 Launchpad Janitor linux (Ubuntu Jammy): status In Progress Fix Released
2022-03-18 15:27:27 Launchpad Janitor cve linked 2021-26401
2022-03-18 15:27:27 Launchpad Janitor cve linked 2022-0001
2022-03-18 15:27:27 Launchpad Janitor cve linked 2022-0435
2022-03-18 15:27:27 Launchpad Janitor cve linked 2022-0516
2022-03-18 15:27:27 Launchpad Janitor cve linked 2022-23960
2022-03-18 15:27:27 Launchpad Janitor cve linked 2022-25636
2022-03-21 15:36:28 Launchpad Janitor linux (Ubuntu Impish): status Fix Committed Fix Released
2022-03-21 15:36:28 Launchpad Janitor cve linked 2022-0492
2022-03-21 15:36:28 Launchpad Janitor cve linked 2022-0742
2022-03-21 15:36:28 Launchpad Janitor cve linked 2022-23222
2022-04-04 19:27:21 Jeff Lane  tags verification-done-impish verification-needed-focal verification-done-focal verification-done-impish
2022-05-12 08:01:34 koba bug task added linux-oem-5.14 (Ubuntu)
2022-05-12 08:02:59 koba linux-oem-5.14 (Ubuntu Focal): status New In Progress
2022-05-12 08:02:59 koba linux-oem-5.14 (Ubuntu Focal): assignee koba (kobako)
2022-05-12 08:03:32 koba bug task added linux-oem-5.17 (Ubuntu)
2022-05-12 08:04:01 koba linux-oem-5.17 (Ubuntu Jammy): status New In Progress
2022-05-12 08:04:01 koba linux-oem-5.17 (Ubuntu Jammy): assignee koba (kobako)
2022-05-12 08:28:20 koba bug added subscriber Canonical Hardware Enablement
2022-05-12 08:28:24 koba tags verification-done-focal verification-done-impish oem-priority originate-from-1967153 somerville verification-done-focal verification-done-impish
2022-05-13 17:59:55 Timo Aaltonen linux-oem-5.17 (Ubuntu Jammy): status In Progress Invalid
2022-05-13 18:00:07 Timo Aaltonen linux-oem-5.14 (Ubuntu Bionic): status New Invalid
2022-05-13 18:00:19 Timo Aaltonen linux-oem-5.14 (Ubuntu Impish): status New Invalid
2022-05-13 18:00:28 Timo Aaltonen linux-oem-5.17 (Ubuntu Impish): status New Invalid
2022-05-13 18:00:40 Timo Aaltonen linux-oem-5.17 (Ubuntu Focal): status New Invalid
2022-05-13 18:00:53 Timo Aaltonen linux-oem-5.17 (Ubuntu Bionic): status New Invalid
2022-05-13 18:01:04 Timo Aaltonen linux-oem-5.17 (Ubuntu): status New Invalid
2022-05-13 18:01:17 Timo Aaltonen linux-oem-5.14 (Ubuntu Jammy): status New Invalid
2022-05-13 18:01:34 Timo Aaltonen linux-oem-5.14 (Ubuntu): status New Invalid
2022-05-13 18:01:59 Timo Aaltonen linux-oem-5.14 (Ubuntu Focal): status In Progress Fix Committed
2022-09-01 09:46:11 Timo Aaltonen linux-oem-5.14 (Ubuntu Focal): status Fix Committed In Progress
2022-09-02 11:41:12 Timo Aaltonen linux-oem-5.14 (Ubuntu Focal): status In Progress Won't Fix
2022-09-02 11:41:37 Timo Aaltonen hwe-next: status New Fix Released