Activity log for bug #1950144

Date Who What changed Old value New value Message
2021-11-08 10:29:15 bugproxy bug added bug
2021-11-08 10:29:18 bugproxy tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin---
2021-11-08 10:29:20 bugproxy ubuntu: assignee Skipper Bug Screeners (skipper-screen-team)
2021-11-08 10:29:24 bugproxy affects ubuntu linux (Ubuntu)
2021-11-08 12:29:54 Frank Heimes bug task added ubuntu-z-systems
2021-11-08 12:30:16 Frank Heimes ubuntu-z-systems: status New Triaged
2021-11-08 12:30:21 Frank Heimes ubuntu-z-systems: importance Undecided High
2021-11-08 12:30:37 Frank Heimes ubuntu-z-systems: assignee Skipper Bug Screeners (skipper-screen-team)
2021-11-08 12:32:49 Frank Heimes nominated for series Ubuntu Hirsute
2021-11-08 12:32:49 Frank Heimes bug task added linux (Ubuntu Hirsute)
2021-11-08 12:32:49 Frank Heimes nominated for series Ubuntu Focal
2021-11-08 12:32:49 Frank Heimes bug task added linux (Ubuntu Focal)
2021-11-08 12:32:49 Frank Heimes nominated for series Ubuntu Jammy
2021-11-08 12:32:49 Frank Heimes bug task added linux (Ubuntu Jammy)
2021-11-08 12:32:49 Frank Heimes nominated for series Ubuntu Impish
2021-11-08 12:32:49 Frank Heimes bug task added linux (Ubuntu Impish)
2021-11-08 12:34:01 Frank Heimes linux (Ubuntu Jammy): status New Fix Released
2021-11-08 12:34:13 Frank Heimes linux (Ubuntu Jammy): assignee Skipper Bug Screeners (skipper-screen-team)
2021-11-09 12:50:36 Frank Heimes linux (Ubuntu Impish): status New In Progress
2021-11-09 12:50:41 Frank Heimes linux (Ubuntu Hirsute): status New In Progress
2021-11-09 12:50:45 Frank Heimes linux (Ubuntu Focal): status New In Progress
2021-11-09 12:51:00 Frank Heimes linux (Ubuntu Impish): assignee Canonical Kernel Team (canonical-kernel-team)
2021-11-09 12:51:13 Frank Heimes linux (Ubuntu Hirsute): assignee Canonical Kernel Team (canonical-kernel-team)
2021-11-09 12:51:24 Frank Heimes linux (Ubuntu Focal): assignee Canonical Kernel Team (canonical-kernel-team)
2021-11-09 12:52:09 Frank Heimes description Description: kernel: unable to read partitions on virtio-block dasd (kvm) Symptom: unable to read partitions on virtio-block dasd (kvm) Problem: verify is called before virtio_finalize_features() , so a transitional s390 virtio device still serves native endian (i.e. big endian) config space, while the driver knows that it is going to accept VERSION_1, so when reading the config space, it assumes it got little endian, and byteswaps. Solution: For QEMU, we can work around the issue by writing out the feature bits with VIRTIO_F_VERSION_1 bit set. We (ab)use the finalize_features config op for this. This isn't enough to address all vhost devices since these do not get the features until FEATURES_OK, however it looks like the affected devices actually never handled the endianness for legacy mode correctly, so at least that's not a regression. SRU Justification: ================== [Impact] * The kernel is unable to read partitions on virtio-block DASD (on KVM). That's a severe situation, since it prevents Ubuntu from starting, if installed on a DASD. This issue can either occur after a fresh installation or after an upgrade. * The virtio specification virtio-v1.1-cs01 states: "Transitional devices MUST detect Legacy drivers by detecting that VIRTIO_F_VERSION_1 has not been acknowledged by the driver." And this is what QEMU as of 6.1 has done relying solely on VIRTIO_F_VERSION_1 for detecting that. * But the specification also says: "... the driver MAY read (but MUST NOT write) the device-specific configuration fields to check that it can support the device ..." before setting FEATURES_OK. * In this case, any transitional device relying solely on VIRTIO_F_VERSION_1 for detecting legacy drivers will return data in legacy format. In particular, this implies that it's in big endian format for big endian guests. This naturally confuses the driver that expects little endian in the modern mode. * VIRTIO_F_VERSION_1 can only be relied on after the feature negotiation is done. * 'verify' is called before virtio_finalize_features(), so a transitional s390 virtio device still serves native endian (i.e. big endian) config space, while the driver knows that it is going to accept VERSION_1, so when reading the config space, it assumes it got little endian, and byteswaps. * For QEMU, we can work around the issue by writing out the feature bits with VIRTIO_F_VERSION_1 bit set. We (ab)use the finalize_features config op for this. This isn't enough to address all vhost devices since these do not get the features until FEATURES_OK, however it looks like the affected devices actually never handled the endianness for legacy mode correctly, so at least that's not a regression. [Fix] * 2f9a174f918e29608564c7a4e8329893ab604fb4 2f9a174f918e "virtio: write back F_VERSION_1 before validate" [Test Case] * Setup an IBM Z or LinuxONE LPAR with Ubuntu Server 20.04 as KVM host. * This Ubuntu KVM host can either be installed on FCP or DASD storage, but at least one DASD disk need to be reserved for a KVM guest. * Now hand over the reserved DASD disk (low-level formatted using dasdfmt and partitioned using fdasd) using 'virtio-block' to a KVM virtual machine (e.g. using a virsh VM config). * Try to install an Ubuntu KVM virtual machine using this DASD disk, that includes the check and read of the partition table. [Where problems could occur] * First of all requested commit contains one additional if statement; and is due tothat relatively traceable. * But the change is in /drivers/virtio/virtio.c, means in common code! * This issue obviously affects big endian systems only. * But if done wrong, it may effect in worst-case little endian systems, too! * But the if statement explicitly checks for '!virtio_legacy_is_little_endian()'. * Only virtio net and virtio blk devices seem to be affected. * And the commit/solutions was in-depth discussed upstream here: https://lore.kernel.org/all/20211011053921.1198936-1-pasic@linux.ibm.com/t/#u [Other] * Patches are upstream accepted with since 5.15-rc6 and tagged for upstream stable #v4.11. Hence jammy is not affected. * Request was to add the patches to focal / 20.04, but to avoid potential regressions on upgrades, the patches need to be added to impish and hirsute, too. * Fortunately cherry-picking the commit works cleanly from all the affected Ubuntu releases. __________ Description: kernel: unable to read partitions on virtio-block dasd (kvm) Symptom: unable to read partitions on virtio-block dasd (kvm) Problem: verify is called before virtio_finalize_features() , so a               transitional s390 virtio device still serves native endian               (i.e. big endian) config space, while the driver knows that it               is going to accept VERSION_1, so when reading the config space,               it assumes it got little endian, and byteswaps. Solution: For QEMU, we can work around the issue by writing out the               feature bits with VIRTIO_F_VERSION_1 bit set. We (ab)use the               finalize_features config op for this. This isn't enough to               address all vhost devices since these do not get the features               until FEATURES_OK, however it looks like the affected devices               actually never handled the endianness for legacy mode correctly,               so at least that's not a regression.
2021-11-09 12:52:16 Frank Heimes ubuntu-z-systems: status Triaged In Progress
2021-11-22 16:27:02 Stefan Bader linux (Ubuntu Impish): importance Undecided Medium
2021-11-22 16:27:07 Stefan Bader linux (Ubuntu Hirsute): importance Undecided Medium
2021-11-22 16:27:12 Stefan Bader linux (Ubuntu Focal): importance Undecided Medium
2021-11-22 16:27:34 Stefan Bader linux (Ubuntu Impish): status In Progress Fix Committed
2021-11-22 16:28:30 Stefan Bader linux (Ubuntu Hirsute): status In Progress Fix Committed
2021-11-22 16:32:01 Stefan Bader linux (Ubuntu Focal): status In Progress Fix Committed
2021-11-22 16:41:21 Frank Heimes ubuntu-z-systems: status In Progress Fix Committed
2021-11-29 18:46:16 Ubuntu Kernel Bot tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-impish
2021-11-29 18:47:45 Ubuntu Kernel Bot tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-impish architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-hirsute verification-needed-impish
2021-11-29 18:49:12 Ubuntu Kernel Bot tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-hirsute verification-needed-impish architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-focal verification-needed-hirsute verification-needed-impish
2021-12-03 16:28:28 Frank Heimes tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-needed-focal verification-needed-hirsute verification-needed-impish architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-done-focal verification-done-hirsute verification-done-impish
2021-12-03 16:28:39 bugproxy tags architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin--- verification-done-focal verification-done-hirsute verification-done-impish architecture-s39064 bugnameltc-195009 severity-high targetmilestone-inin2004 verification-done-focal verification-done-hirsute verification-done-impish
2022-01-04 14:28:24 Launchpad Janitor linux (Ubuntu Focal): status Fix Committed Fix Released
2022-01-04 14:28:24 Launchpad Janitor cve linked 2020-26541
2022-01-04 14:28:24 Launchpad Janitor cve linked 2021-4002
2022-01-04 14:34:32 Launchpad Janitor linux (Ubuntu Hirsute): status Fix Committed Fix Released
2022-01-04 14:34:32 Launchpad Janitor cve linked 2021-43267
2022-01-04 14:36:58 Launchpad Janitor linux (Ubuntu Impish): status Fix Committed Fix Released
2022-01-04 16:31:42 Frank Heimes ubuntu-z-systems: status Fix Committed Fix Released