qemu-arm-static swallows the program name, and shifts all arguments left
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
qemu (Ubuntu) |
Fix Released
|
Critical
|
Christian Ehrhardt | ||
Impish |
Fix Released
|
Undecided
|
Christian Ehrhardt |
Bug Description
[Impact]
* The kernel changed handling of binfmt-P and that breaks qemu
using binfmt registering TCG based emulators to run foreign
architecture code.
* This happened late 5.11.0-20 -> 5.13.0-20 and even as of today
cloud-images start with the former :-/ That hides this unless you
properly upgrade and reboot.
* This was already reported and fixed in 1:6.1+dfsg-5 and this is
backporting this fix
[Test Plan]
* Based on the great report and evolving on that there is a small
but very useful test.
1. System (Kernel + userspace dependent, so it needs to be a VM)
=> Get a VM of the target release to test
2. Install dependencies
$ sudo apt update
$ sudo apt upgrade -y
$ sudo apt install -y qemu-user-static debootstrap
3. Prep foreign arch chroots for armhf
(other architectures would work as well)
# I tested this with various userspace versions, but behavior
# only depends on kernel + qmeu-user-static
$ sudo debootstrap --foreign --verbose --arch=armhf jammy armTest-J
4. Run something int he chroot, which will trigger binfmt -> qemu
$ sudo chroot armTest-I echo 1 2 3
Bad result: 2 3
Good case with fix: 1 2 3
There also is an extended testcase in the report, but it tests and achieves the same, so I picked this simpler one. Everyone is welcome to test his own case once this is up for SRU verification.
[Where problems could occur]
* This is changing binfmt integration for qemu-user-static, so of the
many things qemu does we do not have to think about classic
virtualization or even emulation. Only cross arch emulation through
binfmt (like the test examples with qemu-user-static + foreign arch
debootstrap) are what will be affected.
[Other Info]
* Older qemu seems non-affected by using binfmt differently back then.
* While this is actually a regression-
qemu, the new qemu code works fine with the older kernel behavior as
well.
---- ---- ----
Problem with qemu-arm-static version 1:6.0+dfsg-
When using qemu-arm-static for cross-debootstrap for an ARM machine argument 0 of the invoked ARM program does not contain the name of the executable but argument 1, argument 2 is passed as argument 1 and so forth.
Thus all program calls are missing the 1st argument.
Eample: A shell script test.sh
-- Start -------
#!/bin/sh
echo "\$0=$0"
echo "\$1=$1"
echo "\$2=$2"
echo "\$*=$*"
-- End -------
Running the script as "./test.sh a bb ccc dddd" emits:
$0=./test.sh
$1=a
$2=bb
$*=a bb ccc dddd
Reproduce the erroneous behavior of qemu-arm-static
Create a directory armTest. Enter:
sudo debootstrap --foreign --verbose --arch=armhf impish armTest
sudo cp test.sh armTest
sudo cp -v /usr/bin/
Now run
sudo chroot armTest ./test.sh a bb ccc dddd
The script is now run by the shell for the ARM architecture, and is invoked by qemu-arm-static.
The script now emits
$0=a
$1=bb
$2=ccc
$*=bb ccc dddd
argument 0 (the program name) disappeared, and all arguments are shifted up.
Evidence that actually qemu-arm-static is the culprit:
In /etc/apt/
sudo apt-get update
sudo apt-get install qemu-user-
Confirm that you want to downgrade the program to version 1:5.2+dfsg-
Copy the downgraded qemu-arm-static to armTest/usr/bin. Repeat the chroot command. Now the result is the same as invoking test.sh directly under your x86_64 architecture.
This bug does not only affect shell scripts. Apparently all programs are affected by the shifted program arguments.
This bug makes a complete cross-debootstrap impossible because the second debootstrap stage is run as chroot in the target architecture. Further steps to prepare a ready-to-boot image/SD card are equally impossible.
ProblemType: Bug
DistroRelease: Ubuntu 21.10
Package: qemu-user-static 1:6.0+dfsg-
ProcVersionSign
Uname: Linux 5.13.0-20-generic x86_64
ApportVersion: 2.20.11-0ubuntu70
Architecture: amd64
CasperMD5CheckR
CurrentDesktop: LXQt
Date: Wed Oct 20 13:35:52 2021
KvmCmdLine: COMMAND STAT EUID RUID PID PPID %CPU COMMAND
MachineType: VMware, Inc. VMware Virtual Platform
ProcKernelCmdLine: BOOT_IMAGE=
SourcePackage: qemu
UpgradeStatus: Upgraded to impish on 2021-10-16 (3 days ago)
dmi.bios.date: 11/12/2020
dmi.bios.release: 4.6
dmi.bios.vendor: Phoenix Technologies LTD
dmi.bios.version: 6.00
dmi.board.name: 440BX Desktop Reference Platform
dmi.board.vendor: Intel Corporation
dmi.board.version: None
dmi.chassis.
dmi.chassis.type: 1
dmi.chassis.vendor: No Enclosure
dmi.chassis.
dmi.ec.
dmi.modalias: dmi:bvnPhoenixT
dmi.product.name: VMware Virtual Platform
dmi.product.
dmi.sys.vendor: VMware, Inc.
Related branches
- Paride Legovini (community): Approve
- Canonical Server: Pending requested
- git-ubuntu import: Pending requested
-
Diff: 88 lines (+55/-2)2 files modifieddebian/changelog (+7/-0)
debian/patches/linux-user-binfmt-P.diff (+48/-2)
- Paride Legovini (community): Approve
- Canonical Server packageset reviewers: Pending requested
- git-ubuntu import: Pending requested
-
Diff: 88 lines (+55/-2)2 files modifieddebian/changelog (+7/-0)
debian/patches/linux-user-binfmt-P.diff (+48/-2)
tags: | added: qemu-arm-static qemu-user-static |
Changed in qemu (Ubuntu): | |
importance: | Undecided → High |
description: | updated |
Changed in qemu (Ubuntu): | |
assignee: | nobody → Christian Ehrhardt (paelzer) |
description: | updated |
Changed in qemu (Ubuntu Impish): | |
assignee: | nobody → Christian Ehrhardt (paelzer) |
Hello and thanks for this bug report. I can confirm this issue, reproduced by:
# With qemu-user-static 1:5.2+dfsg- 9ubuntu3. 2 (Hirsute):
$ sudo chroot armTest /usr/bin/echo 1 2 3
1 2 3
# With qemu-user-static 1:6.0+dfsg- 2expubuntu1 (Impish/Jammy):
$ sudo chroot armTest /usr/bin/echo 1 2 3
2 3
which is clearly wrong. (Note: I didn't copy qemu-arm-static in the chroot, I don't think that's needed.) @Christian: any idea of what could be going on here?