Kernel 5.13 broke my Ubuntu 21.10 on RPI4 8Gb

[Jose Netto]

Hello all,

Ubuntu 21.10 on RPI4 and Kernel 5.13 broke my system when I use a USB SSD install.

Lets see all steps:

I downloaded Ubuntu 21.10 desktop img from ubuntu daily builds and installed it on a USB SSD (about a month ago).

Every things were all wright an for me it was better than 21.04. I was very happy.

Until last week, when Update manager came with a kernel 5.13 version. As usual I applied all updates and rebooted.

Reboot took a long time and terminated in a character screen and was written Busybox..... initram... Type help for... For what??? I simply didn't know how to deal with that info.

Than I decide to install Ubuntu 21.04, but firefox latest version was 90 and I would be in trouble to recover firefox (90.0.2) data files form 21.10 and use it.

So I decide to give ubuntu 21.10 another try, and installed it on a SDcard. set up finished lets update it all and reboot. Reboot successfully and than I copied SDcard to my USB SSD using SD Card Copy from Raspberry PI OS.

Reboot took a long time and terminated in a character screen and was written Busybox..... initram... Type help for...

Oh my God, expending a weekend to have a 21.10 working....

New try... Installed 21.10 again on RPI4 on USB SSD but... but in update manager UNMARKed all kernel 5.13 updates.

Now I am running Ubuntu 21.10 for 3 days but Kernel installed is 5.11.

I Hope this report could be useful for someone. And waiting for a fix.

Thanks,

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1943097/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[William Wilson (jawn-smith)]

I'm seeing this issue with the arm64 server image as well.

[William Wilson (jawn-smith)]

My logs over serial: https://paste.ubuntu.com/p/Wp4YdqyjJy/

[Jose Netto (vieirabrazil)]

Hi Everybody,

Today I tried to boot from bugged SSD and after keying ESC in the long wait for boot, I have got this new info:

Gave up waiting for root file system device. Common problems
 - Boot args (cat /proc/cmdline)
   - check rootdelay= did the system wait long enough?)
  - Missing modules (cat /proc/modules; ls /dev)
ALERT! LABEL=writable does not exist. Dropping to a shell!

BusyBox v1.30.1 (Ubuntu 1:1.30.1-6ubuntu2)....

See shots attached.

Question: Is Ubuntu thinking writable is BTRFS?

Thanks,

[Jose Netto (vieirabrazil)]

Hi Everybody,

More shots attached.

Thanks,

[William Wilson (jawn-smith)]

If I wait long enough I see:
mdadm: No devices listed in conf file were found.
Gave up waiting for root file system device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! LABEL=writable does not exist. Dropping to a shell!

And then end up in an initramfs shell.

[Jose Netto (vieirabrazil)]

Hi William,

Our error msgs are almost the same.

Is your computer a RPI4?

Do you use BTRFS?

What is wrong in that log you posted?

Thanks,

[William Wilson (jawn-smith)]

Jose, I am also using the Pi4 8GB. The real issue in these logs is that the system is failing to find the root file system to boot to. That's what is causing the "Gave up waiting for root file system device" log and causing the system to drop to an initramfs shell.

[Paul Broadhead (pjbroad)]

I'm also experiencing this after the latest 21.10 update on two builds on the same Raspberry Pi 400, booting from USB. I added a comment on https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1937924 which sounded similar. I have restored a bootable system by mounting the devices on another machine and replacing initrd.img and vmlinuz on the boot partition with the .bak versions.

[Juerg Haefliger (juergh)]

Thanks all for the reports. I'll take a look next week.

[Juerg Haefliger (juergh)]

I believe the problem is that with 5.13 we introduced a linux-modules-extra package and the USB storage drivers ended up in that package rather than linux-modules.

[Juerg Haefliger (juergh)]

The workaround for now is to manually install linux-modules-extra after upgrading to 5.13 but *before* rebooting.

[satmandu (satadru-umich)]

Could we have the usb-storage modules moved back out of linux-modules-extra? Alternately ensure that linux-modules-extra is installed on upgrades so installs don't break?

I don't think you want to break installs which don't have that installed. Booting from USB is officially supported on Raspberry Pi devices now, and ubuntu maybe shouldn't consider that extra-ordinary behavior requiring extra modules.

[Jose Netto (vieirabrazil)]

Hi Juerg,

The workaround is not working for me.

I took a new .img from Daily Builds (kernel 5.13 already installed) and made a SDCard install.

sudo apt update

sudo apt install linux-modules-extra-5.13.0-1006-raspi

rebooted.

Working.

Using another OS, in gparted copied SDCard to a SSD.

This system does not boot, only a black screen.

Hi Satmandu,

You are right, theorically. Indeed the system does not boot.

Thanks to all,

[Juerg Haefliger (juergh)]

The fix [1] is committed as indicated by the status of the task.

[1] https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux-raspi/+git/impish/commit/?h=Ubuntu-raspi-5.13.0-1007.8&id=162acbb01741e376605d7b04194e1952d49e711a

[satmandu (satadru-umich)]

My system does indeed boot with this fix!

Unfortunately, there is some other kernel breakage with linux-image-5.13.0-1007-raspi which is killing any display output and forcing me to do a hard reset to reboot the system.

So I've opened a separate bug for that here: https://bugs.launchpad.net/ubuntu/+source/linux-raspi/+bug/1944397

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1943097

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-raspi - 5.13.0-1007.8

---------------
linux-raspi (5.13.0-1007.8) impish; urgency=medium

  * impish/linux-raspi: 5.13.0-1007.8 -proposed tracker (LP: #1943746)

  * impish/linux-raspi: Update to upstream raspberrypi rpi-5.13.y (2021-08-27)
    (LP: #1942881)
    - bcm2835: Allow compressed frames to set sizeimage (#4386)
    - media: i2c: imx477: Fix for long exposure limit calculations
    - ASoC: hdmi-codec: Rework to support more controls
    - snd: iec958: split status creation and fill
    - ASoC: hdmi-codec: Add iec958 controls
    - ASoC: hdmi-codec: Add a prepare hook
    - drm/vc4: Make vc4_crtc_get_encoder public
    - drm/vc4: crtc: Add encoder to vc4_crtc_config_pv prototype
    - drm/vc4: crtc: Rework the encoder retrieval code (again)
    - drm/vc4: crtc: Add some logging
    - drm/vc4: Leverage the load tracker on the BCM2711
    - drm/vc4: Increase the core clock based on HVS load
    - clk: bcm2835: Pass DT node to rpi_firmware_get
    - ARM: dts: Add firmware reference to sdhost node
    - mmc: sdhost: Pass DT pointer to rpi_firmware_get
    - drm/vc4: Register HDMI codec
    - drm/vc4: hdmi: Remove redundant variables
    - media: i2c: imx290: Support 60fps in 2 lane operation
    - media: i2c: imx290: Fix the pixel rate at 148.5Mpix/s
    - media: i2c: imx290: Fix clock setup register assignments
    - drm/vc4: Fix timings for interlaced modes
    - drm/vc4: Don't create hvs_load_tracker on fkms
    - media: rpivid: Fix H265 aux ent reuse of the same slot
    - Support RPi DPI interface in mode6 for 18-bit color
    - overlays: Add dpi18cpadhi, vc4-kms-dpi-at056tn53v1
    - drm: vc4: Fix pixel-wrap issue with DVP teardown
    - drm/vc4: Increase the core clock to a minimum of 500MHz
    - media: i2c: ov9281: Remove override of subdev name
    - drm/vc4: hdmi: Use a fixed rate for the HSM clock on BCM2835
    - drm/vc4: hdmi: Enable the scrambler on reconnection
    - configs: Add CONFIG_MACVTAP=m (arm64 only)
    - staging: vc04_services: isp: Set the YUV420/YVU420 format stride to 64 bytes
    - Documentation: devicetree: Add documentation for imx378 sensor
    - overlays: Add overlay for imx378 sensor
    - media: i2c: imx477: Extend driver to support imx378 sensor
    - overlays: Make i2c-rtc and i2c-rtc-gpio share RTCs
    - dt-bindings: clk: raspberrypi: Remove unused property
    - dt-bindings: display: vc4: Add phandle to the firmware
    - firmware: raspberrypi: Add RPI_FIRMWARE_NOTIFY_DISPLAY_DONE
    - drm/vc4: Remove conflicting framebuffers before callind bind_all
    - drm/vc4: Notify the firmware when DRM is in charge
    - ARM: dts: rpi: Add the firmware node to vc4
    - drm/vc4: hdmi: Put the device on error in pre_crtc_configure
    - drm/vc4: hdmi: Split the CEC disable / enable functions in two
    - drm/vc4: hdmi: Make sure the device is powered with CEC
    - drm/vc4: hdmi: Add missing clk_disable_unprepare on error path
    - drm/vc4: hdmi: Warn if we access the controller while disabled
    - drm/vc4: hdmi: Move initial register read after pm_runtime_get
    - vc4/drv: Only notify firmware of display done with kms
    - dwc_otg: Update NetBSD usb.h header li...

[Jose Netto (vieirabrazil)]

Thanks Juerg,

Now it is booting... kernel 5.13.

But now Desktop is frizzing with no return, no mouse movement and no keyboard response. Once during sudo apt update, other during sudo apt install vlc and other time during updating Region and Language...

So I gave up...

What do I do, Open another report???

Thanks again,

[Juerg Haefliger (juergh)]

Probably the same issue as LP: #1944397 but I can't be certain without seeing logs.