core20 install never completes on CM3 and rpi3a+

[Paul Larson]

Trying to boot the core20 image from http://cdimage.ubuntu.com/ubuntu-core/20/beta/20200513.2/ on a cm3, it gets as far as "Installing the system, please wait for reboot" but never completes. I've given it over an hour but it's still stuck there.

[Paul Larson (pwlars)]

See attached serial log from the initial boot. It never gets as far as rebooting after the install

[Michael Vogt (mvo)]

Hey, you mentioned in LP:#1878943 that removing "dtoverlay=vc4-fkms-v3d,cma-256 line from config.txt" has helped, does this trick work here too?

[Paul Larson (pwlars)]

I don't think so. I've had it booted for over 30 min. and on the serial console I've only reached:
Installing the system, please wait for reboot
[ 933.649361] systemd[1]: Starting Cleanup of Temporary Directories...
[ 933.699984] systemd[1]: systemd-tmpfiles-clean.service: Succeeded.
[ 933.702369] systemd[1]: Finished Cleanup of Temporary Directories.

[Paul Larson (pwlars)]

I'm seeing something exactly like this on rpi3a+ also, even when I remove the cma line. I'm seeing it with both the arm64 and the armhf image

[Paul Larson (pwlars)]

Added dmesg, journalctl -e, and snap debug changes

[Ian Johnson (anonymouse67)]

I have reproduced this in a VM by changing snap-bootstrap to mount tmpfs ubuntu-data in install mode with size=100M. The issue here is that we mount /writable for install mode as a tmpfs and there's not enough space in the tmpfs to copy all the snaps. the solution is to use a symlink in install mode's tmpfs /var/lib/snapd/snaps/ to /var/lib/snapd/seed/systems/<label>/snaps/<name>.snap, and then also follow that symlink in makeBootable20RunMode when we copy the files to ubuntu-data. I have a branch locally implementing this I will propose tomorrow.

[Ian Johnson (anonymouse67)]

PR open: https://github.com/snapcore/snapd/pull/8711

[Michael Vogt (mvo)]

Ians PRs have landed. I can can boot a Pi3 in armhf mode with pi r64, snapd r7978 but I don't have the hardware to test on the cm3. My Pi3 has 1Gb of ram.

[Ian Johnson (anonymouse67)]

Hi Paul, can you try to reproduce this with the current beta images on the CM3 and/or CM3+ with 512 MB of RAM?

[Paul Larson (pwlars)]

I just realized I talked to Ian about this but didn't update the bug yet. This seemed to work fine on the 20200529.3 beta image with both cm3 (1GB) and rpi3a+ (512MB)