Missing initrd.img-core in os.snap

[Paolo Pisati]

I'm puzzled by this new "behaviour" of the kernel plugin in snapcraft - it's reproducible in every version i tried (2.35...2.38).

$ cat snapcraft.yaml
name: kernel
version: 4.13.16+
summary: Linux kernel
description: The upstream Linux kernel
grade: stable
confinement: strict
type: kernel

parts:
  kernel:
    plugin: kernel
    source: .
    source-type: git
    kernel-with-firmware: false

$ cd linux-vanilla (any recent vanilla linux tree will work, i'm using stable/linux-4.13.y FWIW)
$ make defconfig
$ snapcraft
...
Generating driver initrd for kernel release: 4.13.16+
Parallel unsquashfs: Using 4 processors
2 inodes (2 blocks) to write

[===========================================================================================================================================|] 2/2 100%

created 0 files
created 4 directories
created 2 symlinks
created 0 devices
created 0 fifos
...
Traceback (most recent call last): [3252/9137]
  File "/home/flag/canonical/snapcraft/bin/snapcraft", line 36, in <module>
    obj=dict(project=snapcraft.ProjectOptions()))
  File "/usr/lib/python3/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1037, in invoke
    return Command.invoke(self, ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/flag/canonical/snapcraft/snapcraft/cli/__init__.py", line 124, in run
    ctx.forward(lifecyclecli.commands['snap'])
  File "/usr/lib/python3/dist-packages/click/core.py", line 552, in forward
    return self.invoke(cmd, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/flag/canonical/snapcraft/snapcraft/cli/lifecycle.py", line 140, in snap
    project_options, directory=directory, output=output)
  File "/home/flag/canonical/snapcraft/snapcraft/internal/lifecycle/_packer.py", line 45, in snap
    execute('prime', project_options)
  File "/home/flag/canonical/snapcraft/snapcraft/internal/lifecycle/_runner.py", line 80, in execute
    _Executor(config, project_options).run(step, part_names)
  File "/home/flag/canonical/snapcraft/snapcraft/internal/lifecycle/_runner.py", line 175, in run
    self._run_step(step, part, part_names)
  File "/home/flag/canonical/snapcraft/snapcraft/internal/lifecycle/_runner.py", line 212, in _run_step
    getattr(part, step)()
  File "/home/flag/canonical/snapcraft/snapcraft/internal/pluginhandler/__init__.py", line 330, in build
    self.plugin.build()
  File "/home/flag/canonical/snapcraft/snapcraft/plugins/kbuild.py", line 258, in build
    self.do_install()
  File "/home/flag/canonical/snapcraft/snapcraft/plugins/kernel.py", line 473, in do_install
    self._make_initrd()
  File "/home/flag/canonical/snapcraft/snapcraft/plugins/kernel.py", line 286, in _make_initrd
    initrd_unpacked_path = self._unpack_generic_initrd()
  File "/home/flag/canonical/snapcraft/snapcraft/plugins/kernel.py", line 255, in _unpack_generic_initrd
    mime_type = mime_detector.file(os.path.realpath(tmp_initrd_path))
  File "/home/flag/.local/lib/python3.5/site-packages/magic.py", line 134, in file
    return str(r, 'utf-8')
TypeError: coercing to str: need a bytes-like object, NoneType found

so i added the following debug patch and snapped again:

diff --git a/snapcraft/plugins/kernel.py b/snapcraft/plugins/kernel.py
index c1e81e4..e388c2c 100644
--- a/snapcraft/plugins/kernel.py
+++ b/snapcraft/plugins/kernel.py
@@ -60,6 +60,7 @@ import glob
 import logging
 import magic
 import os
+import pty
 import shutil
 import subprocess
 import tempfile
@@ -248,6 +249,11 @@ class KernelPlugin(kbuild.KBuildPlugin):
             mime_detector.load()
             # Make sure we're getting the mime type of the actual initrd, not
             # a symbolic link.
+ logger.info('tmp_initrd_path: {}\n'.format(os.path.realpath(tmp_initrd_path)))
+ myfile = os.path.realpath(tmp_initrd_path)
+ if not os.path.exists(myfile):
+ logger.info('Missing initrd, spwaning a shell')
+ pty.spawn("/bin/dash")
             mime_type = mime_detector.file(os.path.realpath(tmp_initrd_path))
             if not mime_type:
                 raise RuntimeError(

$ snapcraft
...
created 0 files
created 4 directories
created 2 symlinks
created 0 devices
created 0 fifos
tmp_initrd_path: /tmp/tmp27aod6nv/squashfs-root/boot/initrd.img-core

Missing initrd, spwaning a shell

$ find /tmp/tmp27aod6nv/squashfs-root/
/tmp/tmp27aod6nv/squashfs-root/
/tmp/tmp27aod6nv/squashfs-root/usr
/tmp/tmp27aod6nv/squashfs-root/usr/lib
/tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd
/tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd/initrd.img-core-0.7.43+ppa23
/tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd/initrd.img-core

$ ls -la /tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd
total 8
drwxr-xr-x 2 flag flag 4096 nov 30 18:38 .
drwxr-xr-x 3 flag flag 4096 nov 30 18:38 ..
lrwxrwxrwx 1 flag flag 29 nov 30 18:37 initrd.img-core -> ../../../boot/initrd.img-core
lrwxrwxrwx 1 flag flag 42 nov 30 18:37 initrd.img-core-0.7.43+ppa23 -> ../../../boot/initrd.img-core-0.7.43+ppa23

$ cat /tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd/initrd.img-core
cat: /tmp/tmp27aod6nv/squashfs-root/usr/lib/ubuntu-core-generic-initrd/initrd.img-core: No such file or directory

Manually unsquashing os.snap, there's this:

$ unsquashfs -d foobar ./parts/kernel/src/os.snap

$ find foobar -name \initrd\*
foobar/boot/initrd.img-core-0.7.43+ppa23
foobar/boot/initrd.img-core
foobar/lib/systemd/system/initrd-switch-root.service
foobar/lib/systemd/system/initrd-parse-etc.service
foobar/lib/systemd/system/initrd-fs.target
foobar/lib/systemd/system/initrd-root-fs.target
foobar/lib/systemd/system/initrd-cleanup.service
foobar/lib/systemd/system/initrd-switch-root.target
foobar/lib/systemd/system/initrd-udevadm-cleanup-db.service
foobar/lib/systemd/system/initrd.target
foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core-0.7.43+ppa23
foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core

$ ls -la foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core
lrwxrwxrwx 1 flag flag 29 nov 30 18:37 foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core -> ../../../boot/initrd.img-core

$ md5sum foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core
e4b9d74334b4f40ec07e1ee20c48462b foobar/usr/lib/ubuntu-core-generic-initrd/initrd.img-core

Was initrd.img-core moved from usr/lib/ubuntu-core-generic-initrd to boot without updating the kernel plugin?

[Paolo Pisati (p-pisati)]

And indeed, the attached patch fixes it - can i have any confirmation about this?

And what do we want to do to avoid any future breakage like this? I think building a kernel snap should be part of the snapcraft and snapd test pipeline.

[Oliver Grawert (ogra)]

https://github.com/snapcore/snapcraft/pull/1839

[Sergio Schvezov (sergiusens)]

Added snapd so they can add testing so this doesn't break again (or implement split initrd)

[Ian Johnson (anonymouse67)]

This seems to be resolved, and in any case the initrd got way more complicated for UC20 so whatever we did for uc18 isn't applicable for uc20