udf does not support using aarch64 kernel snap with armhf 32 bits user space

[Yann Sionneau]

fallen@fallen-ThinkPad-X260:~/dev/packages/images$ sudo ./ubuntu-device-flash --verbose core 16 -o rpi2-all-snap.img --channel stable --enable-ssh --gadget ../paros_gadget/paros_1.0_all.snap --kernel ../paros_kernel/paros-kernel_3.10.67_arm64.snap --os ubuntu-core
Determining gadget configuration
Parallel unsquashfs: Using 4 processors
4 inodes (4 blocks) to write

/tmp/gadget531796560/gadget/fake-gadget/1.0-fake
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/boot-assets
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/boot-assets/uboot.env
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/meta
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/meta/snap.yaml
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/paros_1.0_all.snap
/tmp/gadget531796560/gadget/fake-gadget/1.0-fake/tegra210-paros-hw00.dtb

created 4 files
created 3 directories
created 0 symlinks
created 0 devices
created 0 fifos

Partitioning...
Formatting...
Mounting...
Provisioning...
46.60 MB / 46.60 MB [==================================================================================================================================================================] 100.00 % 531.80 KB/s 1m29s
Installing ../paros_gadget/paros_1.0_all.snap
Installing ../paros_kernel/paros-kernel_3.10.67_arm64.snap
Unmounting...
failed to install "../paros_kernel/paros-kernel_3.10.67_arm64.snap" from "stable": ../paros_kernel/paros-kernel_3.10.67_arm64.snap failed to install: package's supported architectures (arm64) is incompatible with this system (armhf)

See : https://lists.ubuntu.com/archives/snappy-devel/2016-May/001808.html

[Oliver Grawert (ogra)]

you could try to cheat and just re-pack the kernel snap with "architectures: armhf" in the snap.yaml (using unsquashfs, changing meta/snap.yaml in the unpacked binary and running "snapcraft snap $path_to_unpack_dir"), then ubuntu-device-flash should not complain anymore

[Pedro Coca (pcoca)]

Re-packing the kernel snap editing the metadata gives a CRC error:

fallen@fallen-ThinkPad-X260:~/dev/packages/images$ sudo
./ubuntu-device-flash --verbose core 16 -o paros.img --channel edge
--gadget ../paros_gadget/paros_1.0_all.snap --kernel
../paros_kernel/paros-kernel_3.10.67_armhf.snap --os ubuntu-core
--enable-ssh
[sudo] Mot de passe de fallen :
Determining gadget configuration
Parallel unsquashfs: Using 4 processors
4 inodes (4 blocks) to write

/tmp/gadget328243825/gadget/fake-gadget/1.0-fake
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/boot-assets
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/boot-assets/uboot.env
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/meta
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/meta/snap.yaml
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/paros_1.0_all.snap
/tmp/gadget328243825/gadget/fake-gadget/1.0-fake/tegra210-paros-hw00.dtb

created 4 files
created 3 directories
created 0 symlinks
created 0 devices
created 0 fifos

Partitioning...
Formatting...
Mounting...
Provisioning...
46.55 MB / 46.55 MB
[=================================================================================================================]
100.00 % 2.78 MB/s 16s
Installing ../paros_gadget/paros_1.0_all.snap
Installing ../paros_kernel/paros-kernel_3.10.67_armhf.snap
Installing ubuntu-core
46.55 MB / 46.55 MB
[=================================================================================================================]
100.00 % 2.58 MB/s 18s
Unmounting...
bad CRC: 4138496679 != 2339728981

[Pedro Coca (pcoca)]

The CRC error was related to one of the flags of mkenvimage (-r)

./mkenvimage -s 2600 -o uboot.env $CWD/uboot.txt

But running mkenvimage with -r and re-running the whole process of the kernel snap build with modified metadata now gives another error:

Unmounting...
panic: runtime error: index out of range

goroutine 1 [running, locked to thread]:
panic(0x94f960, 0xc82000e0c0)
    /usr/lib/go-1.6/src/runtime/panic.go:464 +0x3e6
github.com/mvo5/uboot-go/uenv.parseData(0xc82051e005, 0x752b, 0x7dfb, 0x9217b12c)
    /home/egon/devel/go/src/github.com/mvo5/uboot-go/uenv/env.go:95 +0x2ea
github.com/mvo5/uboot-go/uenv.Open(0xc82015dbc0, 0x33, 0x0, 0x0, 0x0)
    /home/egon/devel/go/src/github.com/mvo5/uboot-go/uenv/env.go:80 +0x440
github.com/ubuntu-core/snappy/partition.(*uboot).SetBootVar(0xecf4b8, 0xa0b560, 0xd, 0xc8203adac3, 0x13, 0x0, 0x0)
    /home/egon/devel/go/src/github.com/ubuntu-core/snappy/partition/uboot.go:53 +0x6a
main.(*Snapper).install(0xeb2828, 0xc820186080, 0x1e, 0x0, 0x0)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:265 +0x1532
main.(*Snapper).setup(0xeb2828, 0xc82019f7c8, 0x0, 0x0, 0x0, 0x0)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:630 +0x26f3
main.(*Snapper).deploy(0xeb2828, 0xc82019f7c8, 0x0, 0x0, 0x0, 0x0)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:658 +0x290
main.(*Snapper).create(0xeb2828, 0x0, 0x0)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:815 +0xb8e
main.(*CoreCmd).Execute(0xeb2800, 0xc82013e000, 0x1, 0xf, 0x0, 0x0)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/core.go:83 +0x3b1
github.com/jessevdk/go-flags.(*Parser).ParseArgs(0xc8200fe820, 0xc82000a0f0, 0xf, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/egon/devel/go/src/github.com/jessevdk/go-flags/parser.go:301 +0xa10
main.execute(0xc82000a0f0, 0xf, 0xf)
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/main.go:64 +0x3c9
main.main()
    /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/main.go:46 +0x3b

[Oliver Grawert (ogra)]

the uboot.env file needs to be exactly 128k and redundant ... the mkenvimage command you use to create must be:

mkenvimage -r -s 131072 -o uboot.env uboot.env.in

note that you need to adjust your u-boot build config similar to http://paste.ubuntu.com/16199863/ in case your u-boot binary uses a different size ...

[Yann Sionneau (yann-sionneau)]

I recompiled my u-boot with the mentioned config changes, then I regenerated my uboot.env with -r -s 131072.
Here is what I get :
Unmounting...
panic: runtime error: index out of range

goroutine 1 [running, locked to thread]:
panic(0x94f960, 0xc82000e0c0)
 /usr/lib/go-1.6/src/runtime/panic.go:464 +0x3e6
github.com/mvo5/uboot-go/uenv.parseData(0xc82056a005, 0x1fffb, 0x3fdfb, 0x7419fb55)
 /home/egon/devel/go/src/github.com/mvo5/uboot-go/uenv/env.go:95 +0x2ea
github.com/mvo5/uboot-go/uenv.Open(0xc8200566c0, 0x33, 0x0, 0x0, 0x0)
 /home/egon/devel/go/src/github.com/mvo5/uboot-go/uenv/env.go:80 +0x440
github.com/ubuntu-core/snappy/partition.(*uboot).SetBootVar(0xecf4b8, 0xa0b560, 0xd, 0xc820389893, 0x13, 0x0, 0x0)
 /home/egon/devel/go/src/github.com/ubuntu-core/snappy/partition/uboot.go:53 +0x6a
main.(*Snapper).install(0xeb2828, 0xc820136060, 0x1e, 0x0, 0x0)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:265 +0x1532
main.(*Snapper).setup(0xeb2828, 0xc8204f17c8, 0x0, 0x0, 0x0, 0x0)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:630 +0x26f3
main.(*Snapper).deploy(0xeb2828, 0xc8204f17c8, 0x0, 0x0, 0x0, 0x0)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:658 +0x290
main.(*Snapper).create(0xeb2828, 0x0, 0x0)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/snappy.go:815 +0xb8e
main.(*CoreCmd).Execute(0xeb2800, 0xc82000a1e0, 0x1, 0xf, 0x0, 0x0)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/core.go:83 +0x3b1
github.com/jessevdk/go-flags.(*Parser).ParseArgs(0xc820012c30, 0xc82000a0f0, 0xf, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0)
 /home/egon/devel/go/src/github.com/jessevdk/go-flags/parser.go:301 +0xa10
main.execute(0xc82000a0f0, 0xf, 0xf)
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/main.go:64 +0x3c9
main.main()
 /home/egon/devel/go/src/launchpad.net/goget-ubuntu-touch-all-snaps/ubuntu-device-flash/main.go:46 +0x3b

[Oliver Grawert (ogra)]

well, the CRC error is gone, the remaining error looks more like it is content related, how does your uboot.txt file look like ?

[Yann Sionneau (yann-sionneau)]

fallen@fallen-ThinkPad-X260:~/dev/snappy/paros/snappy_image/u-boot/tools$ cat uboot.txt
arch=arm
baudrate=115200
board=paros
board_name=paros
boot_a_script=load 29882{devtype} 29882{devnum}:29882{bootpart} 29882{scriptaddr}
29882{prefix}29882{script}; source 29882{scriptaddr}
boot_extlinux=sysboot 29882{devtype} 29882{devnum}:29882{bootpart} any 29882{scriptaddr}
29882{prefix}extlinux/extlinux.conf
boot_net_usb_start=usb start
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_targets=mmc1 mmc0 usb0 pxe dhcp
bootcmd=run distro_bootcmd
bootcmd_dhcp=run boot_net_usb_start; if dhcp 29882{scriptaddr}
29882{boot_script_dhcp}; then source 29882{scriptaddr}; fi
bootcmd_mmc0=setenv devnum 0; run mmc_boot
bootcmd_mmc1=setenv devnum 1; run mmc_boot
bootcmd_pxe=run boot_net_usb_start; dhcp; if pxe get; then pxe boot; fi
bootcmd_usb0=setenv devnum 0; run usb_boot
bootdelay=2
cpu=armv8
dfu_alt_info=kernel ram 0x80080000;dtb ram 0x83000000;initramfs ram
0x84000000
dfu_boot=dfu 0 ram 0; booti 0x80080000 - 0x83000000
distro_bootcmd=for target in 29882{boot_targets}; do run bootcmd_29882{target}; done
fdt_addr_r=0x82000000
fdt_copy_node_names=/memory-controller@7001b000
fdt_copy_src_addr=0x83080000
fdt_high=ffffffffffffffff
initrd_high=ffffffffffffffff
kernel_addr_r=0x80080000
loadaddr=0x80080000
lp0_vec=0x1000@0xff2bf000
mmc_boot=if mmc dev 29882{devnum}; then setenv devtype mmc; run
scan_dev_for_boot_part; fi
nvdumper_reserved=0xff23f000
pxefile_addr_r=0x90100000
ramdisk_addr_r=0x83100000
revision=hw00
scan_dev_for_boot=echo Scanning 29882{devtype} 29882{devnum}:29882{bootpart}...; for
prefix in 29882{boot_prefixes}; do run scan_dev_for_extlinux; run
scan_dev_for_scripts; done
scan_dev_for_boot_part=part list 29882{devtype} 29882{devnum} -bootable
devplist; env exists devplist || setenv devplist 1; for bootpart in
29882{devplist}; do if fstype 29882{devtype} 29882{devnum}:29882{bescan_dev_for_extlinux=if test -e 29882{devtype} 29882{devnum}:29882{bootpart}
29882{prefix}extlinux/extlinux.conf; then echo Found
29882{prefix}extlinux/extlinux.conf; run boot_extlinux; echo SCRIPT FAIL
scan_dev_for_scripts=for script in 29882{boot_scripts}; do if test -e
29882{devtype} 29882{devnum}:29882{bootpart} 29882{prefix}29882{script}; then echo Found
U-Boot script 29882{prefix}29882{script}; run boot_a_sce
scriptaddr=0x90000000
soc=tegra210
stderr=serial
stdin=serial
stdout=serial
usb_boot=usb start; if usb dev 29882{devnum}; then setenv devtype usb; run
scan_dev_for_boot_part; fi
vendor=parrot
fallen@fallen-ThinkPad-X260:~/dev/snappy/paros/snappy_image/u-boot/tools$ hexdump -C uboot.env | head -n2
00000000 55 fb 19 74 01 61 72 63 68 3d 61 72 6d 00 62 61 |U..t.arch=arm.ba|
00000010 75 64 72 61 74 65 3d 31 31 35 32 30 30 00 62 6f |udrate=115200.bo|

[Yann Sionneau (yann-sionneau)]

Hmm the uboot.txt file got corrupted somehow, here is another try:
fallen@fallen-ThinkPad-X260:~/dev/snappy/paros/snappy_image/u-boot/tools$ ./mkenvimage -r -s 131072 -o uboot.env uboot.txt
fallen@fallen-ThinkPad-X260:~/dev/snappy/paros/snappy_image/u-boot/tools$ uboot-go uboot.env print
panic: invalid line "${prefix}${script}; source ${scriptaddr}" (corrupted file?)

goroutine 1 [running]:
panic(0x4d7280, 0xc8200742e0)
 /usr/lib/go-1.6/src/runtime/panic.go:464 +0x3e6
github.com/mvo5/uboot-go/uenv.parseData(0xc8200da005, 0x93f, 0x3fdfb, 0xc82003fb6a)
 /home/fallen/gocode/src/github.com/mvo5/uboot-go/uenv/env.go:97 +0x38e
github.com/mvo5/uboot-go/uenv.Open(0x7fff77a2336f, 0x9, 0x0, 0x0, 0x0)
 /home/fallen/gocode/src/github.com/mvo5/uboot-go/uenv/env.go:82 +0x4a8
main.main()
 /home/fallen/gocode/src/github.com/mvo5/uboot-go/main.go:19 +0x605

fallen@fallen-ThinkPad-X260:~/dev/snappy/paros/snappy_image/u-boot/tools$ cat uboot.txt

arch=arm
baudrate=115200
board=paros
board_name=paros
boot_a_script=load ${devtype} ${devnum}:${bootpart} ${scriptaddr}
${prefix}${script}; source ${scriptaddr}
boot_extlinux=sysboot ${devtype} ${devnum}:${bootpart} any ${scriptaddr}
${prefix}extlinux/extlinux.conf
boot_net_usb_start=usb start
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_targets=mmc1 mmc0 usb0 pxe dhcp
bootcmd=run distro_bootcmd
bootcmd_dhcp=run boot_net_usb_start; if dhcp ${scriptaddr}
${boot_script_dhcp}; then source ${scriptaddr}; fi
bootcmd_mmc0=setenv devnum 0; run mmc_boot
bootcmd_mmc1=setenv devnum 1; run mmc_boot
bootcmd_pxe=run boot_net_usb_start; dhcp; if pxe get; then pxe boot; fi
bootcmd_usb0=setenv devnum 0; run usb_boot
bootdelay=2
bootfstype=ext4
cpu=armv8
devnum=0
devplist=1
devtype=mmc
dfu_alt_info=kernel ram 0x80080000;dtb ram 0x83000000;initramfs ram
0x84000000
dfu_boot=dfu 0 ram 0; booti 0x80080000 - 0x83000000
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
fdt_addr_r=0x82000000
fdt_copy_node_names=/memory-controller@7001b000
fdt_copy_src_addr=0x83080000
fdt_high=ffffffffffffffff
initrd_high=ffffffffffffffff
kernel_addr_r=0x80080000
loadaddr=0x80080000
lp0_vec=0x1000@0xff2bf000
mmc_boot=if mmc dev ${devnum}; then setenv devtype mmc; run
scan_dev_for_boot_part; fi
nvdumper_reserved=0xff23f000
pxefile_addr_r=0x90100000
ramdisk_addr_r=0x83100000
revision=hw00
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${bootpart}...; for
prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run
scan_dev_for_scripts; done
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable
devplist; env exists devplist || setenv devplist 1; for bootpart in
${devplist}; do if fstype ${devtype} ${devnum}:${bootparte
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${bootpart}
${prefix}extlinux/extlinux.conf; then echo Found
${prefix}extlinux/extlinux.conf; run boot_extlinux; echo SCRIPT FAILED: coi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e
${devtype} ${devnum}:${bootpart} ${prefix}${script}; then echo Found
U-Boot script ${prefix}${script}; run boot_a_script; ee
scriptaddr=0x90000000
soc=tegra210
stderr=serial
stdin=serial
stdout=serial
u...

[Oliver Grawert (ogra)]

aha, so you are not having any of teh required snappy_* vars set at all ...

see http://bazaar.launchpad.net/~snappy-dev/snappy-hub/snappy-systems/view/head:/pi2/boot-assets/uboot.env.in

you want the "loadfiles" bits (adjusted for your system (specifically the fdt loading part i guess))
as well as all the snappy_* vars (you can just copy/paste these)

then point your actual boot command to "run snappy_boot"

[Oliver Grawert (ogra)]

hmm ... you also want whats in "mmcargs" to set the right root=

also what mkenvimage do you use (your u-boot build should have created one, it is best to use that over something from the archive)

[Oliver Grawert (ogra)]

also: the error you get above looks like there are line-wraps where they shouldn't be in your uboot.txt

[Yann Sionneau (yann-sionneau)]

Allright, I fixed the linewraps issues and now the image is generated o/
Thanks!
Ok I see I need to put some snappy_* variables, is this documented somewhere?
What variables to use, and what they are supposed to contain?

[Oliver Grawert (ogra)]

not yet, no, i'm working on a porting guide that will cover these basics, for now just do what i described above (the loading needs to be adapted for your system, all snappy_* vars can be taken 1:1 and you need to get the root= variable populated properly)

[Pedro Coca (pcoca)]

The snappy_* vars would be similar to this ones:

snappy_boot=if test "${snappy_mode}" = "try"; then if test "${snappy_trial_boot}" = "1"; then setenv snappy_os "${snappy_good_os}"; setenv snappy_kernel "${snappy_good_kernel}"; saveenv; else setenv snappy_trial_boot 1; saveenv; fi; fi; run loadfiles; setenv mmcroot "/dev/disk/by-label/writable ${snappy_cmdline} snappy_os=${snappy_os} snappy_kernel=${snappy_kernel}"; run mmcargs; bootz ${loadaddr} ${initrd_addr}:${initrd_size} 0x100
snappy_cmdline=net.ifnames=0 init=/lib/systemd/systemd ro panic=-1 fixrtc
snappy_mode=regular
snappy_trial_boot=0

snappy_ab=a would not be relevant as this is UC16

[Yann Sionneau (yann-sionneau)]

Maybe I should open a new bug but now I'm able to generate paros.img image with ubuntu-device-flash.
But then when I flash it and boot it I get issues from the init of initrd.
initrd cannot mount the gadget/kernel/core snaps because it searches for them in the wrong paths.
It searches at /tmpmnt_writable/system-data/var/lib/snappy/snaps/file.snap
But they are in /tmpmnt_writable/system-data/var/lib/snapd/snaps/file.snap

I guess this is coming from UDF generating wrong files in initrd?
How can I fix this?

Thanks!

[Oliver Grawert (ogra)]

this sounds like you are using a very outdated initrd rather

[Oliver Grawert (ogra)]

the initrd should be shipped in the kernel snap, you need an updated initrd and you need the squashfs module inside to actually be able to mount the snaps.
best would be if you used the snapcraft kernel plugin to build your kernel snap, it will pull the latest initrd that is available at build time add modules as defined in snapcraft.yaml

[Yann Sionneau (yann-sionneau)]

Aaaaah OK, right, I need to update my kernel snap.
It's been some time since I last built it (using snapcraft).

I'll rebuild it, thanks!

[Yann Sionneau (yann-sionneau)]

Issue was that os.snap (containing initrd.img) was being downloaded in self.sourcedir + 'os.snap'.
Meaning the os.snap was being downloaded (and cached) in the kernel source dir.
Which I didn't clean and which "snapcraft clean" didn't clean either.

I guess this is not the correct behaviour.
So now the image is flashed and boots correctly o/