squashfs url matching is too strict

[Scott Moser]

==== Begin SRU Template ===
[Impact]
Squashfs images published by Ubuntu on http://cloud-images.ubuntu.com
end in '.squashfs' and on http://images.maas.io/ephemeral-v3/
the files are named 'squashfs'.

When you boot an initramfs with rooturl it would automatically
determine/assume that content is squashfs if it ended in
'.squash' or '.squashfs'.

The filter was just too strict and would thus not match
  http://images.maas.io/ephemeral-v3/..../squashfs

That meant a usage would have to re-publish the url.

[Test Case]
## setup system to build maas images
$ sudo apt-get install -qy --no-install-recommends bzr
$ bzr branch lp:maas-images
$ cd maas-images
$ ./system-setup

## download the reference squashfs images.
$ arch=amd64
$ burl=http://cloud-images.ubuntu.com/server
# download reference squashfs images
$ for rel in xenial zesty artful; do
  file=../$rel-server-cloudimg-$arch.squashfs;
  [ -f "$file" ] && continue;
  wget "$burl/$rel/current/${file#*/}" -O "$file.tmp" &&
    mv "$file.tmp" "$file"
done

## build maas image output. This creates a squashfs image and kernel/initramfs
## files that have the updated packages inside.
## It does take a while to run, and lots of cpu and IO.
## You can reduce the kernels built by commenting out 'lowlatency' and 'edge'
## kernels:
## sed -i -e '/^[ ].*\(lowlatency\|edge\)/s/^/#/' conf/meph-v3.yaml
# either set 'proposed' or add the repo via M2E_ADD_REPOS
$ proposed="--proposed"
$ export M2E_ADD_REPOS=ppa:smoser/cloud-initramfs-tools
$ for rel in xenial zesty artful; do
  img=../$rel-server-cloudimg-$arch.squashfs
  PATH=$PWD/bin:$PATH ./bin/meph2-build $proposed --config=conf/meph-v3.yaml \
        $arch $rel $(date +"%Y%m%d") $img ../$rel.d/ 2>&1 |
        tee ../$rel.log
  done

## After running, you will have files like:
$ find xenial.d/xenial -type f | sort
xenial.d/xenial/amd64/20171221/ga-16.04/generic/boot-initrd
xenial.d/xenial/amd64/20171221/ga-16.04/generic/boot-kernel
xenial.d/xenial/amd64/20171221/hwe-16.04/generic/boot-initrd
xenial.d/xenial/amd64/20171221/hwe-16.04/generic/boot-kernel
xenial.d/xenial/amd64/20171221/squashfs
xenial.d/xenial/amd64/20171221/squashfs.manifest

## You can verify in the $rel.log files you have 'Setting up cloud-initramfs-'
$ grep '^Setting up overlayroot' xenial.log
Setting up overlayroot (0.27ubuntu1.5~ppa16.04.1) ...

## Now download the test-boot script, provide it a directory and
## the squashfs, kernel, initramfs

$ ./test-boot test-x xenial.d/xenial/amd64/20171221/squashfs \
    xenial.d/xenial/amd64/20171221/ga-16.04/generic/boot-kernel \
    xenial.d/xenial/amd64/20171221/ga-16.04/generic/boot-initrd

$ r=zesty; ./test-boot work.d $r.d/*/amd64/*/squashfs $r.d/*/amd64/*/ga-*/generic/boot-kernel $r.d/*/amd64/*/ga-*/generic/boot-initrd
Now, do:
  cd work.d
  ./boot

## Running ./boot will serve the squashfs image over http, and then
## boot the kernel and initrd with a kernel command line of
## root=http://<url>/squashfs
## and
## ds=nocloud-net;seedfrom=http://10.0.2.2:9999/
## which will the read the 'user-data' and 'meta-data' files in that dir.
##
## The result is that you can log in with 'ubuntu' and 'passw0rd'

## Then collect some information
# you'll see the 'LABEL=' entry is commented out.
$ cat /etc/fstab
# local-fs.target should be fine
$ systemctl status local-fs.target --no-pager
# just for record
$ cat /proc/cmdline

[Regression Potential]
Very low.
If a url does not match, rooturl would just complain and do nothing.
This just extends the things that match.

[Other Info]
This does seem to be low importance. In fact it is, but its jsut part
of syncing cloud-initramfs-tools across xenial, artful, zesty with bionic.

The fix applied here is a backport what is currently in bionic to
each release. in xenial, that means a new binary package
(cloud-initramfs-updateroot).

==== End SRU Template ===

The code that automatically determines a squashfs url is just to strict.
You can explicitly declare it as squash with:
   root=squashfs:http://<>/...
but currently the automatic matching requires it to end in .squashfs or .squash.

This doesn't match urls like
  <url>/squashfs
  <url>/my-squash

which are as obviously squashfs urls as others.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.40ubuntu1

---------------
cloud-initramfs-tools (0.40ubuntu1) bionic; urgency=medium

  * rooturl: adjust squashfs url matching to not require '.'.
    Previously it would match <url>*.squashfs or <url>*.squash
    Now it will match <url>*squashfs or <url>*.squash.
    (LP: #1728048)
  * overlayroot: Comment out the read-only line for /. This hopes to
    avoid systemd confusion and transient ordering cycle. (LP: #1723183)

 -- Scott Moser <email address hidden> Fri, 27 Oct 2017 09:44:49 -0400

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted cloud-initramfs-tools into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.39ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted cloud-initramfs-tools into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.35ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted cloud-initramfs-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-initramfs-tools/0.27ubuntu1.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.39ubuntu1.1

---------------
cloud-initramfs-tools (0.39ubuntu1.1) artful; urgency=medium

  * sync with upstream at 0.40ubuntu1
    - rooturl: adjust squashfs url matching to not require '.'.
      Previously it would match <url>*.squashfs or <url>*.squash
      Now it will match <url>*squashfs or <url>*.squash. (LP: #1728048)
    - overlayroot: avoid confusing systemd by commenting out read-only
      line for /. (LP: #1723183)
    - overlayroot: remove unused variable 'rel_root'.

 -- Scott Moser <email address hidden> Fri, 15 Dec 2017 15:54:28 -0500

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for cloud-initramfs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-initramfs-tools - 0.27ubuntu1.5

---------------
cloud-initramfs-tools (0.27ubuntu1.5) xenial; urgency=medium

  * sync with upstream at 0.40ubuntu1
    - whitespace changes.
    - rooturl: adjust squashfs url matching to not require '.'.
      Previously it would match <url>*.squashfs or <url>*.squash
      Now it will match <url>*squashfs or <url>*squash. (LP: #1728048)
    - updateroot/*, debian/control: new package 'cloud-initramfs-updateroot'
    - debian/README.source: update upstream information for move to git.
    - overlayroot: change from storing password in
      /run/initramfs/overlayroot.passwd rather than randomly generated
      /run/initramfs/overlayroot.XXXXXX
    - overlayroot: avoid confusing systemd (LP: #1723183)
       + Mark / as 'noauto' in the newly written /etc/fstab
       + Comment out the read-only line for /.

 -- Scott Moser <email address hidden> Fri, 15 Dec 2017 15:57:35 -0500