PXE install prefixes /cdrom to squashfs location, fails to find it

[Tony Maro]

[Impact]
This impacts any network installs specifying live-installer/net-image (ie. using live-installer to install using a squashfs instead of debootstrap)

[Test case]
1) Boot on netinstall image to grub menu; it 'e' for edit;
2) Add 'live-installer/net-image=http://cdimage.ubuntu.com/ubuntu-server/xenial/daily/current/xenial-server-amd64.squashfs' to the linux line.
3) Hit F10 to boot with the changes.

Installation should complete successfully.

[Regression Potential]
Minimal; this is meant to fix a bug that was introduced that currently breaks all uses of net-image that don't use a path relative to /cdrom, which used to be possible; and only enforces the proper handling of URIs as absolute paths instead of prepending them with '/cdrom'. URI-like syntax in typical filesystem paths is not something that is supported.

---

Attempting to use PXE to install 16.04 results in an error dialog that states "The failing step is: Install the system". Checking syslog shows that it has prefixed the URL to the filesystem.squashfs with "/cdrom" and failed to find the file.

base-installer: info: Downloading /cdromhttp://192.168.1.251/ubuntu-1604.amd64/install/filesystem.squashfs to /tmp/live-installer/filesystem.squashfs

Setting the path to the filesystem.squashfs by either the "append" option in the PXE default file, or as a preseed option in the kickstart file both have the same result.

My PXE menu option:

LABEL Ubuntu 16.04 Desktop
MENU LABEL Xenial 16.04 Desktop
KERNEL 1604/ubuntu-installer/amd64/linux
append vga=normal ks=http://192.168.1.251/ks-1604-64-desktop.cfg initrd=1604/ubuntu-installer/amd64/initrd.gz live-installer/net-image=http://192.168.1.251/ubuntu-1604.amd64/install/filesystem.squashfs biosdevname=0

I found a changelog for live-installer 51ubuntu1 dated 4/18/16 that states it strips /cdrom/ prefix from all places, but this does not appear to be happening. The log line points to base-installer rather than live-installer as the cause.

This is from the 16.04 server release ISO, running base-installer_1.158ubuntu2_amd64 and live-installer_51ubuntu2_amd64

I would expect that locating the live-installer filesystem.squashfs would not prefix the specified URL with /cdrom. Previous versions of Ubuntu did not do this.

[kunal kalyanpur (kunaltheniceguy)]

I am facing the similar issue while attempting to PXE boot Ubuntu 16.04 LTS version over corporate Network
This looks like a 100% bug because I followed the exact same steps as Ubuntu 14.x and 15.x however earlier versions PXE install
smoothly whereas Ubuntu 16.04 LTS version errors out saying "installation error ocurred" and logs show live installer image not found , Can't open /usr/lib/fetch-url//cdromhttp

As you can see its prefixing cdrom which should not happen. I have spent hours testing and finding workarounds for this however this seems like a bug in the code. Request you to fix this. We are currently doing a POC on 16.04 LTS and if everything goes successfully, we may need to mass deploy this in our environment. We can't be deploying using CDROM on every machine and need to a mechanism where PXE via netboot works smoothly

Request the developers to have a look at this and come up with a patch or fix for this.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robert Hrovat (robi-hipnos)]

Same here! We are prepairing for mass clean installs from 12.04 to 16.04 with kickstart method over PXE and came across this problem.

[Mathias Radtke (raddirad)]

I am facing the same problem.

can deploy Ubuntu 16.04 over PXE

[Tony Maro (tonymaro)]

Has anyone been able to come up with a workaround for this yet? I'm completely stumped. I even tried the net installer from 14.04 to no avail.

[Mathias Radtke (raddirad)]

I would recommend to try the net.installer package from 15.10. At least some 15.10 packages worked for me on a highly experimental 16.04 build

[Robert Hrovat (robi-hipnos)]

I tried a bit different kickstart line and it works!

In ubuntu-installer/amd64/boot-screens/txt.cfg:

default install
label install
 menu label ^Install
 menu default
 kernel ubuntu-installer/amd64/linux
 append http_proxy="http://192.168.1.1:80" ks=http://192.168.1.1/kickstart/ks.cfg vga=normal initrd=ubuntu-installer/amd64/initrd.gz -- quiet

[Tony Maro (tonymaro)]

Robert where are you specifying the filesystem.squashfs file? In the ks.cfg file?

Are you providing any other options in the ks file to enable the proxy there as well?

I still get the same error, (prefixed with /cdrom) even with inserting the http_proxy option. I can confirm it uses the proxy to pull the kickstart file, then the rest of the install is pulled without use of the proxy.

[Tony Maro (tonymaro)]

Ah I found another workaround / hack someone left on a comment in my original askubuntu question.

It's definitely a hack but it got me around this bug. I extracted the initrd.gz file served by my TFTP site using this guide: http://www.thegeekstuff.com/2009/07/how-to-view-modify-and-recreate-initrd-img/

Then, I edited the included bin/fetch-url file to add the following lines before the proto= line near the bottom:

# Remove /cdrom prefix:
prefix="/cdrom"
url=${url#$prefix}

Then I repackaged the initrd.gz file, copied it back to my tftp server and everything worked.

Definitely a nasty hack, but it got the image working for me.

[Robert Hrovat (robi-hipnos)]

I didn't. I don't have filesystem.squashfs.

[Mathias Radtke (raddirad)]

@tonymaro

thank you, this did the trick.
Local PXE install works fine now

[Leo-Chen (leo-chen)]

Dear
We still have seen same this issue ,Please help to fix this problem

[jeffrey leung (jefleung)]

I am also running into this issue with the added /cdrom prefixed to the url with 16.04 image.

[Mathieu Trudel-Lapierre (cyphermox)]

There are multiple users confirming this -- Triaged/High; I'll fix this.

[seth (sethsbarnard)]

@tonymaro

Its ugly but it works. Thanks!

Just a note to others, you still need to specify the live-installer/net-image location in your txt.cfg file(or by other means).

[Launchpad Janitor (janitor)]

This bug was fixed in the package live-installer - 51ubuntu3

---------------
live-installer (51ubuntu3) yakkety; urgency=medium

  * Don't impose a prefix on places paths that are already URIs. (LP: #1575652)

 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 25 May 2016 16:38:38 -0400

[Steve Langasek (vorlon)]

Hello Tony, or anyone else affected,

Accepted live-installer into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/live-installer/51ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Launchpad Janitor (janitor)]

This bug was fixed in the package live-installer - 51ubuntu2.1

---------------
live-installer (51ubuntu2.1) xenial; urgency=medium

  * Don't impose a prefix on places paths that are already URIs. (LP: #1575652)

 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 30 Jun 2016 15:58:14 -0400

[Jay (jay-londhe)]

Hello All,

I tested pxe boot today with "Ubuntu 16.04.1 LTS".
Still I get same error.

root@pxeserver:/srv/tftp# ll
total 1104
drwxrwxr-x 4 root root 4096 Sep 23 10:36 ./
drwxr-xr-x 3 root root 4096 Sep 22 16:52 ../
drwxr-xr-x 2 root root 4096 Sep 23 11:40 grub/
-rw-r--r-- 1 root root 1107320 Sep 23 10:31 grubnetx64.efi.signed
lrwxrwxrwx 1 root root 47 Aug 2 04:21 ldlinux.c32 -> ubuntu-installer/amd64/boot-screens/ldlinux.c32
lrwxrwxrwx 1 root root 33 Aug 2 04:21 pxelinux.0 -> ubuntu-installer/amd64/pxelinux.0
lrwxrwxrwx 1 root root 35 Aug 2 04:21 pxelinux.cfg -> ubuntu-installer/amd64/pxelinux.cfg/
drwxrwxr-x 3 root root 4096 Aug 2 04:21 ubuntu-installer/
-rw-rw-r-- 1 root root 58 Aug 2 04:21 version.info
root@pxeserver:/srv/tftp# more grub/grub.cfg
menuentry "Install Ubuntu" {
set gfxpayload=keep
linux /ubuntu-installer/amd64/linux gfxpayload=800x600x16,800x600 -- quiet
initrd /ubuntu-installer/amd64/initrd.gz
}

After partitioning error is shown as mentioned below
http://www.michaelm.info/blog/?p=1378

base-installer: error: Could not find any live images. WARNING **: Configuring 'live-installer' failed with error code 1. WARNING **: Menu item ...

Kindly suggest how to fix this error.

Regards,
Jay

[Jay (jay-londhe)]

It worked.
Below is my grub.cfg

menuentry "Install Ubuntu" {
set gfxpayload=keep
linux /ubuntu-installer/amd64/linux live-installer/net-image=http://192.168.1.1/ubuntu/install/filesystem.squashfs hostname=unassigned netcfg/choose_interface=auto gfxpayload=800x600x16,800x600 --- auto=true url=http://192.168.1.1/preseed.cfg quiet
initrd /ubuntu-installer/amd64/initrd.gz
}

[Thom Yorke (unholymachine)]

This bug is still plaguing me. I tried the hack of editing the initrd and it did not fix the issue. Apparently there is a proposed test package fix but I do not know how to apply it. I don't even see the installer TRYING to access the filesystem.squashfs file when I check ctrl-alt-f4. I'm completely at a loss here, I've been banging my head against the wall for days trying to fix this. I've tried uploading the squashfs file to a server, I've tried serving it from the tftp server , this is all after the hack mind you, and no joy. If someone could please help I would greatly appreciate it.

Thanks

[Thom Yorke (unholymachine)]

I tried installing "live-installer" following Steve Langasek (vorlon)'s instructions for the fix but it says the package cannot be found?