Strange usb device format and failure to boot on UEFI system

Bug #1557894 reported by Brian Burch on 2016-03-16
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
usb-creator (Ubuntu)
High
Unassigned

Bug Description

I want to run 16.04 on a new ASUS T300 Chi transformer notebook, which is pre-installed with windows 10.

I started by making a DVD iso from the beta distribution, then booted and installed it on a spare partition of my old Dell 1558 Studio laptop. The system runs fine in 64-bit mode, but the BIOS is non-EFI. I then applied the latest updates to the Dell HDD 16.04 system before running usb-creator. I've done this several times, always starting by DD-ing zeros over the first 200MB of the target 8mb usb stick.

The creator reports successful completion, but gparted doesn't like the contents of the usb stick after the image has been created. The partition table has been changed from type=msdos to type=mac, and the sector size from 512 to 2048. The capacity changed from 7.7GB to 57GB! There are two partitions: sdb1 is called "Ubuntu Gnome 16.04 LTS, with an "unknown" file system type and a size of ONLY 4KB; 1.22GB unallocated; sdb2 is fat16 and its 2.31 MB extent is almost full. No flags are set on either partition. Whenever I interact with gparted it throws a pop-up that says "The driver descriptor says the physical blocksize is 2048, but linux says it is 512 - cancel or ignore".

I am amazed to report this image boots successfully on my Dell with its "legacy" BIOS. The ubuntu live/install options page displays and it eventually boots the live system, which superficially seems to run OK.

The ASUS uses American Megatrends UEFI BIOS, which I have updated to the latest version 207. I have enabled the CSM (Compatibility Support Module - legacy BIOS) and disabled both fast start and secure boot. It boots the usb stick in non-EFI mode as desired (proved by not getting the grub2 efi menu). The "man=keyboard" splash panel appears, then it drops to a tty login briefly before the new ubuntu "circling logo" appears. After several minutes, it drops back to a tty screen with the following:

 (initramfs) Unable to find a medium containing a live system.

I also have a usb stick with knoppix 7.0 which successfully boots in 64-bit non-efi mode on both the Dell and the Asus, so I am reluctant to blame the Asus legacy bios at this stage. Why is gparted reporting such a strange format after creating the usb image?

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: usb-creator-gtk 0.3.2
ProcVersionSignature: Ubuntu 4.4.0-13.29-generic 4.4.5
Uname: Linux 4.4.0-13-generic x86_64
ApportVersion: 2.20-0ubuntu3
Architecture: amd64
CurrentDesktop: GNOME
Date: Wed Mar 16 16:11:25 2016
InstallationDate: Installed on 2016-03-14 (2 days ago)
InstallationMedia: Ubuntu-GNOME 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160225.1)
SourcePackage: usb-creator
UpgradeStatus: No upgrade log present (probably fresh install)

Brian Burch (brian-pingtoo) wrote :
Brian Burch (brian-pingtoo) wrote :

I forgot to say the initramfs message occurs when the external usb mouse and keyboard have become unresponsive, so I cannot use busybox to inspect the state of the system. I have no alternative except to power off the machine.

Brian Burch (brian-pingtoo) wrote :

I have discovered the "strange" device format of the image is a hybrid iso, and gparted simply does not recognise this as meaningful. I was able to mount the iso as follows:

  sudo mount -o loop ./image.iso iso-contents/

... and then the mounted directory allows me read-only access to the file system within the image.

As I said above, the Asus EFI CSM BIOS detects and executes the hybridiso's bootloader. However, something goes wrong further into the process. Perhaps this is because the hybrid iso's file system also contains efi files and their presence confuses the BIOS. I will continue to investigate.

Brian Burch (brian-pingtoo) wrote :

More knowledge acquired... I used simple dd to create a usb from debian-live-8.3.0-amd64-gnome-desktop.iso. This image has NO EFI structure or files at all, so it can only boot in legacy bios mode. When the asus bios has CSM enabled this live image boots perfectly.

I think the evidence is pointing to something wrong with the EFI content of the ubuntu iso. Even when it starts off with CSM being asked to boot the legacy content, the presence of the EFI stuff is confusing the bios later, probably when it is asked to boot the live linux kernel.

Brian Burch (brian-pingtoo) wrote :

This has proved to be a very frustrating bug and I worry that less knowledgeable and persistent users will encounter the same situation and give ubuntu a bad name. After all, EFI is here to stay and we all want ubuntu to used more widely with each new release. 16.04 is to be the next LTS and my short experience with both betas has convinced me that it is a huge improvement over 14.04.

The good news it that I have installed 16.04 beta2 on the Asus T300 chi transformer notebook. I will write more about this later. The bad news is that I found it impossible to install directly from a usb.

The internet is littered with dead-ends, bad advice and out-of-date instructions when dealing with the initramfs message "unable to find a medium containing a live file system"... and not just for ubuntu.

In my case (and perhaps many others), I found the message was issued by the script casper, when ${livefs_root} has not been set by the function mountroot(). This function calls find_livefs(), which is also in the casper script. I even tried setting the casper kernel boot parameter ignore_uuid in grub.cfg, but the message was still generated. I then deleted the quiet boot option and changed splash to nosplash. Now I could see the console log as it was generated.

My problem determination was made much more difficult by the fact that busybox would not read from my keyboard (PS/2 into a USB dongle). Eventually, I tried popping the keyboard dongle out and back into the USB hub port - the log showed the device was recognised and then I could enter commands to see what was going on!

Most importantly, the ubuntu live iso usb device was NOT mounted at the time casper had given up looking for the live filesystem.squashfs. No wonder it gave up!

I spent a long time messing around - trying different manufacturers usb sticks, manually mounting the device on /cdrom, pre-unsquashing the live system, and even trying to satisfy the unsquashed symlinks for /initrd.img and /vmlinuz. Nothing worked because I was unable to find a way to resume initialisation of the final stage of the bootstrap process. (Perhaps someone more knowledgeable could tell me?)

Brian Burch (brian-pingtoo) wrote :

Finally, I realised that the USB stick was probably getting "lost" at the same time as my USB keyboard. However, I didn't need to pop the stick because I could manually mount it under busybox. Unfortunately, I didn't know how to make casper restart once it had given up!

Now for the good news - how I installed ubuntu 16.04 beta2 (only useful for stubborn people!)...

First, I re-enabled the CSM (Compatibility Support Manager component of the UEFI BIOS) by accessing BIOS from windows (you really need to keep windows until ubuntu is properly installed!) I then booted my legacy-bios knoppix 7.0 USB system. I used gparted to create four new partitions on the solid state disk - I had previously shrunk the windows partiton using its disk manager. My new partitions were for linux-swap, / and /boot (both ext4), and a 2GB unformatted partition. I then used knoppix to DD the ubuntu live iso image onto the new unmounted unformatted partition.

When I rebooted the machine to windows I found that I was able to restart using the "device" which was my live iso from its new SDD partition. The system BIOS (not sure whether the CSM or UEFI component was responsible) was happy to recognise the hybrid iso on the SDD, and process its contents. It apparently found the grub2 efi file in my partition, because I hadn't already copied it to /dev/sda1/EFI/BOOT/. The presence of a grub menu proved the BIOS had booted the efi "personality" of the live iso. I selected the "install ubuntu" menu option, and the installer started running. I told it to use the boot, root and swap partitions I had already prepared and installation ran OK.

I crossed my fingers when I was prompted to restart the system, and was very relieved to find my new ubuntu system booted successfully. The wifi connection came up (no ethernet port on my machine) and I ran apt-get update and upgrade without a hitch. I then installed gparted to examine the disk - the three original partitions were intact (EFI, windows recovery and windows C:). I was very impressed to see that ubuntu had automatically mounted the /dev/sda1 EFI partition on its own /boot/efi/ path, and grub had been installed into the "real" EFI partition.

The default grub installation had successfully created a directory structure on sda1 which allowed me to boot windows and the BIOS. I changed grub.cfg (on sda1) to not hide its menu and use a long timeout. I then confirmed I could use grub to boot windows, ubuntu (again!) and even the system BIOS menu. I disabled the CSM and rebooted to grub and then into ubuntu.

Everything was working normally with minimal effort, so a lot of hard work from others has made the 16.04 installation process work brilliantly with EFI and co-existence with windows. The only hurdle is bypassing this problem of running the live iso from a usb stick. I still can't do that!

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in usb-creator (Ubuntu):
status: New → Confirmed
Ivan Larionov (xeron-oskom) wrote :

Looks like this is how new USB creator works.

I'm not sure why they use dd and hybrid ISO now. Old variant with copying files wasn't working good enough?

Changed in usb-creator (Ubuntu):
importance: Undecided → High
spm2011 (spm2011) wrote :

Possible duplicate of 1589028

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers