usb drives imaged with usb-creator fail to boot on some systems with mass storage emulation off

[Tobin Davis]

Binary package hint: usb-creator

When using usb-creator-[gtk|kde] to image a drive, some systems will fail to boot unless the system bios is set to "USB Mass Storage Emulation Type" to "All Fixed Disc" from "Auto Detect". This is reproducible on the following Intel boards: D945GCFL D945GCFL2, DP35DP, DQ33HS. All systems have been verified to have the latest available bios from Intel. Until this setting is made, the only message received is "Boot Error".

This used to work with usb-creator for Jaunty. I have retried images that worked with Jaunty (through Karmic Beta) to verify this is not an image issue. This affects UNR, Moblin-remix, Desktop-i386, Desktop-amd64, and probably others as well.

ProblemType: Bug
Architecture: i386
Date: Thu Oct 22 12:16:56 2009
DistroRelease: Ubuntu 9.10
NonfreeKernelModules: nvidia
Package: usb-creator 0.2.11
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-14.48-generic
SourcePackage: usb-creator
Uname: Linux 2.6.31-14-generic i686

[Evan (ev)]

When you say, "I have retried images that worked with Jaunty ", do you mean that you used CDs from Jaunty, but the version of usb-creator from Karmic, or that you used both images and usb-creator itself from Jaunty?

[Tobin Davis (gruemaster)]

I mean that images that worked with the Jaunty usb-creator no longer work with the karmic usb-creator-[gtk|kde]. Essentially, the new usb-creator package is not properly writing the mbr of the usb device.

[bingo (stefandrude)]

I just wanted to confirm the problem described for an Intel DG965WH based board. Images written with Jaunty worked fine so far.
Also, I was wondering why the USB Creator does not use a "brute force" approach when trying to format the USB memory stick.

[Evan (ev)]

Define brute force. It tells devicekit-disks to clear the partition table, create a new MBR partition table and single partition with the boot flag set, then clears the code area of the MBR.

[Yeeshkull (yeeshkull)]

I am also having problems booting/installing Jaunty & Karmic from a USB drive with an older Compaq laptop w/ Phoenix Bios. All went well with Puppy Linux, though.

[bingo (stefandrude)]

@ Evan

Brute force to me would mean that the USB Creator can format a USB Stick as needed. The USB Stick is a memory based device after all, it is normally not write protected, and if I tell the USB Creator to format it I don't expect to see error messages or indications that formatting did not succeed. This happened to me with USB Creator a number of times with sticks of size 2 GByte and 8 GByte.

[komputes (komputes)]

For those of you who have a key that does not boot on every computer, I may have found something in Bug #326701 woth a test:
usb-creator does it is not set to LBA (partition bigger than 512mb). To solve the problem use gparted and enable the LBA "flag"

Can anyone confirm that this allows them to boot their key on a system which previously had issues?

[markusl (markus-mnslaker)]

In a freshly patched Karmic, usb-creator-kde does indeed set the LBA flag on my 2GB USB data stick. fdisk(1) prints the partition table as follows:

Command (m for help): p

Disk /dev/sdf: 2002 MB, 2002747392 bytes
62 heads, 62 sectors/track, 1017 cylinders
Units = cylinders of 3844 * 512 = 1968128 bytes
Disk identifier: 0x00094c11

   Device Boot Start End Blocks Id System
/dev/sdf1 * 1 1017 1954643 c W95 FAT32 (LBA)

Command (m for help):

Not that it does any good: my Dell Inspiron still prints "Boot error" when I try to boot from the pen drive. Unfortunately, the Dell doesn't have a "USB Mass Storage Emulation Type" option, and so I'm left unable to boot from the pen drive.

[John S. Gruber (jsjgruber)]

Boot Error is issued from the syslinux boot sector, the first sector of the FAT file system created when usb-creator formats it. The error means that the code in the boot sector can't figure out what sector it was loaded into and therefore can't find the rest of the boot code (the very next sector and following). This problem occurs way before anything .iso specific happens.

If the BIOS correctly interprets the usb stick as a hard disk it will start execution from the MBR which will set some registers and find and start the sector I mention above. One register points to the booting partition table which contains the starting sector of the partition--and there should be no problem. (If real interested see ldlinux.asm in the syslinux package.)

If the BIOS interprets the flash drive in another way, perhaps zip drive emulation, it may skip the MBR and start execution with the syslinux boot sector. Without having a pointer to the partition table entry, the code there relies on a field (bsHidden) in that sector which is set during partition formatting--the hidden sector field. It contains the offset of the partition on the disk as a whole.

------------------
If I may speculate:

In working on an another bug I found out that older versions of mkdosfs in the dosfstools package set that value to 63, the usual offset of the first partition on a modern disk since most start on the second track of the device and most modern devices have a tracksize of 63 sectors. Formatting thumbdrives with usb-creator when one of those older packages was installed worked for me. I determined that more recent versions of mkdosfs set that field to 0, which obviously won't do in the zip drive emulation case. This was changed upstream in mkdosfs to eliminate a regression when formatting diskettes (or, more importantly, diskette images) and made it into Ubuntu.

I've submitted a patch to mkdosfs to set the field correctly in both the diskette and mass storage cases but it hasn't been reviewed yet. Please see LP: #398241.

To check the possibility that this incorrect field is causing this problem one could install the mkdosfs package in my ppa, or a very old Ubuntu one (from Hardy perhaps, I don't remember how far one has to go back) and reformat the partition.

It may also be possible to format the disk partition manually using the mkdosfs -h option, eg. sudo mkdiskfs -h 63 (partion-device). This doesn't work with my BIOS because it needs other fields to match the traditional standard. One could also format the partition with using one of several operating systems originating from the state of Washington in the USA to get a de facto standard format.

To see the wrong sector number by eye: sudo dd if=/dev/sdxn bs=512 count=1 | hd

and examine the 4 byte little-endian value at offset 0x1c. As I said above, it is supposed to be the sector offset of the partition /dev/sdxn on the device /dev/sdx.

If setting the hidden sector field fixes the problem this bug is a duplicate of LP: #398241 and the package dosfstools is responsible.

Note well that it only matters when the partition is formatted by usb-creator or more directly, not when usb-creator installs a new image on an al...

[komputes (komputes)]

Some other systems that have been reported as failing to boot usb-creator imaged drives:

Motherboard Asrock K7S8X
BIOS American Megatrends Inc. P1.90 (07/11/2003)

Dell Latitude D620

Acer Aspire One

Testing a bootable USB key (which works on most machines) would be helpful to confirm and start building a database of particular systems where usb drives imaged with usb-creator fails to boot.

[richud (richud.com)]

Bug also effects D965GF Intel boards (Still in Natty 11.04 startup disk creator)

However there is a solution and it is because of 'faulty' FAT32 boot sector, the offending bits to fix are offset (from boot sector start)
1A h (Number of Heads) [this as always been correct in the things ive looked at, but its something to check, 255 heads = FF in below example ]
1C h (Number of Hidden Sectors in Partition) [in below example is 2048 = 80 00, when wrong usually 00 00]
40 h (Logical Drive Number of Partition) [should be 80, when wrong usually 00]

You can manually edit these or use ms-sys (http://ms-sys.sourceforge.net/) to fix by doing (assume usb stick is sdd)
./ms-sys -p /dev/sdd1
you will see something like

 ./ms-sys -p /dev/sdd1
Start sector 2048 (nr of hidden sectors) successfully written to /dev/sdd1
Physical disk drive id 0x80 (C:) successfully written to /dev/sdd1
Number of heads (255) successfully written to /dev/sdd1

You can also use this to fix your own bootable sticks if you are formatting with mkfs.vfat

[richud (richud.com)]

incidently if you are using FAT16 , 1A and 1C same to fix, but its offset 24 h instead of 40 h for the "Logical Drive Number of Partition

[xoristzatziki (simsonbike-bugs)]

As I already asked in question 165187 my problem is that a Live USB with "user documents and settings be stored on a reserved extra space" makes it unusable.

Please confirm that the creation of a live usb with "discard on shutdown...." is also not working.

[Bryce Harrington (bryce)]

Can confirm this bug still affects a DG965OT. Both natty and oneiric images tested, received "Boot error".

Setting "USB Mass Storage Emulation Type" to "All Fixed Disc" from "Auto Detect" as documented worked around the issue.

Guessing next step in getting this bug resolved would be to test the patch 0001-Add-options-and-make-dos-boot-sector-more-compatible.patch on deb #552673. I may do that if I have time, unless anyone has better suggestions.

[kingttx (kingttx-tomslinux)]

I have this same error on an older Alienware Area51.

[Michael Langerhorst (m-langerhorst)]

Had the same problem with various PC and Notebooks such as:
ASUS M2400N and ASUS P50IJ.
I also found a solution which worked fine for me:

sudo dd if=/dev/sdX1 of=~/usb-stick.img bs=10M
sudo dd of=/dev/sdX if=~/usb-stick.img bs=10M
sudo rm ~/usb-stick.img

found here:
http://wiki.ubuntuusers.de/Live-USB?highlight=startmedien