No persistence with USB Creator in Ubuntu 8.10

[Slyron]

I'm using Ubuntu 8.10, with all the latest updates.

My problem is I cannot get Ubuntu on a USB "WITH" persistence. USB Creator GUI works fine and does in fact install Ubuntu. It boots fine, but basically I only get a LiveCD type functionality out of it. If I try saving anything in the home folder, the USB becomes unbootable. The error message at startup says "Push F1 to retry boot...."

Has anyone else noticed this? Perhaps I'm doing something wrong.

[Lupine (thelupine)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. This bug did not have a package associated with it, which is important for ensuring that it gets looked at by the proper developers. You can learn more about finding the right package at https://wiki.ubuntu.com/Bugs/FindRightPackage . I have classified this bug as a bug in usb-creator. Also, I've been unable to reproduce your issue on Ubuntu 8.10. Here are the steps I took:

1.) Plugged in a Corsair 32GB USB disk
2.) Clicked System -> Administration -> USB start up disk
3.) Clicked Other... and browsed to my ubuntu-8.10-desktop-i386.iso file
4.) Left the default of 128MB "Stored in reserved extra space" setting
5.) Clicked Make Startup Disk
6.) Once done, removed USB disk and placed it in my Fujitsu U810 laptop
7.) Booted into LIVE environment
8.) Created a test.txt file in the Documents folder, saved and closed it
9.) Shutdown laptop and unplugged USB disk
10.) Plugged USB disk back in and booted back into LIVE environment
11.) Checked the Documents folder, and test.txt was still there

Where these similar to the steps you took?

[Rich (cuda-rich)]

I repro the "no persistence" on several separate ubuntu machines. I don't recall the attempts to write making the stick unbootable though.

Was using a 4GB corsair, freshly formatted fat32. ubuntu 8.10 and 9.04 AMD64. I maxed out the persistence file size for my stick (about 3.1 GB). usb-creator proceeded to write the iso to the disk but at the end a small window with a red warning circle with a white horizontal line appeared. There was no text message. I closed that out & rebooted. It booted into the live system properly. I created a small text file on the desktop and one in / . neither one persisted through a subsequent clean reboot.

As a test I later manually created a casper-rw file in the root of my usb stick (there was not one present) and persistence started working. I see the casper packages are not a dependency of the usb-creator package. Lupine: do you have them installed? I do not.

there's a thread over at ubuntu forums where several people report problems w/ persistence: http://ubuntuforums.org/showthread.php?t=1077904

[jruss (jruss)]

I think the original bug report may be the same problem as https://bugs.launchpad.net/ubuntu/+source/usb-creator/+bug/284085. If the usb is using FAT32 (aka vfat) then there is a 4 GB file size limitation, that can cause problems when creating a file casper-rw file that is too big. The casper-rw file won't be created but usb-creator exits cleanly and the USB will boot up fine, but won't be persistent.

[Slyron (slyron84)]

I think that some USB flash drives will work for this and others will not. When I originally filed this bug report, I used an 8gig PNY attache'

later on I tried with a 16gig Corsair flash voyager and it worked fine... persistence and all.

[jruss (jruss)]

I have tracked down a more serious problem. The current version of usb-creator in jaunty beta tries to create the casper-rw file as one large block, when actually it should use a reasonable block size like 1k and then create right number of blocks. This is what is causing the "dd: invalid number" errors in ~/.usb-creator.log and causing the casper-rw file not to be created. If you surpass the 4 GB file size you will get a file too large error (or something like that). There are patches with this bug that should address both of these issues.

https://bugs.launchpad.net/ubuntu/+source/usb-creator/+bug/321544

[Slyron (slyron84)]

I guess they are planning on relying on the ext4 filesystem right? B/c I think you can have huge single files. I'm not sure about that at all though.

[jruss (jruss)]

They will have to stick with FAT because that is what syslinux supports. All modern file sytems including ntfs and ext3 already support files larger than 4 GB. This most recent issue I brought up was the block size which is different than the file size.

[Max (max-goedjen)]

After a few experiments, it seems like roughly 2047MB is the limit for how big persistence can be set to (it's somewhere in between 1025MB and 2047MB, this much I can confirm; I just tried it with 1GB and it worked after two failures with 2GB), if it's above that threshold it just skips creating it (for me it just jumped from about 25% to done).

[Tokenekie (tokenekie)]

@Max:

Yes I can confirm I have the same problems trying to create a live-persistent USB with a 4GB drive.

For details, see duplicate bug: https://bugs.launchpad.net/ubuntu/+source/usb-creator/+bug/364970

[jruss (jruss)]

You guys can easily fix your problem by applying the patch in the bug I linked to above: https://bugs.launchpad.net/ubuntu/+source/usb-creator/+bug/321544. If you have the 2 GB (2048 MB) limit it is because usb-creator is telling dd to make a single block that is 2048 MB. However, dd will only allow blocksizes < 2048 MB. This is the cause of the bug. Even without the 2048 MB limit having really large blocksize is a bad idea anyway. You can easily just change that one line in the script (as is shown in the patch) to tell it to use a reasonable block size like 1 KB and then just make the right number of blocks to get the size you want. You will still be limited to a 4 GB persistence file though, because this is the largest file you can create on a vfat file system. They are unlikely to change from vfat because that is what syslinux uses and it allows the usb stick to still be read in Windows.

[Tokenekie (tokenekie)]

Hi Jruss,

Why would it be bad to create a blocksize greater than 2GB? My usb is formated in Fat 32, which is capable of handling volume sizes much greater than that. Plus usb-creator doesn't complain if the format of the drive is Fat 32 vs Fat 16.

[jruss (jruss)]

Blocksize is not the same as volume/partition/file size (http://en.wikipedia.org/wiki/Block_%28data_storage%29). A file system usually has a block size associated with it. In general terms, when data is pulled off the device it will usually pull off the whole block at the same time. Since related data is usually close by, the future accesses won't have to go back to this disk because that block is already cached in memory. The volume size is the product of the blocksize and the number of blocks.

Persistence works by creating a large file (casper-rw) with dd. The 2048 MB block size limit is a limitation of dd. Almost nobody would be using a block size that large, unless they are only working with large multi-gigabyte files of contiguous data. Using large blocks will be inefficient in most normal situations. This is why most filesystems default to a blocksize in the 1KB-4KB range. The fact that usb-creator is trying to create persistence files with an arbitrary block size is definitely a bug.

Now as far as FAT32 is concerned it is capable of have larger than 2 GB volume sizes, but it does have a hard limit of 4 GB (2^32) for a single file. Usb-creator creates a file called casper-rw with dd and formats it to ext3. This file is mounted by ubuntu at boot time and is used to store persistent changes. However the casper-rw file is a single file saved on the FAT32 filesystem of the USB drive, and thus it cannot be larger than 4 GB.

With the current blocksize bug in usb creator you will get an error about 'invalid blocksize' in your usb-creator log, but it will exit normally. If you fix usb-creator so that it uses a reasonable block size like 1 KB, then you will be able to go about 2048 MB without an error. However, if you go above 4 GB then you will run into the file size limitation of FAT32. You will see an error in your usb-creator log where it calls dd. It will write the file right up to 4 GB and then it will give some error about the wrong file size or out of space or something like that. The 4 GB limit is not a bug and is a technological limit of FAT32. Hopefully the patches in the bug report I linked to will get incorporated to make this clear to the user.

[komputes (komputes)]

This bug had not been updated in over a year, and deals with older software (8.10 Intrepid) which has reached end of life. If you still have this issue in the current LTS, please feel free to open a new bug. If there are any unresolved issues discussed in this bug, feel free to get in touch with me.