Comment 0 for bug 321544

Binary package hint: usb-creator

usb-creator fails to create a persistence file as large as requested, producing the message dd: writing "`/media/disk/casper-rw': File too large". Creating the largest file can take 3 hours 25 minutes. This consumes most of one core in my two core system, mostly in system state.

Running 0.1.11 on jaunty alpha 3

Analysis:

The casper-rw file will contain the persistence file system and is created on a FAT file system since that is what is compatible with SYSLINUX (and its sibling ISOLINUX used for live-cds). http://en.wikipedia.org/wiki/FAT_file_system states that the maximum file size for a FAT file system is 4GB minus 1 byte and this is the filesize actually created by the package. However in its user interface usb-creator unrealistically offers persistence sizes larger than this when used on larger usb memory sticks.

In the bazaar trunk of usb-creator at revision 54 (and for Intrepid) the blocksize was 1M. After that it became possible to specify the size in bytes. In revision 60 the blocksize (bs) was set to 1 byte. For the largest persistent filesize the dd command executes 4 billion output operations when using this blocksize.

Since the casper-rw file will be made into an ext3 file system, any bytes greater than n*filesystemblocksize will be wasted. For an ext3 file system this filesystemblocksize will be 1K, 2K, or 4K. For a file of several gigabytes one would expect the blocksize to be 4K bytes.

Thus the maximum file size offered by usb-creator should be 4G minus 4K. This should get rid of the error message and produce more realistic expectations for those using usb-creator.

The persistence size offered by the user can be divided by 1024 and that number of 1024 byte blocks can be written. Any smaller granularity will be wasted as the EXT file system will necessarily ignore it. I suggest using 1024 here to give maximum precision for small persistence file sizes with 1K blocksizes. Using a dd blocksize of 1024 reduces the persistence file creation time down to just 6 minutes so further increases in dd blocksize to 2K or 4K are probably unnecessary. If there is a use case for creating a persistent file with one byte accuracy a second dd command can be added with a blocksize of 1 byte using the seek parameter to finish the last odd 1-1023 bytes (persistence-size%1024).

On the FAT file system it may suffice to seek and write to the last byte of the persistence file, but this doesn't save much time because of all the writes to the FAT tables for allocation, and it could produce hard to find errors should the FAT file system ever be replaced with one which supports files with holes (as software evolves). I tried this dd command as a test but persistence file creation still took several minutes.

Note that with the blocksize change any persistence size passed manually to the included install command will have to be in bytes, dd command shorthand of, say, 3K or 6M or 4G will again create an type error.

I built and tested a package sucessfully with the proposed patches I'm including.