This appears to be caused (at least, as reported to me by Marc Deslauriers) by stray files left over from a previous Desktop image copy on the USB drive; if it has not been formatted before writing the Server image files to the drive.
If you're seeing this issue and are able to reproduce it easily with your USB drive, please see if you have a "casper" directory on the USB drive, if that drive has been configured to contain a server installation image. The casper directory can be removed from a USB drive if it's meant to be used to install the Ubuntu Server (and only Ubuntu Server; the Ubuntu Desktop installer *requires* that directory and will fail if it's not there).
This appears to be caused (at least, as reported to me by Marc Deslauriers) by stray files left over from a previous Desktop image copy on the USB drive; if it has not been formatted before writing the Server image files to the drive.
If you're seeing this issue and are able to reproduce it easily with your USB drive, please see if you have a "casper" directory on the USB drive, if that drive has been configured to contain a server installation image. The casper directory can be removed from a USB drive if it's meant to be used to install the Ubuntu Server (and only Ubuntu Server; the Ubuntu Desktop installer *requires* that directory and will fail if it's not there).