Images that are > 3G in size fail on a system with enough RAM for tmpfs

[Clint Byrum]

The heuristic used to check whether tmpfs can be used does not take into account that the image may be bigger than DIB_MIN_TMPFS GB. Once the image goes over around DIB_MIN_TMPFS*0.5, the final step that converts it to a raw disk image via loopback will fail because the tmpfs will fill up while copying everything onto the loopback.

dib should check for enough space left on the tmpfs to also store the loopback file, and if there isn't enough, build the loopback in a new non-tmpfs temp directory.

[Clint Byrum (clint-fewbar)]

Critical because this is breaking TripleO CI.

[Robert Collins (lifeless)]

I thought we did a mv, precisely to avoid this failure mode :(

[Robert Collins (lifeless)]

I've made the nova flavor for h1.large 16G memory, see if this fixes it.

[Robert Collins (lifeless)]

We're getting green tests so that does appear to be a valid workaround

[Clint Byrum (clint-fewbar)]

FYI, the reason mv doesn't fix the problem entirely is that there is no synchronous connection between the mv and the growth of the loopback file. One hypothesis is that ext4 extents cause a large section of the file to be pre-allocated all at once.

It kind of doesn't matter though because the heuristic also just assumes that images will be ~ 2G.. which is clearly false.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/91545
Committed: https://git.openstack.org/cgit/openstack/diskimage-builder/commit/?id=3fada2566a366c8bc1b668e91c8412b8fb63fd7a
Submitter: Jenkins
Branch: master

commit 3fada2566a366c8bc1b668e91c8412b8fb63fd7a
Author: Derek Higgins <email address hidden>
Date: Thu May 1 10:22:15 2014 +0100

    Build raw image in separate tmpfs

    If the tmpfs being used to hold the image filesystem is close to being
    full then housing the raw disk image in the same place can be
    problematic as it tends to grow faster then the filesystem shrinks when
    data is being moved into it.

    Putting them both into separate tmpfs's will allow there to be an
    overlap where they jointly use more then the size limit for a singe tmpfs.

    Change-Id: Ia17ca357d3b865d70a2d3e13e0479d008ca5f924
    Closes-Bug: #1289582