Uploading images using euca-publish-tarball sets image size = 0 when using Swift storage

[Andrew Glen-Young]

glance 2011.3~rc~20110908.r1005-0ubuntu1
nova 2011.3~rc~20110909.r1155-0ubuntu1
swift 1.4.3-0ubuntu1

Until recently, I was using nova with glance + the filesystem backend without any problems. After switching to using swift as the backend all images uploaded are registered with a size of zero. This prevents the images from being used to launch instances.

Tests that I have performed:

1. Upload via nova using 'euca-publish-tarball' results in a registered file size of zero bytes.
2. Upload the same image using the glance command line client (without passing in the file size) results in the file being registered correctly with the correct file size.
3. Upload as in (1) and manually change the file size in the registry's database allows the image to be downloaded and booted correctly.

During tests (1) and (3) the logs indicate that glance.registry sets size to zero in the metadata, but the size is set correctly in (2).

When I query swift for the images directly, they report the correct sizes and can be downloaded directly.

[Jay Pipes (jaypipes)]

Almost positive this is due to Nova not using the latest Glance client (which does a file seek and sets the Content-Length of the request properly...

I'll check with Monty and Vish and see what's up. In the meantime, could you check on your Nova box (the one running nova-compute) that your Glance library has a glance.client.add_image() method that has a section that looks like this?

        if image_data:
            body = image_data
            headers['content-type'] = 'application/octet-stream'
            image_size = self._get_image_size(image_data)
            if image_size:
                headers['x-image-meta-size'] = image_size
                headers['content-length'] = image_size
        else:
            body = None

Thanks!
-jay

[Andrew Glen-Young (aglenyoung)]

On Thu 15-09-2011 17:05 ›, Jay Pipes wrote:
> Almost positive this is due to Nova not using the latest Glance client
> (which does a file seek and sets the Content-Length of the request
> properly...
>
> I'll check with Monty and Vish and see what's up. In the meantime, could
> you check on your Nova box (the one running nova-compute) that your
> Glance library has a glance.client.add_image() method that has a section
> that looks like this?
>
> if image_data:
> body = image_data
> headers['content-type'] = 'application/octet-stream'
> image_size = self._get_image_size(image_data)
> if image_size:
> headers['x-image-meta-size'] = image_size
> headers['content-length'] = image_size
> else:
> body = None
>
>

FWIW all my tests using glance have been from the machine with nova-*
installed on it.

The glance client library I have has a slightly different definition:

    def add_image(self, image_meta=None, image_data=None):
        [...]

        if image_data:
            body = image_data
            headers['content-type'] = 'application/octet-stream'
            # For large images, we need to supply the size of the
            # image file. See LP Bug #827660.
            if hasattr(image_data, 'seek') and hasattr(image_data, 'tell'):
                try:
                    image_data.seek(0, os.SEEK_END)
                    image_size = image_data.tell()
                    image_data.seek(0)
                    headers['x-image-meta-size'] = image_size
                    headers['content-length'] = image_size
                except IOError, e:
                    if e.errno == errno.ESPIPE:
                        # Illegal seek. This means the user is trying
                        # to pipe image data to the client, e.g.
                        # echo testdata | bin/glance add blah..., or
                        # that stdin is empty
                        pass
                    else:
                        raise
        else:
            body = None

        [...]

I've included log excerpts below in case they are of any use:

2011-09-15 16:17:39 DEBUG [glance.api.v1.images] Uploading image data for image 142 to swift store
2011-09-15 16:17:39 DEBUG [glance.api.v1.images] Got request with no content-length and no x-image-meta-size header
2011-09-15 16:17:39 DEBUG [glance.store.swift] Creating Swift connection with (auth_address=https://127.0.0.1/auth/v1.0/, user=glance:glance, key=XXXXXXXXXXXXXXXXXX, snet=False)
2011-09-15 16:17:39 DEBUG [glance.store.swift] Adding image object '142' to Swift
2011-09-15 16:18:37 DEBUG [glance.api.v1.images] Updating image 142 data. Checksum set to 7cefaa146c26e5947f872e5aa1d57ba0, size set to 0

--
Regards.
Andrew Glen-Young

[Jay Pipes (jaypipes)]

Gotcha. OK, this is indeed due to an older, buggy version of Glance's client code on the Nova box, and is a duplicate of bug #845788. A new package should be uploaded to PyPI for Glance (see https://bugs.launchpad.net/bugs/809663)

[Jay Pipes (jaypipes)]

FYI, the reason this works with the Glance client and not with the Nova client is because you are doing "glance add ... < myfile" in the Glance CLI client, which correctly calls add_image() which does a file seek. But, Nova's client actually reserves an image identifier with a call to POST /images, then uploads the image file (after being decrypted and decompressed from nova-objectstore) using a PUT /images/<ID> call. It is the latter call to update_image() that the bug referenced above fixes.

Cheers!
-jay