glance image download does not decompress the content as curl does

[Attila Fazekas]

When you download this image with curl, you get a qcow2 file, NOT a gz file.
https://fedorapeople.org/groups/magnum/fedora-21-atomic-5.qcow2

When you use
glance --os-image-api-version=1 image-create --copy-from https://fedorapeople.org/groups/magnum/fedora-21-atomic-5.qcow2 --container-format bare --disk-format qcow2 --property os_distro=fedora-atomic --property os_type=linux --min-disk 8 --min-ram 512 --name fedora-21-atomic-5 --is-public True

It will be stored as gz compressed file, and when you or nova downloads it will be gzip compressed as well.
glance image-download <id> --file test.qcow2.gz

glance MUST not ask for gzip compression when it is unable to handle it.
glance SHOULD be able handle compressed content.

Note:
I had rbd backend.

I do not have this issue with the https://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2 url, because the server refuses to compress , regardless to what the client requests.

[Nikhil Komawar (nikhil-komawar)]

Hi Attila Fazekas,

I don't see this being enforced by Glance. The way your images are getting compressed are through the gzip middleware that can be optionally enabled using the right filter in your glance-api.paste.ini file.
https://github.com/openstack/glance/blob/master/etc/glance-api-paste.ini#L76

Can you please check and verify is that's still the case?

Also, can you share some of these details about your request?
Headers Content-Type and Accept-Encoding if present and their values.

[Ian Cordasco (icordasc)]

Piotr, we also need the response headers. Accept-Encoding affects the *transfer* of the data. In other words, by the client saying that it accepts `gzip` encoding, that will be applied by the server to save bandwidth and on streaming the download it should be decoded automatically by our underlying HTTP library. The way it determines that is by looking at the Content-Encoding response header. If that's not present, there's nothing for the library to unzip. If the Content-Type, however, is gzip, that's a *completely* different story.

Can you retrieve those response headers?

[Piotr Misiak (piotr-misiak)]

Here you have all requests and responses headers:

HEAD /images/PLATFORM_010.vdi HTTP/1.1
Host: XXXXXXXXXXX
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.9.1

HTTP/1.1 200 OK
Date: Thu, 09 Feb 2017 15:17:26 GMT
Connection: keep-alive
X-Content-Type-Options: nosniff
Last-Modified: Wed, 08 Feb 2017 14:45:16 GMT
ETag: "310b00000-54805e9ae020c-gzip"
Accept-Ranges: bytes
Cache-Control: max-age=1209600
Expires: Thu, 23 Feb 2017 15:17:26 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Server: Apache/2

GET /images/PLATFORM_010.vdi HTTP/1.1
Host: XXXXXXXXXXXX
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.9.1

HTTP/1.1 200 OK
Date: Thu, 09 Feb 2017 15:17:26 GMT
Transfer-Encoding: chunked
Connection: keep-alive
X-Content-Type-Options: nosniff
Last-Modified: Wed, 08 Feb 2017 14:45:16 GMT
ETag: "310b00000-54805e9ae020c-gzip"
Accept-Ranges: bytes
Cache-Control: max-age=1209600
Expires: Thu, 23 Feb 2017 15:17:26 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Server: Apache/2

[Ian Cordasco (icordasc)]

Piotr, one last question: Is this just when talking to v1 of the API or does it happen with both v1 and v2?

[Piotr Misiak (piotr-misiak)]

I have Mitaka version and I don't see any option in CLI client to specify a remote URL during image creation in Glance API v2, so I can't test it. If it is possible tell me how and I will test v2.