Metadata would ideally properly differentiate between None and the empty string

[justinsb]

This is probably low priority, as it's fairly easy to work around, but this will bite us one day when we have a property when null is not the same as the empty string.

The way metadata is returns is inconsistent between the images detail list and the metadata headers on the image...

In particular, below you can see that disk_format is null in the JSON, but in the header it is returned as the empty string. I think the header should not be returned, because otherwise you can't tell Null and "" apart. For now, it's fairly easy to assume that "" => None though, but this may not always be valid.

---

Extract from the details list:

{"images": [{"status": "active", "name": "Z}'DM^5z^yMM;a3lY2[@9D7sIs\\pax", "deleted": false, "container_format": null, "created_at": "2012-02-16T19:10:26", "disk_format": null, "updated_at": "2012-02-16T19:10:41", "properties": {}, "min_disk": 0, "protected": false, "id": "3fd5cb85-64de-4557-beb8-775c47d023dc", "checksum": "e9ceb5025565e99ef8bac91f612a2e2f", "owner": "d14f636a44394931b717fd30f4a1ba03", "is_public": false, "deleted_at": null, "min_ram": 0, "size": 4202895},

....
But then if we do a HEAD:

3 > HEAD http://192.168.71.1:9292/v1/images/3fd5cb85-64de-4557-beb8-775c47d023dc
3 > Accept: application/json
3 > X-Auth-Token: c82aa86c-85e8-4734-b348-76e95d9485f0
3 * Client in-bound response
3 < 200
3 < X-Image-Meta-Checksum: e9ceb5025565e99ef8bac91f612a2e2f
3 < X-Image-Meta-Owner: d14f636a44394931b717fd30f4a1ba03
3 < X-Image-Meta-Deleted_at:
3 < Content-Length: 0
3 < X-Image-Meta-Deleted: False
3 < X-Image-Meta-Name: Z}'DM^5z^yMM;a3lY2[@9D7sIs\pax
3 < Location: http://192.168.71.1:9292/v1/images/3fd5cb85-64de-4557-beb8-775c47d023dc
3 < Etag: e9ceb5025565e99ef8bac91f612a2e2f
3 < X-Image-Meta-Created_at: 2012-02-16T19:10:26
3 < X-Image-Meta-Protected: False
3 < Connection: keep-alive
3 < X-Image-Meta-Min_disk: 0
3 < Date: Thu, 16 Feb 2012 19:29:03 GMT
3 < X-Image-Meta-Id: 3fd5cb85-64de-4557-beb8-775c47d023dc
3 < X-Image-Meta-Min_ram: 0
3 < X-Image-Meta-Disk_format:
3 < X-Image-Meta-Status: active
3 < X-Image-Meta-Updated_at: 2012-02-16T19:10:41
3 < X-Image-Meta-Is_public: False
3 < X-Image-Meta-Size: 4202895
3 < Content-Type: text/html; charset=UTF-8
3 < X-Image-Meta-Container_format:
3 <

[Jay Pipes (jaypipes)]

I think you may have stumbled upon a much larger problem... An image should never be able to be created with a disk_format or container_format of null :(

[Jay Pipes (jaypipes)]

Critical since this is a data loss/corruption bug.

[Jay Pipes (jaypipes)]

Justin, could you create a test case that is able to reproduce the behaviour you are seeing? This is very concerning for me... as there should be default values inserted if no disk_format or container_format is given.

[justinsb (justin-fathomdb)]

I'm using the API, not the glance command line tool, so if it's enforced in the command line tool that'll be why.

Also, I'm using SQLite (running devstack) so if it's enforced using a DB constraint, that could be why.

Here's the log from POSTing an image without metadata, and below that I ran glance show and the container_format and disk_format are null/None.

Is it because I'm running SQLite / going via the API? I didn't realize these were required; where is it supposed to be enforced?

git log on glance shows I last updated on Feb 11th, if that matters (commit 56efd271ac08b4c0e0ce273b9cbc0c5952af05fa)

22 > POST http://192.168.71.1:9292/v1/images
22 > Accept: application/json
22 > x-image-meta-name: _?pl=i:EQ)zI9w9iQt?5j)
22 > X-Auth-Token: c82aa86c-85e8-4734-b348-76e95d9485f0
hjbz�c�ݏ`��w,� �&� )�
��%�y�˚�?�|�m
^,�l��{
ˋo�]�
��M�����C��|�XT���
L�i��]��z��h�`�����.���s�'����
�Ss9E,�Q��
22 * Client in-bound response
22 < 201
22 < Date: Thu, 16 Feb 2012 20:27:53 GMT
22 < Content-Length: 463
22 < Location: http://192.168.71.1:9292/v1/images/c9c21225-7166-4809-bd09-9719633db6d3
22 < Etag: bb65589093b4b3c28481388e5585c8bf
22 < Content-Type: application/json
22 < Connection: keep-alive
22 <
{"image": {"status": "active", "name": "_?pl=i:EQ)zI9w9iQt?5j)", "deleted": false, "container_format": null, "created_at": "2012-02-16T20:27:53", "disk_format": null, "updated_at": "2012-02-16T20:27:53", "properties": {}, "min_disk": 0, "protected": false, "id": "c9c21225-7166-4809-bd09-9719633db6d3", "checksum": "bb65589093b4b3c28481388e5585c8bf", "owner": "d14f636a44394931b717fd30f4a1ba03", "is_public": false, "deleted_at": null, "min_ram": 0, "size": 128}}

---

stack@openstack1:~/devstack$ glance show c9c21225-7166-4809-bd09-9719633db6d3
URI: http://192.168.71.1:9292/v1/images/c9c21225-7166-4809-bd09-9719633db6d3
Id: c9c21225-7166-4809-bd09-9719633db6d3
Public: No
Protected: No
Name: _?pl=i:EQ)zI9w9iQt?5j)
Status: active
Size: 128
Disk format: None
Container format: None
Minimum Ram Required (MB): 0
Minimum Disk Required (GB): 0
Owner: d14f636a44394931b717fd30f4a1ba03

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/4272

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/4272
Committed: http://github.com/openstack/glance/commit/62c913c3ad4db1a03461f51304fac885196df8ca
Submitter: Jenkins
Branch: master

commit 62c913c3ad4db1a03461f51304fac885196df8ca
Author: Eoghan Glynn <email address hidden>
Date: Fri Feb 17 13:46:49 2012 +0000

    Require container & disk formats on image create

    Fixes lp 933702

    For images created via the glance CLI, the container and disk formats
    were previously defaulted if not explicitly set. However if created via
    the python or REST APIs, these attributes were not defaulted if unset.

    There is no real sensible default for these formats, so now an image
    create fails with 400 "Bad Request" if the format metadata are missing.

    Also we ensure unset image metadata are not reported in x-image-meta-*
    headers in order to disambiguate None and empty string values.

    Change-Id: I8189383f5f9adf42a8cdac7f8dc7e9327baf46da