Metadata would ideally properly differentiate between None and the empty string

Bug #933702 reported by justinsb on 2012-02-16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Eoghan Glynn

Bug Description

This is probably low priority, as it's fairly easy to work around, but this will bite us one day when we have a property when null is not the same as the empty string.

The way metadata is returns is inconsistent between the images detail list and the metadata headers on the image...

In particular, below you can see that disk_format is null in the JSON, but in the header it is returned as the empty string. I think the header should not be returned, because otherwise you can't tell Null and "" apart. For now, it's fairly easy to assume that "" => None though, but this may not always be valid.


Extract from the details list:

{"images": [{"status": "active", "name": "Z}'DM^5z^yMM;a3lY2[@9D7sIs\\pax", "deleted": false, "container_format": null, "created_at": "2012-02-16T19:10:26", "disk_format": null, "updated_at": "2012-02-16T19:10:41", "properties": {}, "min_disk": 0, "protected": false, "id": "3fd5cb85-64de-4557-beb8-775c47d023dc", "checksum": "e9ceb5025565e99ef8bac91f612a2e2f", "owner": "d14f636a44394931b717fd30f4a1ba03", "is_public": false, "deleted_at": null, "min_ram": 0, "size": 4202895},

But then if we do a HEAD:

3 > HEAD
3 > Accept: application/json
3 > X-Auth-Token: c82aa86c-85e8-4734-b348-76e95d9485f0
3 * Client in-bound response
3 < 200
3 < X-Image-Meta-Checksum: e9ceb5025565e99ef8bac91f612a2e2f
3 < X-Image-Meta-Owner: d14f636a44394931b717fd30f4a1ba03
3 < X-Image-Meta-Deleted_at:
3 < Content-Length: 0
3 < X-Image-Meta-Deleted: False
3 < X-Image-Meta-Name: Z}'DM^5z^yMM;a3lY2[@9D7sIs\pax
3 < Location:
3 < Etag: e9ceb5025565e99ef8bac91f612a2e2f
3 < X-Image-Meta-Created_at: 2012-02-16T19:10:26
3 < X-Image-Meta-Protected: False
3 < Connection: keep-alive
3 < X-Image-Meta-Min_disk: 0
3 < Date: Thu, 16 Feb 2012 19:29:03 GMT
3 < X-Image-Meta-Id: 3fd5cb85-64de-4557-beb8-775c47d023dc
3 < X-Image-Meta-Min_ram: 0
3 < X-Image-Meta-Disk_format:
3 < X-Image-Meta-Status: active
3 < X-Image-Meta-Updated_at: 2012-02-16T19:10:41
3 < X-Image-Meta-Is_public: False
3 < X-Image-Meta-Size: 4202895
3 < Content-Type: text/html; charset=UTF-8
3 < X-Image-Meta-Container_format:
3 <

Jay Pipes (jaypipes) wrote :

I think you may have stumbled upon a much larger problem... An image should never be able to be created with a disk_format or container_format of null :(

Changed in glance:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → essex-4
Jay Pipes (jaypipes) wrote :

Critical since this is a data loss/corruption bug.

Jay Pipes (jaypipes) wrote :

Justin, could you create a test case that is able to reproduce the behaviour you are seeing? This is very concerning for me... as there should be default values inserted if no disk_format or container_format is given.

justinsb (justin-fathomdb) wrote :

I'm using the API, not the glance command line tool, so if it's enforced in the command line tool that'll be why.

Also, I'm using SQLite (running devstack) so if it's enforced using a DB constraint, that could be why.

Here's the log from POSTing an image without metadata, and below that I ran glance show and the container_format and disk_format are null/None.

Is it because I'm running SQLite / going via the API? I didn't realize these were required; where is it supposed to be enforced?

git log on glance shows I last updated on Feb 11th, if that matters (commit 56efd271ac08b4c0e0ce273b9cbc0c5952af05fa)

22 > POST
22 > Accept: application/json
22 > x-image-meta-name: _?pl=i:EQ)zI9w9iQt?5j)
22 > X-Auth-Token: c82aa86c-85e8-4734-b348-76e95d9485f0
hjbz�c�ݏ`��w,� �&� )�
22 * Client in-bound response
22 < 201
22 < Date: Thu, 16 Feb 2012 20:27:53 GMT
22 < Content-Length: 463
22 < Location:
22 < Etag: bb65589093b4b3c28481388e5585c8bf
22 < Content-Type: application/json
22 < Connection: keep-alive
22 <
{"image": {"status": "active", "name": "_?pl=i:EQ)zI9w9iQt?5j)", "deleted": false, "container_format": null, "created_at": "2012-02-16T20:27:53", "disk_format": null, "updated_at": "2012-02-16T20:27:53", "properties": {}, "min_disk": 0, "protected": false, "id": "c9c21225-7166-4809-bd09-9719633db6d3", "checksum": "bb65589093b4b3c28481388e5585c8bf", "owner": "d14f636a44394931b717fd30f4a1ba03", "is_public": false, "deleted_at": null, "min_ram": 0, "size": 128}}


stack@openstack1:~/devstack$ glance show c9c21225-7166-4809-bd09-9719633db6d3
Id: c9c21225-7166-4809-bd09-9719633db6d3
Public: No
Protected: No
Name: _?pl=i:EQ)zI9w9iQt?5j)
Status: active
Size: 128
Disk format: None
Container format: None
Minimum Ram Required (MB): 0
Minimum Disk Required (GB): 0
Owner: d14f636a44394931b717fd30f4a1ba03

Eoghan Glynn (eglynn) on 2012-02-17
Changed in glance:
assignee: nobody → Eoghan Glynn (eglynn)
Eoghan Glynn (eglynn) on 2012-02-17
Changed in glance:
status: Confirmed → In Progress

Submitter: Jenkins
Branch: master

commit 62c913c3ad4db1a03461f51304fac885196df8ca
Author: Eoghan Glynn <email address hidden>
Date: Fri Feb 17 13:46:49 2012 +0000

    Require container & disk formats on image create

    Fixes lp 933702

    For images created via the glance CLI, the container and disk formats
    were previously defaulted if not explicitly set. However if created via
    the python or REST APIs, these attributes were not defaulted if unset.

    There is no real sensible default for these formats, so now an image
    create fails with 400 "Bad Request" if the format metadata are missing.

    Also we ensure unset image metadata are not reported in x-image-meta-*
    headers in order to disambiguate None and empty string values.

    Change-Id: I8189383f5f9adf42a8cdac7f8dc7e9327baf46da

Changed in glance:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2012-02-29
Changed in glance:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in glance:
milestone: essex-4 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers