mysql may have duplicate entry for resource

[ZhiQiang Fan]

run devstack with latest code on ubuntu 14.04 using mysql as backend, I have observed that there is a chance that resource table has duplicate entry. for example, I have clear all data, then with 3 images for 2 metrics, mysql shows

+-------------+--------------------------------------+---------+----------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+
| internal_id | resource_id | user_id | project_id | source_id | resource_metadata | metadata_hash |
+-------------+--------------------------------------+---------+----------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+
| 77 | 559a584b-b147-4629-9ff8-401728cc5aa9 | NULL | f952728cbc8a46e0870c1c41bd3234ec | openstack | {"status": "active", "name": "cirros-0.3.2-x86_64-uec", "deleted": false, "container_format": "ami", "created_at": "2015-01-30T05:24:49", "disk_format": "ami", "updated_at": "2015-01-30T05:24:49", "properties": {"kernel_id": "6d2c38a6-0b07-4566-8a4f-cd53a04beca3", "ramdisk_id": "7b4d553b-eacb-483b-93e1-1bd4bcb16f4b"}, "protected": false, "checksum": "4eada48c2843d2a262c814ddc92ecf2c", "min_disk": 0, "is_public": true, "deleted_at": null, "min_ram": 0, "size": 25165824} | 4529d35d56b4150d2cdd0a0cb0115498 |
| 78 | 7b4d553b-eacb-483b-93e1-1bd4bcb16f4b | NULL | f952728cbc8a46e0870c1c41bd3234ec | openstack | {"status": "active", "name": "cirros-0.3.2-x86_64-uec-ramdisk", "deleted": false, "container_format": "ari", "created_at": "2015-01-30T05:24:48", "disk_format": "ari", "updated_at": "2015-01-30T05:24:48", "properties": {}, "protected": false, "checksum": "68085af2609d03e51c7662395b5b6e4b", "min_disk": 0, "is_public": true, "deleted_at": null, "min_ram": 0, "size": 3723817} | afdaab3dd44e2f33f8432602dc903ee1 |
| 79 | 7b4d553b-eacb-483b-93e1-1bd4bcb16f4b | NULL | f952728cbc8a46e0870c1c41bd3234ec | openstack | {"status": "active", "name": "cirros-0.3.2-x86_64-uec-ramdisk", "deleted": false, "container_format": "ari", "created_at": "2015-01-30T05:24:48", "disk_format": "ari", "updated_at": "2015-01-30T05:24:48", "properties": {}, "protected": false, "checksum": "68085af2609d03e51c7662395b5b6e4b", "min_disk": 0, "is_public": true, "deleted_at": null, "min_ram": 0, "size": 3723817} | afdaab3dd44e2f33f8432602dc903ee1 |
| 80 | 6d2c38a6-0b07-4566-8a4f-cd53a04beca3 | NULL | f952728cbc8a46e0870c1c41bd3234ec | openstack | {"status": "active", "name": "cirros-0.3.2-x86_64-uec-kernel", "deleted": false, "container_format": "aki", "created_at": "2015-01-30T05:24:47", "disk_format": "aki", "updated_at": "2015-01-30T05:24:47", "properties": {}, "protected": false, "checksum": "836c69cbcd1dc4f225daedbab6edc7c7", "min_disk": 0, "is_public": true, "deleted_at": null, "min_ram": 0, "size": 4969360} | 37cf77e4b24b2e75b6c7655fc063acb6 |
| 81 | 6d2c38a6-0b07-4566-8a4f-cd53a04beca3 | NULL | f952728cbc8a46e0870c1c41bd3234ec | openstack | {"status": "active", "name": "cirros-0.3.2-x86_64-uec-kernel", "deleted": false, "container_format": "aki", "created_at": "2015-01-30T05:24:47", "disk_format": "aki", "updated_at": "2015-01-30T05:24:47", "properties": {}, "protected": false, "checksum": "836c69cbcd1dc4f225daedbab6edc7c7", "min_disk": 0, "is_public": true, "deleted_at": null, "min_ram": 0, "size": 4969360} | 37cf77e4b24b2e75b6c7655fc063acb6 |
+-------------+--------------------------------------+---------+----------------------------------+-----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------+

note that there are two duplicate entries for the two resources, I don't know why

any explanation?

[gordon chung (chungg)]

i'm guessing it's because of this: https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/sqlalchemy/models.py#L172-L178

[gordon chung (chungg)]

if the above is correct, i created this bug: https://bugs.launchpad.net/ceilometer/+bug/1353039

[ZhiQiang Fan (aji-zqfan)]

I thought it would be multiple collector workers write data to db then our logic is if there is no such resource, then we insert a new one, I'm afraid there is a race condition when there is a new resource with multiple metrics

[gordon chung (chungg)]

i think it's possible to just add distinct to query?... that or we make fields smaller to enable uniquecontrainst... the second option is better but definitely bigger

[gordon chung (chungg)]

marking this as a byproduct of this: https://bugs.launchpad.net/ceilometer/+bug/1353039