errors in db log when i run glance index

Bug #1276114 reported by gustavo panizzo on 2014-02-04
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Undecided
Unassigned

Bug Description

after some 1-2 weeks of use glance starts to send garbage to the db and finally glance won't work anymore until a reinitialization takes place (drop db, sync_db, re upload imgs). on my production db it started to happen today again

i'm using postgresql 9.1, python-psycopg2 2.4.5-1, havana release of glance

postgres log

Feb 4 08:39:02 infra01 postgres[30429]: [4-1] 2014-02-04 08:39:02 ART ERROR: invalid input syntax for type timestamp: "" at character 4367
Feb 4 08:39:02 infra01 postgres[30429]: [4-2] 2014-02-04 08:39:02 ART STATEMENT: SELECT anon_1.anon_2_images_created_at AS anon_1_anon_2_images_created_at, anon_1.anon_2_images_updated_at AS anon_1_anon_2_images_updated_at, anon_1.anon_2_images_deleted_at AS anon_1_anon_2_images_deleted_at, anon_1.anon_2_images_deleted AS anon_1_anon_2_images_deleted, anon_1.anon_2_images_id AS anon_1_anon_2_images_id, anon_1.anon_2_images_name AS anon_1_anon_2_images_name, anon_1.anon_2_images_disk_format AS anon_1_anon_2_images_disk_format, anon_1.anon_2_images_container_format AS anon_1_anon_2_images_container_format, anon_1.anon_2_images_size AS anon_1_anon_2_images_size, anon_1.anon_2_images_status AS anon_1_anon_2_images_status, anon_1.anon_2_images_is_public AS anon_1_anon_2_images_is_public, anon_1.anon_2_images_checksum AS anon_1_anon_2_images_checksum, anon_1.anon_2_images_min_disk AS anon_1_anon_2_images_min_disk,
Feb 4 08:39:02 infra01 postgres[30429]: [4-3] anon_1.anon_2_images_min_ram AS anon_1_anon_2_images_min_ram, anon_1.anon_2_images_owner AS anon_1_anon_2_images_owner, anon_1.anon_2_images_protected AS anon_1_anon_2_images_protected, image_properties_1.created_at AS image_properties_1_created_at, image_properties_1.updated_at AS image_properties_1_updated_at, image_properties_1.deleted_at AS image_properties_1_deleted_at, image_properties_1.deleted AS image_properties_1_deleted, image_properties_1.id AS image_properties_1_id, image_properties_1.image_id AS image_properties_1_image_id, image_properties_1.name AS image_properties_1_name, image_properties_1.value AS image_properties_1_value, image_locations_1.created_at AS image_locations_1_created_at, image_locations_1.updated_at AS image_locations_1_updated_at, image_locations_1.deleted_at AS image_locations_1_deleted_at, image_locations_1.deleted AS image_locations_1_deleted,
Feb 4 08:39:02 infra01 postgres[30429]: [4-4] image_locations_1.id AS image_locations_1_id, image_locations_1.image_id AS image_locations_1_image_id, image_locations_1.value AS image_locations_1_value, image_locations_1.meta_data AS image_locations_1_meta_data
Feb 4 08:39:02 infra01 postgres[30429]: [4-5] #011FROM (SELECT anon_2.images_created_at AS anon_2_images_created_at, anon_2.images_updated_at AS anon_2_images_updated_at, anon_2.images_deleted_at AS anon_2_images_deleted_at, anon_2.images_deleted AS anon_2_images_deleted, anon_2.images_id AS anon_2_images_id, anon_2.images_name AS anon_2_images_name, anon_2.images_disk_format AS anon_2_images_disk_format, anon_2.images_container_format AS anon_2_images_container_format, anon_2.images_size AS anon_2_images_size, anon_2.images_status AS anon_2_images_status, anon_2.images_is_public AS anon_2_images_is_public, anon_2.images_checksum AS anon_2_images_checksum, anon_2.images_min_disk AS anon_2_images_min_disk, anon_2.images_min_ram AS anon_2_images_min_ram, anon_2.images_owner AS anon_2_images_owner, anon_2.images_protected AS anon_2_images_protected
Feb 4 08:39:02 infra01 postgres[30429]: [4-6] #011FROM (SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.status AS images_status, images.is_public AS images_is_public, images.checksum AS images_checksum, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected
Feb 4 08:39:02 infra01 postgres[30429]: [4-7] #011FROM images
Feb 4 08:39:02 infra01 postgres[30429]: [4-8] #011WHERE images.is_public = true OR images.owner = 'af41371701004068b2532695ed57f16f' UNION SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.status AS images_status, images.is_public AS images_is_public, images.checksum AS images_checksum, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected
Feb 4 08:39:02 infra01 postgres[30429]: [4-9] #011FROM images JOIN image_members ON images.id = image_members.image_id
Feb 4 08:39:02 infra01 postgres[30429]: [4-10] #011WHERE image_members.deleted = false AND image_members.member = 'af41371701004068b2532695ed57f16f') AS anon_2
Feb 4 08:39:02 infra01 postgres[30429]: [4-11] #011WHERE anon_2.images_deleted = false AND anon_2.images_status != 'killed' AND (CASE WHEN (anon_2.images_created_at IS NOT NULL) THEN anon_2.images_created_at ELSE '' END < '2014-01-28T16:08:16.845874'::timestamp OR CASE WHEN (anon_2.images_created_at IS NOT NULL) THEN anon_2.images_created_at ELSE '' END = '2014-01-28T16:08:16.845874'::timestamp AND CASE WHEN (anon_2.images_id IS NOT NULL) THEN anon_2.images_id ELSE '' END < '38fe03a3-8ef2-4445-8754-585d3e84b2ac') ORDER BY anon_2.images_created_at DESC, anon_2.images_id DESC
Feb 4 08:39:02 infra01 postgres[30429]: [4-12] #011 LIMIT 10) AS anon_1 LEFT OUTER JOIN image_properties AS image_properties_1 ON anon_1.anon_2_images_id = image_properties_1.image_id LEFT OUTER JOIN image_locations AS image_locations_1 ON anon_1.anon_2_images_id = image_locations_1.image_id ORDER BY anon_1.anon_2_images_created_at DESC, anon_1.anon_2_images_id DESC

$ glance index
ID Name Disk Format Container Format Size
------------------------------------ ------------------------------ -------------------- -------------------- --------------
86d2fa74-aa86-435c-bce8-84a3acf12000 web2py-testing-environment qcow2 ovf 1401683968
ba234ba1-6ab6-45b1-a820-cfb0811a85c1 Win2008r2 qcow2 ovf 3280732160
af19bcb6-2621-46b4-bb4e-bb33a52dd8f8 Win7x86 qcow2 ovf 3374317568
b3ca4b0d-df9d-4f0c-8518-7bbe9e173a2d Win2012 qcow2 ovf 4284743680
11bf41ef-2478-49d9-9262-f751c1f37e13 Win7x64 qcow2 ovf 4608753664
cc75052c-3d5f-4774-ab50-abcf82554e60 Win2012-sql qcow2 ovf 13241942016
d89e01c0-b862-418f-a19d-66b504a6ff04 mojo-testing-environment ami ami 3125346304
bee5e021-3786-441b-9133-e590a78b21de wheezy-14-01-17 raw bare 996999168
b7d0db05-0b86-4707-8511-7f99a2fc32ea initrd-mockup ari ari 17541543
38fe03a3-8ef2-4445-8754-585d3e84b2ac kernel-mockup aki aki 4100880
Request returned failure status.
HTTPInternalServerError (HTTP 500)

gustavo panizzo (gfa) wrote :

a diff of the db's dump btw the time of the error and the previous day shows no differences

To old, and no one else has encountered this issue ever since.

Changed in glance:
status: New → Incomplete
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers