Glance image upload failure in CI as swift isn't running

Bug #1567444 reported by Steven Hardy
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Unassigned

Bug Description

Seeing failures in CI such as http://logs.openstack.org/78/296578/1/check-tripleo/gate-tripleo-ci-f22-upgrades/bd70058/console.html

2016-04-07 08:35:11.455 | + /tmp/tripleo-ci/scripts/tripleo.sh --overcloud-images
2016-04-07 08:35:11.460 | + sudo dd of=/var/log/image_build.txt
2016-04-07 09:05:05.003 | 500 Internal Server Error: Failed to upload image ff9d8cca-48bd-4c77-ad73-5eb501280555 (HTTP 500)
2016-04-07 09:05:05.051 | 3934+25358 records in
2016-04-07 09:05:05.051 | 9208+1 records out
2016-04-07 09:05:05.052 | 4714974 bytes (4.7 MB) copied, 1793.59 s, 2.6 kB/s
2016-04-07 09:05:05.100 | Transferred: sent 4204, received 44264 bytes, in 2900.4 seconds
2016-04-07 09:05:05.100 | Bytes per second: sent 1.4, received 15.3

Glance log shows this, appears Swift isn't running, there's no swift.log.

2016-04-07 09:05:03.612 20881 INFO swiftclient [req-1afaf1aa-3ed8-423f-8bdd-2f07e4fd26cf a0387a4caab14ed59ee4193d3969b671 f70ffa8afea04aac82cf8bb774c823df - - -] RESP BODY: <html><h1>Service Unavailable</h1><p>The server is currently unavailable. Please try again at a later time.</p></html>
2016-04-07 09:05:04.614 20881 ERROR glance_store._drivers.swift.store [req-1afaf1aa-3ed8-423f-8bdd-2f07e4fd26cf a0387a4caab14ed59ee4193d3969b671 f70ffa8afea04aac82cf8bb774c823df - - -] Failed to add object to Swift.
Got error from Swift: put_object('glance', 'ff9d8cca-48bd-4c77-ad73-5eb501280555', ...) failure and no ability to reset contents for reupload..
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils [req-1afaf1aa-3ed8-423f-8bdd-2f07e4fd26cf a0387a4caab14ed59ee4193d3969b671 f70ffa8afea04aac82cf8bb774c823df - - -] Failed to upload image ff9d8cca-48bd-4c77-ad73-5eb501280555
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils Traceback (most recent call last):
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance/api/v1/upload_utils.py", line 110, in upload_data_to_store
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils context=req.context)
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 346, in store_add_to_backend
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils verifier=verifier)
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 226, in op_checker
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils return store_op_fun(store, *args, **kwargs)
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils File "/usr/lib/python2.7/site-packages/glance_store/_drivers/swift/store.py", line 662, in add
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils raise glance_store.BackendException(msg)
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils BackendException: Failed to add object to Swift.
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils Got error from Swift: put_object('glance', 'ff9d8cca-48bd-4c77-ad73-5eb501280555', ...) failure and no ability to reset contents for reupload..
2016-04-07 09:05:04.616 20881 ERROR glance.api.v1.upload_utils·
2016-04-07 09:05:04.811 20881 DEBUG glance.registry.client.v1.api [req-1afaf1aa-3ed8-423f-8bdd-2f07e4fd26cf a0387a4caab14ed59ee4193d3969b671 f70ffa8afea04aac82cf8bb774c823df - - -] Updating image metadata for image ff9d8cca-48bd-4c77-ad73-5eb501280555... update_image_metadata /usr/lib/python2.7/site-packages/glance/registry/client/v1/api.py:170

Steven Hardy (shardy)
Changed in tripleo:
status: New → Triaged
importance: Undecided → Critical
Steven Hardy (shardy)
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.