Glance

Failed to create image due to exception NotAuthenticated

Bug #1530328 reported by javeme
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
Won't Fix
Undecided
Unassigned

Bug Description

Failed to create image due to exception NotAuthenticated.

If we take a long time to upload a large image to store, it may trigger an exception "NotAuthenticated: Authentication required" and the image has been stuck in SAVING state, this due to the token expires after a period of time(default is expiration=3600s -- 1 hour, this can be modified in the keystone conf file).

PS:
Image Size: 80G
Take Time: 2 hours

Icehouse
Ceph backend

The error log below:
2015-12-12 18:45:00.416 6370 INFO glance.registry.client.v1.client [b7ee5de9-addf-457c-88a8-552476044f0d 44edf16aa49f4469a76c90c984fe7ce7 b10adc7c9afd4506b26d42779c77b2f4 - - -] Registry client request PUT /images/9366c0a1-40a9-4f4a-a821-2acac905ecd5 raised NotAuthenticated
2015-12-12 18:45:00.417 6370 ERROR glance.api.v1.upload_utils [b7ee5de9-addf-457c-88a8-552476044f0d 44edf16aa49f4469a76c90c984fe7ce7 b10adc7c9afd4506b26d42779c77b2f4 - - -] Failed to upload image 9366c0a1-40a9-4f4a-a821-2acac905ecd5
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/api/v1/upload_utils.py", line 152, in upload_data_to_store
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils from_state='saving')
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/api.py", line 169, in update_image_metadata
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils from_state=from_state)
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/client.py", line 190, in update_image
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils headers=headers)
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/client.py", line 107, in do_request
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils **kwargs)
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 69, in wrapped
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils return func(self, *args, **kwargs)
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 386, in do_request
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils headers=copy.deepcopy(headers))
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 83, in wrapped
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils return func(self, method, url, body, headers)
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 527, in _do_request
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils raise exception.NotAuthenticated(res.read())
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils NotAuthenticated: Authentication required
2015-12-12 18:45:00.417 6370 TRACE glance.api.v1.upload_utils
2015-12-12 18:45:00.447 6370 INFO glance.registry.client.v1.client [b7ee5de9-addf-457c-88a8-552476044f0d 44edf16aa49f4469a76c90c984fe7ce7 b10adc7c9afd4506b26d42779c77b2f4 - - -] Registry client request PUT /images/9366c0a1-40a9-4f4a-a821-2acac905ecd5 raised NotAuthenticated
2015-12-12 18:45:00.448 6370 ERROR glance.api.v1.upload_utils [b7ee5de9-addf-457c-88a8-552476044f0d 44edf16aa49f4469a76c90c984fe7ce7 b10adc7c9afd4506b26d42779c77b2f4 - - -] Unable to kill image 9366c0a1-40a9-4f4a-a821-2acac905ecd5:
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils Traceback (most recent call last):
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/api/v1/upload_utils.py", line 76, in safe_kill
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils _kill(req, image_id, from_state)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/api/v1/upload_utils.py", line 61, in _kill
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils from_state=from_state)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/api.py", line 169, in update_image_metadata
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils from_state=from_state)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/client.py", line 190, in update_image
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils headers=headers)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/registry/client/v1/client.py", line 107, in do_request
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils **kwargs)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 69, in wrapped
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils return func(self, *args, **kwargs)
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 386, in do_request
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils headers=copy.deepcopy(headers))
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils File "/usr/lib/python2.6/site-packages/glance/common/client.py", line 83, in wrapped
2015-12-12 18:45:00.448 6370 TRACE glance.api.v1.upload_utils return func(self, method, url, body, headers)

See original description
javeme (javaloveme)
description: updated
Revision history for this message
Niall Bunting (niall-bunting) wrote :

As posted on your review (https://review.openstack.org/#/c/262770/) this is being solved by trusts.

Changed in glance:
status: New → Won't Fix
Revision history for this message
Niall Bunting (niall-bunting) wrote :

Here is the solution proposed: https://review.openstack.org/#/c/241986/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on glance (master)

Change abandoned by javeme (<email address hidden>) on branch: master
Review: https://review.openstack.org/262770

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.