Glance

Cache prefetcher can't work with trusted-auth registry

Bug #1502991 reported by Hemanth Makkapati on 2015-10-05

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Glance	Won't Fix	Undecided	Hemanth Makkapati

Bug Description

As described below, Cache prefetcher fails to work when the registry is deployed in trusted-auth mode.
Cache prefetcher should send necessary identity headers to registry when pulling image metadata.

$ curl -X PUT http://127.0.0.1:9292/v1/queued_images/7c383ea4-f108-43c4-9d1e-02c8a6a1385c -H 'X-Auth-Token: ABCD' -i
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 4
X-Openstack-Request-Id: req-f3012c1b-a4ff-4ec2-a88b-3048bee0938d
Date: Mon, 05 Oct 2015 17:54:57 GMT

$ curl -X GET http://127.0.0.1:9292/v1/queued_images -H 'X-Auth-Token: ABCD' -i
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 59
X-Openstack-Request-Id: req-a5cb12a1-6126-425c-9cff-d540c3a01f4e
Date: Mon, 05 Oct 2015 17:55:16 GMT

{"queued_images": ["7c383ea4-f108-43c4-9d1e-02c8a6a1385c"]}

$ /opt/stack/glance/glance/cmd/cache_prefetcher.py
2015-10-05 17:55:31.226 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.sheepdog.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.226 9296 DEBUG glance_store.backend [-] Attempting to import store no_conf _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.227 9296 DEBUG glance_store.backend [-] Attempting to import store http _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.227 9296 DEBUG glance_store.backend [-] Attempting to import store file _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.227 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.http.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.227 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.rbd.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.227 9296 DEBUG glance_store.backend [-] Attempting to import store rbd _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.gridfs.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store s3 _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store vmware _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.cinder.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store gridfs _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.228 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.filesystem.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.229 9296 DEBUG glance_store.backend [-] Attempting to import store cinder _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.229 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.swift.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.229 9296 DEBUG glance_store.backend [-] Attempting to import store swift _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.229 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.vmware_datastore.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.229 9296 DEBUG glance_store.backend [-] Attempting to import store sheepdog _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.230 9296 DEBUG glance_store.backend [-] Attempting to import store glance.store.s3.Store _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.230 9296 DEBUG glance_store.backend [-] Registering options for group glance_store register_opts /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:78
2015-10-05 17:55:31.230 9296 DEBUG glance_store.backend [-] Registering options for group glance_store register_opts /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:78
2015-10-05 17:55:31.230 9296 DEBUG glance_store.backend [-] Attempting to import store http _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.231 9296 DEBUG glance_store.capabilities [-] Store glance_store._drivers.http.Store doesn't support updating dynamic storage capabilities. Please overwrite 'update_capabilities' method of the
store to implement updating logics if needed. update_capabilities /usr/local/lib/python2.7/dist-packages/glance_store/capabilities.py:98
2015-10-05 17:55:31.231 9296 DEBUG glance_store.backend [-] Registering store http with schemes ('http', 'https') create_stores /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:196
2015-10-05 17:55:31.231 9296 DEBUG glance_store.driver [-] Late loading location class glance_store._drivers.http.StoreLocation get_store_location_class /usr/local/lib/python2.7/dist-packages/glance_store/driver
.py:95
2015-10-05 17:55:31.231 9296 DEBUG glance_store.location [-] Registering scheme http with {'location_class': <class 'glance_store._drivers.http.StoreLocation'>, 'store': <glance_store._drivers.http.Store object
at 0x7f5b35074b90>, 'store_entry': 'http'} register_scheme_map /usr/local/lib/python2.7/dist-packages/glance_store/location.py:90
2015-10-05 17:55:31.231 9296 DEBUG glance_store.location [-] Registering scheme https with {'location_class': <class 'glance_store._drivers.http.StoreLocation'>, 'store': <glance_store._drivers.http.Store object
at 0x7f5b35074b90>, 'store_entry': 'http'} register_scheme_map /usr/local/lib/python2.7/dist-packages/glance_store/location.py:90
2015-10-05 17:55:31.231 9296 DEBUG glance_store.backend [-] Attempting to import store file _load_store /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:149
2015-10-05 17:55:31.232 9296 DEBUG glance_store.capabilities [-] Store glance_store._drivers.filesystem.Store doesn't support updating dynamic storage capabilities. Please overwrite 'update_capabilities' method
of the store to implement updating logics if needed. update_capabilities /usr/local/lib/python2.7/dist-packages/glance_store/capabilities.py:98
2015-10-05 17:55:31.233 9296 DEBUG glance_store.backend [-] Registering store file with schemes ('file', 'filesystem') create_stores /usr/local/lib/python2.7/dist-packages/glance_store/backend.py:196
2015-10-05 17:55:31.233 9296 DEBUG glance_store.driver [-] Late loading location class glance_store._drivers.filesystem.StoreLocation get_store_location_class /usr/local/lib/python2.7/dist-packages/glance_store/
driver.py:95
2015-10-05 17:55:31.233 9296 DEBUG glance_store.location [-] Registering scheme file with {'location_class': <class 'glance_store._drivers.filesystem.StoreLocation'>, 'store': <glance_store._drivers.filesystem.S
tore object at 0x7f5b31678050>, 'store_entry': 'file'} register_scheme_map /usr/local/lib/python2.7/dist-packages/glance_store/location.py:90
2015-10-05 17:55:31.233 9296 DEBUG glance_store.location [-] Registering scheme filesystem with {'location_class': <class 'glance_store._drivers.filesystem.StoreLocation'>, 'store': <glance_store._drivers.filesy
stem.Store object at 0x7f5b31678050>, 'store_entry': 'file'} register_scheme_map /usr/local/lib/python2.7/dist-packages/glance_store/location.py:90
2015-10-05 17:55:31.235 9296 INFO glance.image_cache [-] Image cache loaded driver 'sqlite'.
2015-10-05 17:55:31.236 9296 DEBUG glance.image_cache.prefetcher [-] Found 1 images to prefetch run /opt/stack/glance/glance/image_cache/prefetcher.py:75
2015-10-05 17:55:31.326 9296 DEBUG glance.common.client [req-c5b6db8d-1a1a-4ca6-8d46-a3fd94882052 - - - - -] Constructed URL: http://104.239.166.31:9191/images/7c383ea4-f108-43c4-9d1e-02c8a6a1385c _construct_url
/opt/stack/glance/glance/common/client.py:402
2015-10-05 17:55:31.458 9296 DEBUG glance.common.client [req-c5b6db8d-1a1a-4ca6-8d46-a3fd94882052 - - - - -] Constructed URL: http://104.239.166.31:9191/images/7c383ea4-f108-43c4-9d1e-02c8a6a1385c _construct_url
/opt/stack/glance/glance/common/client.py:402
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client [req-c5b6db8d-1a1a-4ca6-8d46-a3fd94882052 - - - - -] Registry client request GET /images/7c383ea4-f108-43c4-9d1e-02c8a6a1385c raised NotAuthenticated
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client Traceback (most recent call last):
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client File "/opt/stack/glance/glance/registry/client/v1/client.py", line 121, in do_request
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client **kwargs)
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client File "/opt/stack/glance/glance/common/client.py", line 74, in wrapped
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client return func(self, *args, **kwargs)
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client File "/opt/stack/glance/glance/common/client.py", line 377, in do_request
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client headers=copy.deepcopy(headers))
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client File "/opt/stack/glance/glance/common/client.py", line 88, in wrapped
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client return func(self, method, url, body, headers)
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client File "/opt/stack/glance/glance/common/client.py", line 519, in _do_request
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client raise exception.NotAuthenticated(res.read())
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client NotAuthenticated: 401 Unauthorized
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client
2015-10-05 17:55:31.462 9296 ERROR glance.registry.client.v1.client
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main
    result = function(*args, **kwargs)
  File "/opt/stack/glance/glance/image_cache/prefetcher.py", line 46, in fetch_image_into_cache
    image_meta = registry.get_image_metadata(ctx, image_id)
  File "/opt/stack/glance/glance/registry/client/v1/api.py", line 160, in get_image_metadata
    return c.get_image(image_id)
  File "/opt/stack/glance/glance/registry/client/v1/client.py", line 158, in get_image
    res = self.do_request("GET", "/images/%s" % image_id)
  File "/opt/stack/glance/glance/registry/client/v1/client.py", line 136, in do_request
    'exc_name': exc_name})
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/opt/stack/glance/glance/registry/client/v1/client.py", line 121, in do_request
    **kwargs)
  File "/opt/stack/glance/glance/common/client.py", line 74, in wrapped
    return func(self, *args, **kwargs)
  File "/opt/stack/glance/glance/common/client.py", line 377, in do_request
    headers=copy.deepcopy(headers))
  File "/opt/stack/glance/glance/common/client.py", line 88, in wrapped
    return func(self, method, url, body, headers)
  File "/opt/stack/glance/glance/common/client.py", line 519, in _do_request
    raise exception.NotAuthenticated(res.read())
NotAuthenticated: 401 Unauthorized

This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the cr
edentials required.

See original description

Hemanth Makkapati (hemanth-makkapati) on 2015-10-05

Changed in glance:
assignee:	nobody → Hemanth Makkapati (hemanth-makkapati)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2015-10-05: Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/231138

Changed in glance:
status:	New → In Progress

Flavio Percoco (flaper87) on 2015-10-21

description:

updated

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-01-18: Change abandoned on glance (master)

Change abandoned by Glance Bot (<email address hidden>) on branch: master
Review: https://review.openstack.org/231138

Revision history for this message

Abhishek Kekane (abhishek-kekane) wrote on 2021-06-17:

Marking as Won't Fix as registry is now removed from glance code base.

Changed in glance:
status:	In Progress → Won't Fix

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.