Only Cinder Service failed in Devstack

Bug #1840757 reported by Yeshwanth Allampati
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Incomplete
Undecided
Unassigned

Bug Description

cinder --debug list
DEBUG:keystoneauth.session:REQ: curl -g -i -X GET http://172.16.65.216/identity -H "Accept: application/json" -H "User-Agent: cinder keystoneauth1/3.13.1 python-requests/2.22.0 CPython/2.7.12"
DEBUG:keystoneauth.session:RESP: [300] Connection: close Content-Length: 272 Content-Type: application/json Date: Tue, 20 Aug 2019 09:58:56 GMT Location: http://172.16.65.216/identity/v3/ Server: Apache/2.4.18 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-f6e3d03c-e130-4132-92e7-223aa50950e0
DEBUG:keystoneauth.session:RESP BODY: {"versions": {"values": [{"status": "stable", "updated": "2019-01-22T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.12", "links": [{"href": "http://172.16.65.216/identity/v3/", "rel": "self"}]}]}}
DEBUG:keystoneauth.session:GET call to http://172.16.65.216/identity used request id req-f6e3d03c-e130-4132-92e7-223aa50950e0
DEBUG:keystoneauth.identity.v3.base:Making authentication request to http://172.16.65.216/identity/v3/auth/tokens
DEBUG:keystoneauth.identity.v3.base:{"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "a3171cbad8174416b6e96b59d95abcf8", "name": "member"}, {"id": "84c09c62c24d44a48253b8032aa02a45", "name": "reader"}, {"id": "9fbfa33ea06144419bd2af2c95da64c0", "name": "admin"}], "expires_at": "2019-08-20T10:58:56.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "5cb1426e43cd4f2db6bb428a6051622a", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://172.16.65.216/volume/v2/5cb1426e43cd4f2db6bb428a6051622a", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "1e470461085b47d9a91a492b7e1a85b0"}], "type": "volumev2", "id": "24f0ef3a4a8a4fabbc18317dcedd1847", "name": "cinderv2"}, {"endpoints": [{"url": "http://172.16.65.216/identity", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "569020847a9047d1938c9949d08d0827"}, {"url": "http://172.16.65.216/identity", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "80fd4d208c2849e9b72d749cc8cd74c8"}], "type": "identity", "id": "26cc41c907294149af0ce70c6773457e", "name": "keystone"}, {"endpoints": [{"url": "http://172.16.65.216/volume/v3/5cb1426e43cd4f2db6bb428a6051622a", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "76438f6e854e4810934d89b4258895ea"}], "type": "block-storage", "id": "73b9580de53249ca9824e6613f277f10", "name": "cinder"}, {"endpoints": [{"url": "http://172.16.65.216/volume/v3/5cb1426e43cd4f2db6bb428a6051622a", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "0cfd94d5b9b84b18834726d920dc214d"}], "type": "volumev3", "id": "9d06e237e64b4878843c8c10b651bdcc", "name": "cinderv3"}, {"endpoints": [{"url": "http://172.16.65.216:8080", "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "be3a85511bda496f8eaf79f186109c8b"}, {"url": "http://172.16.65.216:8080/v1/AUTH_5cb1426e43cd4f2db6bb428a6051622a", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "d675a549da5a41e2a4bb218171d6e86e"}], "type": "object-store", "id": "a86140ab7c17459d94bab8b52b148ace", "name": "swift"}, {"endpoints": [{"url": "http://172.16.65.216:9696/", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "65b0f4f37e034d1a89473869a255bdc2"}], "type": "network", "id": "ac32c00ffa19440b83f906f2ed83510d", "name": "neutron"}, {"endpoints": [{"url": "http://172.16.65.216/compute/v2/5cb1426e43cd4f2db6bb428a6051622a", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b8d5b7b95c534c318641c2e909110f59"}], "type": "compute_legacy", "id": "adb8efa06d2e4bf989f0a673c761e4ad", "name": "nova_legacy"}, {"endpoints": [{"url": "http://172.16.65.216/image", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "66f786ff4baf42f19eda1caad307c24f"}], "type": "image", "id": "b75cc5d60c574d9db377e7fd46bd4bf9", "name": "glance"}, {"endpoints": [{"url": "http://172.16.65.216/placement", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c640625db180470383f2a09cc5d4076a"}], "type": "placement", "id": "c5fac4a49a324c27986f7f900d09c333", "name": "placement"}, {"endpoints": [{"url": "http://172.16.65.216/compute/v2.1", "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "96b61d8d01054ddc9272fdabb5127ca5"}], "type": "compute", "id": "f81c8d3727d04c75b4c9aee437c82701", "name": "nova"}], "user": {"domain": {"id": "default", "name": "Default"}, "password_expires_at": null, "name": "admin", "id": "d645bb574c394feeb2390140ed981f8c"}, "audit_ids": ["1Bj5tZitTL6zBVSADeUk2Q"], "issued_at": "2019-08-20T09:58:56.000000Z"}}
DEBUG:keystoneauth:REQ: curl -g -i -X GET http://172.16.65.216/volume/v3/5cb1426e43cd4f2db6bb428a6051622a/volumes/detail -H "Accept: application/json" -H "User-Agent: python-cinderclient" -H "X-Auth-Token: {SHA256}ea56d6805d291da2599f5ab1f38619c63ab356a88bde552a4b719de562e143ea"
DEBUG:keystoneauth:RESP: [404] Connection: close Content-Length: 334 Content-Type: text/html; charset=iso-8859-1 Date: Tue, 20 Aug 2019 09:58:56 GMT Server: Apache/2.4.18 (Ubuntu)
DEBUG:keystoneauth:RESP BODY: Omitted, Content-Type is set to text/html; charset=iso-8859-1. Only application/json responses have their bodies logged.
DEBUG (shell:911) Not Found (HTTP 404)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/shell.py", line 906, in main
    for item in sys.argv[1:]])
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/shell.py", line 726, in main
    args.func(self.cs, args)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/v3/shell.py", line 418, in do_list
    sort_dir=args.sort_dir, sort=args.sort)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line 325, in list
    return self._list(url, resource_type, limit=limit)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/base.py", line 80, in _list
    resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 200, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 191, in _cs_request
    return self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 177, in request
    raise exceptions.from_response(resp, body)
NotFound: Not Found (HTTP 404)
DEBUG:cinderclient.shell:Not Found (HTTP 404)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/shell.py", line 906, in main
    for item in sys.argv[1:]])
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/shell.py", line 726, in main
    args.func(self.cs, args)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/v3/shell.py", line 418, in do_list
    sort_dir=args.sort_dir, sort=args.sort)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", line 325, in list
    return self._list(url, resource_type, limit=limit)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/base.py", line 80, in _list
    resp, body = self.api.client.get(url)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 200, in get
    return self._cs_request(url, 'GET', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 191, in _cs_request
    return self.request(url, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/cinderclient/client.py", line 177, in request
    raise exceptions.from_response(resp, body)
NotFound: Not Found (HTTP 404)
ERROR: Not Found (HTTP 404)

Revision history for this message
Sean McGinnis (sean-mcginnis) wrote :

stach.sh will fail if one service does not start, and last I knew cinder was not the last service that gets stacked. Can you explain more than just a stack trace?

Changed in cinder:
status: New → Incomplete
Revision history for this message
Yeshwanth Allampati (yeshwanth-allampati) wrote :

The Devstack Stein 'stack.sh' ran successfully and this happend after the devstack is installed.

At first, I configured another backend and It worked fine.

Later I tried to use barbican as key manager and changed the 'cinder.conf' accordingly.

(Like this https://docs.openstack.org/cinder/pike/configuration/block-storage/volume-encryption.html)

After I changed it, cinder commands didnt work. So, I reset the conf and after I restarted the Cinder Services, First I lost '/etc/cinder/cinder-api-uwsgi.ini' file.

I managed to restore it from another devstack Stein Setup.

After that, whenever I run cinder commands, I get Error 404 Not Found.

Revision history for this message
Yeshwanth Allampati (yeshwanth-allampati) wrote :

Thanks For the Concern. I was able to restore the cinder service by doing unstack.sh and stack.sh.

Revision history for this message
Yeshwanth Allampati (yeshwanth-allampati) wrote :

But Unable to spot the cause of the problem

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.