In Mitaka Bay stuck in Create_IN_Progress state and conductor gives error

[Prateek khushalani]

I manually installed Magnum-api and Magnum-conductor services on Openstack Mitaka enviornment. It had the following components pre-installed :-

1. Cinder
2. Neutron-lbaas

The baymodel got created successfully and when I tried to craete the a kubernetes bay following happened:-

1. Kube-master VM got created.
2. Loadbalancer VIP got created
3. Health-checks on port 2379 got created(ETCD) service

I was able to login to the VM and even tried to run kubectl commands it was working. However the status of the bay in "magnum bay-list is stuck in CREATE-IN-PROGRESS".

After checking the logs the magnum conductor gave this error :-

----------------------------------------------------------------------------
2016-12-29 23:39:52.615 18042 DEBUG keystoneclient.v3.client [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Authorization failed. get_raw_token_from_identity_service /usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py:303
2016-12-29 23:39:52.616 18042 WARNING magnum.service.periodic [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Ignore error [reason: A user and password or token is required.] when syncing up bay status.
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic Traceback (most recent call last):
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 82, in sync_bay_status
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic stacks = osc.heat().stacks.list(global_tenant=True,filters={'id': bay_stack_ids})
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 173, in wrapped
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic client=func.__name__, message="reason: %s" % sys.exc_info()[1])
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic AuthorizationFailure: reason: A user and password or token is required.
2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic
2016-12-29 23:39:52.672 18042 DEBUG heatclient.common.http [-] curl -g -i -X GET -H 'X-Auth-Token: {SHA1}3b4d35d453f34ffc48f478117920dc7def84ee1c' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://controller:5000/v3' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://controller:8004/v1/1f4ff4d2c3684ffab7d983f3740e7411/stacks/ade48de7-c0df-4b13-994a-c7de6d572a09 log_curl_request /usr/lib/python2.7/dist-packages/heatclient/common/http.py:129
2016-12-29 23:39:52.754 18042 DEBUG heatclient.common.http [-]
----------------------------------------------------------------------------

Here is the magnum.conf for refference:-

----------------------------------------------------------------------------
[DEFAULT]
transport_url = rabbit://openstack:password@10.10.10.10
debug=true
verbose=true
periodic_global_stack_list= True
pybasedir = /usr/lib/python2.7/dist-packages/magnum
bindir = /usr/bin
state_path = /var/lib/magnum

[api]
host = 10.10.10.10

[certificates]
cert_manager_type = local

[cinder_client]
region_name = RegionOne

[database]
connection = mysql+pymysql://magnum:password@10.10.10.10/magnum

[keystone_authtoken]
auth_version = v3
auth_uri = http://10.10.10.10:5000/v3
auth_url = http://10.10.10.10:35357/v3
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = magnum
password = password

[oslo_concurrency]
lock_path = /var/lock/magnum

[oslo_messaging_notifications]
driver = messaging

[oslo_messaging_rabbit]
rabbit_host = 10.10.10.10
rabbit_userid = openstack
rabbit_password = password

[trust]
trustee_domain_name = magnum
trustee_domain_id = 8b673a7e21f74588b8edb28541bb0877
trustee_domain_admin_name = magnum_domain_admin
trustee_domain_admin_id = c95849d2d09a4b718f7bb5d2aee77dd5
trustee_domain_admin_password = password

----------------------------------------------------------------------------

It seems magnum conductor is not using keystone v3

[Prateek khushalani (prateek-khushalani)]

Found the fix in the Mitaka release will send it for review

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/416171

[OpenStack Infra (hudson-openstack)]

Change abandoned by Spyros Trigazis (strigazi) (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/416171
Reason: Mitaka is EOL