Activity log for bug #1653206

Date Who What changed Old value New value Message
2016-12-30 08:34:17 Prateek khushalani bug added bug
2016-12-30 09:01:20 Prateek khushalani description I manually installed Magnum-api and Magnum-conductor services on Openstack Mitaka enviornment. It had the following components pre-installed :- 1. Cinder 2. Neutron-lbaas The baymodel got created successfully and when I tried to craete the a kubernetes bay following happened:- 1. Kube-master VM got created. 2. Loadbalancer VIP got created 3. Health-checks on port 2379 got created(ETCD) service I was able to login to the VM and even tried to run kubectl commands it was working. However the status of the bay in "magnum bay-list is stuck in CREATE-IN-PROGRESS". After checking the logs the magnum conductor gave this error :- ---------------------------------------------------------------------------- 2016-12-29 23:39:52.615 18042 DEBUG keystoneclient.v3.client [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Authorization failed. get_raw_token_from_identity_service /usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py:303 2016-12-29 23:39:52.616 18042 WARNING magnum.service.periodic [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Ignore error [reason: A user and password or token is required.] when syncing up bay status. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic Traceback (most recent call last): 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 82, in sync_bay_status 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic stacks = osc.heat().stacks.list(global_tenant=True,filters={'id': bay_stack_ids}) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 173, in wrapped 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic client=func.__name__, message="reason: %s" % sys.exc_info()[1]) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic AuthorizationFailure: reason: A user and password or token is required. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic 2016-12-29 23:39:52.672 18042 DEBUG heatclient.common.http [-] curl -g -i -X GET -H 'X-Auth-Token: {SHA1}3b4d35d453f34ffc48f478117920dc7def84ee1c' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://A1-39978-controller:5000/v3' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://A1-39978-controller:8004/v1/1f4ff4d2c3684ffab7d983f3740e7411/stacks/ade48de7-c0df-4b13-994a-c7de6d572a09 log_curl_request /usr/lib/python2.7/dist-packages/heatclient/common/http.py:129 2016-12-29 23:39:52.754 18042 DEBUG heatclient.common.http [-] ---------------------------------------------------------------------------- Here is the magnum.conf for refference:- ---------------------------------------------------------------------------- [DEFAULT] transport_url = rabbit://openstack:password@10.10.10.10 debug=true verbose=true periodic_global_stack_list= True pybasedir = /usr/lib/python2.7/dist-packages/magnum bindir = /usr/bin state_path = /var/lib/magnum [api] host = 10.10.10.10 [certificates] cert_manager_type = local [cinder_client] region_name = RegionOne [database] connection = mysql+pymysql://magnum:password@10.10.10.10/magnum [keystone_authtoken] auth_version = v3 auth_uri = http://10.10.10.10:5000/v3 auth_url = http://10.10.10.10:35357/v3 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = magnum password = password [oslo_concurrency] lock_path = /var/lock/magnum [oslo_messaging_notifications] driver = messaging [oslo_messaging_rabbit] rabbit_host = 10.10.10.10 rabbit_userid = openstack rabbit_password = password [trust] trustee_domain_name = magnum trustee_domain_id = 8b673a7e21f74588b8edb28541bb0877 trustee_domain_admin_name = magnum_domain_admin trustee_domain_admin_id = c95849d2d09a4b718f7bb5d2aee77dd5 trustee_domain_admin_password = password ---------------------------------------------------------------------------- It seems magnum conductor is not using keystone v3 I manually installed Magnum-api and Magnum-conductor services on Openstack Mitaka enviornment. It had the following components pre-installed :- 1. Cinder 2. Neutron-lbaas The baymodel got created successfully and when I tried to craete the a kubernetes bay following happened:- 1. Kube-master VM got created. 2. Loadbalancer VIP got created 3. Health-checks on port 2379 got created(ETCD) service I was able to login to the VM and even tried to run kubectl commands it was working. However the status of the bay in "magnum bay-list is stuck in CREATE-IN-PROGRESS". After checking the logs the magnum conductor gave this error :- ---------------------------------------------------------------------------- 2016-12-29 23:39:52.615 18042 DEBUG keystoneclient.v3.client [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Authorization failed. get_raw_token_from_identity_service /usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py:303 2016-12-29 23:39:52.616 18042 WARNING magnum.service.periodic [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Ignore error [reason: A user and password or token is required.] when syncing up bay status. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic Traceback (most recent call last): 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 82, in sync_bay_status 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic stacks = osc.heat().stacks.list(global_tenant=True,filters={'id': bay_stack_ids}) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 173, in wrapped 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic client=func.__name__, message="reason: %s" % sys.exc_info()[1]) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic AuthorizationFailure: reason: A user and password or token is required. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic 2016-12-29 23:39:52.672 18042 DEBUG heatclient.common.http [-] curl -g -i -X GET -H 'X-Auth-Token: {SHA1}3b4d35d453f34ffc48f478117920dc7def84ee1c' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://A1-39978-controller:5000/v3' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://controller:8004/v1/1f4ff4d2c3684ffab7d983f3740e7411/stacks/ade48de7-c0df-4b13-994a-c7de6d572a09 log_curl_request /usr/lib/python2.7/dist-packages/heatclient/common/http.py:129 2016-12-29 23:39:52.754 18042 DEBUG heatclient.common.http [-] ---------------------------------------------------------------------------- Here is the magnum.conf for refference:- ---------------------------------------------------------------------------- [DEFAULT] transport_url = rabbit://openstack:password@10.10.10.10 debug=true verbose=true periodic_global_stack_list= True pybasedir = /usr/lib/python2.7/dist-packages/magnum bindir = /usr/bin state_path = /var/lib/magnum [api] host = 10.10.10.10 [certificates] cert_manager_type = local [cinder_client] region_name = RegionOne [database] connection = mysql+pymysql://magnum:password@10.10.10.10/magnum [keystone_authtoken] auth_version = v3 auth_uri = http://10.10.10.10:5000/v3 auth_url = http://10.10.10.10:35357/v3 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = magnum password = password [oslo_concurrency] lock_path = /var/lock/magnum [oslo_messaging_notifications] driver = messaging [oslo_messaging_rabbit] rabbit_host = 10.10.10.10 rabbit_userid = openstack rabbit_password = password [trust] trustee_domain_name = magnum trustee_domain_id = 8b673a7e21f74588b8edb28541bb0877 trustee_domain_admin_name = magnum_domain_admin trustee_domain_admin_id = c95849d2d09a4b718f7bb5d2aee77dd5 trustee_domain_admin_password = password ---------------------------------------------------------------------------- It seems magnum conductor is not using keystone v3
2016-12-30 09:02:31 Prateek khushalani description I manually installed Magnum-api and Magnum-conductor services on Openstack Mitaka enviornment. It had the following components pre-installed :- 1. Cinder 2. Neutron-lbaas The baymodel got created successfully and when I tried to craete the a kubernetes bay following happened:- 1. Kube-master VM got created. 2. Loadbalancer VIP got created 3. Health-checks on port 2379 got created(ETCD) service I was able to login to the VM and even tried to run kubectl commands it was working. However the status of the bay in "magnum bay-list is stuck in CREATE-IN-PROGRESS". After checking the logs the magnum conductor gave this error :- ---------------------------------------------------------------------------- 2016-12-29 23:39:52.615 18042 DEBUG keystoneclient.v3.client [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Authorization failed. get_raw_token_from_identity_service /usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py:303 2016-12-29 23:39:52.616 18042 WARNING magnum.service.periodic [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Ignore error [reason: A user and password or token is required.] when syncing up bay status. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic Traceback (most recent call last): 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 82, in sync_bay_status 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic stacks = osc.heat().stacks.list(global_tenant=True,filters={'id': bay_stack_ids}) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 173, in wrapped 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic client=func.__name__, message="reason: %s" % sys.exc_info()[1]) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic AuthorizationFailure: reason: A user and password or token is required. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic 2016-12-29 23:39:52.672 18042 DEBUG heatclient.common.http [-] curl -g -i -X GET -H 'X-Auth-Token: {SHA1}3b4d35d453f34ffc48f478117920dc7def84ee1c' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://A1-39978-controller:5000/v3' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://controller:8004/v1/1f4ff4d2c3684ffab7d983f3740e7411/stacks/ade48de7-c0df-4b13-994a-c7de6d572a09 log_curl_request /usr/lib/python2.7/dist-packages/heatclient/common/http.py:129 2016-12-29 23:39:52.754 18042 DEBUG heatclient.common.http [-] ---------------------------------------------------------------------------- Here is the magnum.conf for refference:- ---------------------------------------------------------------------------- [DEFAULT] transport_url = rabbit://openstack:password@10.10.10.10 debug=true verbose=true periodic_global_stack_list= True pybasedir = /usr/lib/python2.7/dist-packages/magnum bindir = /usr/bin state_path = /var/lib/magnum [api] host = 10.10.10.10 [certificates] cert_manager_type = local [cinder_client] region_name = RegionOne [database] connection = mysql+pymysql://magnum:password@10.10.10.10/magnum [keystone_authtoken] auth_version = v3 auth_uri = http://10.10.10.10:5000/v3 auth_url = http://10.10.10.10:35357/v3 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = magnum password = password [oslo_concurrency] lock_path = /var/lock/magnum [oslo_messaging_notifications] driver = messaging [oslo_messaging_rabbit] rabbit_host = 10.10.10.10 rabbit_userid = openstack rabbit_password = password [trust] trustee_domain_name = magnum trustee_domain_id = 8b673a7e21f74588b8edb28541bb0877 trustee_domain_admin_name = magnum_domain_admin trustee_domain_admin_id = c95849d2d09a4b718f7bb5d2aee77dd5 trustee_domain_admin_password = password ---------------------------------------------------------------------------- It seems magnum conductor is not using keystone v3 I manually installed Magnum-api and Magnum-conductor services on Openstack Mitaka enviornment. It had the following components pre-installed :- 1. Cinder 2. Neutron-lbaas The baymodel got created successfully and when I tried to craete the a kubernetes bay following happened:- 1. Kube-master VM got created. 2. Loadbalancer VIP got created 3. Health-checks on port 2379 got created(ETCD) service I was able to login to the VM and even tried to run kubectl commands it was working. However the status of the bay in "magnum bay-list is stuck in CREATE-IN-PROGRESS". After checking the logs the magnum conductor gave this error :- ---------------------------------------------------------------------------- 2016-12-29 23:39:52.615 18042 DEBUG keystoneclient.v3.client [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Authorization failed. get_raw_token_from_identity_service /usr/lib/python2.7/dist-packages/keystoneclient/v3/client.py:303 2016-12-29 23:39:52.616 18042 WARNING magnum.service.periodic [req-3a651119-aa5b-4688-9426-ae1f51c23f0f - - - - -] Ignore error [reason: A user and password or token is required.] when syncing up bay status. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic Traceback (most recent call last): 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/service/periodic.py", line 82, in sync_bay_status 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic stacks = osc.heat().stacks.list(global_tenant=True,filters={'id': bay_stack_ids}) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic File "/usr/lib/python2.7/dist-packages/magnum/common/exception.py", line 173, in wrapped 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic client=func.__name__, message="reason: %s" % sys.exc_info()[1]) 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic AuthorizationFailure: reason: A user and password or token is required. 2016-12-29 23:39:52.616 18042 ERROR magnum.service.periodic 2016-12-29 23:39:52.672 18042 DEBUG heatclient.common.http [-] curl -g -i -X GET -H 'X-Auth-Token: {SHA1}3b4d35d453f34ffc48f478117920dc7def84ee1c' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://controller:5000/v3' -H 'Accept: application/json' -H 'User-Agent: python-heatclient' http://controller:8004/v1/1f4ff4d2c3684ffab7d983f3740e7411/stacks/ade48de7-c0df-4b13-994a-c7de6d572a09 log_curl_request /usr/lib/python2.7/dist-packages/heatclient/common/http.py:129 2016-12-29 23:39:52.754 18042 DEBUG heatclient.common.http [-] ---------------------------------------------------------------------------- Here is the magnum.conf for refference:- ---------------------------------------------------------------------------- [DEFAULT] transport_url = rabbit://openstack:password@10.10.10.10 debug=true verbose=true periodic_global_stack_list= True pybasedir = /usr/lib/python2.7/dist-packages/magnum bindir = /usr/bin state_path = /var/lib/magnum [api] host = 10.10.10.10 [certificates] cert_manager_type = local [cinder_client] region_name = RegionOne [database] connection = mysql+pymysql://magnum:password@10.10.10.10/magnum [keystone_authtoken] auth_version = v3 auth_uri = http://10.10.10.10:5000/v3 auth_url = http://10.10.10.10:35357/v3 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = magnum password = password [oslo_concurrency] lock_path = /var/lock/magnum [oslo_messaging_notifications] driver = messaging [oslo_messaging_rabbit] rabbit_host = 10.10.10.10 rabbit_userid = openstack rabbit_password = password [trust] trustee_domain_name = magnum trustee_domain_id = 8b673a7e21f74588b8edb28541bb0877 trustee_domain_admin_name = magnum_domain_admin trustee_domain_admin_id = c95849d2d09a4b718f7bb5d2aee77dd5 trustee_domain_admin_password = password ---------------------------------------------------------------------------- It seems magnum conductor is not using keystone v3
2016-12-30 11:32:13 Prateek khushalani summary In Mitaka Bay stuck in Create_IN_Prgress state and conductor gives error In Mitaka Bay stuck in Create_IN_Progress state and conductor gives error
2017-01-03 07:57:07 Prateek khushalani magnum: assignee Prateek khushalani (prateek-khushalani)