Juniper Openstack

contrail-kube-manager container fails saying user contrail is not found

Bug #1689509 reported by Vedamurthy Joshi on 2017-05-09

This bug affects 1 person

	Status	Importance	Assigned to	Milestone
Juniper Openstack	Status tracked in Trunk
R4.0	Fix Committed	Critical	Rudra Rugge	Juniper Openstack r4.0.0.0-fcs "r4.0.0.0"
Trunk	Fix Committed	Critical	Rudra Rugge	Juniper Openstack r4.1.0.0-fcs "r4.1"

Bug Description

R4.0 Build 3073

contrail-kube-manager container is failing to come up

2017-05-09T08:58:58.478629000Z TASK [contrail/common : Make sure contrail log directory exists with appropriate ownership] ***
2017-05-09T08:58:58.802251000Z fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "gid": 0, "group": "root", "mode": "0755", "msg": "chown failed: failed to look up user contrail", "owner": "root", "path": "/var/log/contrail", "size": 6, "state": "directory", "uid": 0}

Below is the docker log

2017-05-09T08:48:36.200669000Z
2017-05-09T08:52:01.302549000Z + set -e
2017-05-09T08:52:01.302878000Z + PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
2017-05-09T08:52:01.303134000Z + DAEMON=/usr/bin/supervisord
2017-05-09T08:52:01.303379000Z + CONFIG=/etc/contrail/contrail-kubernetes.conf
2017-05-09T08:52:01.303664000Z + SERVICE=kubernetes
2017-05-09T08:52:01.303914000Z + ANSIBLE_INVENTORY=all-in-one
2017-05-09T08:52:01.304154000Z + test -x /usr/bin/supervisord
2017-05-09T08:52:01.304401000Z + DAEMON_OPTS='-n -c /etc/contrail/supervisord_kubernetes.conf '
2017-05-09T08:52:01.304682000Z + ulimit -s unlimited
2017-05-09T08:52:01.304923000Z + ulimit -c unlimited
2017-05-09T08:52:01.305159000Z + ulimit -d unlimited
2017-05-09T08:52:01.305372000Z + ulimit -v unlimited
2017-05-09T08:52:01.305653000Z + ulimit -n 4096
2017-05-09T08:52:01.305894000Z + contrailctl config sync -c kubemanager -F -v -t configure
2017-05-09T08:52:01.937603000Z
2017-05-09T08:52:01.937938000Z PLAY [Setup Contrail Kubernetes Network manager] *******************************
2017-05-09T08:52:01.996410000Z
2017-05-09T08:52:01.996768000Z TASK [setup] *******************************************************************
2017-05-09T08:52:02.619626000Z ok: [localhost]
2017-05-09T08:52:02.623480000Z
2017-05-09T08:52:02.624029000Z TASK [include_vars] ************************************************************
2017-05-09T08:52:02.647309000Z ok: [localhost]
2017-05-09T08:52:02.652159000Z
2017-05-09T08:52:02.652482000Z TASK [common : make sure /var/run/utmp exist] **********************************
2017-05-09T08:52:02.943870000Z ok: [localhost]
2017-05-09T08:52:02.944194000Z
2017-05-09T08:52:02.944468000Z TASK [common : Findout if current machine is a container or a machine] *********
2017-05-09T08:52:03.256374000Z changed: [localhost]
2017-05-09T08:52:03.260081000Z
2017-05-09T08:52:03.260349000Z TASK [common : Setting a fact whether the code is running in a docker container or not] ***
2017-05-09T08:52:03.307204000Z ok: [localhost]
2017-05-09T08:52:03.310685000Z
2017-05-09T08:52:03.310938000Z TASK [common : Setting a default for deployment_platform] **********************
2017-05-09T08:52:03.363317000Z ok: [localhost]
2017-05-09T08:52:03.363628000Z
2017-05-09T08:52:03.363879000Z TASK [common : Assuming default service manager] *******************************
2017-05-09T08:52:03.414432000Z ok: [localhost]
2017-05-09T08:52:03.414785000Z
2017-05-09T08:52:03.415084000Z TASK [common : Setting a default for enable_lb] ********************************
2017-05-09T08:52:03.459803000Z ok: [localhost]
2017-05-09T08:52:03.466821000Z
2017-05-09T08:52:03.467070000Z TASK [common : Setting a default for enable_controller] ************************
2017-05-09T08:52:03.527370000Z ok: [localhost]
2017-05-09T08:52:03.527693000Z
2017-05-09T08:52:03.527946000Z TASK [common : Setting a default for enable_analytics] *************************
2017-05-09T08:52:03.579914000Z ok: [localhost]
2017-05-09T08:52:03.580171000Z
2017-05-09T08:52:03.580416000Z TASK [common : Setting a fact no_upstart when deployment platform is docker] ***
2017-05-09T08:52:03.628348000Z ok: [localhost]
2017-05-09T08:52:03.630642000Z
2017-05-09T08:52:03.630903000Z TASK [common : Setting fact to override ansible_service_mgr for docker. This is to wrongly detect service manager as systemd when ubuntu container run on top of centos] ***
2017-05-09T08:52:03.676203000Z skipping: [localhost]
2017-05-09T08:52:03.676627000Z
2017-05-09T08:52:03.676895000Z TASK [common : Setting default for controller_components] **********************
2017-05-09T08:52:03.723245000Z ok: [localhost]
2017-05-09T08:52:03.725945000Z
2017-05-09T08:52:03.726219000Z TASK [common : update hosts file] **********************************************
2017-05-09T08:52:04.077592000Z ok: [localhost] => (item={'key': u'{{ ansible_hostname }}', 'value': u'127.0.0.1'})
2017-05-09T08:52:04.082105000Z
2017-05-09T08:52:04.082467000Z TASK [contrail/common : include OS family specific package dependencies] *******
2017-05-09T08:52:04.156451000Z included: /contrail-ansible-internal/playbooks/roles/contrail/common/tasks/install/Debian.yml for localhost
2017-05-09T08:52:04.166270000Z
2017-05-09T08:52:04.166748000Z TASK [contrail/common : Check if source SSL Cert directory present] ************
2017-05-09T08:52:04.371921000Z ok: [localhost]
2017-05-09T08:52:04.372268000Z
2017-05-09T08:52:04.372773000Z TASK [contrail/common : Create SSL directories on container] *******************
2017-05-09T08:52:04.416452000Z skipping: [localhost] => (item=certs)
2017-05-09T08:52:04.425921000Z skipping: [localhost] => (item=private)
2017-05-09T08:52:04.426195000Z
2017-05-09T08:52:04.426457000Z TASK [contrail/common : Copy SSL files for use by container] *******************
2017-05-09T08:52:04.474463000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/certs/ca-cert.pem', u'src': u'/etc/contrailctl/ssl/ca-cert.pem'})
2017-05-09T08:52:04.480277000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/certs/server.pem', u'src': u'/etc/contrailctl/ssl/server.pem'})
2017-05-09T08:52:04.485773000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/private/server-privkey.pem', u'src': u'/etc/contrailctl/ssl/server-privkey.pem'})
2017-05-09T08:52:04.490341000Z
2017-05-09T08:52:04.490811000Z TASK [contrail/kube_manager : include] *****************************************
2017-05-09T08:52:04.578525000Z included: /contrail-ansible-internal/playbooks/roles/contrail/kube_manager/tasks/install/Debian.yml for localhost
2017-05-09T08:52:04.594336000Z
2017-05-09T08:52:04.594825000Z TASK [contrail/common : Load OS distribution specific code] ********************
2017-05-09T08:52:04.640990000Z
2017-05-09T08:52:04.641453000Z TASK [contrail/kube_manager : Include contrail service manager specific code] **
2017-05-09T08:52:04.714530000Z included: /contrail-ansible-internal/playbooks/roles/contrail/kube_manager/tasks/install/supervisord.yml for localhost
2017-05-09T08:52:04.736378000Z
2017-05-09T08:52:04.736902000Z TASK [contrail/common : Make sure contrail log directory exists with appropriate ownership] ***
2017-05-09T08:52:05.047703000Z fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "gid": 0, "group": "root", "mode": "0755", "msg": "chown failed: failed to look up user contrail", "owner": "root", "path": "/var/log/contrail", "size": 6, "state": "directory", "uid": 0}
2017-05-09T08:52:05.049393000Z to retry, use: --limit @/contrail-ansible-internal/playbooks/contrail_kube_manager.retry
2017-05-09T08:52:05.049855000Z
2017-05-09T08:52:05.050228000Z PLAY RECAP *********************************************************************
2017-05-09T08:52:05.050671000Z localhost : ok=17 changed=1 unreachable=0 failed=1
2017-05-09T08:52:05.051052000Z
2017-05-09T08:58:54.960105000Z + set -e
2017-05-09T08:58:54.960439000Z + PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
2017-05-09T08:58:54.960760000Z + DAEMON=/usr/bin/supervisord
2017-05-09T08:58:54.961001000Z + CONFIG=/etc/contrail/contrail-kubernetes.conf
2017-05-09T08:58:54.961252000Z + SERVICE=kubernetes
2017-05-09T08:58:54.961498000Z + ANSIBLE_INVENTORY=all-in-one
2017-05-09T08:58:54.961808000Z + test -x /usr/bin/supervisord
2017-05-09T08:58:54.962046000Z + DAEMON_OPTS='-n -c /etc/contrail/supervisord_kubernetes.conf '
2017-05-09T08:58:54.962310000Z + ulimit -s unlimited
2017-05-09T08:58:54.962576000Z + ulimit -c unlimited
2017-05-09T08:58:54.962884000Z + ulimit -d unlimited
2017-05-09T08:58:54.963130000Z + ulimit -v unlimited
2017-05-09T08:58:54.963397000Z + ulimit -n 4096
2017-05-09T08:58:54.963669000Z + contrailctl config sync -c kubemanager -F -v -t configure
2017-05-09T08:58:55.591648000Z
2017-05-09T08:58:55.592046000Z PLAY [Setup Contrail Kubernetes Network manager] *******************************
2017-05-09T08:58:55.651580000Z
2017-05-09T08:58:55.652093000Z TASK [setup] *******************************************************************
2017-05-09T08:58:56.295938000Z ok: [localhost]
2017-05-09T08:58:56.299011000Z
2017-05-09T08:58:56.299295000Z TASK [include_vars] ************************************************************
2017-05-09T08:58:56.332046000Z ok: [localhost]
2017-05-09T08:58:56.336336000Z
2017-05-09T08:58:56.336628000Z TASK [common : make sure /var/run/utmp exist] **********************************
2017-05-09T08:58:56.647145000Z ok: [localhost]
2017-05-09T08:58:56.647462000Z
2017-05-09T08:58:56.647729000Z TASK [common : Findout if current machine is a container or a machine] *********
2017-05-09T08:58:56.992400000Z changed: [localhost]
2017-05-09T08:58:56.992811000Z
2017-05-09T08:58:56.993070000Z TASK [common : Setting a fact whether the code is running in a docker container or not] ***
2017-05-09T08:58:57.044667000Z ok: [localhost]
2017-05-09T08:58:57.044982000Z
2017-05-09T08:58:57.045306000Z TASK [common : Setting a default for deployment_platform] **********************
2017-05-09T08:58:57.101304000Z ok: [localhost]
2017-05-09T08:58:57.101789000Z
2017-05-09T08:58:57.102222000Z TASK [common : Assuming default service manager] *******************************
2017-05-09T08:58:57.150914000Z ok: [localhost]
2017-05-09T08:58:57.153824000Z
2017-05-09T08:58:57.154255000Z TASK [common : Setting a default for enable_lb] ********************************
2017-05-09T08:58:57.204322000Z ok: [localhost]
2017-05-09T08:58:57.207005000Z
2017-05-09T08:58:57.207344000Z TASK [common : Setting a default for enable_controller] ************************
2017-05-09T08:58:57.255304000Z ok: [localhost]
2017-05-09T08:58:57.258900000Z
2017-05-09T08:58:57.259160000Z TASK [common : Setting a default for enable_analytics] *************************
2017-05-09T08:58:57.309634000Z ok: [localhost]
2017-05-09T08:58:57.309985000Z
2017-05-09T08:58:57.310258000Z TASK [common : Setting a fact no_upstart when deployment platform is docker] ***
2017-05-09T08:58:57.361433000Z ok: [localhost]
2017-05-09T08:58:57.361778000Z
2017-05-09T08:58:57.362079000Z TASK [common : Setting fact to override ansible_service_mgr for docker. This is to wrongly detect service manager as systemd when ubuntu container run on top of centos] ***
2017-05-09T08:58:57.408290000Z skipping: [localhost]
2017-05-09T08:58:57.408625000Z
2017-05-09T08:58:57.408845000Z TASK [common : Setting default for controller_components] **********************
2017-05-09T08:58:57.456331000Z ok: [localhost]
2017-05-09T08:58:57.461153000Z
2017-05-09T08:58:57.461414000Z TASK [common : update hosts file] **********************************************
2017-05-09T08:58:57.843877000Z ok: [localhost] => (item={'key': u'{{ ansible_hostname }}', 'value': u'127.0.0.1'})
2017-05-09T08:58:57.850007000Z
2017-05-09T08:58:57.850265000Z TASK [contrail/common : include OS family specific package dependencies] *******
2017-05-09T08:58:57.910300000Z included: /contrail-ansible-internal/playbooks/roles/contrail/common/tasks/install/Debian.yml for localhost
2017-05-09T08:58:57.918200000Z
2017-05-09T08:58:57.918487000Z TASK [contrail/common : Check if source SSL Cert directory present] ************
2017-05-09T08:58:58.102306000Z ok: [localhost]
2017-05-09T08:58:58.102710000Z
2017-05-09T08:58:58.102961000Z TASK [contrail/common : Create SSL directories on container] *******************
2017-05-09T08:58:58.156980000Z skipping: [localhost] => (item=private)
2017-05-09T08:58:58.157299000Z skipping: [localhost] => (item=certs)
2017-05-09T08:58:58.157542000Z
2017-05-09T08:58:58.157823000Z TASK [contrail/common : Copy SSL files for use by container] *******************
2017-05-09T08:58:58.206925000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/certs/ca-cert.pem', u'src': u'/etc/contrailctl/ssl/ca-cert.pem'})
2017-05-09T08:58:58.212228000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/certs/server.pem', u'src': u'/etc/contrailctl/ssl/server.pem'})
2017-05-09T08:58:58.228816000Z skipping: [localhost] => (item={u'dest': u'/etc/contrail/ssl/private/server-privkey.pem', u'src': u'/etc/contrailctl/ssl/server-privkey.pem'})
2017-05-09T08:58:58.229285000Z
2017-05-09T08:58:58.229640000Z TASK [contrail/kube_manager : include] *****************************************
2017-05-09T08:58:58.315464000Z included: /contrail-ansible-internal/playbooks/roles/contrail/kube_manager/tasks/install/Debian.yml for localhost
2017-05-09T08:58:58.332291000Z
2017-05-09T08:58:58.332691000Z TASK [contrail/common : Load OS distribution specific code] ********************
2017-05-09T08:58:58.382888000Z
2017-05-09T08:58:58.383330000Z TASK [contrail/kube_manager : Include contrail service manager specific code] **
2017-05-09T08:58:58.455434000Z included: /contrail-ansible-internal/playbooks/roles/contrail/kube_manager/tasks/install/supervisord.yml for localhost
2017-05-09T08:58:58.478224000Z
2017-05-09T08:58:58.478629000Z TASK [contrail/common : Make sure contrail log directory exists with appropriate ownership] ***
2017-05-09T08:58:58.802251000Z fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "gid": 0, "group": "root", "mode": "0755", "msg": "chown failed: failed to look up user contrail", "owner": "root", "path": "/var/log/contrail", "size": 6, "state": "directory", "uid": 0}
2017-05-09T08:58:58.804325000Z to retry, use: --limit @/contrail-ansible-internal/playbooks/contrail_kube_manager.retry
2017-05-09T08:58:58.804835000Z
2017-05-09T08:58:58.805225000Z PLAY RECAP *********************************************************************
2017-05-09T08:58:58.805591000Z localhost : ok=17 changed=1 unreachable=0 failed=1
2017-05-09T08:58:58.805954000Z
[root@testbed-1-vm1 contrailctl]#