[periodic/check][master] multinode fs016/019 & scenario-10 job failed while deploying overcloud with Permission denied: '/var/lib/mistral/overcloud/ceph-ansible

Bug #1815411 reported by chandan kumar on 2019-02-11
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Critical
Unassigned

Bug Description

Multinode fs016/f019 job in promotion pipeline and scenario-10 (fs038) in check jobs are failing while deploying overcloud with following errors:
2019-02-11 00:48:03 | There was an issue creating /var/lib/mistral/overcloud/ceph-ansible as requested: [Errno 13] Permission denied: '/var/lib/mistral/overcloud/ceph-ansible'
2019-02-11 00:48:03 | failed: [undercloud] (item=/var/lib/mistral/overcloud/ceph-ansible/host_vars) => {
2019-02-11 00:48:03 | "changed": false,
2019-02-11 00:48:03 | "item": "/var/lib/mistral/overcloud/ceph-ansible/host_vars",
2019-02-11 00:48:03 | "path": "/var/lib/mistral/overcloud/ceph-ansible/host_vars",
2019-02-11 00:48:03 | "state": "absent"
2019-02-11 00:48:03 | }
2019-02-11 00:48:03 |
2019-02-11 00:48:03 | MSG:
2019-02-11 00:48:03 |
2019-02-11 00:48:03 | There was an issue creating /var/lib/mistral/overcloud/ceph-ansible as requested: [Errno 13] Permission denied: '/var/lib/mistral/overcloud/ceph-ansible'
2019-02-11 00:48:03 | failed: [undercloud] (item=/var/lib/mistral/overcloud/ceph-ansible/fetch_dir) => {
2019-02-11 00:48:03 | "changed": false,
2019-02-11 00:48:03 | "item": "/var/lib/mistral/overcloud/ceph-ansible/fetch_dir",
2019-02-11 00:48:03 | "path": "/var/lib/mistral/overcloud/ceph-ansible/fetch_dir",
2019-02-11 00:48:03 | "state": "absent"
2019-02-11 00:48:03 | }
2019-02-11 00:48:03 |
2019-02-11 00:48:03 | MSG:
2019-02-11 00:48:03 |
2019-02-11 00:48:03 | There was an issue creating /var/lib/mistral/overcloud/ceph-ansible as requested: [Errno 13] Permission denied: '/var/lib/mistral/overcloud/ceph-ansible'

Below is the logs:
fs016: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/e511b59/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-11_00_48_03

fs019: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/7f72339/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-11_00_36_57

scenario10:
http://logs.openstack.org/98/604298/224/check/tripleo-ci-centos-7-scenario010-multinode-oooq-container/c19dd2f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-02-11_01_46_49

It is coming only where multinode ceph is getting tested.

Reason: ceph-ansible is now invoked on the undercloud host with user tripleo-admin, so either this user will need write access to /var/lib/mistral/overcloud/ceph-ansible, or the working directory needs to be moved to ~tripleo-admin/overcloud, or some of those
external_deploy_tasks will need to be become:true

Earlier it was running as the mistral user inside the mistral-executor container, which was really not ideal. mistral user is the owner of that directory, so it was working earlier.
It needs to be fixed.

https://review.openstack.org/#/c/636086/ might fix the issue.
Changes made to address this bug https://bugs.launchpad.net/tripleo/+bug/1813832 leads to this error

description: updated
description: updated
description: updated
description: updated
summary: - [periodic/check][master] multinode fs016/019 job failed while deploying
- overcloud with Permission denied: '/var/lib/mistral/overcloud/ceph-
- ansible
+ [periodic/check][master] multinode fs016/019 & scenario-10 job failed
+ while deploying overcloud with Permission denied:
+ '/var/lib/mistral/overcloud/ceph-ansible
Changed in tripleo:
status: Triaged → In Progress
Quique Llorente (quiquell) wrote :

fs016 and fs019 are fixed now

wes hayutin (weshayutin) wrote :
Changed in tripleo:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers