kolla-ansible CI: tacker-conductor dies always (Train only)

Bug #1845142 reported by Radosław Piliszek
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla-ansible
Fix Released
High
Radosław Piliszek
Train
Fix Released
High
Radosław Piliszek
tacker
Invalid
Undecided
Unassigned

Bug Description

Since 2019-09-22 tacker-conductor always fails NFV jobs:
https://zuul.opendev.org/t/openstack/builds?job_name=kolla-ansible-centos-source-scenario-nfv&branch=master

Note: there is no ubuntu variant of this job.

Revision history for this message
Radosław Piliszek (yoctozepto) wrote : Re: CI: tacker-conductor dies always (Train only)

Two errors in logs:

2019-09-24 02:44:32.275 7 ERROR glance_store._drivers.filesystem [-] Unable to create datadir: /var/lib/glance/images: OSError: [Errno 13] Permission denied: '/var/lib/glance'

2019-09-24 02:44:32.277 7 ERROR tacker.conductor.conductor_server [-] Config option 'vnf_package_csar_path' is not configured correctly. VNF package CSAR path directory /var/lib/tacker/vnfpackages/ doesn't exist

summary: - CI: tacker-conductor dies always
+ CI: tacker-conductor dies always (Train only)
Revision history for this message
Radosław Piliszek (yoctozepto) wrote :

These upstream changes seem related (merged during that time and named with 'CSAR'): https://review.opendev.org/#/q/topic:bp/tosca-csar-mgmt-driver+status:merged+project:openstack/tacker

No relevant changes in kolla-ansible.

summary: - CI: tacker-conductor dies always (Train only)
+ kolla-ansible CI: tacker-conductor dies always (Train only)
Revision history for this message
Radosław Piliszek (yoctozepto) wrote :
Mark Goddard (mgoddard)
Changed in tacker:
status: New → Confirmed
Revision history for this message
Radosław Piliszek (yoctozepto) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.opendev.org/684275

Changed in kolla-ansible:
assignee: nobody → Eduardo Gonzalez (egonzalez90)
status: Triaged → In Progress
Changed in kolla-ansible:
assignee: Eduardo Gonzalez (egonzalez90) → Radosław Piliszek (yoctozepto)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/684275
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=b96ade3cf01009d822f85744efee523127f2674c
Submitter: Zuul
Branch: master

commit b96ade3cf01009d822f85744efee523127f2674c
Author: Eduardo Gonzalez <email address hidden>
Date: Tue Sep 24 12:57:58 2019 +0200

    [train] Add required Tacker Conductor config and docs

    Tacker requires config for storing CSAR vnf packages.
    This patch adds it as well as relevant docs.
    Only one Tacker Conductor is deployed by default due to
    lack of a shared filesystem.

    Change-Id: Iad391f35105e79fa9319502256528990915df9b7
    Co-authored-by: Radosław Piliszek <email address hidden>
    Closes-Bug: #1845142

Changed in kolla-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 9.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 9.0.0.0rc1 release candidate.

Revision history for this message
Radosław Piliszek (yoctozepto) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to kolla-ansible (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/695804

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (master)

Reviewed: https://review.opendev.org/695804
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=c5f9ea959092019aa44eb96224fd365b8025bba6
Submitter: Zuul
Branch: master

commit c5f9ea959092019aa44eb96224fd365b8025bba6
Author: Radosław Piliszek <email address hidden>
Date: Sat Nov 23 21:46:11 2019 +0100

    [Train] Fix Tacker config for the local Glance store

    It turned out the previous fix ([1]) was incomplete.
    Additionally, it seems we have to limit Tacker server
    to one instance co-located with conductor.

    [1] https://review.opendev.org/684275
    commit b96ade3cf01009d822f85744efee523127f2674c

    Change-Id: I9ce27d5f68f32ef59e245960e23336ae5c5db905
    Closes-bug: #1853715
    Related-bug: #1845142

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to kolla-ansible (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/697048

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to kolla-ansible (stable/train)

Reviewed: https://review.opendev.org/697048
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=c7458b006fe1dc0ccba9460c7090a50a6ac4d3d0
Submitter: Zuul
Branch: stable/train

commit c7458b006fe1dc0ccba9460c7090a50a6ac4d3d0
Author: Radosław Piliszek <email address hidden>
Date: Sat Nov 23 21:46:11 2019 +0100

    [Train] Fix Tacker config for the local Glance store

    It turned out the previous fix ([1]) was incomplete.
    Additionally, it seems we have to limit Tacker server
    to one instance co-located with conductor.

    [1] https://review.opendev.org/684275
    commit b96ade3cf01009d822f85744efee523127f2674c

    Change-Id: I9ce27d5f68f32ef59e245960e23336ae5c5db905
    Closes-bug: #1853715
    Related-bug: #1845142
    (cherry picked from commit c5f9ea959092019aa44eb96224fd365b8025bba6)

tags: added: in-stable-train
Hiroyuki Jo (joxyuki)
Changed in tacker:
status: Confirmed → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers