[master][promotion] centos 7 container build job deploying undercloud with usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory

Bug #1814851 reported by chandan kumar on 2019-02-06
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
Quique Llorente

Bug Description

Periodic master promotion centos7 container build job got failed while undercloud at step 5:
https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/f56417e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2019-02-06_05_58_14

Error running ['podman', 'run', '--name', 'nova_cellv2_discover_hosts', '--label', 'config_id=tripleo_step5', '--label', 'container_name=nova_cellv2_discover_hosts', '--label', 'managed_by=paunch', '--label', 'config_data={\"start_order\": 1, \"command\": \"/docker-config-scripts/pyshim.sh /docker-config-scripts/nova_cell_v2_discover_host.py\", \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/nova/etc/my.cnf.d/:/etc/my.cnf.d/:ro\", \"/var/lib/config-data/nova/etc/nova/:/etc/nova/:ro\", \"/var/log/containers/nova:/var/log/nova\", \"/var/lib/docker-config-scripts/:/docker-config-scripts/\"], \"image\": \"192.168.24.1:8787/tripleomaster/centos-binary-nova-compute-ironic:tripleo-ci-testing-updated-20190206052058\", \"detach\": false, \"net\": \"host\"}', '--net=host', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/lib/config-data/nova/etc/my.cnf.d/:/etc/my.cnf.d/:ro', '--volume=/var/lib/config-data/nova/etc/nova/:/etc/nova/:ro', '--volume=/var/log/containers/nova:/var/log/nova', '--volume=/var/lib/docker-config-scripts/:/docker-config-scripts/', '192.168.24.1:8787/tripleomaster/centos-binary-nova-compute-ironic:tripleo-ci-testing-updated-20190206052058', '/docker-config-scripts/pyshim.sh', '/docker-config-scripts/nova_cell_v2_discover_host.py']. [1]",
2019-02-06 05:58:14 | "stdout: (cellv2) Waiting for service to register",
2019-02-06 05:58:14 | "(cellv2) Waiting for service to register",

While checking the rabbitmq podman container logs, we found that:
https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/f56417e/logs/undercloud/var/log/extra/podman/containers/rabbitmq/stdout.log.txt.gz

Running command: '/usr/lib/rabbitmq/bin/rabbitmq-server'
+ exec /usr/lib/rabbitmq/bin/rabbitmq-server
/usr/local/bin/kolla_start: line 18: /usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory

https://review.openstack.org/#/c/634405 adds the workaround for the rabbitmq-server might be causing the issue:

Changed in tripleo:
assignee: nobody → Quique Llorente (quiquell)

Fix proposed to branch: master
Review: https://review.openstack.org/635108

Changed in tripleo:
status: Triaged → In Progress

Reviewed: https://review.openstack.org/635108
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=72041379dbabd08f2f56895df165760d25d980dd
Submitter: Zuul
Branch: master

commit 72041379dbabd08f2f56895df165760d25d980dd
Author: Quique Llorente <email address hidden>
Date: Wed Feb 6 08:38:59 2019 +0100

    Append rabbitmq_packages_append contents

    The review [1] override the fix later on the jinja template, this review
    replace the override by an append, and also check if there is nothing to
    append and add an empty list.

    Closes-Bug: #1814851

    [1] https://review.openstack.org/#/c/634405

    Change-Id: I35cbde7e3d44ce071a962cea278680bd5e453ec2

Changed in tripleo:
status: In Progress → Fix Released

This issue was fixed in the openstack/tripleo-common 10.4.0 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers