openvswitch is installed by tripleo but ovn-controller is failing

Bug #1921097 reported by Kamil Sambor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Undecided
Kamil Sambor

Bug Description

Description of problem:
openvswitch is installed by tripleo but not started on a controller containing just neutron_api which leads to failure during the OVS->OVN migration.
Attempt migration from OVS->OVN

Actual results:
ovn_controller container failing to start on non existent /run/openvswitch directory:

2021-03-17 14:05:47,314 INFO: 35043 -- Image already exists: director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-neutron-server-ovn:16.1.3\n2021-03-17 14:05:50,411 ERROR: 35042 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-ovn_controller', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', 'NAME=ovn_controller', '--env', 'HOSTNAME=ctrl-net-c-04', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpi9ii6nqu:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--volume', '/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-ovn-controller:16.1.3'] run failed after Error: error checking path \"/run/openvswitch\": stat /run/openvswitch: no such file or directory\n attempt(s): 1\n2021-03-17 14:05:50,412 WARNING: 35042 -- Retrying running container: ovn_controller\n2021-03-17 14:05:53,559 ERROR: 35042 -- ['/usr/bin/podman', 'run', '--user', '0', '--name', 'container-puppet-ovn_controller', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,vs_config,exec', '--env', 'NAME=ovn_controller', '--env', 'HOSTNAME=ctrl-net-c-04', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--env', 'DEBUG=False', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpi9ii6nqu:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro', '--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro', '--volume', '/dev/log:/dev/log:rw', '--rm', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-ovn_controller.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--volume', '/etc/sysconfig/modules:/etc/sysconfig/modules', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', 'director-c.infra.prod.upshift.rdu2.redhat.com:8787/rhosp-rhel8/openstack-ovn-controller:16.1.3'] run failed after Error: error checking path \"/run/openvswitch\": stat /run/openvswitch: no such file or directory\n attempt(s): 2

Expected results:
OVN migration succeeds and openvswitch is started out of the box

Kamil Sambor (ksambor)
Changed in tripleo:
assignee: nobody → Kamil Sambor (ksambor)
summary: - openvswitch is installed by tripleo but ovn-controller is failingr
+ openvswitch is installed by tripleo but ovn-controller is failing
Kamil Sambor (ksambor)
Changed in tripleo:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786968

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/ussuri)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786968
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/48c44479616b5a2a971b27c5b4d196397d8ea58d
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 48c44479616b5a2a971b27c5b4d196397d8ea58d
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/ussuri)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786969
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/d81c5544d24b6567f1615f6065d340278f317604
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit d81c5544d24b6567f1615f6065d340278f317604
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

tags: added: in-stable-ussuri
tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/786970
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/c9fa94dd52cc20855586fb13e3ba51c2c2662f06
Submitter: "Zuul (22348)"
Branch: stable/train

commit c9fa94dd52cc20855586fb13e3ba51c2c2662f06
Author: Kamil Sambor <email address hidden>
Date: Wed Mar 24 11:46:35 2021 +0100

    Add systemd dependency to openvswitch to ovn-controller

    Ovn controller need openvswitch to be running properly. This
    patch adds the required depends_on so paunch can add the dependency to
    the systemd service file.

    Closes-bug: #1921097
    Change-Id: I2e54771f0a01d22ce95530bef146ea368189db24
    (cherry picked from commit 51059676b6b6d73648a6ffb3c6364a6f3b5ed42f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 14.1.0

This issue was fixed in the openstack/tripleo-heat-templates 14.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 13.3.0

This issue was fixed in the openstack/tripleo-heat-templates 13.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.6.0

This issue was fixed in the openstack/tripleo-heat-templates 11.6.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 12.4.4

This issue was fixed in the openstack/tripleo-heat-templates 12.4.4 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers