tripleo

  1. Bug #1895822
  2. Comment #16

Comment 16 for bug 1895822

Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

If we look at job closely it fails on https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_resources.yml#L283-L291

If we look at interfaces before upgrade they will be as

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:ef:f4:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.23/24 brd 192.168.122.255 scope global dynamic noprefixroute ens3
       valid_lft 3147sec preferred_lft 3147sec
    inet6 fe80::5054:ff:feef:f4eb/64 scope link
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:25:a0:f8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.229/24 brd 192.168.122.255 scope global dynamic noprefixroute ens4
       valid_lft 3271sec preferred_lft 3271sec
    inet6 fe80::5054:ff:fe25:a0f8/64 scope link
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether c6:c0:26:55:35:d3 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 02:85:da:68:e5:4e brd ff:ff:ff:ff:ff:ff
    inet 192.168.24.2/24 scope global br-ex
       valid_lft forever preferred_lft forever
    inet6 fe80::85:daff:fe68:e54e/64 scope link
       valid_lft forever preferred_lft forever
6: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 02:85:da:68:e5:4e brd ff:ff:ff:ff:ff:ff
    inet 192.168.24.1/24 brd 192.168.24.255 scope global br-ctlplane
       valid_lft forever preferred_lft forever
    inet 192.168.24.3/32 brd 192.168.24.255 scope global br-ctlplane
       valid_lft forever preferred_lft forever
    inet6 fe80::85:daff:fe68:e54e/64 scope link
       valid_lft forever preferred_lft forever
7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:6f:46:31:8f:42 brd ff:ff:ff:ff:ff:ff

After upgrade run they will be as

: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:ef:f4:eb brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.23/24 brd 192.168.122.255 scope global dynamic noprefixroute ens3
       valid_lft 2369sec preferred_lft 2369sec
    inet6 fe80::5054:ff:feef:f4eb/64 scope link
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:25:a0:f8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.229/24 brd 192.168.122.255 scope global dynamic noprefixroute ens4
       valid_lft 2358sec preferred_lft 2358sec
    inet6 fe80::5054:ff:fe25:a0f8/64 scope link
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether c6:c0:26:55:35:d3 brd ff:ff:ff:ff:ff:ff
5: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 02:85:da:68:e5:4e brd ff:ff:ff:ff:ff:ff
    inet6 fe80::85:daff:fe68:e54e/64 scope link
       valid_lft forever preferred_lft forever
7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether fa:6f:46:31:8f:42 brd ff:ff:ff:ff:ff:ff
46: br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 02:85:da:68:e5:4e brd ff:ff:ff:ff:ff:ff
    inet 192.168.24.1/24 brd 192.168.24.255 scope global br-ctlplane
       valid_lft forever preferred_lft forever
    inet 192.168.24.1/32 brd 192.168.24.1 scope global br-ctlplane
       valid_lft forever preferred_lft forever
    inet 192.168.24.3/32 brd 192.168.24.255 scope global br-ctlplane
       valid_lft forever preferred_lft forever
    inet6 fe80::85:daff:fe68:e54e/64 scope link
       valid_lft forever preferred_lft forever

So ips on interfaces are different. So, when I made interfaces in the same way as they were before upgrade ping test passed.

If we look at deployment script it looks like

tripleo_deploy.sh

---%<---
#!/bin/bash
# This file is managed by ansible
set -xeo pipefail

export DEPLOY_CONTROL_VIP=192.168.24.3
export DEPLOY_DEPLOYMENT_USER=zuul
export DEPLOY_LOCAL_IP=192.168.24.1/24
export DEPLOY_OUTPUT_DIR=/home/zuul
export DEPLOY_ROLES_FILE=/usr/share/openstack-tripleo-heat-templates/roles/Standalone.yaml
export DEPLOY_STACK=standalone
export DEPLOY_STANDALONE_ROLE=Standalone
export DEPLOY_TEMPLATES=/usr/share/openstack-tripleo-heat-templates
export DEPLOY_TIMEOUT_ARG=90
openstack tripleo deploy \
   --templates $DEPLOY_TEMPLATES \
   --standalone \
   --yes \
   --output-dir $DEPLOY_OUTPUT_DIR \
   --stack $DEPLOY_STACK \
   --standalone-role $DEPLOY_STANDALONE_ROLE \
   --timeout $DEPLOY_TIMEOUT_ARG \
   -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml \
   -e /home/zuul/containers-prepare-parameters.yaml \
   -e /home/zuul/standalone_parameters.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
   -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \
   -r $DEPLOY_ROLES_FILE \
   --deployment-user $DEPLOY_DEPLOYMENT_USER \
   --local-ip $DEPLOY_LOCAL_IP \
   --control-virtual-ip $DEPLOY_CONTROL_VIP > /home/zuul/standalone_deploy.log 2>&1
---%<---

which comes from https://github.com/openstack/tripleo-operator-ansible/blob/master/plugins/modules/tripleo_shell_script.py

However, if we look standalone-upgrade.sh it looks like

PROMPT_ANSWER=""
if openstack tripleo upgrade --help | grep -qe "--yes"; then
    PROMPT_ANSWER="--yes"
fi

sudo yum update -y \
    python-tripleoclient \
    openstack-tripleo-common \
    openstack-tripleo-heat-templates

sudo openstack tripleo upgrade $PROMPT_ANSWER \
  --templates \
  --local-ip=192.168.24.1/24 \
  --control-virtual-ip=192.168.24.3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml \
  -r /usr/share/openstack-tripleo-heat-templates/roles/Standalone.yaml \
  -e "/home/zuul/containers-prepare-parameters-upgrade.yaml" \
  -e "/home/zuul/standalone_parameters_upgrade.yaml" \
  -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \
  --output-dir /home/zuul \
  --standalone

and it comes from quickstart-extras https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/standalone-upgrade/tasks/main.yml#L27-L38
and generates the following upgrade script

 cat standalone-upgrade.sh
PROMPT_ANSWER=""
if openstack tripleo upgrade --help | grep -qe "--yes"; then
    PROMPT_ANSWER="--yes"
fi

sudo yum update -y \
    python-tripleoclient \
    openstack-tripleo-common \
    openstack-tripleo-heat-templates

sudo openstack tripleo upgrade $PROMPT_ANSWER \
  --templates \
  --local-ip=192.168.24.1/24 \
  --control-virtual-ip=192.168.24.3 \
  -e /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-tripleo.yaml \
  -r /usr/share/openstack-tripleo-heat-templates/roles/Standalone.yaml \
  -e "/home/zuul/containers-prepare-parameters-upgrade.yaml" \
  -e "/home/zuul/standalone_parameters_upgrade.yaml" \
  -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \
  --output-dir /home/zuul \
  --standalone

So, as we can see deployment and upgrade scripts are different. So, as experiment I made the upgrade script from deployment just sed -i -e 's/openstack\ tripleo\ deploy/openstack\ tripleo\ upgrade/' tripleo_deploy.sh

and ran it the upgrade passed, the interfaces were before and after upgrade were identical and ping test passed.