[update] Fip test may start before the fip is ready and return false positive

Bug #1895829 reported by Sofer Athlan-Guyot
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Sofer Athlan-Guyot

Bug Description

Hi,

first noticed there https://bugzilla.redhat.com/show_bug.cgi?id=1878688

So we started to see <10seconds packets loss during train update. Looking closer to the ping log, we saw that we were missing the first packets.

It seems we start the ping test before the vm is ready. Now that we have 0% tolerance for ping loss during update this kind of false positive appears.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-upgrade (master)

Fix proposed to branch: master
Review: https://review.opendev.org/752232

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-upgrade (master)

Reviewed: https://review.opendev.org/752232
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=306aa3b1996daddd2a81f6045ebbe47550bbc128
Submitter: Zuul
Branch: master

commit 306aa3b1996daddd2a81f6045ebbe47550bbc128
Author: Sofer Athlan-Guyot <email address hidden>
Date: Wed Sep 16 14:33:35 2020 +0200

    Block on fip creation before starting the ping test.

    We wait for the fip to be reachable during ~1minutes before starting
    the ping test. We fail if it's not reachable in that time frame.

    Change-Id: I20cecd06834ef424599d9e281f0d6a6434f584a5
    Closes-Bug: #1895829

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-upgrade (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752639

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-upgrade (stable/ussuri)

Reviewed: https://review.opendev.org/752639
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=f9cc51478bfbcde96a81917754d320f2ba8ca03f
Submitter: Zuul
Branch: stable/ussuri

commit f9cc51478bfbcde96a81917754d320f2ba8ca03f
Author: Sofer Athlan-Guyot <email address hidden>
Date: Wed Sep 16 14:33:35 2020 +0200

    Block on fip creation before starting the ping test.

    We wait for the fip to be reachable during ~1minutes before starting
    the ping test. We fail if it's not reachable in that time frame.

    Change-Id: I20cecd06834ef424599d9e281f0d6a6434f584a5
    Closes-Bug: #1895829
    (cherry picked from commit 306aa3b1996daddd2a81f6045ebbe47550bbc128)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-upgrade (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/752663

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-upgrade (stable/train)

Reviewed: https://review.opendev.org/752663
Committed: https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=b7edf3112ee545b276db76417711b418d6d4e448
Submitter: Zuul
Branch: stable/train

commit b7edf3112ee545b276db76417711b418d6d4e448
Author: Sofer Athlan-Guyot <email address hidden>
Date: Wed Sep 16 14:33:35 2020 +0200

    Block on fip creation before starting the ping test.

    We wait for the fip to be reachable during ~1minutes before starting
    the ping test. We fail if it's not reachable in that time frame.

    Change-Id: I20cecd06834ef424599d9e281f0d6a6434f584a5
    Closes-Bug: #1895829
    (cherry picked from commit 306aa3b1996daddd2a81f6045ebbe47550bbc128)
    (cherry picked from commit f9cc51478bfbcde96a81917754d320f2ba8ca03f)

tags: added: in-stable-train
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.