bluebox gate generates invalid password in keepalived logs

[Steven Dake]

bluebox gate blocks at waiting for VIP to become available:
http://logs.openstack.org/40/299540/1/check/gate-kolla-dsvm-deploy-centos-source/6c69e15/console.html#_2016-04-11_09_06_37_454

This is presumably because keepalived cannot create the VIP. There is an invalid password error here:
http://logs.openstack.org/40/299540/1/check/gate-kolla-dsvm-deploy-centos-source/6c69e15/logs/kolla/keepalived/keepalived.log

This only happens on bluebox gates.

[Steven Dake (sdake)]

rechecks are suitable until 4/15 release, at which point we should elevate the priority of this gate bug.

[Hui Kang (huikang27)]

not sure if this patch is relevant: https://review.openstack.org/294291

- Hui

[Jeffrey Zhang (jeffrey4l)]

Hui,

No, see the keepalived log file

```
<6>2016-04-11T09:06:54Z node1 Keepalived_vrrp[22]: VRRP_Instance(kolla_internal_vip_51) ignoring received advertisment...
<6>2016-04-11T09:06:55Z node1 Keepalived_vrrp[22]: receive an invalid passwd!
<6>2016-04-11T09:06:55Z node1 Keepalived_vrrp[22]: bogus VRRP packet received on eth0 !!!
<6>2016-04-11T09:06:55Z node1 Keepalived_vrrp[22]: VRRP_Instance(kolla_internal_vip_51) ignoring received advertisment...
<6>2016-04-11T09:06:56Z node1 Keepalived_vrrp[22]: receive an invalid passwd!
<6>2016-04-11T09:06:56Z node1 Keepalived_vrrp[22]: bogus VRRP packet received on eth0 !!!
<6>2016-04-11T09:06:56Z node1 Keepalived_vrrp[22]: VRRP_Instance(kolla_internal_vip_51) ignoring received advertisment...
<6>2016-04-11T09:06:57Z node1 Keepalived_vrrp[22]: receive an invalid passwd!
<6>2016-04-11T09:06:57Z node1 Keepalived_vrrp[22]: bogus VRRP packet received on eth0 !!!
<6>2016-04-11T09:06:57Z node1 Keepalived_vrrp[22]: VRRP_Instance(kolla_internal_vip_51) ignoring received advertisment...
<6>2016-04-11T09:06:58Z node1 Keepalived_vrrp[22]: receive an invalid passwd!
```

if is caused by the another keepavlied instance exist with different auth.

[Steven Dake (sdake)]

note openstack-infra said their firewall rules would prevent leaking of multicast packets. Worth verifying if that is the case and reporting back here.

[Jeffrey Zhang (jeffrey4l)]

https://review.openstack.org/304080

Fix is here

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/304080
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=f93ed9a935c9e9b159cb42c548f4d6d3383f36f2
Submitter: Jenkins
Branch: master

commit f93ed9a935c9e9b159cb42c548f4d6d3383f36f2
Author: Jeffrey Zhang <[e-mail]>
Date: Mon Apr 11 20:11:33 2016 +0800

    Use a random virtual_router_id in the keepalived

    Use a random virtual_router_id in case there are multi keepalived
    instances exist at the same time in the gate.

    Closes-Bug: #1568818
    Change-Id: Iebb3dec088a21820e641b63bb2a5f40586d1df8c

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/kolla 3.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/357997

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/357998

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/357998
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c600eb6ee994ac66fdaa94332e167866b599e271
Submitter: Jenkins
Branch: stable/liberty

commit c600eb6ee994ac66fdaa94332e167866b599e271
Author: Jeffrey Zhang <[e-mail]>
Date: Mon Apr 11 20:11:33 2016 +0800

    Use a random virtual_router_id in the keepalived

    Use a random virtual_router_id in case there are multi keepalived
    instances exist at the same time in the gate.

    Closes-Bug: #1568818
    Change-Id: Iebb3dec088a21820e641b63bb2a5f40586d1df8c
    (cherry picked from commit f93ed9a935c9e9b159cb42c548f4d6d3383f36f2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/357997
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=c07c8b4cfada3345ab33d638e202b9272fa8f5a6
Submitter: Jenkins
Branch: stable/mitaka

commit c07c8b4cfada3345ab33d638e202b9272fa8f5a6
Author: Jeffrey Zhang <[e-mail]>
Date: Mon Apr 11 20:11:33 2016 +0800

    Use a random virtual_router_id in the keepalived

    Use a random virtual_router_id in case there are multi keepalived
    instances exist at the same time in the gate.

    Closes-Bug: #1568818
    Change-Id: Iebb3dec088a21820e641b63bb2a5f40586d1df8c
    (cherry picked from commit f93ed9a935c9e9b159cb42c548f4d6d3383f36f2)

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/359450

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kolla 2.0.3 release.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christian Berendt (<email address hidden>) on branch: master
Review: https://review.openstack.org/359450