| 2015-05-12 21:34:32 |
amit surana |
bug |
|
|
added bug |
| 2015-05-12 21:34:46 |
amit surana |
nominated for series |
|
juniperopenstack/r2.20 |
|
| 2015-05-12 21:34:46 |
amit surana |
bug task added |
|
juniperopenstack/r2.20 |
|
| 2015-05-12 21:34:46 |
amit surana |
nominated for series |
|
juniperopenstack/trunk |
|
| 2015-05-12 21:34:46 |
amit surana |
bug task added |
|
juniperopenstack/trunk |
|
| 2015-05-12 21:34:55 |
amit surana |
juniperopenstack/r2.20: importance |
Undecided |
High |
|
| 2015-05-12 21:35:06 |
amit surana |
juniperopenstack/r2.20: milestone |
|
r2.20-fcs |
|
| 2015-05-12 21:35:16 |
amit surana |
juniperopenstack/trunk: milestone |
r2.20-fcs |
r2.30-fcs |
|
| 2015-05-12 21:39:44 |
amit surana |
description |
Consider a case where the controller/compute node is connected to the IP fabric via a 2 interface bond (LAG). The bond assumes the mac address of one of the slave interfaces (usually the first one that comes up).
On the compute, the vhost0 interface also gets assigned the same MAC address as that of the bond interface. This MAC address is also used by keepalived/haproxy as the MAC address corresponding to the VIP.
Now, if the bond interface flaps (networking service was restarted for instance) and the MAC address of the bond interface changes to that of a different slave interface, it is seen that vhost0 interface still points to the old MAC and this breaks all the connectivity to the compute. As far as the KA/HAproxy is concerned, though the services are running, the VIP isn't owned by any of the nodes, and so this functionality also breaks.
The above scenario was simulated on the solution testbed by having the fab script add a static route to all the nodes on a working cluster. After adding the static route the fab add_static_route script restarts the networking service; this flaps the bond interface and caused the MAC address of the bond interface to change, which in turn lead to the above noted issues.
If vrouter/keepalive services are restarted, the issue is resolved - which points towards a potential solution.
bond0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
em1 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
em2 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
vhost0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
==restart networking==
bond0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
em1 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
em2 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
vhost0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a |
R2.20 build 13 Ubuntu 14.0.4.
Flapping a bond interface can cause its MAC address to change. Other
services that rely on this MAC address, should also be restarted so
that they can attach to the new MAC address. Details below.
Consider a case where the controller/compute node is connected to the IP fabric via a 2 interface bond (LAG). The bond assumes the mac address of one of the slave interfaces (usually the first one that comes up).
On the compute, the vhost0 interface also gets assigned the same MAC address as that of the bond interface. This MAC address is also used by keepalived/haproxy as the MAC address corresponding to the VIP.
Now, if the bond interface flaps (networking service was restarted for instance) and the MAC address of the bond interface changes to that of a different slave interface, it is seen that vhost0 interface still points to the old MAC and this breaks all the connectivity to the compute. As far as the KA/HAproxy is concerned, though the services are running, the VIP isn't owned by any of the nodes, and so this functionality also breaks.
The above scenario was simulated on the solution testbed by having the fab script add a static route to all the nodes on a working cluster. After adding the static route the fab add_static_route script restarts the networking service; this flaps the bond interface and causes the MAC address of the bond interface to change, which in turn leads to the above noted issues.
If vrouter/keepalive services are restarted, the issue is resolved.
bond0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
em1 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
em2 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
vhost0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a
==restart networking==
bond0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
em1 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
em2 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8b
vhost0 Link encap:Ethernet HWaddr 08:9e:01:d9:27:8a |
|
| 2015-05-13 00:34:51 |
Ashish Ranjan |
juniperopenstack/trunk: assignee |
|
Hari Prasad Killi (haripk) |
|
| 2015-05-13 00:35:01 |
Ashish Ranjan |
juniperopenstack/r2.20: assignee |
|
Hari Prasad Killi (haripk) |
|
| 2015-05-13 00:35:04 |
Ashish Ranjan |
information type |
Proprietary |
Public |
|
| 2015-05-15 05:27:23 |
Ashish Ranjan |
tags |
|
vrouter |
|
| 2015-05-15 05:29:15 |
Nagabhushana R |
tags |
vrouter |
provisioning |
|
| 2015-06-09 21:28:26 |
Ignatious Johnson Christopher |
bug |
|
|
added subscriber Senthilnathan Murugappan |
| 2015-06-18 03:38:21 |
Nagabhushana R |
juniperopenstack/r2.20: assignee |
Hari Prasad Killi (haripk) |
Ignatious Johnson Christopher (ijohnson-x) |
|
| 2015-06-18 03:38:31 |
Nagabhushana R |
juniperopenstack/trunk: assignee |
Hari Prasad Killi (haripk) |
Ignatious Johnson Christopher (ijohnson-x) |
|
| 2015-06-18 23:15:12 |
OpenContrail Admin |
tags |
provisioning |
blocker provisioning |
|
| 2015-06-19 07:06:29 |
Vinay Mahuli |
juniperopenstack/r2.20: status |
New |
In Progress |
|
| 2015-06-19 07:09:19 |
Vinay Mahuli |
juniperopenstack/trunk: status |
New |
In Progress |
|
| 2015-06-19 07:45:56 |
OpenContrail Admin |
juniperopenstack/r2.20: status |
In Progress |
Fix Committed |
|
| 2015-06-19 22:10:39 |
OpenContrail Admin |
juniperopenstack/trunk: status |
In Progress |
Fix Committed |
|
| 2015-06-19 23:54:18 |
Vinay Mahuli |
juniperopenstack/trunk: status |
Fix Committed |
In Progress |
|
| 2015-06-20 00:06:42 |
Vinay Mahuli |
juniperopenstack/r2.20: status |
Fix Committed |
In Progress |
|
| 2015-06-20 03:15:05 |
OpenContrail Admin |
juniperopenstack/trunk: status |
In Progress |
Fix Committed |
|
| 2015-06-20 05:08:34 |
OpenContrail Admin |
juniperopenstack/r2.20: status |
In Progress |
Fix Committed |
|
