[DVR][L3 HA] Unable to ping 8.8.8.8 from VM without floating ip

Bug #1582739 reported by Ann Taraday
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Brian Haley

Bug Description

On the environment with L3 HA and DVR enabled all pings from VM without floating ip to 8.8.8.8 were lost.

This happened because router_centralized_snat port was binded to the wrong host were l3 agent was in standby state.

root@node-4:~# neutron l3-agent-list-hosting-router a1de4263-08af-48cb-a9b5-400ebcd3ac1a
+--------------------------------------+-------------------+----------------+-------+----------+
| id | host | admin_state_up | alive | ha_state |
+--------------------------------------+-------------------+----------------+-------+----------+
| ae474016-48ee-4121-88b7-71b65f2a4244 | node-4.domain.tld | True | :-) | standby |
| b93959c3-51da-45ae-8af7-aa90671953b4 | node-5.domain.tld | True | :-) | active |
| ca082545-5c1a-4eb5-882b-b35bf76f9350 | node-2.domain.tld | True | :-) | standby |
+--------------------------------------+-------------------+----------------+-------+----------+
root@node-4:~# neutron port-show cd1e7af6-0aa7-444b-b0b3-37d04325655f
+-----------------------+-----------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | node-2.domain.tld |
| binding:profile | {} |
| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": true} |
| binding:vif_type | ovs |
| binding:vnic_type | normal |
| created_at | 2016-05-17T08:55:58 |
| description | |
| device_id | a1de4263-08af-48cb-a9b5-400ebcd3ac1a |
| device_owner | network:router_centralized_snat |
| dns_name | |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "fe4e6901-f5ce-41b1-a7ce-582d70fe310e", "ip_address": "10.100.0.4"} |
| id | cd1e7af6-0aa7-444b-b0b3-37d04325655f |
| mac_address | fa:16:3e:f4:1d:6a |
| name | |
| network_id | 7e9e27d7-d331-4b09-8b28-04a0d5173af7 |
| port_security_enabled | False |
| security_groups | |
| status | DOWN |
| tenant_id | |
| updated_at | 2016-05-17T11:12:50 |
+-----------------------+-----------------------------------------------------------------------------------+

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/317541

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Undecided → Medium
tags: added: mitaka-backport-potential
Changed in neutron:
assignee: Ann Kamyshnikova (akamyshnikova) → Oleg Bondarev (obondarev)
Changed in neutron:
assignee: Oleg Bondarev (obondarev) → Ann Kamyshnikova (akamyshnikova)
Changed in neutron:
assignee: Ann Kamyshnikova (akamyshnikova) → Brian Haley (brian-haley)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/317541
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=899b6cb0652ee54c08d130ccd7edc3091b7d930b
Submitter: Jenkins
Branch: master

commit 899b6cb0652ee54c08d130ccd7edc3091b7d930b
Author: Ann Kamyshnikova <email address hidden>
Date: Tue May 17 17:25:00 2016 +0300

    Pass ha_router_port flag for _snat_router_interfaces ports

    Currently, router_centralized_snat port can be bound to a host were
    l3-agent is in standby state (L3 HA + DVR case). As a result VM without
    floating ip is unable to reach external network. This change passes
    ha_router_port flag to _ensure_host_set_on_port when called for
    _snat_router_interfaces ports.

    Note: this issue is intermittent, without changes in l3_rpc.py
    unit test does not fail every time.

    Co-Authored-By: Oleg Bondarev <email address hidden>

    Closes-bug: #1582739
    Change-Id: I74bad578361ed7eac8cc6c740b06b66ab1530cd5

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/324108

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/mitaka)

Reviewed: https://review.openstack.org/324108
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1bef59101e7c3dd60e1c046367e070f1b8ca9b11
Submitter: Jenkins
Branch: stable/mitaka

commit 1bef59101e7c3dd60e1c046367e070f1b8ca9b11
Author: Ann Kamyshnikova <email address hidden>
Date: Tue May 17 17:25:00 2016 +0300

    Pass ha_router_port flag for _snat_router_interfaces ports

    Currently, router_centralized_snat port can be bound to a host were
    l3-agent is in standby state (L3 HA + DVR case). As a result VM without
    floating ip is unable to reach external network. This change passes
    ha_router_port flag to _ensure_host_set_on_port when called for
    _snat_router_interfaces ports.

    Note: this issue is intermittent, without changes in l3_rpc.py
    unit test does not fail every time.

    Co-Authored-By: Oleg Bondarev <email address hidden>

    Closes-bug: #1582739
    Change-Id: I74bad578361ed7eac8cc6c740b06b66ab1530cd5
    (cherry picked from commit 899b6cb0652ee54c08d130ccd7edc3091b7d930b)

tags: added: in-stable-mitaka
Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote : Fix included in openstack/neutron 8.1.2

This issue was fixed in the openstack/neutron 8.1.2 release.

Revision history for this message
Doug Hellmann (doug-hellmann) wrote : Fix included in openstack/neutron 9.0.0.0b2

This issue was fixed in the openstack/neutron 9.0.0.0b2 development milestone.

tags: removed: mitaka-backport-potential
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.