| 2018-03-29 23:50:52 |
Dmitrii Shcherbakov |
bug |
|
|
added bug |
| 2018-03-29 23:54:53 |
Dmitrii Shcherbakov |
bug |
|
|
added subscriber Ante Karamatić |
| 2018-03-30 15:15:43 |
Ryan Beisner |
bug |
|
|
added subscriber Ryan Beisner |
| 2018-04-01 20:44:18 |
Dmitrii Shcherbakov |
description |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet
The right code path seems to be in dvr_local_router.py
https://github.com/openstack/neutron/blob/stable/queens/neutron/agent/l3/dvr_local_router.py#L413
https://github.com/openstack/neutron/blob/stable/queens/neutron/agent/l3/dvr_local_router.py#L623-L632
Based on a quick grep nothing in dvr_fip_ns.py calls internal_network_added so this never gets triggered.
neutron/agent/l3/dvr_edge_ha_router.py|40| def internal_network_added(self, port):
neutron/agent/l3/dvr_edge_ha_router.py|41| # Call RouterInfo's internal_network_added (Plugs the port, adds IP)
neutron/agent/l3/dvr_edge_ha_router.py|42| router_info.RouterInfo.internal_network_added(self, port)
neutron/agent/l3/dvr_edge_router.py|96| def internal_network_added(self, port):
neutron/agent/l3/dvr_edge_router.py|97| super(DvrEdgeRouter, self).internal_network_added(port)
neutron/agent/l3/dvr_edge_router.py|110| self._internal_network_added(
neutron/agent/l3/dvr_edge_router.py|142| self._internal_network_added(
neutron/agent/l3/dvr_local_router.py|398| def internal_network_added(self, port):
neutron/agent/l3/dvr_local_router.py|399| super(DvrLocalRouter, self).internal_network_added(port)
neutron/agent/l3/ha_router.py|331| def internal_network_added(self, port):
neutron/agent/l3/router_info.py|441| def _internal_network_added(self, ns_name, network_id, port_id,
neutron/agent/l3/router_info.py|458| def internal_network_added(self, port):
neutron/agent/l3/router_info.py|466| self._internal_network_added(self.ns_name,
neutron/agent/l3/router_info.py|556| self.internal_network_added(p)
https://github.com/openstack/neutron/blob/stable/queens/neutron/agent/l3/dvr_fip_ns.py |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet |
|
| 2018-04-02 01:21:48 |
OpenStack Infra |
neutron: status |
New |
In Progress |
|
| 2018-04-02 01:21:48 |
OpenStack Infra |
neutron: assignee |
|
Dmitrii Shcherbakov (dmitriis) |
|
| 2018-04-03 18:16:29 |
Swaminathan Vasudevan |
tags |
cpe-onsite |
cpe-onsite l3-dvr-backlog |
|
| 2018-04-03 22:13:31 |
Brian Haley |
bug |
|
|
added subscriber Brian Haley |
| 2018-04-04 14:19:24 |
Dmitrii Shcherbakov |
summary |
[dvr][fast-exit] a route to a tenant network does not get created in fip namespace if an external network is attached after a tenant network have been attached |
[dvr][fast-exit] a route to a tenant network does not get created in fip namespace if an external network is attached after a tenant network have been attached (race condition) |
|
| 2018-04-04 18:38:44 |
Dmitrii Shcherbakov |
description |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
Relevant agent config options:
http://paste.openstack.org/show/718417/
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet |
|
| 2018-04-04 18:40:22 |
Dmitrii Shcherbakov |
description |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
Relevant agent config options:
http://paste.openstack.org/show/718417/
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet |
Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem.
Relevant agent config options:
http://paste.openstack.org/show/718418/
OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
openstack address scope create dev
openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev
openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant
openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet
openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc
openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2
openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1
openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2
openstack network create --internal --provider-network-type vxlan tenantnet
openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet
# -------
# Works in this order when an external network is attached first
openstack router create --disable --no-ha --distributed pubrouter
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
openstack router add subnet pubrouter tenantsubnet
2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne
tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1
424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92
# ------
# Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached
openstack router create --disable --no-ha --distributed pubrouter
openstack router add subnet pubrouter tenantsubnet
openstack router set --disable-snat --external-gateway pubnet --enable pubrouter
# to "fix" this we need to re-trigger the right code path
openstack router remove subnet pubrouter tenantsubnet
openstack router add subnet pubrouter tenantsubnet |
|
| 2018-04-06 22:35:02 |
OpenStack Infra |
neutron: status |
In Progress |
Fix Released |
|
| 2018-04-06 22:46:15 |
Dmitrii Shcherbakov |
bug task added |
|
neutron (Ubuntu) |
|
| 2018-04-09 20:10:32 |
OpenStack Infra |
tags |
cpe-onsite l3-dvr-backlog |
cpe-onsite in-stable-queens l3-dvr-backlog |
|
| 2018-04-16 18:42:28 |
Corey Bryant |
nominated for series |
|
Ubuntu Bionic |
|
| 2018-04-16 18:42:28 |
Corey Bryant |
bug task added |
|
neutron (Ubuntu Bionic) |
|
| 2018-04-16 18:42:28 |
Corey Bryant |
nominated for series |
|
Ubuntu Artful |
|
| 2018-04-16 18:42:28 |
Corey Bryant |
bug task added |
|
neutron (Ubuntu Artful) |
|
| 2018-04-16 18:42:37 |
Corey Bryant |
neutron (Ubuntu Artful): status |
New |
Triaged |
|
| 2018-04-16 18:42:40 |
Corey Bryant |
neutron (Ubuntu Artful): importance |
Undecided |
Medium |
|
| 2018-04-16 18:42:43 |
Corey Bryant |
neutron (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2018-04-16 18:42:46 |
Corey Bryant |
neutron (Ubuntu Bionic): status |
New |
Triaged |
|
| 2018-04-16 18:42:55 |
Corey Bryant |
bug task added |
|
cloud-archive |
|
| 2018-04-16 18:43:08 |
Corey Bryant |
nominated for series |
|
cloud-archive/pike |
|
| 2018-04-16 18:43:08 |
Corey Bryant |
bug task added |
|
cloud-archive/pike |
|
| 2018-04-16 18:43:08 |
Corey Bryant |
nominated for series |
|
cloud-archive/queens |
|
| 2018-04-16 18:43:08 |
Corey Bryant |
bug task added |
|
cloud-archive/queens |
|
| 2018-04-16 18:43:16 |
Corey Bryant |
cloud-archive/pike: status |
New |
Triaged |
|
| 2018-04-16 18:43:21 |
Corey Bryant |
cloud-archive/pike: importance |
Undecided |
Medium |
|
| 2018-04-16 18:43:23 |
Corey Bryant |
cloud-archive/queens: status |
New |
Triaged |
|
| 2018-04-16 18:43:26 |
Corey Bryant |
cloud-archive/queens: importance |
Undecided |
Medium |
|
| 2018-04-18 16:21:50 |
Launchpad Janitor |
neutron (Ubuntu Bionic): status |
Triaged |
Fix Released |
|
| 2018-04-20 18:29:38 |
Corey Bryant |
cloud-archive/queens: status |
Triaged |
Fix Committed |
|
| 2018-05-31 10:04:57 |
Bernard Cafarelli |
tags |
cpe-onsite in-stable-queens l3-dvr-backlog |
cpe-onsite in-stable-queens l3-dvr-backlog neutron-proactive-backport-potential |
|
| 2018-06-22 17:59:53 |
Corey Bryant |
cloud-archive/queens: status |
Fix Committed |
Fix Released |
|
| 2018-08-13 11:47:52 |
OpenStack Infra |
cloud-archive/pike: status |
Triaged |
Fix Committed |
|
| 2018-10-03 16:45:50 |
Corey Bryant |
neutron (Ubuntu Artful): status |
Triaged |
Invalid |
|
| 2020-09-08 14:35:13 |
Chris MacNaughton |
cloud-archive/pike: status |
Fix Committed |
Fix Released |
|
| 2020-09-08 14:35:21 |
Chris MacNaughton |
cloud-archive: status |
Fix Committed |
Fix Released |
|
