Subnets with duplicated CIDRs could be added to one router if multiple commands are executed at the same time

Bug #1535226 reported by Lujin Luo
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
neutron
In Progress
Low
Kevin Benton

Bug Description

I have three controller nodes and the Neutron servers on these controllers are set behind Pacemaker and HAProxy to realize active/active HA using DevStack. MariaDB Galera cluster is used as my database backend.I am using the latest codes.

If one router is going to add two subnets as interface, however these two subnets' CIDRs are duplicated, the expected result is the later API request would fail with error message like this
Bad router request: Cidr 192.166.100.0/24 of subnet bee7663c-f0a0-4120-b556-944af7ca40cf overlaps with cidr 192.166.0.0/16 of subnet 697c82cf-82fd-4187-b460-7046c81f13dc.

But when we run the two commands at the same time, both commands would work and the router would end up with two ports, which have duplicated CIDRs. I have tested for more than 20 times and in only once have I received the expected error message.

How to reproduce

Step 1: Create a router
$ neutron router-create router-subnet-test

Step 2: Create two internal networks
$ neutron net-create net1
$ neutron net-create net2

Step 3: Add one subnet to each of these two networks
$ neutron subnet-create --name subnet1 net1 192.166.100.0/24
$ neutron subnet-create --name subnet2 net2 192.166.0.0/16

Here, we are creating two subnets on different networks with duplicated CIDRs.

Step 4: Add the two subnets as one router's interface at the same time
On controller1:
$ neutron router-interface-add router-subnet-test subnet1
On controller2:
$ neutron router-interface-add router-subnet-test subnet2

Both commands would work and we could see that the router now has two ports, which have duplicated CIDRs
http://paste.openstack.org/show/483838/

In [1], we do have a method to _check_for_dup_router_subnet, but when two API requests arrive at the same time, both checks would validate.

[1] https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L590

Lujin Luo (luo-lujin)
Changed in neutron:
assignee: nobody → Lujin Luo (luo-lujin)
tags: added: l3-ipam-dhcp
Changed in neutron:
importance: Undecided → Medium
Nam (namnh)
Changed in neutron:
assignee: Lujin Luo (luo-lujin) → Nam (namnh)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/287566

Changed in neutron:
status: New → In Progress
Changed in neutron:
importance: Medium → Low
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/287566
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Changed in neutron:
assignee: Nam (namnh) → Kevin Benton (kevinbenton)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/333809
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fbd3578d6494fa3f08d75686783f31454b1b42e5
Submitter: Jenkins
Branch: master

commit fbd3578d6494fa3f08d75686783f31454b1b42e5
Author: Kevin Benton <email address hidden>
Date: Wed Jun 22 18:23:19 2016 -0700

    Delete gw port on exceptions

    Clean up related core plugin ports on routers when there is a failure
    creating the record for the router port.

    The two partial bugs will be fixed after
    I476d3e03c8ee763cc4be6d679fe9f501eb3a19b5 has merged.

    Closes-Bug: #1600344
    Partial-Bug: #1535225
    Partial-Bug: #1535226
    Change-Id: I8dd832f35e20d1ee090ebab921f0deea533b6fc8

tags: added: neutron-proactive-backport-potential
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.