ML2: routers and multiple mechanism drivers

[Sam Morrison]

We have an ML2 environment with linuxbridge and midonet networks. For L3 we use the midonet driver.

If a user tries to bind a linuxbridge port to a midonet router it returns the following error:

{"message":"There is no NeutronPort with ID eddb3d08-97f6-480d-a1a4-9dfe3d22e62c.","code":404}

However the port is created (user can't see it) and doing a router show shows the router as having that interface.

Ideally this should not be allowed and should error out gracefully.

[YAMAMOTO Takashi (yamamoto)]

> However the port is created (user can't see it) and doing a router show shows the router as having that interface.

i don't understand how this can happen. can you provide neutron server log?

[Sam Morrison (sorrison)]

This is the output after running a "neutron router-gateway-set <midonet-router-id> <linuxbridge network>"

2016-03-10 17:50:27.292 10781 INFO neutron.wsgi [req-565add6e-a7db-4fba-a389-ac11ef7bc6f4 d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] 128.250.116.173 - - [10/Mar/2016 17:50:27] "GET /v2.0/routers.json?fields=id&id=e4793db9-7c0d-49f6-bd24-92133b706846 HTTP/1.1" 200 274 0.243358
2016-03-10 17:50:27.328 10781 INFO neutron.wsgi [req-902ac3b4-8791-40fc-91e3-0fba4d53f0c1 d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] 128.250.116.173 - - [10/Mar/2016 17:50:27] "GET /v2.0/networks.json?fields=id&id=ec0080be-fe26-4331-9fc1-16ff2ba1f053 HTTP/1.1" 200 275 0.032465
2016-03-10 17:50:27.335 10781 DEBUG neutron.api.v2.base [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] Request body: {u'router': {u'external_gateway_info': {u'network_id': u'ec0080be-fe26-4331-9fc1-16ff2ba1f053'}}} prepare_request_body /opt/neutron/neutron/api/v2/base.py:654
2016-03-10 17:50:27.362 10781 DEBUG midonet.neutron.services.l3.l3_midonet [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] midonet.neutron.services.l3.l3_midonet.MidonetL3ServicePlugin method update_router called with arguments (<neutron.context.Context object at 0x7f80491cebd0>, u'e4793db9-7c0d-49f6-bd24-92133b706846') {'router': {u'router': {u'external_gateway_info': {u'network_id': u'ec0080be-fe26-4331-9fc1-16ff2ba1f053'}}}} wrapper /opt/liberty/local/lib/python2.7/site-packages/oslo_log/helpers.py:45
2016-03-10 17:50:27.527 10781 DEBUG neutron.db.ipam_non_pluggable_backend [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] Allocated IP - 172.26.82.11 from 172.26.82.12 to 172.26.82.254 _try_generate_ip /opt/neutron/neutron/db/ipam_non_pluggable_backend.py:79
2016-03-10 17:50:27.528 10781 DEBUG neutron.db.db_base_plugin_common [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] Allocated IP 172.26.82.11 (ec0080be-fe26-4331-9fc1-16ff2ba1f053/d1c2f0bf-6697-419c-a674-ddabd8bf2272/efe5e987-99e9-4508-87f0-c3417b1effce) _store_ip_allocation /opt/neutron/neutron/db/db_base_plugin_common.py:102
2016-03-10 17:50:27.552 10781 DEBUG neutron.callbacks.manager [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] Notify callbacks for port, after_create _notify_loop /opt/neutron/neutron/callbacks/manager.py:133
2016-03-10 17:50:27.597 10781 INFO midonetclient.neutron.l3 [req-5ee13bda-1882-46d5-a404-df786e8d9c4f d1fa8867e42444cf8724e65fef1da549 094ae1e2c08f4eddb444a9d9db71ab40 - - -] update_router {'status': u'ACTIVE', 'external_gateway_info': {'network_id': u'ec0080be-fe26-4331-9fc1-16ff2ba1f053', 'enable_snat': True, 'external_fixed_ips': [{'subnet_id': u'd1c2f0bf-6697-419c-a674-ddabd8bf2272', 'ip_address': u'172.26.82.11'}]}, 'name': u'test', 'gw_port_id': u'efe5e987-99e9-4508-87f0-c3417b1effce', 'admin_state_up': True, 'routes': [], 'tenant_id': u'094ae1e2...

[Ihar Hrachyshka (ihar-hrachyshka)]

YAMAMOTO, could you please triage the bug? Thanks.

[YAMAMOTO Takashi (yamamoto)]

midonet's update_router can validate if the given network is compatible or not.
(although it might not be straightforward because midonet has ml2 and monolithic plugin and the l3 plugin is shared between them)

on the other hand, it would be nice if there's a plugin agnostic way to say if a network and a router are compatible or not.
this part can be considered as a neutron issue. probably related to https://bugs.launchpad.net/neutron/+bug/1461133 .

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/297585

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/297585
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=c2ee8fda5c4443c855ae6ef592dbab86bf8d8d7d
Submitter: Jenkins
Branch: master

commit c2ee8fda5c4443c855ae6ef592dbab86bf8d8d7d
Author: YAMAMOTO Takashi <email address hidden>
Date: Fri Mar 25 15:43:32 2016 +0900

    plugin_v2: Show the default provider network_type as "midonet"

    A motivation is to make it look similar to our ML2 driver so that
    our L3 plugin can validate the situation like bug 1555384 easier.

    Related-Bug: #1555384
    Change-Id: Ia3baca5dfeb2b6d045c5aa719d21cd2e5c0b147f

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/313282

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/313282
Committed: https://git.openstack.org/cgit/openstack/networking-midonet/commit/?id=8a9538421c2f1642b5a7458fe3121b3ce4564769
Submitter: Jenkins
Branch: master

commit 8a9538421c2f1642b5a7458fe3121b3ce4564769
Author: YAMAMOTO Takashi <email address hidden>
Date: Fri May 6 14:15:46 2016 +0900

    l3: Reject requests to connect to non-midonet networks gracefully

    This can easily happen with ML2.

    While MidoNet (backend) would reject those requests anyway,
    it's more user-friendly to reject them here.

    Closes-Bug: #1555384
    Change-Id: Ia27c12f2f7f5edd2004671b996cce882ac8a6f79

[Armando Migliaccio (armando-migliaccio)]

This bug is > 180 days without activity. We are unsetting assignee and milestone and setting status to Incomplete in order to allow its expiry in 60 days.

If the bug is still valid, then update the bug status.

[Armando Migliaccio (armando-migliaccio)]

@Sam: can you please elaborate why you think this is a neutron issue to be tracked here?

[Sam Morrison (sorrison)]

@armando is the initial description not enough?

"We have an ML2 environment with linuxbridge and midonet networks. For L3 we use the midonet driver.

If a user tries to bind a linuxbridge port to a midonet router it returns the following error:

{"message":"There is no NeutronPort with ID eddb3d08-97f6-480d-a1a4-9dfe3d22e62c.","code":404}

However the port is created (user can't see it) and doing a router show shows the router as having that interface.

Ideally this should not be allowed and should error out gracefully."

[Sam Morrison (sorrison)]

feedback given

[Armando Migliaccio (armando-migliaccio)]

@Sam: not quite, bear with me though: this bug report links to merged fixes and completed blueprints. Are you saying that you are still experiencing the issue?

It's my understanding that [1,2] would indeed provide the graceful failure as you requested. What else do we need?

Thanks,
Armando

[1] https://review.openstack.org/#/c/313282/
[2] https://review.openstack.org/#/c/297585/

[Sam Morrison (sorrison)]

They are changes to midonet, there needs to be changes to neutron itself to make this more generic so one type of network can't attach to a different type of network in terms of L3 connectivity.

In ML2 there are lots of mechanism drivers, so if you are using more that one you will have issues as you can only specify 1 type of L3 driver. If the mechanism driver for the network is different/incompatible with the L3 driver in use then it won't work.

Hope that's more clear.

[Armando Migliaccio (armando-migliaccio)]

Oh it's clear alright, but I am not sure I agree this must be done in a generic way. We can't boil the ocean.

[Armando Migliaccio (armando-migliaccio)]

IMO, this is an integration issue between Midonet L3 and LinuxBridge L2. In theory L2 and L3 should work without issues, and there are examples where this happens (I recall Astara would work fine with various L2 technology). In this specific instance, we found out an exception to the rule. I don't think we want to embed any general constraint that prevents interoperability.

[Armando Migliaccio (armando-migliaccio)]

Happy to hear what other think but I personally feel this is an invalid neutron issue.

[Kevin Benton (kevinbenton)]

Yeah, an l3 plugin should work with l2 segments by default. If the l3 plugin is exercising some non-standard behavior to perform routing then it should validate that the user didn't try to attach it to a network it didn't support.

Consider the reference l3 ha plugin. As long as you have an interface driver for the l3 agent to use, it can attach to anything and will work with anything that had standard broadcast behavior. It shouldn't have to declare specifically that it works with ovs, Linux bridge, Arista, etc.

[Sam Morrison (sorrison)]

I agree with you and I don't care what project the code needs to go in, it's something as a user they don't care about. As long as neutron makes it possible for a driver to handle this case gracefully then I'd consider this fixed.

Thanks for the discussion