Prefix delegation doesn't work in the described scenario

[Baodong (Robert) Li]

When creating the router and enable pd subnet with the below commands:
    neutron router-create ha-router
    neutron router-gateway-set ha-router public
    neutron router-interface-add ha-router ipv6-pd-subnet

lla timeout on creating with the following trace in l3 agent log:

raceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457, in fire_timers
    timer()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line 58, in __call__
    cb(*args, **kw)
  File "/opt/stack/neutron/neutron/agent/linux/pd.py", line 228, in _ensure_lla_task
    sleep=2)
  File "/opt/stack/neutron/neutron/common/utils.py", line 750, in wait_until_true
    eventlet.sleep(sleep)
  File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line 34, in sleep
    hub.switch()
  File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch
    return self.greenlet.switch()
Timeout: 40 seconds

Note that the order of neutron commands are significant.

Checking the l3 agent log further, gateways were added twice in a row. The later one removed the lla added in the earlier, which caused the timeout.

2016-11-03 11:03:26.517 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:22Z', u'device_owner': u'network:router_gateway', u'revision_number': 5, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'unbound', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'DOWN', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
.....

2016-11-03 11:03:34.338 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'add', 'fe80::f816:3eff:fecc:7a22/64', 'scope', 'link', 'dev', 'qg-95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon /opt/stack/neutron/neutron/agent/linux/utils.py:100

The above log took place when the gw_port was added the first time .

2016-11-03 11:03:34.413 ^[[00;32mDEBUG neutron.agent.l3.router_info [^[[00;36m-^[[00;32m] ^[[01;35m^[[00;32mExternal gateway added: port({u'allowed_address_pairs': [], u'extra_dhcp_opts': [], u'updated_at': u'2016-11-03T15:03:30Z', u'device_owner': u'network:router_gateway', u'revision_number': 8, u'port_security_enabled': False, u'binding:profile': {}, u'binding:vnic_type': u'normal', u'fixed_ips': [{u'subnet_id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'prefixlen': 24, u'ip_address': u'172.24.4.7'}, {u'subnet_id': u'68fc0073-619a-43ca-be62-569842366e48', u'prefixlen': 64, u'ip_address': u'2001:db8::8'}], u'id': u'95b57ead-f411-4f7b-8eda-e2a7b4623ac6', u'security_groups': [], u'binding:vif_details': {u'port_filter': True, u'ovs_hybrid_plug': True}, u'address_scopes': {u'4': None, u'6': None}, u'binding:vif_type': u'ovs', u'mac_address': u'fa:16:3e:2e:2f:58', u'project_id': u'', u'status': u'ACTIVE', u'subnets': [{u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'172.24.4.1', u'cidr': u'172.24.4.0/24', u'id': u'e8cbec96-2ea4-43d5-a717-93b03231d393', u'subnetpool_id': None}, {u'dns_nameservers': [], u'ipv6_ra_mode': None, u'gateway_ip': u'2001:db8::2', u'cidr': u'2001:db8::/64', u'id': u'68fc0073-619a-43ca-be62-569842366e48', u'subnetpool_id': None}], u'binding:host_id': u'ip6-devstack-43', u'description': u'', u'device_id': u'0269742e-56e5-468c-974a-859bb1966a62', u'name': u'', u'admin_state_up': True, u'network_id': u'8da945b9-8ac2-4d1b-840f-094dbd794500', u'tenant_id': u'', u'created_at': u'2016-11-03T15:03:22Z', u'mtu': 1500, u'extra_subnets': []}), interface(qg-95b57ead-f4), ns(qrouter-0269742e-56e5-468c-974a-859bb1966a62)^[[00m ^[[00;33mfrom (pid=28145) _external_gateway_added /opt/stack/neutron/neutron/agent/l3/router_info.py:632^[[00m
......
2016-11-03 11:03:38.513 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0269742e-56e5-468c-974a-859bb1966a62', 'ip', '-6', 'addr', 'del', 'fe80::f816:3eff:fecc:7a22/64', 'dev', 'qg-95b57ead-f4'] from (pid=28145) execute_rootwrap_daemon /opt/stack/neutron/neutron/agent/linux/utils.py:100

the above log took place when the gw_port was added the second time

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/408306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/408306
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e48caf6736426e6cabc94fe3f7fbcdaf26c21dc1
Submitter: Jenkins
Branch: master

commit e48caf6736426e6cabc94fe3f7fbcdaf26c21dc1
Author: Robert Li <email address hidden>
Date: Wed Dec 7 15:54:26 2016 -0500

    Add agent object in router info

    agent object is a member of some sub classes of RouterInfo such as
    HaRouter. This changeset makes it a member of the RouterInfo class
    itself.

    Prior to the change, the agent object has been passed in to some
    methods of RouterInfo that requires it to access the agent object's
    member information. The bugs in concern requires calling the PD object
    that is a member of the agent object to get IPs that need to be
    preserved in the gateway port. Without this change, signatures of the
    methods external_gateway_added() and external_gateway_updated() have
    to be modified to pass in the agent object. And any subclass of
    RouterInfo that overwrites or uses the methods must be changed as
    well. It doesn't seem to make sense considering the subclass such as
    HaRouter has the agent object as one of its members already.

    The changeset fixes the bugs by preserving the LLAs for prefix
    delegation when the gateway port is being updated.

    Closes-Bug: #1639042
    Closes-Bug: #1640271

    Change-Id: I61c6128ed1973deb8440c54234e77a66987d7e28

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/421983

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 10.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Swaminathan Vasudevan (<email address hidden>) on branch: stable/newton
Review: https://review.openstack.org/421983