DM restart changes irb IP address and old IP is no longer usable

[Musa Abdin]

In contrail 4.1.0/RHOSP 10 setup: |

restarting Device manager assigns new IP address to the irb interfaces, while the old IPs are no longer available.

[heat-admin@overcloud-contrailcontroller-0 ~]$ neutron subnet-show 24e8e71e-54fd-483f-bb10-fbe625fd0344
+-------------------+----------------------------------------+
| Field | Value |
+-------------------+----------------------------------------+
| allocation_pools | {"start": "8.8.8.2", "end": "8.8.8.5"} |
| cidr | 8.8.8.0/24 |
| created_at | |
| dns_nameservers | 0.0.0.0 |
| enable_dhcp | False |
| gateway_ip | 8.8.8.1 |
| host_routes | |
| id | 24e8e71e-54fd-483f-bb10-fbe625fd0344 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | 24e8e71e-54fd-483f-bb10-fbe625fd0344 |
| network_id | 18067a99-ed12-4a74-8b50-92577dc27bdb |
| tenant_id | db666c41aefc4a4581655e3c2e02627c |
| updated_at | |
+-------------------+----------------------------------------+
[heat-admin@overcloud-contrailcontroller-0 ~]$ curl -H "X-Auth-Token: `openstack token issue -c id -f value`" http://192.168.122.11:8082/virtual-network/18067a99-ed12-4a74-8b50-92577dc27bdb/subnet-ip-count -H "Content-Type: application/json" -d '{"subnet_list": ["24e8e71e-54fd-483f-bb10-fbe625fd0344"]}' |python -m json.tool
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 79 100 22 100 57 142 370 --:--:-- --:--:-- --:--:-- 372
{
    "ip_count_list": [
        2
    ]
}
[heat-admin@overcloud-contrailcontroller-0 ~]$ sudo systemctl restart contrail-device-manager
[heat-admin@overcloud-contrailcontroller-0 ~]$ curl -H "X-Auth-Token: `openstack token issue -c id -f value`" http://192.168.122.11:8082/virtual-network/18067a99-ed12-4a74-8b50-92577dc27bdb/subnet-ip-count -H "Content-Type: application/json" -d '{"subnet_list": ["24e8e71e-54fd-483f-bb10-fbe625fd0344"]}' |python -m json.tool
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
100 79 100 22 100 57 152 395 --:--:-- --:--:-- --:--:-- 398
{
    "ip_count_list": [
        3
    ]
}
[heat-admin@overcloud-contrailcontroller-0 ~]$ neutron port-create --name testsrv01 --fixed-ip subnet_id=24e8e71e-54fd-483f-bb10-fbe625fd0344,ip_address=8.8.8.3 `openstack network show sriov-test -c id -f value`
An unknown exception occurred.
Neutron server returns request_ids: ['req-38e57c68-57ae-40b9-a1a1-58a71bb5536c']
[heat-admin@overcloud-contrailcontroller-0 ~]$

on the mx

[edit]
musa# show groups __contrail__ interfaces irb.7
May 19 11:30:43
family inet {
    /* Allocated IPv4 Address from Subnet: 8.8.8.0/24 */
    address 8.8.8.3/24 {
        virtual-gateway-address 8.8.8.1;
    }
}

[edit]
musa#
May 19 11:32:52

after the restart:

[edit]
musa#
May 19 11:32:52

[edit]
musa# show groups __contrail__ interfaces irb.7
May 19 11:35:05
family inet {
    /* Allocated IPv4 Address from Subnet: 8.8.8.0/24 */
    address 8.8.8.4/24 {
        virtual-gateway-address 8.8.8.1;
    }
}

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43415
Submitter: Suresh Balineni (<email address hidden>)

[Musa Abdin (musamustafa)]

Tried the fix,
when restarting the DM, and the ip_count_list doesn't increase anymore, however the previous stale IP's are still there.
is there any way to clean those stale IP's?

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43415
Committed: http://github.com/Juniper/contrail-controller/commit/eaaa3d2f01aed4141da4901c35aef59eba0d9f21
Submitter: Zuul (<email address hidden>)
Branch: R4.1

commit eaaa3d2f01aed4141da4901c35aef59eba0d9f21
Author: sbalineni <email address hidden>
Date: Wed May 30 14:14:29 2018 -0700

[DM]: Fix for reading DM private DB keyspace columns.

Currently DM invokes get_range() with column count 0.
This call does not return persisted ip-addresses.
Similar to this, ae id allocations also not read properly.
Fixed.

Change-Id: I37ad9dae8aec13a38cd810f15bc1265109c841b2
Closes-Bug: #1772160

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43432
Submitter: Suresh Balineni (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/43433
Submitter: Suresh Balineni (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43432
Committed: http://github.com/Juniper/contrail-controller/commit/d9230022f7c9e323aa24f759f5a5a2736af47aa2
Submitter: Zuul v3 CI (<email address hidden>)
Branch: R5.0

commit d9230022f7c9e323aa24f759f5a5a2736af47aa2
Author: sbalineni <email address hidden>
Date: Wed May 30 14:14:29 2018 -0700

[DM]: Fix for reading DM private DB keyspace columns.

Currently DM invokes get_range() with column count 0.
This call does not return persisted ip-addresses.
Similar to this, ae id allocations also not read properly.
Fixed.

Change-Id: I37ad9dae8aec13a38cd810f15bc1265109c841b2
Closes-Bug: #1772160
(cherry picked from commit eaaa3d2f01aed4141da4901c35aef59eba0d9f21)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/43433
Committed: http://github.com/Juniper/contrail-controller/commit/7f0794723d857031da74089d52007376138a2933
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 7f0794723d857031da74089d52007376138a2933
Author: sbalineni <email address hidden>
Date: Wed May 30 14:14:29 2018 -0700

[DM]: Fix for reading DM private DB keyspace columns.

Currently DM invokes get_range() with column count 0.
This call does not return persisted ip-addresses.
Similar to this, ae id allocations also not read properly.
Fixed.

Change-Id: I37ad9dae8aec13a38cd810f15bc1265109c841b2
Closes-Bug: #1772160
(cherry picked from commit eaaa3d2f01aed4141da4901c35aef59eba0d9f21)