intermittent failures in setup_physical_bridges

[Armando Migliaccio]

http://logs.openstack.org/39/388939/18/check/gate-neutron-dsvm-functional-ubuntu-xenial/71d901a/testr_results.html.gz

http://logs.openstack.org/50/402750/31/check/gate-neutron-dsvm-functional-ubuntu-xenial/6c1d7b4/testr_results.html.gz

http://logs.openstack.org/04/395504/15/check/gate-neutron-dsvm-functional-ubuntu-xenial/dc17d0b/testr_results.html.gz

http://logs.openstack.org/09/422309/1/check/gate-neutron-dsvm-functional-ubuntu-xenial/e0c8a6c/testr_results.html.gz

http://logs.openstack.org/04/395504/13/check/gate-neutron-dsvm-functional-ubuntu-xenial/ea7661f/testr_results.html.gz

http://logs.openstack.org/28/340228/30/check/gate-neutron-dsvm-functional-ubuntu-xenial/f229bb2/testr_results.html.gz

http://logs.openstack.org/02/416302/13/check/gate-neutron-dsvm-functional-ubuntu-xenial/d4d757f/testr_results.html.gz

[Armando Migliaccio (armando-migliaccio)]

We regressed somehow since Jan 12th

[Armando Migliaccio (armando-migliaccio)]

This does not happen on stable [0], thus I wonder if the smoking gun is [1].

[0] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22in%20test_trunk_creation_with_subports%5C%22
[1] https://review.openstack.org/#/c/373973/

[Armando Migliaccio (armando-migliaccio)]

Or this one:

https://review.openstack.org/#/c/402140

[Victor Morales (electrocucaracha)]

I noticed in this patch[1]. I've two theories on that, given the last issues on memory consumption[2] the sleep time is not enough for releasing resources so the solution could be increase or are more attempts on this function[3] or maybe it's a bug on ip net tool.

[1] https://review.openstack.org/#/c/360799/
[2] https://bugs.launchpad.net/neutron/+bug/1656386
[3] https://github.com/openstack/neutron/blob/master/neutron/cmd/netns_cleanup.py#L255-L265

[Armando Migliaccio (armando-migliaccio)]

It might be too early to claim victory, but since [1] landed, this might have gone away. Kevin and I looked at [1] as the potential culprit of this woes.

[1] https://review.openstack.org/#/c/421325/

[Armando Migliaccio (armando-migliaccio)]

This might as well another manifestation of bug 1654287. We'll keep an eye on it before closing.

[Armando Migliaccio (armando-migliaccio)]

This is gone away.

[Miguel Angel Ajo (mangelajo)]

Thanks armando.