[SWARM][8.0] cinder_nova_partition_preservation: VM is not accessible via ip from admin network

[Vladimir Jigulin]

Scenario:
1. Create a cluster
2. Add 3 nodes with controller and mongo roles
3. Add a node with compute and cinder roles
4. Provision nodes
5. Create an OS volume and OS instance
6. Mark 'cinder' partition to be preserved
7. Mark 'vm' partition to be preserved
8. Reinstall the compute node
9. Run network verification
10. Run OSTF
11. Verify that the volume is present and has 'available' status after the node reinstallation
12. Verify that the VM is available and pingable after the node reinstallation <-- error
or use cinder_nova_partition_preservation from fuel-qa

Error Message:
test-serv128939413 VM is not accessible via its <FloatingIP fixed_ip=None, id=ab6f0425-cf0b-4034-9b3b-c2dc3a18b7c6, instance_id=None, ip=10.109.3.132, pool=admin_floating_net> floating ip

PortBindingFailed: Binding failed for port 20b2ab5d-4359-45c3-8eed-7ab7f8852584, please check neutron logs for more information.

Logs:
node-2:~# grep ERROR /var/log/neutron-all.log : http://termbin.com/g29j
node-3:~# grep ERROR /var/log/nova-all.log : http://termbin.com/slg3
node-3:~# grep ERROR /var/log/neutron-all.log : http://termbin.com/j6ch

[Vladimir Jigulin (vjigulin)]

in neutron logs some traces with
UnboundLocalError: local variable 'routers' referenced before assignment
error probably caused by
https://review.fuel-infra.org/gitweb?p=openstack/neutron.git;a=commitdiff;h=4eb0437b8f52eba6b68152986801838756a0763e;hp=-c

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: openstack-ci/fuel-8.0/liberty
Change author: Denis V. Meltsaykin <email address hidden>
Review: https://review.fuel-infra.org/33137

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/33137
Submitter: Pkgs Jenkins <email address hidden>
Branch: openstack-ci/fuel-8.0/liberty

Commit: c3d842abfb6c3f7bdf8b1ccaa59dba0b15b13aa3
Author: Denis V. Meltsaykin <email address hidden>
Date: Sat Apr 15 20:05:04 2017

Remove accidentaly restored parts

Initialy, with the change Id675910c2a0b862bfb9e6f4fdaf3cd9fe337e52f
there was removed a bunch of code from neutron/agent/l3/agent.py but
then it was accidentaly restored in
I198eb530c9bce42da89a20d1196ec4b9d616eb2d. This leads to a mix of two
conflicting versions, what obviously doesn't work as expected.

Change-Id: I47459ec4b347be50236e0b7992b58dd698900a58
Closes-Bug: #1681753

[Ekaterina Shutova (eshutova)]

Checked latest runs. Test is not stable now, sometimes failed with the same error even with fix:
https://patching-ci.infra.mirantis.net/view/8.0.swarm/job/8.0.system_test.ubuntu.partition_preservation/66/testReport/(root)/cinder_nova_partition_preservation/cinder_nova_partition_preservation/

[Denis Meltsaykin (dmeltsaykin)]

VM sometimes cannot obtain an IP-address. This is not persistent and needs more time to investigate. Moving to 8.0-mu-5

[Denis Meltsaykin (dmeltsaykin)]

So far all the subsequent tests were successful and I didn't have a chance to troubleshoot them. Moving to Incomplete until it occurs again.