EMC-POC:Nexus plugin: fail to restart Neutron server in scaled VLAN

[Danny Choi]

Nexus plugin replay parameers:
   - switch_heartbeat_time = 30
   - switch_replay_count = 3

VLAN scaling parameters:
   - 9 tenants
   - Each tenant with 5 networks
   - Each network with 2 VMs in 2 separate Compute nodes
   - Total 45 networks (VLANs), 90 VMs

With this scaled config, at the Controller+Network node, restart neutron-server would fail.
Same problem for rebooting the node.
Problem does not happen if replay is disabled.

[root@bxb-ds-46 ~]# openstack-service restart neutron
Job for neutron-server.service failed. See 'systemctl status neutron-server.service' and 'journalctl -xn' for details.
[root@bxb-ds-46 ~]# systemctl status neutron-server.service
neutron-server.service - OpenStack Neutron Server
   Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled)
   Active: failed (Result: timeout) since Mon 2015-06-22 07:29:25 EDT; 40s ago
 Main PID: 23064

Jun 22 07:29:25 bxb-ds-46 systemd[1]: neutron-server.service operation timed out. Terminating.
Jun 22 07:29:25 bxb-ds-46 systemd[1]: Failed to start OpenStack Neutron Server.
Jun 22 07:29:25 bxb-ds-46 systemd[1]: Unit neutron-server.service entered failed state.
[root@bxb-ds-46 ~]# journalctl -xn
-- Logs begin at Sun 2015-06-21 09:59:18 EDT, end at Mon 2015-06-22 07:30:12 EDT. --
Jun 22 07:30:01 bxb-ds-46 object-auditor[23438]: Begin object audit "forever" mode (ALL)
Jun 22 07:30:01 bxb-ds-46 object-auditor[23438]: Object audit (ALL) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0
Jun 22 07:30:01 bxb-ds-46 object-auditor[23437]: Begin object audit "forever" mode (ZBF)
Jun 22 07:30:01 bxb-ds-46 object-auditor[23437]: Object audit (ZBF) "forever" mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: Beginning replication run
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: Replication run OVER
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: Attempted to replicate 0 dbs in 0.02240 seconds (0.00000/s)
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: Removed 0 dbs
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: 0 successes, 0 failures
Jun 22 07:30:12 bxb-ds-46 account-replicator[2465]: no_change:0 ts_repl:0 diff:0 rsync:0 diff_capped:0 hashmatch:0 empty:0
[root@bxb-ds-46 ~]# openstack-service status | grep neutron-server
neutron-server (pid 0) is failed
[root@bxb-ds-46 ~]#