[nsxv][scale] Server error on create and list routers

[Andrey Setyaev]

NSX Version: 6.1.4, Build: 2691049
Neutron NSX plugin 2015.1.2 (package is built using git tag 2015.1.2)

InternalServerError: An unknown exception 500 occurred: <?xml version="1.0" encoding="UTF-8"?>
<error><details>all controllers are inactive</details><errorCode>100</errorCode></error>

rally test: NeutronNetworks.create_and_list_routers

NSXv manager informs that all nsxv controllers are in inactive state.

[Igor Zinovik (izinovik)]

This error was found during Rally test:
   NeutronNetworks.create_and_list_routers:
     -
       args:
         network_create_args:
         subnet_create_args:
         subnet_cidr_start: "1.1.0.0/30"
         subnets_per_network: 2
         router_create_args:
       runner:
         type: "constant"
         times: {{smoke or 15}}
         concurrency: {{smoke or 5}}
       context:
         network: {}
         users:
           tenants: {{smoke or 3}}
           users_per_tenant: {{smoke or 2}}
         quotas:
           neutron:
              network: -1
              subnet: -1
              router: -1
       sla:
         failure_rate:
           max: 20

In this test rally constantly create routers 15 times.
Setup of this test involved 3 OpenStack controllers, on each Neutron server was configured with Neutron NSX plugin.
HAproxy on controllers was configured in active/passive mode.

[Igor Zinovik (izinovik)]

Moving bug to upstream project. Puppet deployment manifests seems to be not related to the described problem.