[10.0-mitaka]Smoke and BVT tests failed requesting auth session: Internal Server Error (HTTP 500)

[Alexander Makarov]

Reproduced on https://product-ci.infra.mirantis.net/job/10.0-mitaka.main.ubuntu.smoke_neutron/165/:

2016-08-02 16:58:23,827 - ERROR __init__.py:67 -- verify_network raised: InternalServerError('Internal Server Error (HTTP 500)',)
Traceback: Traceback (most recent call last):
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/__init__.py", line 60, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/models/fuel_web_client.py", line 1637, in verify_network
    task = self.run_network_verify(cluster_id)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/__init__.py", line 60, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/models/fuel_web_client.py", line 1308, in run_network_verify
    return self.client.verify_networks(cluster_id)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/__init__.py", line 60, in wrapped
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/models/nailgun_client.py", line 108, in verify_networks
    json=self.get_networks(cluster_id)
  File "/home/jenkins/workspace/10.0-mitaka.main.ubuntu.smoke_neutron/fuelweb_test/models/nailgun_client.py", line 85, in _put
    return self.session.put(url=url, **kwargs)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 672, in put
    return self.request(url, 'PUT', **kwargs)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
    return wrapped(*args, **kwargs)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 559, in request
    raise exceptions.from_response(resp, method, url)
InternalServerError: Internal Server Error (HTTP 500)

Keystone logs end right before errors begin.

==============================
UPD(vkhlyunev)
nailgun/app.log:
2016-08-02 16:58:03.353 ERROR [7f5ca7cb1880] (base) Unexpected exception occured
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 306, in content_json
    resp = func(cls, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nailgun/extensions/network_manager/handlers/network_configuration.py", line 228, in PUT
    task = task_manager.execute(data, vlan_ids)
  File "/usr/lib/python2.7/site-packages/nailgun/task/manager.py", line 1124, in execute
    self.cluster)
  File "/usr/lib/python2.7/site-packages/nailgun/task/task.py", line 2080, in get_config
    if iface.bond and iface.bond.mode in lacp_modes:
AttributeError: 'NoneType' object has no attribute 'bond'

[Timur Nurlygayanov (tnurlygayanov)]

Nailgun API returns 500 status code after the request from fuel-qa code, FuelQA team, could you please take a look?

Thank you!

[Vladimir Khlyunev (vkhlyunev)]

from nailgun/app.log

2016-08-02 16:58:03.353 ERROR [7f5ca7cb1880] (base) Unexpected exception occured
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 306, in content_json
    resp = func(cls, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nailgun/extensions/network_manager/handlers/network_configuration.py", line 228, in PUT
    task = task_manager.execute(data, vlan_ids)
  File "/usr/lib/python2.7/site-packages/nailgun/task/manager.py", line 1124, in execute
    self.cluster)
  File "/usr/lib/python2.7/site-packages/nailgun/task/task.py", line 2080, in get_config
    if iface.bond and iface.bond.mode in lacp_modes:
AttributeError: 'NoneType' object has no attribute 'bond'

what is expected from qa here?

[Georgy Kibardin (gkibardin)]

According to the snapshot no controller has an interface associated with a public network. Figuring why.

[Georgy Kibardin (gkibardin)]

However, this doesn't seem to be connected with original 500 error from keystone client.

[Georgy Kibardin (gkibardin)]

QA guys, could you please double check configuration of the system under the test so that there is at least one public interface per controller?

[Nastya Urlapova (aurlapova)]

Moved to "Invalid" because https://product-ci.infra.mirantis.net/job/10.0.main.ubuntu.bvt_2/1384/ is green, and this issue looks fixed.