Neutron-api fails ocata to pike upgrade

[Chris MacNaughton]

#############################################################################
Create 1 * tiny cirros and 1 * small precise instances on the overcloud
#############################################################################

2017-10-16 11:56:27 [INFO] Pulling secrets from /tmp/tmp.pemDYZ6Ddv/LOCAL/mojo-openstack-specs/specs/full_stack/next_openstack_upgrade/ocata to /tmp/tmp.pemDYZ6Ddv/mojo-openstack-specs/xenial/osci-mojo/local
2017-10-16 11:56:27 [WARNING] Automatic secrets phase ran but secrets directory /tmp/tmp.pemDYZ6Ddv/LOCAL/mojo-openstack-specs/specs/full_stack/next_openstack_upgrade/ocata does not exist!
2017-10-16 11:56:27 [INFO] Running script simple_os_checks.py
2017-10-16 11:56:34 [WARNING] No debug log matching debug-logs found. Using default.
2017-10-16 11:56:35 [ERROR] 2017-10-16 11:56:31 [INFO] Using keystone API V2 for overcloud auth
2017-10-16 11:56:33 [INFO] Deleting key mojo
2017-10-16 11:56:33 [INFO] Creating key mojo
Traceback (most recent call last):
  File "/tmp/tmp.pemDYZ6Ddv/mojo-openstack-specs/xenial/osci-mojo/spec/specs/full_stack/next_openstack_upgrade/ocata/simple_os_checks.py", line 89, in <module>
    sys.exit(main(sys.argv))
  File "/tmp/tmp.pemDYZ6Ddv/mojo-openstack-specs/xenial/osci-mojo/spec/specs/full_stack/next_openstack_upgrade/ocata/simple_os_checks.py", line 73, in main
    mojo_os_utils.add_neutron_secgroup_rules(neutronc, project_id)
  File "/tmp/tmp.pemDYZ6Ddv/mojo-openstack-specs/xenial/osci-mojo/spec/helper/tests/utils/mojo_os_utils.py", line 707, in add_neutron_secgroup_rules
    for group in neutron_client.list_security_groups().get('security_groups'):
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 953, in list_security_groups
    retrieve_all, **_params)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 367, in list
    for r in self._pagination(collection, path, **params):
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 382, in _pagination
    res = self.get(path, params=params)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 352, in get
    headers=headers, params=params)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 329, in retry_request
    headers=headers, params=params)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 280, in do_request
    resp, replybody = self.httpclient.do_request(action, method, body=body)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/client.py", line 342, in do_request
    return self.request(url, method, **kwargs)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/neutronclient/client.py", line 330, in request
    resp = super(SessionClient, self).request(*args, **kwargs)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 189, in request
    return self.session.request(url, method, **kwargs)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 698, in request
    resp = send(**kwargs)
  File "/var/lib/jenkins/tools/0/charm-test-infra/.tox/clients/local/lib/python2.7/site-packages/keystoneauth1/session.py", line 772, in _send_request
    raise exceptions.ConnectFailure(msg)
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://172.17.104.107:9696/v2.0/security-groups: ('Connection aborted.', BadStatusLine("''",))

[David Ames (thedac)]

It seems we might be missing a DB migration.

After upgrade the neutron-server process keeps dying with:

2017-11-09 00:19:56.883 29705 WARNING stevedore.named [req-998ddd50-b9b9-48fd-a63c-2390687fb179 - - - - -] Could not load neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters [req-998ddd50-b9b9-48fd-a63c-2390687fb179 - - - - -] DBAPIError exception wrapped from (pymysql.err.InternalError) (1054, u"Unknown column 'vpnservices.flavor_id' in 'field list'") [SQL: u'SELECT vpnservices.project_id AS vpnservices_project_id, vpnservices.id AS vpnservices_id, vpnservices.name AS vpnservices_name, vpnservices.description AS vpnservices_description, vpnservices.status AS vpnservices_status, vpnservices.admin_state_up AS vpnservices_admin_state_up, vpnservices.external_v4_ip AS vpnservices_external_v4_ip, vpnservices.external_v6_ip AS vpnservices_external_v6_ip, vpnservices.subnet_id AS vpnservices_subnet_id, vpnservices.router_id AS vpnservices_router_id, vpnservices.flavor_id AS vpnservices_flavor_id \nFROM vpnservices']: InternalError: (1054, u"Unknown column 'vpnservices.flavor_id' in 'field list'")
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters context)
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters result = self._query(query)
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters conn.query(q)
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 856, in query
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1057, in _read_query_result
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters result.read()
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1340, in read
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet()
2017-11-09 00:19:57.134 29705 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2....

[David Ames (thedac)]

This is Ocata to Pike upgrades

A tertiary issue is that the pike amulet tests in neutron-api are not enabled.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/519512
Committed: https://git.openstack.org/cgit/openstack/charm-neutron-api/commit/?id=1ee9f465435c893613af86dd9f99ddbdfc2701d4
Submitter: Zuul
Branch: master

commit 1ee9f465435c893613af86dd9f99ddbdfc2701d4
Author: David Ames <email address hidden>
Date: Mon Nov 13 16:26:30 2017 -0800

    Fix Ocata to Pike upgrade

    When upgrading from Ocata to Pike the context was not recognizing
    Pike until a subsequent hook run. Resetting os release fixes this.

    Enable xenia-pike amulet test.

    Closes-Bug: #1723981

    Change-Id: I67ec257f0a91cf4108de54a2cd93ab0cc3663376