Paramiko 2.0 does not work with Nova

[git-harry]

Paramiko 2.0 has just been release and breaks nova which currently has a requirement for 'paramiko>=1.16.0 # LGPL'. Why does the upper constraints file not protect against this?

2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions [req-2e14bc90-999a-46ba-95f6-de817548014e 8fbdaf628d114c32a81f7587df939771 be553343f9674e61b53b6b2516d4be2f - - -] Unexpected exception in API method
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/api/openstack/compute/keypairs.py", line 111, in create
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return self._create(req, body)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/api/openstack/compute/keypairs.py", line 132, in _create
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions context, user_id, name, key_type)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/exception.py", line 110, in wrapped
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions payload)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions self.force_reraise()
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions six.reraise(self.type_, self.value, self.tb)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return f(self, context, *args, **kw)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/compute/api.py", line 4046, in create_key_pair
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions user_id, key_type)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/compute/api.py", line 4068, in _generate_key_pair
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions return crypto.generate_key_pair()
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/crypto.py", line 152, in generate_key_pair
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions key = generate_key(bits)
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions File "/openstack/venvs/nova-master/lib/python2.7/site-packages/nova/crypto.py", line 144, in generate_key
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions key = paramiko.RSAKey(vals=(rsa.e, rsa.n))
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions TypeError: __init__() got an unexpected keyword argument 'vals'
2016-04-29 14:56:58.381 2175 ERROR nova.api.openstack.extensions
2016-04-29 14:56:58.395 2175 INFO nova.api.openstack.wsgi [req-2e14bc90-999a-46ba-95f6-de817548014e 8fbdaf628d114c32a81f7587df939771 be553343f9674e61b53b6b2516d4be2f - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

[Jesse Pretorius (jesse-pretorius)]

Weird, upper-constraints for master is currently at 1.16.0 so the package in the venv should be constrained to paramiko>=1.16.0,<=1.16.0

[Kevin Carter (kevin-carter)]

The issue we have with paramiko is that we have an unbound requirement as found here: [ https://github.com/openstack/openstack-ansible/blob/master/requirements.txt#L3 ]. we need to update this item to match upstream.

[Kevin Carter (kevin-carter)]

An update to move our pin has been submitted here: [ https://review.openstack.org/#/c/312294 ] which will unblock the integrated gate. That said, if we isolate ansible as found within this PR: [ https://review.openstack.org/#/c/304840/ ] we can remove the lot of the Ansible specific requirements we've been carrying.

[Kevin Carter (kevin-carter)]

To answer the question "why is this happening":

The cause of this problem is because we have a requirement set and anything we set will take precedence over all else we have our own upper constraints file and if we want to have this as not installed or simply limited we can achieve that by adding packages to [ https://github.com/openstack/openstack-ansible/blob/kilo/global-requirement-pins.txt ].

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/312294
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=e971e15997aab2c9d8d1e0cf106a67ee703a3409
Submitter: Jenkins
Branch: master

commit e971e15997aab2c9d8d1e0cf106a67ee703a3409
Author: Kevin Carter <email address hidden>
Date: Tue May 3 19:57:55 2016 -0500

    Integrated updates after the multi-distro changes

    * Corrected the repo server log directory. This change updates the repos
      server log directory which was being incorrectly indexed. The old container
      build process would create a service log directory based on "properties".
      The "properties" log directory is unused or is re-defined by the various
      service roles. This change simply corrects the log paths which the
      `rsyslog_client` role uses to ensure all logs are shipped to the right
      places.
    * Add a log directory creation task to the utility container. This play
      incorrectly assumed that the log directory based on "properties" would
      be automatically created. This update simply makes that more explicit.
    * paramiko has been pinned to match global requirements and use a version
      <2. This will ressolve runtime issues in Newton with an unbound
      requirements.

    Closes-Bug: #1576755
    Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/312584
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=0c46a99b381c448082bbdc4e82e2a55121b2384e
Submitter: Jenkins
Branch: stable/mitaka

commit 0c46a99b381c448082bbdc4e82e2a55121b2384e
Author: Kevin Carter <email address hidden>
Date: Tue May 3 19:57:55 2016 -0500

    Integrated updates after the multi-distro changes

    * Corrected the repo server log directory. This change updates the repos
      server log directory which was being incorrectly indexed. The old container
      build process would create a service log directory based on "properties".
      The "properties" log directory is unused or is re-defined by the various
      service roles. This change simply corrects the log paths which the
      `rsyslog_client` role uses to ensure all logs are shipped to the right
      places.
    * Add a log directory creation task to the utility container. This play
      incorrectly assumed that the log directory based on "properties" would
      be automatically created. This update simply makes that more explicit.
    * paramiko has been pinned to match global requirements and use a version
      <2. This will ressolve runtime issues in Newton with an unbound
      requirements.

    Closes-Bug: #1576755
    Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
    Signed-off-by: Kevin Carter <email address hidden>
    (cherry picked from commit e971e15997aab2c9d8d1e0cf106a67ee703a3409)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/312588
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=b230ae12ab5bf20c1420168b80a1c2cd327e7304
Submitter: Jenkins
Branch: kilo

commit b230ae12ab5bf20c1420168b80a1c2cd327e7304
Author: Kevin Carter <email address hidden>
Date: Wed May 4 10:09:33 2016 -0500

    Cap paramiko to <2

    paramiko has been pinned to match global requirements and use a version
    <2. This change ensures we don't have runtime issues in nova.

    Change-Id: I6b122d749ad128a4ebc94029151c797867a3b872
    Closes-Bug: #1576755
    Signed-off-by: Kevin Carter <email address hidden>

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/312587
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=77b1c7c4efdbe5b00802569ebb1d0f5ba908184d
Submitter: Jenkins
Branch: liberty

commit 77b1c7c4efdbe5b00802569ebb1d0f5ba908184d
Author: Kevin Carter <email address hidden>
Date: Wed May 4 10:09:33 2016 -0500

    Cap paramiko to <2

    paramiko has been pinned to match global requirements and use a version
    <2. This change ensures we don't have runtime issues in nova.

    Change-Id: I6b122d749ad128a4ebc94029151c797867a3b872
    Closes-Bug: #1576755
    Signed-off-by: Kevin Carter <email address hidden>

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.15 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 12.0.12 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 13.1.0 release.

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/openstack-ansible 14.0.0.0b1 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 14.0.0.0b2 development milestone.