Barbican test failures with pykmip 0.4.1 in master and stable/liberty

[Arun Kant]

In Barbican, 3 unit test fails in test_kmip.py when pykmip 0.4.1 version is used instead of 0.4.0. Our internal gate picked up that version and saw the test failures.

pykmip 0.4.1 version is a valid version as per test-requirements dependency i.e. "pykmip>=0.4.0"

https://github.com/openstack/barbican/blob/stable/liberty/test-requirements.txt#L8.

This issue is there with stable/liberty as well as in current master branch. Openstack gates are passing as it still getting 0.4.0 version of pykmip. This can be verified in https://review.openstack.org/#/c/252578/.

This error can be reproduced locally by setting pykmip in test-requirements to 0.4.1 version "pykmip==0.4.1". When same setup has pykmip version pinned to 0.4.0 version, tests passes (as seen in openstack gate builds).

We might want to exclude 0.4.1 version in barbican test-requirements and global requirements.

Failure 1:
barbican.tests.plugin.test_kmip.WhenTestingKMIPSecretStore.test_credential_StringException: Empty attachments:
....
....

Traceback (most recent call last):
  File "barbican/tests/plugin/test_kmip.py", line 862, in test_credential
    actual_credential.credential_type.value)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != <CredentialType.USERNAME_AND_PASSWORD: 1>

Failure 2:

barbican.tests.plugin.test_kmip.WhenTestingKMIPSecretStore.test_store_certificate_secret_assert_called_no_key_spec_StringException: Empty attachments:
....
....

Traceback (most recent call last):
  File "barbican/tests/utils.py", line 327, in wrapper
    return func(self, *test_args, **test_kwargs)
  File "barbican/tests/plugin/test_kmip.py", line 606, in test_store_certificate_secret_assert_called
    actual_secret.certificate_type.value)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != <CertificateTypeEnum.X_509: 1>

Failure 3:

barbican.tests.plugin.test_kmip.WhenTestingKMIPSecretStore.test_store_certificate_secret_assert_called_rsa_StringException: Empty attachments:
....
....

Traceback (most recent call last):
  File "barbican/tests/utils.py", line 327, in wrapper
    return func(self, *test_args, **test_kwargs)
  File "barbican/tests/plugin/test_kmip.py", line 606, in test_store_certificate_secret_assert_called
    actual_secret.certificate_type.value)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/jenkins/workspace/gate-barbican-python27/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 1 != <CertificateTypeEnum.X_509: 1>

[Arun Kant (arukant)]

Looks like the issue is resolved in master with this patch. https://github.com/openstack/barbican/commit/c6e806bb7824e370a60df623a2b874ca0f8239af

And confirmed that stable/liberty branch has that issue. So may be above commit needs to backport.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/255368

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/255368
Committed: https://git.openstack.org/cgit/openstack/barbican/commit/?id=3ee60c62f945b5f86f62c14ee204d88584546748
Submitter: Jenkins
Branch: stable/liberty

commit 3ee60c62f945b5f86f62c14ee204d88584546748
Author: Peter Hamilton <email address hidden>
Date: Fri Dec 4 08:49:39 2015 -0500

    Remove low-level PyKMIP test asserts

    This change removes two low-level asserts that test the internals of
    PyKMIP core. The referenced portions of PyKMIP have been modified in the
    latest release. Once the latest release has been published, these asserts
    will be added back in, modified to work properly with the new version of
    PyKMIP.

    Partial-Bug: #1522856
    Closes-Bug: #1524132
    Change-Id: I50b9a4ad935d6458c5dc42f8184a724c78a28a59
    (cherry picked from commit c6e806bb7824e370a60df623a2b874ca0f8239af)