warlock 1.3.0 breaks a few gates

[Steve Martinelli]

Our functional test functional.tests.image.v2.test_image.ImageTests.test_image_unset is now failing:

Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
  File "functional/tests/image/v2/test_image.py", line 71, in test_image_unset
self.openstack('image set --tag 01 ' + self.NAME)
  File "functional/common/test.py", line 53, in openstack
return execute('openstack ' + cmd, fail_ok=fail_ok)
  File "functional/common/test.py", line 42, in execute result_err)
tempest.lib.exceptions.CommandFailed: Command 'openstack image set --tag 01 7fbbb8c5da634c54aea88473e4e3c16b' returned non-zero exit status 1.
stdout:

stderr:
WARNING: openstackclient.common.utils is deprecated and will be removed after Jun 2017. Please use osc_lib.utils
400 Bad Request
Invalid JSON pointer for this resource: '/tags/0'
(HTTP 400)

[Steve Martinelli (stevemar)]

warlock 1.3.0 was released over the weekend and looks like the likely root cause https://pypi.python.org/pypi/warlock

[Steve Martinelli (stevemar)]

we can use https://bugs.launchpad.net/python-glanceclient/+bug/1596644 to track this as well

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/334531
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=044a46ed5f040020cf80adc1ace987802344e87d
Submitter: Jenkins
Branch: master

commit 044a46ed5f040020cf80adc1ace987802344e87d
Author: Steve Martinelli <email address hidden>
Date: Mon Jun 27 11:26:18 2016 -0400

    skip image tag tests

    the command ``image set tag <tag> <image>`` is now failing
    skip the test so our gate isn't broken.

    Change-Id: I05e415b212e76aef62c1b0b966863573eff338e9
    Partial-Bug: #1596573

[Steve Martinelli (stevemar)]

See https://review.openstack.org/#/c/334613/ for a requirements patch

[Steve Martinelli (stevemar)]

glanceclient unit tests fail with:

======================================================================
FAIL: glanceclient.tests.unit.v2.test_metadefs_objects.TestObjectController.test_update_object
tags: worker-2
----------------------------------------------------------------------
Traceback (most recent call last):
  File "glanceclient/tests/unit/v2/test_metadefs_objects.py", line 298, in test_update_object
    obj = self.controller.update(NAMESPACE1, OBJECT1, **properties)
  File "glanceclient/v2/metadefs.py", line 343, in update
    obj = self.get(namespace, object_name)
  File "glanceclient/v2/metadefs.py", line 367, in get
    return self.model(**body)
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/warlock/core.py", line 35, in __init__
    base_class.__init__(self, *args, **kwargs)
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/warlock/model.py", line 34, in __init__
    self.validate(d)
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/warlock/model.py", line 141, in validate
    jsonschema.validate(obj, self.schema, resolver=self.resolver)
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/validators.py", line 478, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/validators.py", line 122, in validate
    for error in self.iter_errors(*args, **kwargs):
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/validators.py", line 98, in iter_errors
    for error in errors:
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/_validators.py", line 291, in properties_draft4
    schema_path=property,
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/validators.py", line 114, in descend
    for error in self.iter_errors(instance, schema):
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/validators.py", line 98, in iter_errors
    for error in errors:
  File "/opt/stack/python-glanceclient/.tox/py27/local/lib/python2.7/site-packages/jsonschema/_validators.py", line 195, in ref
    with validator.resolver.resolving(ref) as resolved:
AttributeError: type object 'SchemaBasedModel' has no attribute 'resolving'

[Steve Martinelli (stevemar)]

One funny thing I noticed was that when I tested this manually, it seemed fine:

(war) python-glanceclient$ glance image-tag-update 14e6fc40-f236-4e1f-86da-d3bda0af0baa tttt
+--------------------------------------+-------------------+
| ID | Tags |
+--------------------------------------+-------------------+
| 14e6fc40-f236-4e1f-86da-d3bda0af0baa | [u'tttt', u't5t'] |
+--------------------------------------+-------------------+
(war) python-glanceclient$ glance image-show 14e6fc40-f236-4e1f-86da-d3bda0af0baa
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| checksum | eb9139e4942121f22bbc2afc0400b2a4 |
| container_format | ami |
| created_at | 2016-06-27T18:12:29Z |
| disk_format | ami |
| id | 14e6fc40-f236-4e1f-86da-d3bda0af0baa |
| kernel_id | d5649c23-50c5-4db1-af17-b031cfab535d |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros-0.3.4-x86_64-uec |
| owner | 1037fcd24a704897966647fb20b12836 |
| protected | False |
| ramdisk_id | aa9d4c45-e391-4d8d-b2d5-7224749547c8 |
| size | 25165824 |
| status | active |
| tags | ["tttt", "t5t"] |
| updated_at | 2016-06-27T18:23:47Z |
| virtual_size | None |
| visibility | public |
+------------------+--------------------------------------+
(war) $ pip freeze
Babel==2.3.4
debtcollector==1.5.0
funcsigs==1.0.2
functools32==3.2.3.post2
iso8601==0.1.11
jsonpatch==1.14
jsonpointer==1.10
jsonschema==2.5.1
keystoneauth1==2.8.0
monotonic==1.1
msgpack-python==0.4.7
netaddr==0.7.18
netifaces==0.10.4
oslo.config==3.11.0
oslo.i18n==3.7.0
oslo.serialization==2.9.0
oslo.utils==3.13.0
pbr==1.10.0
positional==1.1.1
prettytable==0.7.2
-e git+http://github.com/openstack/python-glanceclient@d168bd73eb5965e31483aec041e6c9a1047ad540#egg=python_glanceclient
python-keystoneclient==3.1.0
pytz==2016.4
requests==2.10.0
six==1.10.0
stevedore==1.15.0
warlock==1.3.0
wrapt==1.10.8

[Travis Tripp (travis-tripp)]

No fix needed for Horizon if glanceclient is fixed, but this does affect Horizon.

[Cyril Roelandt (cyril-roelandt)]

Confirmed for glance: https://bugs.launchpad.net/glance/+bug/1596598

See Model.validate() warlock/model.py: if self.resolver is set, it is passed to jsonschema.validate(). Here is the diff:

@@ -130,6 +137,9 @@ class Model(dict):
     def validate(self, obj):
         """Apply a JSON schema to an object"""
         try:
- jsonschema.validate(obj, self.schema)
+ if self.resolver is not None:
+ jsonschema.validate(obj, self.schema, resolver=self.resolver)
+ else:
+ jsonschema.validate(obj, self.schema)
         except jsonschema.ValidationError as exc:
             raise exceptions.ValidationError(str(exc))

Here, self.resolver is a SchemaBasedModel instance. We should probably write a "resolving" method for this class, WDYT?

[Travis Tripp (travis-tripp)]

Duplicate of https://bugs.launchpad.net/searchlight/+bug/1596598, so removing searchlight from that one.

[Sabari Murugesan (smurugesan)]

Warlock 1.3.0 introduced a new kwarg 'resolver' in model_factory method
but changed its position with the 'base_class' kwarg. Since we
were calling the model_factory method with positional arg, this
broke the model creation.

I uploaded https://review.openstack.org/#/c/338630/ for glanceclient.

[Steve Martinelli (stevemar)]

We are now using upper constraints in OSC functional tests, so we are OK for now.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/338630
Committed: https://git.openstack.org/cgit/openstack/python-glanceclient/commit/?id=d7db97c92605a11e312d17cdcdd8ac363bc10924
Submitter: Jenkins
Branch: master

commit d7db97c92605a11e312d17cdcdd8ac363bc10924
Author: Sabari Kumar Murugesan <email address hidden>
Date: Wed Jul 6 18:02:28 2016 -0700

    Fix warlock model creation

    Commands like glance md-namespace-show <namespace> fail because
    of a breaking change in warlock 1.3.0's model creation factory
    method.

    Warlock introduced a new kwarg 'resolver' in model_factory method
    but changed its position with the 'base_class' kwarg. Since we
    were calling the model_factory method with positional arg, this
    broke the model creation.

    Closes-Bug: #1596573

    Change-Id: Ic7821f4fdb1b752e0c7ed2bc486299a06bf485c1

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-glanceclient 2.4.0 release.

[Jon Gjengset (jonhoo)]

Unless I'm mistaken, python-glanceclient still carries a warlock!=1.3.0 bound in its requirements.txt: https://git.openstack.org/cgit/openstack/python-glanceclient/tree/requirements.txt. Shouldn't this now be removed?

[Gary W. Smith (gary-w-smith)]

Removing horizon. As indicated in the comments above, horizon was affected by the broken glanceclient, but is no longer affected due to the fix.