TypeError: unhashable type: 'list' with redis driver

[Ben Nemec]

This was reported on the mailing list as affecting ceilometer. It appears that the fix for the memcached bug (https://github.com/openstack/oslo.cache/commit/685a05edfef24cb2300df24aa3417ad0ecd2c2c2) broke the redis backend because it can't handle a list for the url argument.

Nov 04 13:12:28.735348 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base Traceback (most recent call last):
Nov 04 13:12:28.735494 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/opt/stack/new/ceilometer/ceilometer/pipeline/base.py", line 125, in __init__
Nov 04 13:12:28.735629 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base self.publishers.append(publisher_manager.get(p))
Nov 04 13:12:28.735770 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/opt/stack/new/ceilometer/ceilometer/pipeline/base.py", line 180, in get
Nov 04 13:12:28.735915 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base 'ceilometer.%s.publisher' % self._purpose)
Nov 04 13:12:28.736070 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/opt/stack/new/ceilometer/ceilometer/publisher/__init__.py", line 36, in get_publisher
Nov 04 13:12:28.736205 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base return loaded_driver.driver(conf, parse_result)
Nov 04 13:12:28.736363 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/opt/stack/new/ceilometer/ceilometer/publisher/gnocchi.py", line 228, in __init__
Nov 04 13:12:28.736705 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base conf, cache_region)
Nov 04 13:12:28.736857 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/oslo_cache/core.py", line 225, in configure_cache_region
Nov 04 13:12:28.737014 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base '%s.' % conf.cache.config_prefix)
Nov 04 13:12:28.737148 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 591, in configure_from_config
Nov 04 13:12:28.737282 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base "%sreplace_existing_backend" % prefix, False),
Nov 04 13:12:28.737415 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/region.py", line 423, in configure
Nov 04 13:12:28.737564 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base _config_prefix
Nov 04 13:12:28.737697 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/api.py", line 89, in from_config_dict
Nov 04 13:12:28.737831 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base for key in config_dict
Nov 04 13:12:28.739015 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/backends/redis.py", line 109, in __init__
Nov 04 13:12:28.739168 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base self.client = self._create_client()
Nov 04 13:12:28.739306 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/dogpile/cache/backends/redis.py", line 129, in _create_client
Nov 04 13:12:28.739443 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base return redis.StrictRedis.from_url(**args)
Nov 04 13:12:28.739581 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 486, in from_url
Nov 04 13:12:28.739725 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base connection_pool = ConnectionPool.from_url(url, db=db, **kwargs)
Nov 04 13:12:28.739860 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 833, in from_url
Nov 04 13:12:28.740010 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base url = urlparse(url)
Nov 04 13:12:28.740145 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/lib/python2.7/urlparse.py", line 143, in urlparse
Nov 04 13:12:28.740290 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base tuple = urlsplit(url, scheme, allow_fragments)
Nov 04 13:12:28.740430 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base File "/usr/lib/python2.7/urlparse.py", line 176, in urlsplit
Nov 04 13:12:28.740563 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base cached = _parse_cache.get(key, None)
Nov 04 13:12:28.740700 ubuntu-xenial-vexxhost-sjc1-0000319443 ceilometer-agent-notification[12547]: 2018-11-04 13:12:28.730 13121 ERROR ceilometer.pipeline.base TypeError: unhashable type: 'list'

[Ben Nemec (bnemec)]

It looks like dogpile actually documents both list and string urls, presumably depending on the driver. Maybe we need to have some driver-specific logic in oslo.cache to handle this?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/615979
Committed: https://git.openstack.org/cgit/openstack/oslo.cache/commit/?id=356cdc13b6aac02c688dba4c050acfc795678ffa
Submitter: Zuul
Branch: master

commit 356cdc13b6aac02c688dba4c050acfc795678ffa
Author: Morgan Fainberg <email address hidden>
Date: Tue Nov 6 11:30:55 2018 -0800

    Only split url arg with memcache backend

    Only split the url arg with the memcache backend, the list form breaks
    the redis backend driver / library.

    Change-Id: Id0e1d9351a92ba862e48f088ff15d1f77b6105d9
    Closes-Bug: #1801967

[OpenStack Infra (hudson-openstack)]

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: master
Review: https://review.openstack.org/615960

[OpenStack Infra (hudson-openstack)]

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: stable/rocky
Review: https://review.openstack.org/615961

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.cache 1.31.2 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/624765

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/624765
Committed: https://git.openstack.org/cgit/openstack/oslo.cache/commit/?id=5f42092e2cba1d9e9711d819a2792dd264a887fd
Submitter: Zuul
Branch: stable/rocky

commit 5f42092e2cba1d9e9711d819a2792dd264a887fd
Author: Morgan Fainberg <email address hidden>
Date: Tue Nov 6 11:30:55 2018 -0800

    Only split url arg with memcache backend

    Only split the url arg with the memcache backend, the list form breaks
    the redis backend driver / library.

    Change-Id: Id0e1d9351a92ba862e48f088ff15d1f77b6105d9
    Closes-Bug: #1801967
    (cherry picked from commit 356cdc13b6aac02c688dba4c050acfc795678ffa)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.cache 1.30.3 release.