grenade job fail due to missing memcache module
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Identity (keystone) |
Invalid
|
Undecided
|
Unassigned | ||
devstack |
New
|
Undecided
|
Unassigned |
Bug Description
We saw it quite often recently that grenade jobs are failing due to missing memcache module while keystone is starting.
Error message:
2019-03-29 00:22:39.361 | + lib/keystone:
2019-03-29 00:22:41.201 | [01;31mCRITICAL keystone [[00;36m-[01;31m] [01;35m
2019-03-29 00:22:41.201 | ERROR keystone [01;35m
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m sys.exit(main())
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m cli.main(
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m CONF.command.
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m klass = cls()
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m self.bootstrapper = bootstrap.
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m backends.
2019-03-29 00:22:41.201 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m cache.configure
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/opt/stack/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m cache.configure
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m '%s.' % conf.cache.
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m "%sreplace_
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m _config_prefix
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m for key in config_dict
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m super(MemcacheArgs, self)._
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m self._imports()
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m File "/usr/local/
2019-03-29 00:22:41.202 | ERROR keystone [01;35m[00m import memcache # noqa
2019-03-29 00:22:41.202 | ERROR keystone [01;35m
Logstash query which I used to find similar examples: http://
Looks that it happens at least for 10 runs in last 7 days.
Changed in keystone: | |
status: | New → Confirmed |
importance: | Undecided → Critical |
Changed in keystone: | |
status: | Confirmed → Invalid |
This is an issue in devstack. You can see python-memcached is installed using python2.7:
http:// logs.openstack. org/47/ 641747/ 8/check/ neutron- grenade- dvr-multinode/ f6fe13c/ logs/old/ devstacklog. txt#_2019- 03-29_06_ 15_08_086
While keystone is installed with python3:
http:// logs.openstack. org/47/ 641747/ 8/check/ neutron- grenade- dvr-multinode/ f6fe13c/ logs/old/ devstacklog. txt#_2019- 03-29_06_ 15_13_680
For some reason the check to see whether it is compatible with python3 is failing:
http:// logs.openstack. org/47/ 641747/ 8/check/ neutron- grenade- dvr-multinode/ f6fe13c/ logs/old/ devstacklog. txt#_2019- 03-29_06_ 15_07_676
Even though the metadata is correct on pypi:
https:/ /pypi.org/ project/ python- memcached/
The same problem is occurring for other packages too, so if it wasn't keystone it probably would have been something else:
http:// logs.openstack. org/47/ 641747/ 8/check/ neutron- grenade- dvr-multinode/ f6fe13c/ logs/old/ devstacklog. txt#_2019- 03-29_06_ 12_59_809