[SRU] Alarms fail on Rocky
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Aodh |
Fix Released
|
Undecided
|
Corey Bryant | ||
OpenStack AODH Charm |
Invalid
|
Undecided
|
Unassigned | ||
Ubuntu Cloud Archive |
Fix Released
|
Undecided
|
Unassigned | ||
Rocky |
Fix Released
|
Undecided
|
Unassigned | ||
oslo.i18n |
Confirmed
|
Medium
|
Unassigned | ||
aodh (Ubuntu) |
Fix Released
|
Undecided
|
Unassigned | ||
Cosmic |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Impact]
In a rocky deployment aodh is failing to raise alarms due to what seems like a locale issue. If I purge packages:
apt-get purge aodh-api aodh-evaluator aodh-expirer aodh-notifier aodh-listener python3-aodh libapache2-
Then disable the AodhCharmRocky class and rerun install so that it pulls in the py2 packages then things start working.
The error from /var/log/
http://
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR oslo_db.
ERROR aodh.evaluator [-] alarm state update failed: oslo_db.
ERROR aodh.evaluator Traceback (most recent call last):
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator context)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator cursor.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator query = self.mogrify(query, args)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator query = query % self._escape_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return dict((key, conn.literal(val)) for (key, val) in args.items())
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return dict((key, conn.literal(val)) for (key, val) in args.items())
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return self.escape(obj, self.encoders)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return "'" + self.escape_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return converters.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return value.translate
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator self.has_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator fallback=True)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator mofiles = find(domain, localedir, languages, all=True)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator for nelang in _expand_lang(lang):
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator loc = locale.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator code = localename.lower()
ERROR aodh.evaluator AttributeError: 'list' object has no attribute 'lower'
ERROR aodh.evaluator
ERROR aodh.evaluator The above exception was the direct cause of the following exception:
ERROR aodh.evaluator
ERROR aodh.evaluator Traceback (most recent call last):
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator self._storage_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator alarm.as_dict())
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator update_op.exec_()
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator self._do_exec()
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator self._execute_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator mapper=self.mapper)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator bind, close_with_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return meth(self, multiparams, params)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return connection.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator compiled_sql, distilled_params
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator context)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator util.raise_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator reraise(
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator raise value.with_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator context)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator cursor.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator query = self.mogrify(query, args)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator query = query % self._escape_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return dict((key, conn.literal(val)) for (key, val) in args.items())
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return dict((key, conn.literal(val)) for (key, val) in args.items())
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return self.escape(obj, self.encoders)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return "'" + self.escape_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return converters.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator return value.translate
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator self.has_
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator fallback=True)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator mofiles = find(domain, localedir, languages, all=True)
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator for nelang in _expand_lang(lang):
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator loc = locale.
ERROR aodh.evaluator File "/usr/lib/
ERROR aodh.evaluator code = localename.lower()
ERROR aodh.evaluator oslo_db.
ERROR aodh.evaluator
[Test Case]
1) Deploy openstack with aodh (we're deploying with the development versions of the openstack charms which deploy py3 payloads)
2) Once deployed/configured create an alarm with:
openstack alarm create \
--type event \
--name instance_off \
--description 'Instance powered OFF' \
--event-type "compute.
--enable True \
--query "traits.
--alarm-action 'log://' \
--ok-action 'log://' \
--insufficien
3) create a server instance
4) power server instance off to trigger alarm
5) ensure alarm is successfully raised with no traceback (see /var/log/
[Regression Potential]
The fix is a minimal fix that should result in low regression potential.
description: | updated |
summary: |
- Alarms fail on Rocky + [SRU] Alarms fail on Rocky |
Changed in cloud-archive: | |
status: | New → Fix Committed |
Changed in charm-aodh: | |
status: | New → Invalid |
Changed in oslo.i18n: | |
status: | New → Invalid |
The problem looks to be that pymysql/ converters. py is calling oslo_i18n's translate() method with a list argument (named _escape_table). translate expects desired_locale (string) to be passed in, not a list.
1) The call to translate is in /usr/lib/ python3/ dist-packages/ pymysql/ converters. py: translate( _escape_ table).
value.
2) This calls into the translate() function in /usr/lib/ python3/ dist-packages/ oslo_i18n/ _message. py
3) Looking at the argument that gets passed to 'def translate(self, desired_ locale= None)' looks like the _escape_table:
desired_ locale= ['\\0', '\x01', '\x02', '\x03', '\x04', '\x05', '\x06', '\x07', '\x08', '\t', '\\n', '\x0b', '\x0c', '\\r', '\x0e', '\x0f', '
\x10', '\x11', '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x18', '\x19', '\\Z', '\x1b', '\x1c', '\x1d', '\x1e', '\x1f', ' ', '!', '\\"', '#', '$', '%', '&', "\\'", '(', ')', '*', '+', ',', '-', '.',
'/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?', '@', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W'
, 'X', 'Y', 'Z', '[', '\\\\', ']', '^', '_', '`', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~', '\x7
f']
4) languages ends up as a nested list in /usr/lib/ python3/ dist-packages/ oslo_i18n/ _message. py: translation( domain,
localedir= locale_ dir,
languages= [desired_ locale] ,
fallback= True)
lang = gettext.
5) languages are looped through in /usr/lib/ python3. 6/gettext. py:
for lang in languages:
for nelang in _expand_lang(lang):
6) _expand_lang calls local.normalize with a list (_escape_table) and that causes the "AttributeError: 'list' object has no attribute 'lower'": normalize( loc)
locale.
The normalize() call should instead be getting called with a locale string, something such as: locale. normalize( 'EN_US. utf-8') .