CI: periodic jobs fail because of "Data too long for column.." in heat-engine

[Sagi (Sergey) Shnaidman]

From undercloud /var/log/heat/heat-engine.log:

DataError: (1406, u"Data too long for column 'resource_properties' at row 1")

2016-09-05 07:10:39.950 30564 INFO heat.engine.resource [req-a0cf7868-05ea-4f32-83f8-dd4dd9835238 5d11b9ee681b4ef0859d4745613e9bc6 80026f267bfe431da138fb77d7b09f20 - - -] creating TemplateResource "1" Stack "ove
rcloud-Controller-zvvkan24kgpc" [97d3ae7e-213a-4023-984a-41fabbcca64f]

2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters [req-a0cf7868-05ea-4f32-83f8-dd4dd9835238 5d11b9ee681b4ef0859d4745613e9bc6 80026f267bfe431da138fb77d7b09f20 - - -] DBAPIError exception wrapped from (pymysql.err.DataError) (1406, u"Data too long for column 'resource_properties' at row 1") [SQL: u'INSERT INTO event (created_at, updated_at, stack_id, uuid, resource_action, resource_status, resource_name, physical_resource_id, resource_status_reason, resource_type, resource_properties) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: (datetime.datetime(2016, 9, 5, 7, 10, 40, 56220), None, '97d3ae7e-213a-4023-984a-41fabbcca64f', 'f5db0449-1748-4b33-bf39-8c852cd6fd25', 'CREATE', 'IN_PROGRESS', u'1', None, 'state changed', u'OS::TripleO::Controller', '\x80\x02}q\x01...

[really big string]

2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters context)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 146, in execute
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters result = self._query(query)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 296, in _query
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters conn.query(q)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 781, in query
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 942, in _read_query_result
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters result.read()
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1138, in read
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet()
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 906, in _read_packet
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters packet.check_error()
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 367, in check_error
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters err.raise_mysql_exception(self._data)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters _check_mysql_exception(errinfo)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters File "/usr/lib/python2.7/site-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters raise errorclass(errno, errorvalue)
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'resource_properties' at row 1")
2016-09-05 07:10:40.082 30564 ERROR oslo_db.sqlalchemy.exc_filters

[Sagi (Sergey) Shnaidman (sshnaidm)]

String that is supposed to be inserted into DB is attached

[Thomas Herve (therve)]

This really ought to be have been fixed by https://bugs.launchpad.net/heat/+bug/1552431. It's be interesting to see the table schema or if there is something going on with the db setup. We limit right now at 65535, and the linked string is only 28k, so ought to fit in that. Are you sure that's the complete string? Maybe the constant we use is too big.