So still not entirely gold. I got this today:
Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: e1904824-33e7-40a4-9938-8bbab39e0b7d │·
{u'body': {u'exception': u'(pymysql.err.OperationalError) (2006, "MySQL server has gone away (error(32, \'Broken pipe\'))") [SQL: u\'INSERT INTO `Queues` (project, name, metadata) VALUES (│·
%(project)s, %(name)s, %(metadata)s)\'] [parameters: {\'project\': u\'5ac518c14589482da9d97c4ca4a58b6a\', \'name\': u\'d9eecd49-a283-4bcb-86f9-bfd2a33e9857\', \'metadata\': bytearray(b\'{}│·
\')}]', u'error': u'Unexpected error.'}, u'headers': {u'status': 500}, u'request': {u'action': u'queue_create', u'body': {u'queue_name': u'd9eecd49-a283-4bcb-86f9-bfd2a33e9857'}, u'api': u│·
'v2', u'headers': {u'Client-ID': u'a355b207-54e7-4fcf-8468-94175313ba5f', u'X-Project-ID': u'5ac518c14589482da9d97c4ca4a58b6a'}}}
And my setting was already changed server-side:
MariaDB [(none)]> show variables like 'max_allowed_packet';
+--------------------+----------+
| Variable_name | Value |
+--------------------+----------+
| max_allowed_packet | 33554432 |
+--------------------+----------+
1 row in set (0.00 sec)
Maybe the size is not enough?
The zaqar.log for today's failure is:
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver tmp = target(*args, **kwargs)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/zaqar/storage/sqlalchemy/queues.py", line 66, in get_metadata
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver queue = self.driver.run(sel).fetchone()
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/zaqar/storage/sqlalchemy/driver.py", line 81, in run
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver return self.connection.execute(*args, **kwargs)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1046, in execute
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver bind, close_with_result=True).execute(clause, params or {})
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver return meth(self, multiparams, params)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver return connection._execute_clauseelement(self, multiparams, params)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver compiled_sql, distilled_params
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver context)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver exc_info
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver reraise(type(exception), exception, tb=exc_tb)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver context)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver cursor.execute(statement, parameters)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver result = self._query(query)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver conn.query(q)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 840, in query
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver self._execute_command(COMMAND.COM_QUERY, sql)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1064, in _execute_command
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver self._write_bytes(packet)
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver File "/usr/lib/python2.7/site-packages/pymysql/connections.py", line 1016, in _write_bytes
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver raise err.OperationalError(2006, "MySQL server has gone away (%r)" % (e,))
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport.wsgi.driver OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (error(32, 'Broken pipe'))") [SQL: u'SEL
ECT `Queues`.metadata \nFROM `Queues` \nWHERE `Queues`.project = %(project_1)s AND `Queues`.name = %(name_1)s'] [parameters: {u'name_1': 'd9eecd49-a283-4bcb-86f9-bfd2a33e9857', u'project_1
': u'5ac518c14589482da9d97c4ca4a58b6a'}]
So still not entirely gold. I got this today: validations. v1.check_ pre_deployment_ validations. Execution ID: e1904824- 33e7-40a4- 9938-8bbab39e0b 7d │· err.Operational Error) (2006, "MySQL server has gone away (error(32, \'Broken pipe\'))") [SQL: u\'INSERT INTO `Queues` (project, name, metadata) VALUES (│· 482da9d97c4ca4a 58b6a\' , \'name\': u\'d9eecd49- a283-4bcb- 86f9-bfd2a33e98 57\', \'metadata\': bytearray(b\'{}│· a283-4bcb- 86f9-bfd2a33e98 57'}, u'api': u│· 54e7-4fcf- 8468-94175313ba 5f', u'X-Project-ID': u'5ac518c145894 82da9d97c4ca4a5 8b6a'}} }
Started Mistral Workflow tripleo.
{u'body': {u'exception': u'(pymysql.
%(project)s, %(name)s, %(metadata)s)\'] [parameters: {\'project\': u\'5ac518c14589
\')}]', u'error': u'Unexpected error.'}, u'headers': {u'status': 500}, u'request': {u'action': u'queue_create', u'body': {u'queue_name': u'd9eecd49-
'v2', u'headers': {u'Client-ID': u'a355b207-
And my setting was already changed server-side: packet' ; ------- ------- +------ ----+ ------- ------- +------ ----+ ------- ------- +------ ----+
MariaDB [(none)]> show variables like 'max_allowed_
+------
| Variable_name | Value |
+------
| max_allowed_packet | 33554432 |
+------
1 row in set (0.00 sec)
Maybe the size is not enough?
The zaqar.log for today's failure is: .wsgi.driver tmp = target(*args, **kwargs) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ zaqar/storage/ sqlalchemy/ queues. py", line 66, in get_metadata .wsgi.driver queue = self.driver. run(sel) .fetchone( ) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ zaqar/storage/ sqlalchemy/ driver. py", line 81, in run .wsgi.driver return self.connection .execute( *args, **kwargs) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ orm/session. py", line 1046, in execute .wsgi.driver bind, close_with_ result= True).execute( clause, params or {}) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 914, in execute .wsgi.driver return meth(self, multiparams, params) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ sql/elements. py", line 323, in _execute_ on_connection .wsgi.driver return connection. _execute_ clauseelement( self, multiparams, params) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1010, in _execute_ clauseelement .wsgi.driver compiled_sql, distilled_params .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1146, in _execute_context .wsgi.driver context) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1341, in _handle_ dbapi_exception .wsgi.driver exc_info .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ util/compat. py", line 200, in raise_from_cause .wsgi.driver reraise( type(exception) , exception, tb=exc_tb) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ base.py" , line 1139, in _execute_context .wsgi.driver context) .wsgi.driver File "/usr/lib64/ python2. 7/site- packages/ sqlalchemy/ engine/ default. py", line 450, in do_execute .wsgi.driver cursor. execute( statement, parameters) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ pymysql/ cursors. py", line 166, in execute .wsgi.driver result = self._query(query) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ pymysql/ cursors. py", line 322, in _query .wsgi.driver conn.query(q) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ pymysql/ connections. py", line 840, in query .wsgi.driver self._execute_ command( COMMAND. COM_QUERY, sql) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ pymysql/ connections. py", line 1064, in _execute_command .wsgi.driver self._write_ bytes(packet) .wsgi.driver File "/usr/lib/ python2. 7/site- packages/ pymysql/ connections. py", line 1016, in _write_bytes .wsgi.driver raise err.Operational Error(2006, "MySQL server has gone away (%r)" % (e,)) .wsgi.driver OperationalError: (pymysql. err.Operational Error) (2006, "MySQL server has gone away (error(32, 'Broken pipe'))") [SQL: u'SEL a283-4bcb- 86f9-bfd2a33e98 57', u'project_1 82da9d97c4ca4a5 8b6a'}]
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
2017-05-25 08:16:57.149 23412 ERROR zaqar.transport
ECT `Queues`.metadata \nFROM `Queues` \nWHERE `Queues`.project = %(project_1)s AND `Queues`.name = %(name_1)s'] [parameters: {u'name_1': 'd9eecd49-
': u'5ac518c145894