Activity log for bug #1894206

Date Who What changed Old value New value Message
2020-09-04 09:38:33 Chris MacNaughton bug added bug
2020-09-04 09:38:43 Chris MacNaughton bug task added charm-mysql-router
2020-09-04 09:39:42 Chris MacNaughton description 2020-09-04 09:28:48 DEBUG shared-db-relation-changed ERROR: (pymysql.err.OperationalError) (1045, "Access denied for user 'heat'@'10.1.12.19' (using password: YES)") 2020-09-04 09:28:48 DEBUG shared-db-relation-changed (Background on this error at: http://sqlalche.me/e/e3q8) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed Traceback (most recent call last): 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 452, in <module> 2020-09-04 09:28:48 DEBUG shared-db-relation-changed main() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 445, in main 2020-09-04 09:28:48 DEBUG shared-db-relation-changed hooks.execute(sys.argv) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/hookenv.py", line 945, in execute 2020-09-04 09:28:48 DEBUG shared-db-relation-changed self._hooks[hook_name]() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 723, in wrapped_f 2020-09-04 09:28:48 DEBUG shared-db-relation-changed return restart_on_change_helper( 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 747, in restart_on_change_helper 2020-09-04 09:28:48 DEBUG shared-db-relation-changed r = lambda_f() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 724, in <lambda> 2020-09-04 09:28:48 DEBUG shared-db-relation-changed (lambda: f(*args, **kwargs)), restart_map, stopstart, 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 268, in db_changed 2020-09-04 09:28:48 DEBUG shared-db-relation-changed migrate_database() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/heat_utils.py", line 312, in migrate_database 2020-09-04 09:28:48 DEBUG shared-db-relation-changed check_call(['heat-manage', 'db_sync']) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/usr/lib/python3.8/subprocess.py", line 364, in check_call 2020-09-04 09:28:48 DEBUG shared-db-relation-changed raise CalledProcessError(retcode, cmd) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed subprocess.CalledProcessError: Command '['heat-manage', 'db_sync']' returned non-zero exit status 1. 2020-09-04 09:28:49 ERROR juju.worker.uniter.operation runhook.go:136 hook "shared-db-relation-changed" (via explicit, bespoke hook script) failed: exit status 1 On a Focal Ussuri deploy, semi-regularly (one in three on my latest deploy), heat goes into hook errors during it's shared-db-relation-changed hook. 2020-09-04 09:28:48 DEBUG shared-db-relation-changed ERROR: (pymysql.err.OperationalError) (1045, "Access denied for user 'heat'@'10.1.12.19' (using password: YES)") 2020-09-04 09:28:48 DEBUG shared-db-relation-changed (Background on this error at: http://sqlalche.me/e/e3q8) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed Traceback (most recent call last): 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 452, in <module> 2020-09-04 09:28:48 DEBUG shared-db-relation-changed main() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 445, in main 2020-09-04 09:28:48 DEBUG shared-db-relation-changed hooks.execute(sys.argv) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/hookenv.py", line 945, in execute 2020-09-04 09:28:48 DEBUG shared-db-relation-changed self._hooks[hook_name]() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 723, in wrapped_f 2020-09-04 09:28:48 DEBUG shared-db-relation-changed return restart_on_change_helper( 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 747, in restart_on_change_helper 2020-09-04 09:28:48 DEBUG shared-db-relation-changed r = lambda_f() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/charmhelpers/core/host.py", line 724, in <lambda> 2020-09-04 09:28:48 DEBUG shared-db-relation-changed (lambda: f(*args, **kwargs)), restart_map, stopstart, 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/shared-db-relation-changed", line 268, in db_changed 2020-09-04 09:28:48 DEBUG shared-db-relation-changed migrate_database() 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/var/lib/juju/agents/unit-heat-1/charm/hooks/heat_utils.py", line 312, in migrate_database 2020-09-04 09:28:48 DEBUG shared-db-relation-changed check_call(['heat-manage', 'db_sync']) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed File "/usr/lib/python3.8/subprocess.py", line 364, in check_call 2020-09-04 09:28:48 DEBUG shared-db-relation-changed raise CalledProcessError(retcode, cmd) 2020-09-04 09:28:48 DEBUG shared-db-relation-changed subprocess.CalledProcessError: Command '['heat-manage', 'db_sync']' returned non-zero exit status 1. 2020-09-04 09:28:49 ERROR juju.worker.uniter.operation runhook.go:136 hook "shared-db-relation-changed" (via explicit, bespoke hook script) failed: exit status 1