Mistral db-sync failure in CI jobs

Bug #1582651 reported by Sagi (Sergey) Shnaidman
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Steven Hardy

Bug Description

Designate was added as a hard requirement in mistral:
https://github.com/openstack/mistral/commit/ce3b188fc85b6aa1ca6416c161c8df11989fd406

It caused a few periodic jobs to fail:

Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: 'strategies for improved performance.' % expr)
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: Traceback (most recent call last):
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/bin/mistral-db-manage", line 10, in <module>
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: sys.exit(main())
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/db/sqlalchemy/migration/cli.py", line 129, in main
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: CONF.command.func(config, CONF.command.name)
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/db/sqlalchemy/migration/cli.py", line 70, in do_populate
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: action_manager.sync_db()
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/services/action_manager.py", line 83, in sync_db
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: register_action_classes()
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/services/action_manager.py", line 129, in register_action_classes
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: _register_dynamic_action_classes()
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/services/action_manager.py", line 88, in _register_dynamic_action_classes
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: for generator in generator_factory.all_generators():
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/actions/generator_factory.py", line 32, in all_generators
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: mod_action_cls = importutils.import_class(mod_cls_name)
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py", line 30, in import_class
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: __import__(mod_str)
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: File "/usr/lib/python2.7/site-packages/mistral/actions/openstack/actions.py", line 20, in <module>
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: from designateclient import client as designateclient
Notice: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]/returns: ImportError: No module named designateclient
Error: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]: Failed to call refresh: mistral-db-manage --config-file=/etc/mistral/mistral.conf populate returned 1 instead of one of [0]
Error: /Stage[main]/Mistral::Db::Sync/Exec[mistral-db-populate]: mistral-db-manage --config-file=/etc/mistral/mistral.conf populate returned 1 instead of one of [0]
Notice: /Stage[main]/Mistral::Engine/Service[mistral-engine]/ensure: ensure changed 'stopped' to 'running'

2016-05-17 06:31:57.795 | + echo 'puppet apply exited with exit code 6'
2016-05-17 06:31:57.795 | puppet apply exited with exit code 6
2016-05-17 06:31:57.795 | + '[' 6 '!=' 2 -a 6 '!=' 0 ']'
2016-05-17 06:31:57.795 | + exit 6
2016-05-17 06:31:57.795 | [2016-05-17 06:31:57,654] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 6]
2016-05-17 06:31:57.795 |
2016-05-17 06:31:57.795 | [2016-05-17 06:31:57,655] (os-refresh-config) [ERROR] Aborting...
2016-05-17 06:31:57.795 | Traceback (most recent call last):
2016-05-17 06:31:57.795 | File "<string>", line 1, in <module>
2016-05-17 06:31:57.795 | File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 971, in install
2016-05-17 06:31:57.796 | _run_orc(instack_env)
2016-05-17 06:31:57.796 | File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 849, in _run_orc
2016-05-17 06:31:57.796 | _run_live_command(args, instack_env, 'os-refresh-config')
2016-05-17 06:31:57.796 | File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 442, in _run_live_command
2016-05-17 06:31:57.796 | raise RuntimeError('%s failed. See log for details.' % name)
2016-05-17 06:31:57.796 | RuntimeError: os-refresh-config failed. See log for details.
2016-05-17 06:31:57.796 | Command 'instack-install-undercloud' returned non-zero exit status 1
2016-05-17 06:31:57.827 | + false
2016-05-17 06:31:57.827 | Transferred: sent 4196, received 45692 bytes, in 573.0 seconds

jobs:
http://logs.openstack.org/periodic/periodic-tripleo-ci-f22-upgrades/c9a3934/console.html
http://logs.openstack.org/periodic/periodic-tripleo-ci-f22-ha/43fa128/console.html
http://logs.openstack.org/periodic/periodic-tripleo-ci-f22-nonha/af84eae/console.html

Tags: ci mistral
Steven Hardy (shardy)
Changed in tripleo:
status: New → Triaged
importance: Undecided → High
milestone: none → newton-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to instack-undercloud (master)

Fix proposed to branch: master
Review: https://review.openstack.org/317398

Changed in tripleo:
assignee: nobody → Steven Hardy (shardy)
status: Triaged → In Progress
Revision history for this message
Steven Hardy (shardy) wrote :

Alternative solution is to add the dependency to the spec file:

https://review.gerrithub.io/#/c/276918/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on instack-undercloud (master)

Change abandoned by Steven Hardy (<email address hidden>) on branch: master
Review: https://review.openstack.org/317398
Reason: Going with https://review.rdoproject.org/r/#/c/1153/ instead

Revision history for this message
Steven Hardy (shardy) wrote :

https://review.rdoproject.org/r/#/c/1153/ is the packaging fix that should solve this

Changed in tripleo:
milestone: newton-1 → ongoing
milestone: ongoing → newton-1
Revision history for this message
Steven Hardy (shardy) wrote :

This was fixed via the packaging fix mentioned above

Changed in tripleo:
status: In Progress → Fix Committed
Steven Hardy (shardy)
Changed in tripleo:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.