tripleo

Octavia amphora image and keypair not created in overcloud

Bug #1786786 reported by Carlos Goncalves on 2018-08-13

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	Undecided	Carlos Goncalves

Bug Description

Users cannot create load balancers because Octavia amphora image and keypair for SSHing into amphorae are not being created in the overcloud.

2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server [-] Exception during message handling: ComputeBuildException: Failed to build compute instance due to: No Glance images are tagged with amphora-image tag.
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/queue/endpoint.py", line 44, in create_load_balancer
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server self.worker.create_load_balancer(load_balancer_id)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 214, in wrapped_f
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server return self.call(f, *args, **kw)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 295, in call
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server start_time=start_time)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 252, in iter
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server return fut.result()
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server return self.__get_result()
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/tenacity/__init__.py", line 298, in call
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/worker/controller_worker.py", line 319, in create_load_balancer
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server create_lb_tf.run()
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server failures[0].reraise()
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/worker/tasks/compute_tasks.py", line 144, in execute
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server server_group_id=server_group_id, ports=ports)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/controller/worker/tasks/compute_tasks.py", line 98, in execute
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server server_group_id=server_group_id)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/octavia/compute/drivers/nova_driver.py", line 160, in build
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server raise exceptions.ComputeBuildException(fault=e)
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server ComputeBuildException: Failed to build compute instance due to: No Glance images are tagged with amphora-image tag.
2018-07-25 10:20:42.397 22 ERROR oslo_messaging.rpc.server

From /var/lib/mistral/4842bfd0-8a7c-4ff2-b008-e52c6ea9c405/octavia-ansible/octavia-ansible.log

2018-08-13 08:52:17,312 p=14240 u=mistral | [WARNING]: Could not match supplied host pattern, ignoring: undercloud

2018-08-13 08:52:17,314 p=14240 u=mistral | PLAY [undercloud[0]] ***********************************************************
2018-08-13 08:52:17,314 p=14240 u=mistral | skipping: no hosts matched
2018-08-13 08:52:17,319 p=14240 u=mistral | PLAY [octavia_nodes[0]] ********************************************************
2018-08-13 08:52:17,338 p=14240 u=mistral | TASK [Gathering Facts] *********************************************************
2018-08-13 08:52:17,338 p=14240 u=mistral | Monday 13 August 2018 08:52:17 -0400 (0:00:00.105) 0:00:00.105 *********
2018-08-13 08:52:24,123 p=14240 u=mistral | ok: [controller-2]
[...]

Host "undercloud[0]" could not be found when running the playbook tripleo-common/playbooks/octavia-files.yaml [1]. That must be due to the migration of Octavia to external_deploy_task [2] that happened in Rocky cycle.

So, the trouble maker was the renaming of 'undercloud' to 'Undercloud' in tripleo-heat-templates/tree/docker/services/octavia/octavia-deployment-config.yaml.

[root@undercloud-0 mistral]# cat /var/lib/mistral/4842bfd0-8a7c-4ff2-b008-e52c6ea9c405/octavia-ansible/inventory.yaml
octavia_nodes:
  hosts:
    controller-2:
      ansible_user: tripleo-admin
      ansible_host: 192.168.24.19
      ansible_become: true
    controller-1:
      ansible_user: tripleo-admin
      ansible_host: 192.168.24.14
      ansible_become: true
    controller-0:
      ansible_user: tripleo-admin
      ansible_host: 192.168.24.16
      ansible_become: true

Undercloud:
  hosts:
    undercloud-0:
      ansible_host: localhost
      ansible_become: false
      ansible_connection: local

We need to update octavia-files.yaml to reflect that change.

First reported in https://bugzilla.redhat.com/show_bug.cgi?id=1608334

[1] https://github.com/openstack/tripleo-common/blob/336cd3c6364c8c9be88c840253db0d0b54336c75/playbooks/octavia-files.yaml#L2
[2] https://review.openstack.org/#/c/559374/

Tags:

Carlos Goncalves (cgoncalves) on 2018-08-13

tags:	added: rock
tags:	added: rocky-backport-potential removed: rock

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-08-13: Fix proposed to tripleo-common (master)

Fix proposed to branch: master
Review: https://review.openstack.org/591413

Changed in tripleo:
assignee:	nobody → Carlos Goncalves (cgoncalves)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-10-07: Fix proposed to tripleo-common (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/608449

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-10-08: Fix merged to tripleo-common (master)

Reviewed: https://review.openstack.org/591413
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=188e1d4b5d21b09e83f7c0e575fcdc210c08d5fc
Submitter: Zuul
Branch: master

commit 188e1d4b5d21b09e83f7c0e575fcdc210c08d5fc
Author: Carlos Goncalves <email address hidden>
Date: Mon Aug 13 16:26:54 2018 +0200

Fix skip of octavia-undercloud Ansible role

    The migration of Octavia to external_deploy_task [1] changed the
    undercloud group name from 'undercloud' to 'Undecloud' in line with [2].
    Due to that, the execution of role octavia-undercloud was being skipped
    as no host pattern could be matched.

[1] https://review.openstack.org/#/c/559374/
[2] https://review.openstack.org/#/c/588075/

Closes-Bug: #1786786
Depends-On: https://review.openstack.org/580119

Change-Id: I50720c2f1bdb8dce92e297c4d6bfac716bb85b89

Changed in tripleo:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-11-28: Fix included in openstack/tripleo-common 10.1.0

This issue was fixed in the openstack/tripleo-common 10.1.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-01-24: Fix merged to tripleo-common (stable/rocky)

Reviewed: https://review.openstack.org/608449
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=8b40faa71c6043531f88dffe3af837afbf382793
Submitter: Zuul
Branch: stable/rocky

commit 8b40faa71c6043531f88dffe3af837afbf382793
Author: Carlos Goncalves <email address hidden>
Date: Mon Aug 13 16:26:54 2018 +0200

Fix skip of octavia-undercloud Ansible role

[1] https://review.openstack.org/#/c/559374/
[2] https://review.openstack.org/#/c/588075/

Closes-Bug: #1786786
Depends-On: https://review.openstack.org/580119

Change-Id: I50720c2f1bdb8dce92e297c4d6bfac716bb85b89
(cherry picked from commit 188e1d4b5d21b09e83f7c0e575fcdc210c08d5fc)

tags:

added: in-stable-rocky

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-03-14: Fix included in openstack/tripleo-common 9.5.0

This issue was fixed in the openstack/tripleo-common 9.5.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.