AIO with swift gnocchi and ceilometer will not install

Bug #1879192 reported by Dwane Pottratz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Undecided
Dmitriy Rabotyagov

Bug Description

I am trying to use AIO with swift, gnocchi, and ceilometer.

I am following the AIO quick start. https://docs.openstack.org/openstack-ansible/latest/user/aio/quickstart.html

Here are the steps I used:
# git clone https://opendev.org/openstack/openstack-ansible /opt/openstack-ansible
# cd /opt/openstack-ansible
# scripts/bootstrap-ansible.sh
# export SCENARIO='aio_lxc_swift_gnocchi_ceilometer'
# scripts/bootstrap-aio.sh
# cd playbooks
# openstack-ansible setup-everything.yml

I have seen this with AIO as well as a production install. In my production install I have swift working. I add gnocchi to the openstack_user_config.yml. Run 'openstack-ansible setup-everything.yml' and I see that swift also fails to restart. (Sorry didn't catch the error.)

I am getting the following errors with aio1 and aio_swift_proxy_container-xxxxxx

Errors:
TASK [os_swift : Verify md5sum of builder files] *******************************************************************************************************************************************************************************[46/1984]skipping: [aio1] => (item=aio1)
fatal: [aio1]: FAILED! => {"msg": "The conditional check 'hostvars[item]['builder_md5sum'] != md5sum.stdout' failed. The error was: error while evaluating conditional (hostvars[item]['builder_md5sum'] != md5sum.stdout): 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'builder_md5sum'\n\nThe error appears to be in '/etc/ansible/roles/os_swift/tasks/swift_rings_post_distribution_check.yml': line 24, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# As this is post sync all should match and not be empty\n- name: Verify md5sum of builder files\n ^ here\n"}
RUNNING HANDLER [os_swift : Restart swift services] *************************************************************************************************************************************************************************************failed: [aio1] (item={'group': 'swift_acc', 'service_name': 'swift-account-server', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-server /etc/swift/account-server/account-server-2.conf', 'init_config_overrides': {}, 'start_order': 2, 'service_key': 'swift-account-server'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-server /etc/swift/account-server/account-server-2.conf", "group": "swift_acc", "init_config_overrides": {}, "service_key": "swift-account-server", "service_name": "swift-account-server", "start_order": 2}, "msg": "Could not find the requested service swift-account-server:
 host"}
failed: [aio1_swift_proxy_container-aedcf10b] (item={'group': 'swift_proxy', 'service_name': 'swift-proxy-server', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-proxy-server /etc/swift/proxy-server/proxy-server.conf', 'init_config_overrides': {}, 'start_order': 1, 'service_key': 'swift-proxy-server'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-proxy-server /etc/swift/proxy-server/proxy-server.conf", "group": "swift_proxy", "init_config_overrides": {}, "service_key": "swift-proxy-server", "service_name": "swift-proxy-server", "start_order": 1}, "msg": "Could not find the requested service swift-prox
y-server: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-server', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-server /etc/swift/container-server/container-server-2.conf', 'init_config_overrides': {}, 'start_order': 4, 'service_key': 'swift-container-server'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-server /etc/swift/container-server/container-server-2.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-server", "service_name": "swift-container-server", "start_order": 4}, "msg": "Could not find the requested service
 swift-container-server: host"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-server', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-server /etc/swift/object-server/object-server-2.conf', 'init_config_overrides': {
}, 'start_order': 6, 'service_key': 'swift-object-server'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-server /etc/swift/object-server/object-server
-2.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-server", "service_name": "swift-object-server", "start_order": 6}, "msg": "Could not find the requested service swift-object-server: host"}
failed: [aio1] (item={'group': 'swift_acc', 'service_name': 'swift-account-auditor', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-auditor /etc/swift/account-server/account-server.conf', 'init_config_overrides
': {}, 'start_order': 8, 'service_key': 'swift-account-auditor'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-auditor /etc/swift/account-server/acco
unt-server.conf", "group": "swift_acc", "init_config_overrides": {}, "service_key": "swift-account-auditor", "service_name": "swift-account-auditor", "start_order": 8}, "msg": "Could not find the requested service swift-account-audit
or: host"}
failed: [aio1] (item={'group': 'swift_acc', 'service_name': 'swift-account-reaper', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-reaper /etc/swift/account-server/account-server.conf', 'init_config_overrides':
 {}, 'start_order': 9, 'service_key': 'swift-account-reaper'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-reaper /etc/swift/account-server/account-
server.conf", "group": "swift_acc", "init_config_overrides": {}, "service_key": "swift-account-reaper", "service_name": "swift-account-reaper", "start_order": 9}, "msg": "Could not find the requested service swift-account-reaper: hos
t"}
failed: [aio1] (item={'group': 'swift_acc', 'service_name': 'swift-account-replicator', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-replicator /etc/swift/account-server/account-server.conf', 'init_config_ove
rrides': {}, 'start_order': 10, 'service_key': 'swift-account-replicator'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-account-replicator /etc/swift/accoun
t-server/account-server.conf", "group": "swift_acc", "init_config_overrides": {}, "service_key": "swift-account-replicator", "service_name": "swift-account-replicator", "start_order": 10}, "msg": "Could not find the requested service
 swift-account-replicator: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-auditor', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-auditor /etc/swift/container-server/container-server.conf', 'init_config_
overrides': {}, 'start_order': 11, 'service_key': 'swift-container-auditor'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-auditor /etc/swift/conta
iner-server/container-server.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-auditor", "service_name": "swift-container-auditor", "start_order": 11}, "msg": "Could not find the requested ser
vice swift-container-auditor: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-reconciler', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-reconciler /etc/swift/container-server/container-reconciler.conf', 'in
it_config_overrides': {}, 'start_order': 12, 'service_key': 'swift-container-reconciler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-reconciler
/etc/swift/container-server/container-reconciler.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-reconciler", "service_name": "swift-container-reconciler", "start_order": 12}, "msg": "Could
not find the requested service swift-container-reconciler: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-replicator', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-replicator /etc/swift/container-server/container-server-2.conf', 'init
_config_overrides': {}, 'start_order': 13, 'service_key': 'swift-container-replicator'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-replicator /e
tc/swift/container-server/container-server-2.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-replicator", "service_name": "swift-container-replicator", "start_order": 13}, "msg": "Could not
find the requested service swift-container-replicator: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-sync', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-sync /etc/swift/container-server/container-server-2.conf', 'init_config_over
rides': {}, 'start_order': 14, 'service_key': 'swift-container-sync'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-sync /etc/swift/container-serve
r/container-server-2.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-sync", "service_name": "swift-container-sync", "start_order": 14}, "msg": "Could not find the requested service swift-con
tainer-sync: host"}
failed: [aio1] (item={'group': 'swift_cont', 'service_name': 'swift-container-updater', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-updater /etc/swift/container-server/container-server.conf', 'init_config_
overrides': {}, 'start_order': 15, 'service_key': 'swift-container-updater'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-container-updater /etc/swift/conta
iner-server/container-server.conf", "group": "swift_cont", "init_config_overrides": {}, "service_key": "swift-container-updater", "service_name": "swift-container-updater", "start_order": 15}, "msg": "Could not find the requested se$
vice swift-container-updater: host"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-auditor', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-auditor /etc/swift/object-server/object-server-2.conf', 'init_config_overrides'$
 {}, 'start_order': 16, 'service_key': 'swift-object-auditor'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-auditor /etc/swift/object-server/object-$
erver-2.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-auditor", "service_name": "swift-object-auditor", "start_order": 16}, "msg": "Could not find the requested service swift-object-auditor: $
ost"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-expirer', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-expirer /etc/swift/object-server/object-expirer.conf', 'init_config_overrides':
{}, 'start_order': 17, 'service_key': 'swift-object-expirer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-expirer /etc/swift/object-server/object-ex
pirer.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-expirer", "service_name": "swift-object-expirer", "start_order": 17}, "msg": "Could not find the requested service swift-object-expirer: hos
t"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-reconstructor', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-reconstructor /etc/swift/object-server/object-server-2.conf', 'init_config
_overrides': {}, 'start_order': 18, 'service_key': 'swift-object-reconstructor'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-reconstructor /etc/swif
t/object-server/object-server-2.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-reconstructor", "service_name": "swift-object-reconstructor", "start_order": 18}, "msg": "Could not find the reque
sted service swift-object-reconstructor: host"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-replicator', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-replicator /etc/swift/object-server/object-server-2.conf', 'init_config_overr
ides': {}, 'start_order': 19, 'service_key': 'swift-object-replicator'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-replicator /etc/swift/object-ser
ver/object-server-2.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-replicator", "service_name": "swift-object-replicator", "start_order": 19}, "msg": "Could not find the requested service swift
-object-replicator: host"}
failed: [aio1] (item={'group': 'swift_obj', 'service_name': 'swift-object-updater', 'execstarts': '/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-updater /etc/swift/object-server/object-server-2.conf', 'init_config_overrides':
 {}, 'start_order': 20, 'service_key': 'swift-object-updater'}) => {"ansible_loop_var": "item", "changed": false, "item": {"execstarts": "/openstack/venvs/swift-20.1.0.dev235/bin/swift-object-updater /etc/swift/object-server/object-s
erver-2.conf", "group": "swift_obj", "init_config_overrides": {}, "service_key": "swift-object-updater", "service_name": "swift-object-updater", "start_order": 20}, "msg": "Could not find the requested service swift-object-updater: h
ost"}

Dwane Pottratz (dpcsar)
description: updated
Changed in openstack-ansible:
assignee: nobody → Dmitriy Rabotyagov (noonedeadpunk)
Changed in openstack-ansible:
status: New → Confirmed
Changed in openstack-ansible:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to openstack-ansible-os_gnocchi (master)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to openstack-ansible-os_ceilometer (master)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (master)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_swift (master)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/791250
Committed: https://opendev.org/openstack/openstack-ansible-os_swift/commit/2ce00d6c69854a175d876c95b3439863a20a5fe4
Submitter: "Zuul (22348)"
Branch: master

commit 2ce00d6c69854a175d876c95b3439863a20a5fe4
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu May 13 17:34:08 2021 +0300

    Do not collect gnocchi project ID

    Ceilometer middleware now accepts also project names,
    so it's not required to gain project UUID [1].

    We also switching gnocchi project name to `service` for all scenarios
    as it's already whitelisted by default [2]

    [1] https://opendev.org/openstack/ceilometermiddleware/commit/e2bf485044d8f3743da9298a9e461c5808be31f3
    [2] https://opendev.org/openstack/ceilometermiddleware/src/commit/082dabb1b7186f614664f69db32f80918b20d624/ceilometermiddleware/swift.py#L158

    Closes-Bug: #1879192
    Change-Id: I40d3178d1b3bd1234ac34c6fa1d0d2bcb7160720

Changed in openstack-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to openstack-ansible-os_ceilometer (master)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/791259
Committed: https://opendev.org/openstack/openstack-ansible-os_ceilometer/commit/a08e4703936f60f506efc305d402b107e360e971
Submitter: "Zuul (22348)"
Branch: master

commit a08e4703936f60f506efc305d402b107e360e971
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu May 13 17:59:26 2021 +0300

    Remove deprecated options from config

    Gnocchi dispatcher [1] along with api section have been removed
    for a while from ceilometer. We should clean up template as well

    [1] https://opendev.org/openstack/ceilometer/commit/83ffaffcb2cee6a0b19601bc7cefd863685601e2

    Change-Id: Ibbe290ebfd91297080ce2f0b861deece82af42cc
    Related-Bug: #1879192

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to openstack-ansible-os_gnocchi (master)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible-os_gnocchi/+/791254
Committed: https://opendev.org/openstack/openstack-ansible-os_gnocchi/commit/a6a25bf3cd8a84567e8ed7539043a6d08372ecac
Submitter: "Zuul (22348)"
Branch: master

commit a6a25bf3cd8a84567e8ed7539043a6d08372ecac
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu May 13 17:48:32 2021 +0300

    Switch gnocchi service name to service

    Previously we used different from other services project name to protect
    swift from looping. However nowadays ceilometer middleware does
    exclude service project by default. So no futher actions needed
    unless deployer want to measure service project excluding gnocchi
    In this scenario, deployer still can define custom
    `gnocchi_service_project_name` in their user_variables.

    Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/791107
    Change-Id: Ic0ffa6908bfc55ffdb41ea9b8e7435e4dc88ddca
    Related-Bug: #1879192

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible (master)

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible/+/791261
Committed: https://opendev.org/openstack/openstack-ansible/commit/d5c523d428a37a44c3fa01646c33a98fc5c91977
Submitter: "Zuul (22348)"
Branch: master

commit d5c523d428a37a44c3fa01646c33a98fc5c91977
Author: Dmitriy Rabotyagov <email address hidden>
Date: Thu May 13 18:02:11 2021 +0300

    Change order of swift and gnocchi installation

    Ghocchi requires swift to be present for DB migration in case
    Swift is set as driver.

    Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/791107
    Change-Id: I72a72a43488fd3d6923899a6095c988f04c82be9
    Closes-Bug: #1879192

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible 23.0.0.0b1

This issue was fixed in the openstack/openstack-ansible 23.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_swift yoga-eom

This issue was fixed in the openstack/openstack-ansible-os_swift yoga-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_swift wallaby-eom

This issue was fixed in the openstack/openstack-ansible-os_swift wallaby-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_swift xena-eom

This issue was fixed in the openstack/openstack-ansible-os_swift xena-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.