overcloud deployment failed due to "gnocchi-upgrade" tries to run before swift is up and running

Bug #1696052 reported by Artem Hrechanychenko
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Unassigned

Bug Description

Description
===========

overcloud.AllNodesDeploySteps.ControllerDeployment_Step4.0:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: 0819b165-8a47-4645-a925-5b5b4b6671a6
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
  deploy_stdout: |
    ...
    Notice: /Stage[main]/Swift::Storage::Account/Swift::Storage::Generic[account]/Swift::Service[swift-account-replicator]/Service[swift-account-replicator]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Account/Swift::Storage::Generic[account]/Swift::Service[swift-account-auditor]/Service[swift-account-auditor]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Container/Swift::Storage::Generic[container]/Swift::Service[swift-container-server]/Service[swift-container-server]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Container/Swift::Storage::Generic[container]/Swift::Service[swift-container-replicator]/Service[swift-container-replicator]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Container/Swift::Storage::Generic[container]/Swift::Service[swift-container-auditor]/Service[swift-container-auditor]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-server]/Service[swift-object-server]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-replicator]/Service[swift-object-replicator]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Swift::Service[swift-object-auditor]/Service[swift-object-auditor]/ensure: ensure changed 'stopped' to 'running'
    Notice: /Stage[main]/Swift::Deps/Anchor[swift::service::end]: Triggered 'refresh' from 14 events
    Notice: Applied catalog in 241.12 seconds
    (truncated, view all with --long)
  deploy_stderr: |
    ...
       (file & line not available)
    Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored at /etc/puppet/modules/gnocchi/manifests/api.pp:129:5
    Warning: Scope(Class[Aodh]): aodh::rabbit_host, aodh::rabbit_hosts, aodh::rabbit_password, aodh::rabbit_port, aodh::rabbit_userid and aodh::rabbit_virtual_host are deprecated. Please use aodh::default_transport_url instead.
    Warning: Scope(Class[Panko::Api]): This Class is deprecated and will be removed in future releases.
    Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored at /etc/puppet/modules/panko/manifests/api.pp:133:5
    Warning: Scope(Oslo::Messaging::Rabbit[glance_api_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
    Warning: Scope(Oslo::Messaging::Rabbit[glance_registry_config]): The oslo_messaging rabbit_host, rabbit_hosts, rabbit_port, rabbit_userid, rabbit_password, rabbit_virtual_host parameters have been deprecated by the [DEFAULT]\transport_url. Please use oslo::messaging::default::transport_url instead.
    Warning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.
    Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: Failed to call refresh: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf returned 1 instead of one of [0]
    Error: /Stage[main]/Gnocchi::Db::Sync/Exec[gnocchi-db-sync]: gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf returned 1 instead of one of [0]
    (truncated, view all with --long)

[heat-admin@overcloud-controller-0 ~]$ sudo gnocchi-upgrade --config-file /etc/gnocchi/gnocchi.conf
Option "metric_processing_delay" from group "storage" is deprecated. Use option "metric_processing_delay" from group "metricd".
[heat-admin@overcloud-controller-0 ~]$ echo $?
0

[heat-admin@overcloud-controller-0 ~]$ systemctl -l|grep swift
  openstack-swift-account-auditor.service loaded active running OpenStack Object Storage (swift) - Account Auditor
  openstack-swift-account-reaper.service loaded active running OpenStack Object Storage (swift) - Account Reaper
  openstack-swift-account-replicator.service loaded active running OpenStack Object Storage (swift) - Account Replicator
  openstack-swift-account.service loaded active running OpenStack Object Storage (swift) - Account Server
  openstack-swift-container-auditor.service loaded active running OpenStack Object Storage (swift) - Container Auditor
  openstack-swift-container-replicator.service loaded active running OpenStack Object Storage (swift) - Container Replicator
  openstack-swift-container-updater.service loaded active running OpenStack Object Storage (swift) - Container Updater
  openstack-swift-container.service loaded active running OpenStack Object Storage (swift) - Container Server
  openstack-swift-object-auditor.service loaded active running OpenStack Object Storage (swift) - Object Auditor
  openstack-swift-object-expirer.service loaded active running OpenStack Object Storage (swift) - Object Expirer
  openstack-swift-object-replicator.service loaded active running OpenStack Object Storage (swift) - Object Replicator
  openstack-swift-object-updater.service loaded active running OpenStack Object Storage (swift) - Object Updater
  openstack-swift-object.service loaded active running OpenStack Object Storage (swift) - Object Server
  openstack-swift-proxy.service loaded active running OpenStack Object Storage (swift) - Proxy Server

Steps to reproduce
==================
1)Deploy undercloud and perform introspection , --release Ocata
bash quickstart.sh --teardown all -release ocata $HOST

2) deploy overcloud
openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates --libvirt-type qemu --control-flavor oooq_control --compute-flavor oooq_compute -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml

Expected result
===============
Stack successfully created

Actual result
=============
Stack creation failed

Revision history for this message
Juan Antonio Osorio Robles (juan-osorio-robles) wrote :

This is what /var/log/gnocchi/upgrade.log contains http://pastebin.test.redhat.com/491095

Revision history for this message
Artem Hrechanychenko (ahrechan) wrote :

with disabled telemetry service deployment was successfully

Changed in tripleo:
status: New → Fix Released
milestone: none → pike-2
importance: Undecided → Medium
Revision history for this message
Artem Hrechanychenko (ahrechan) wrote :

why fix released?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.