[regression] Swift service fails to start with [Errno 2] No such file or directory: '/etc/swift/object.ring.gz

[Volodymyr Shypyguzov]

Test group: deploy_neutron_vlan_ha
Steps to reproduce:
        1. Create cluster
        2. Add 3 nodes with controller role
        3. Add 2 nodes with compute role
        4. Deploy the cluster
        5. Run network verification
        6. Run OSTF << Fails

Expected result: OSTF tests passed
Actual result: OSTF test fails on checking backends state

In swift-all.log:
#012IOError: [Errno 2] No such file or directory: '/etc/swift/object.ring.gz'

[Alexandr Kostrikov (akostrikov-mirantis)]

Also, reproduce at bonding test cases
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.bonding_ha/109/consoleFull
setting to swarm-blocker

[Dmitry Teselkin (teselkin-d)]

Plz take a look at
* https://bugs.launchpad.net/openstack-manuals/+bug/996944
* https://review.openstack.org/#/c/11614/

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

version

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Igor Yozhikov (iyozhikov)]

Packages? as far as I know and according to Swift manuals? are not responsible for rings creation.
Ring creation and synchronization must be done using CM tools or manually.
Also in deployment logs was found next:
node-1 - primary controller - ring files created
node-4 and node-5 - rest controllers - didn't sync ring files.
Procedure described at https://github.com/openstack/fuel-library/blob/2f9f9516f04de573809b4d1162dfd2b9087ea90f/deployment/puppet/openstack_tasks/manifests/swift/proxy_storage.pp#L421-L440 has not been seen in logs.

Please, reproduce this bug and provide ENV for further investigation.

[Igor Yozhikov (iyozhikov)]

s/?/,/

[Alexandr Kostrikov (akostrikov-mirantis)]

Similar failure on https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.plugins.install_plugin_after_cluster_create/20/testReport/(root)/three_ctrl_enable_installed_after_deploy_redeploy/three_ctrl_enable_installed_after_deploy_redeploy/
 with logs at http://paste.openstack.org/show/497478/

Env is revertable.

[Igor Yozhikov (iyozhikov)]

During more deep investigation made with Dmitry Ilyin found root cause - wrong
puppet sequence of steps.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/318563

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/318563
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=52716670dcd2636828ea49165e24219fd07aea1a
Submitter: Jenkins
Branch: master

commit 52716670dcd2636828ea49165e24219fd07aea1a
Author: Dmitry Ilyin <email address hidden>
Date: Thu May 19 13:50:28 2016 +0300

    Refactor openstack_tasks::swift

    Change-Id: I7ff5778367a30b00975ba41069dd6e11332a5c38
    Related-bug: 1582646

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/320374

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/320374
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=765af856b63f8e99ad8c82f303432d572c399bd3
Submitter: Jenkins
Branch: stable/mitaka

commit 765af856b63f8e99ad8c82f303432d572c399bd3
Author: Dmitry Ilyin <email address hidden>
Date: Thu May 19 13:50:28 2016 +0300

    Refactor openstack_tasks::swift

    cherry picked from commit 52716670dcd2636828ea49165e24219fd07aea1a
    Change-Id: I7ff5778367a30b00975ba41069dd6e11332a5c38
    Related-bug: 1582646

[Alexandr Kostrikov (akostrikov-mirantis)]

There were no reproduces on swarm on iso 9.0 mos iso #416