rsync: change_dir#3 "/etc/swift" failed: No such file or directory

Bug #1360118 reported by Dennis Dmitriev
24
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Bogdan Dobrelya
5.0.x
Fix Committed
High
Bogdan Dobrelya

Bug Description

During HA deployment, when puppet tries rsync files /etc/swift/object.ring.gz and /etc/swift/account.ring.gz , error appear:

rsync: change_dir#3 "/etc/swift" failed: No such file or directory

It is failed because swift is not installed at this moment (it is installed later).

        Exact steps to reproduce (system test 'deploy_ha_vlan', not sure if step #2 is necessary to reproduce the issue):
            1. Create cluster with 3 controller and 2 compute nodes
            2. Set up cluster to use Network VLAN manager with 8 networks
            3. Deploy the cluster

        Expected result: cluster deployment finished successfuly

        Actual result: cluster deployment failed

From the attached diagnostic snapshot, on the node-5/puppet-apply.log (take a look on the log time):

====================================
2014-08-22T03:54:17.914500+01:00 debug: Executing 'rsync -q -a rsync://10.108.37.3/swift_server/object.ring.gz /etc/swift/object.ring.gz'
2014-08-22T03:54:18.037141+01:00 notice: (/Stage[main]/Openstack::Swift::Storage_node/Swift::Ringsync[object]/Rsync::Get[/etc/swift/object.ring.gz]/Exec[rsync /etc/swift/object.ring.gz]/returns) rsync: change_dir#3 "/etc/swift" failed: No such file or directory (2)
...
2014-08-22T03:54:18.048659+01:00 debug: Executing 'rsync -q -a rsync://10.108.37.3/swift_server/account.ring.gz /etc/swift/account.ring.gz'
2014-08-22T03:54:18.170817+01:00 notice: (/Stage[main]/Openstack::Swift::Storage_node/Swift::Ringsync[account]/Rsync::Get[/etc/swift/account.ring.gz]/Exec[rsync /etc/swift/account.ring.gz]/returns) rsync: change_dir#3 "/etc/swift" failed: No such file or directory (2)
...
and 3 minutes later:
...
2014-08-22T03:57:06.825672+01:00 debug: Executing '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install swift'
2014-08-22T03:57:08.206187+01:00 notice: (/Stage[main]/Openstack::Glance/Package[swift]/ensure) ensure changed 'purged' to 'present'
==========================

{u'build_id': u'2014-08-21_16-33-53', u'ostf_sha': u'907f25f8fad39b177bf6a66fba9785afa7dd8008', u'build_number': u'466', u'auth_required': True, u'nailgun_sha': u'25eba6fbb2047f26d9da4d27ffdb742c9c27832a', u'production': u'docker', u'api': u'1.0', u'fuelmain_sha': u'2c63c5024f3a97c873628e1c3a6a30861c6086aa', u'astute_sha': u'ac520b09525af4551e730b1c1f78170fefaf3cb8', u'feature_groups': [u'mirantis'], u'release': u'5.1', u'fuellib_sha': u'bddba1e854a6b0350e844a0baad50816d3cc8e28'}

Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :
Changed in fuel:
importance: Undecided → High
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

Looks like /etc/swift/object.ring.gz and /etc/swift/account.ring.gz are rsync-ed later (at 04:11:39 - 04:11:40), but some puppet dependences are already failed at this moment.

Revision history for this message
Vladimir Kuklin (vkuklin) wrote :
Changed in fuel:
status: New → Confirmed
Revision history for this message
Dennis Dmitriev (ddmitriev) wrote :

It is not network issue so it is not duplicate of https://bugs.launchpad.net/fuel/+bug/1359833
This bug is about too early tries of configuring the swift when it is not installed yet.

Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Bogdan Dobrelya (bogdando)
status: Confirmed → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/116247

Changed in fuel:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/5.0)

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/116261

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/116247
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=fc7822afd3d0de02e7e205b227dd8258943af5a1
Submitter: Jenkins
Branch: master

commit fc7822afd3d0de02e7e205b227dd8258943af5a1
Author: Bogdan Dobrelya <email address hidden>
Date: Fri Aug 22 15:31:11 2014 +0300

    Ensure swift ring sync after its packages installed

    Closes-bug: #1360118

    Change-Id: Ia667f2fd8de16144c25a4f5cc9e849e4d2c69797
    Signed-off-by: Bogdan Dobrelya <email address hidden>

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/5.0)

Reviewed: https://review.openstack.org/116261
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=ee3eded69cdfab9076f6804e29849c7123dd58f6
Submitter: Jenkins
Branch: stable/5.0

commit ee3eded69cdfab9076f6804e29849c7123dd58f6
Author: Bogdan Dobrelya <email address hidden>
Date: Fri Aug 22 15:31:11 2014 +0300

    Ensure swift ring sync after its packages installed

    Closes-bug: #1360118

    Change-Id: Ia667f2fd8de16144c25a4f5cc9e849e4d2c69797
    Signed-off-by: Bogdan Dobrelya <email address hidden>

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.