OSError: [Errno 2] No such file or directory: '/opt/stack/data/cinder/volumes/volume-

Bug #1277362 reported by Attila Fazekas on 2014-02-07
This bug affects 2 people
Affects Status Importance Assigned to Milestone
John Griffith
description: updated
John Griffith (john-griffith) wrote :

This is related to the create_export change. We're doing a double remove somehow it seems in the driver, I'll get it sorted shortly.

Changed in cinder:
status: New → Triaged
importance: Undecided → High
assignee: nobody → John Griffith (john-griffith)
John Griffith (john-griffith) wrote :

Looks like a race between the detach call and the delete call. Result is that when the delete starts the volume_path exists so it tries to go through the target removal process that the detach is already chugging on. By the time delete get's down to remove the volume_path, the detach has finished and already done it for us.

Easy to fix by just adding an if_exists on that removal, but I want to spend a bit of time looking at ways to deal with the race itself.

Fix proposed to branch: master
Review: https://review.openstack.org/72265

Changed in cinder:
status: Triaged → In Progress

Reviewed: https://review.openstack.org/72265
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a9112a9cd8d1286e6c62c1a848c10e4d926a45c2
Submitter: Jenkins
Branch: master

commit a9112a9cd8d1286e6c62c1a848c10e4d926a45c2
Author: John Griffith <email address hidden>
Date: Mon Feb 10 05:21:18 2014 +0000

    Fixup persistence file not found on tgt remove

    After commit: beecd769af02ba5915be827d28a7b46d970e41b0
    we're experiencing a race in the tempest gating tests.

    It appears that there are certain cases where a detach
    and a delete call can overlap causing a race in the
    remove_iscsi_target method. The result is that when
    the slower method gets around to the final removal of
    the persistence file it's no longer there, so we
    hit an unhandled exception.

    This patch does two things:
    1. It adds the volume lock to the detach_volume
    method in the manager (already in place for delete).
    This should mitigate the race and allow us to avoid it.

    2. In the iscsi_remove_target method check if the
    persist file has already been removed when we get
    to the last check. The fact is that if it was somehow
    removed before we got here we don't really care, we're
    just going to delete it anyway.

    Change-Id: Ied45e1a66c8fb79c95ada66f8c59bd2839e200ad
    Closes-Bug: 1277362

Changed in cinder:
status: In Progress → Fix Committed
Thierry Carrez (ttx) on 2014-03-05
Changed in cinder:
milestone: none → icehouse-3
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2014-04-17
Changed in cinder:
milestone: icehouse-3 → 2014.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers