Provision failed in multipath configuration

Bug #1652788 reported by Nastya Urlapova on 2016-12-27
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Vladimir Kozhukalov
Mitaka
High
Vladimir Kozhukalov
Newton
High
Vladimir Kozhukalov

Bug Description

Provision failed with error in astute.log:
"Unexpected error\nUnexpected error while running command.\nCommand: pvcreate --metadatacopies 2 --metadatasize 28m /dev/sde3\nExit code: 5\nStdout: ''\nStderr: 'File descriptor 3 (/run/lock/provision.lock) leaked on pvcreate invocation. Parent PID 7122: /usr/bin/python2.7\\nFile descriptor 4 (/var/log/fuel-agent.log) leaked on pvcreate invocation. Parent PID 7122: /usr/bin/python2.7\\n Device /dev/sde3 not found (or ignored by filtering).\\n'\n",
   :exit_code=>255}}

Scenario:
        1. Create cluster with 1 controller, 1 compute and 1 cinder roles
        2. Run network verification
>>>> 3. Provision the cluster <<< failed here
        4. Verify multipath devices on nodes
        5. Deploy the cluster
        6. Run OSTF

Version:
9.2 snapshot #684

Nastya Urlapova (aurlapova) wrote :
Changed in fuel:
milestone: none → 9.2
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
tags: added: swarm-fail
summary: - Deployment failed in multipath configuration
+ Provision failed in multipath configuration
Changed in fuel:
importance: Undecided → High
status: New → Confirmed
tags: added: area-python
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Vladimir Kozhukalov (kozhukalov)

This issue appears when nailgun-agent starts before device mapper assembles multipath devices and reports them as usual block devices. As a workaround for this issue I suggest to introduce 30 sec delay before starting nailgun-agent first time.

Reviewed: https://review.openstack.org/416661
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=8fbc4d640535387b402aa38cb6227df11c010ff0
Submitter: Jenkins
Branch: stable/mitaka

commit 8fbc4d640535387b402aa38cb6227df11c010ff0
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788

tags: added: in-stable-mitaka

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/418429

Changed in fuel:
status: Confirmed → In Progress
Dmitry Pyzhov (dpyzhov) on 2017-01-11
Changed in fuel:
milestone: 9.2 → 11.0
tags: added: on-verification
tags: removed: on-verification

Reviewed: https://review.openstack.org/418429
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=6b864564bc1a24782343b365de3bc52f7822520c
Submitter: Jenkins
Branch: stable/newton

commit 6b864564bc1a24782343b365de3bc52f7822520c
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788
    (cherry picked from commit 8fbc4d640535387b402aa38cb6227df11c010ff0)

Changed in fuel:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/418430
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=fd0f4664b381067d04c7c26f3e634bc51ab145fe
Submitter: Jenkins
Branch: master

commit fd0f4664b381067d04c7c26f3e634bc51ab145fe
Author: Vladimir Kozhukalov <email address hidden>
Date: Wed Jan 4 19:37:10 2017 +0300

    Fix multipath device search

    When nailgun-agent starts before device mapper
    assembles multipath devices it reports physical disks
    that are used for multipath devices as usual disks.

    This patch does the following

    * filter out devices that are DM_MULTIPATH_DEVICE_PATH = 1
    * run 'udevadm settle' before scanning multipath devices
    * delays scanning multipath devices until building block device info

    Change-Id: I088aede0cf3bd1d16a57e7cdec4e50cab2c19175
    Closes-Bug: #1652788
    (cherry picked from commit 8fbc4d640535387b402aa38cb6227df11c010ff0)

This issue was fixed in the openstack/fuel-nailgun-agent 11.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers