ceph-deploy uses letters instead of IDs

[Andrey Grebennikov]

I use the servers with combination of Integrated+External RAID controllers.
I create OS and Ceph Journals on the disks on the External controller.
I create Ceph OSD on the disks of Integrated controller.

My OS and Ceph Journals utilize /dev/sda, sdb and sdc during provisioning.
My OSDs are using sdd .. sdk

Ceph-deploy adds all OSDs successfully.

When reboot the server, sometimes my Journals appear as /dev/sdk and /dev/sdj, consequently there are no OSD running anymore since they cannot find their Journals.

The issue is described here:
https://wiki.archlinux.org/index.php/persistent_block_device_naming#fstab

I suggest to switch from letters of the devices to UUIDs of them.

[Dmitry Borodaenko (angdraug)]

This can cause data loss after node reboot when Ceph is deployed on nodes with multiple controllers, this fits definition of High priority.

[Oleksiy Molchanov (omolchanov)]

Andrey, please check this bug https://bugs.launchpad.net/fuel/+bug/1416855

Please comment if it was helpful and covered your issue.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/177352

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/177352
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=3c9910f76623233a832383758e2beab63b13618f
Submitter: Jenkins
Branch: master

commit 3c9910f76623233a832383758e2beab63b13618f
Author: Ryan Moe <email address hidden>
Date: Thu Apr 23 07:47:13 2015 -0700

    Use persistent names for journal devices

    On systems with multiple disk controllers drive names
    will not be consistent. On reboot an OSD's journal may
    have changed names which will prevent the OSD from starting.
    Instead we will attempt to use the persistent by-id link
    and fall back to the device name (e.g. /dev/sda) if it's
    not present.

    Change-Id: I18f3c1cb31dedf9e759edf0094ebce1fdb42492d
    Closes-bug: #1441434

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/182742

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/182742
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=801cf96c8e08d87683919e0ae5b83d93e0f81bad
Submitter: Jenkins
Branch: master

commit 801cf96c8e08d87683919e0ae5b83d93e0f81bad
Author: Ryan Moe <email address hidden>
Date: Wed May 13 09:13:24 2015 -0700

    Fix incorrect shell command in ceph OSD Facter function

    When DEVLINKS was on the first line of udevadm output
    this function would fail to correctly find a journal
    device name. This would result in an invalid ceph-deploy
    command line. An empty string as the journal device name
    would cause the same issue and this case has been fixed
    as well.

    Change-Id: I785fd463fab781ac1b769fb2eee5c6abf0ef6261
    Co-Authored-By: Mykola Golub <email address hidden>
    Closes-bug: #1453959
    Related-bug: #1441434

[Mykola Golub (mgolub)]

api: '1.0'
astute_sha: 96801c5bccb14aa3f2a0d7f27f4a4b6dd2b4a548
auth_required: true
build_id: 2015-05-19_10-05-51
build_number: '437'
feature_groups:
- mirantis
fuel-library_sha: 2814c51668f487e97e1449b078bad1942421e6b9
fuel-ostf_sha: 9ce1800749081780b8b2a4a7eab6586583ffaf33
fuelmain_sha: 68796aeaa7b669e68bc0976ffd616709c937187a
nailgun_sha: 593c99f2b46cf52b2be6c7c6e182b6ba9f2232cd
openstack_version: 2014.2.2-6.1
production: docker
python-fuelclient_sha: e19f1b65792f84c4a18b5a9473f85ef3ba172fce
release: '6.1'
release_versions:
  2014.2.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: 96801c5bccb14aa3f2a0d7f27f4a4b6dd2b4a548
      build_id: 2015-05-19_10-05-51
      build_number: '437'
      feature_groups:
      - mirantis
      fuel-library_sha: 2814c51668f487e97e1449b078bad1942421e6b9
      fuel-ostf_sha: 9ce1800749081780b8b2a4a7eab6586583ffaf33
      fuelmain_sha: 68796aeaa7b669e68bc0976ffd616709c937187a
      nailgun_sha: 593c99f2b46cf52b2be6c7c6e182b6ba9f2232cd
      openstack_version: 2014.2.2-6.1
      production: docker
      python-fuelclient_sha: e19f1b65792f84c4a18b5a9473f85ef3ba172fce
      release: '6.1'

On my OSD nodes there is no by-id link so it falls back to the device name:

root@node-3:~# ceph-disk list
/dev/sda :
 /dev/sda1 other, 21686148-6449-6e6f-744e-656564454649
 /dev/sda2 other, ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
 /dev/sda3 other, ext2, mounted on /boot
 /dev/sda4 ceph journal
 /dev/sda5 ceph data, active, cluster ceph, osd.4
 /dev/sda6 other, LVM2_member
 /dev/sda7 other, LVM2_member
 /dev/sda8 other, iso9660
/dev/sr0 other, unknown

root@node-3:~# ls -l /var/lib/ceph/osd/ceph-4/journal
lrwxrwxrwx 1 root root 9 May 21 06:42 /var/lib/ceph/osd/ceph-4/journal -> /dev/sda4

root@node-3:~# udevadm info -q property -n /dev/sda5 | awk 'BEGIN {FS="="} {if ($1 == "DEVLINKS") print $2}'
/dev/disk/by-partlabel/primary /dev/disk/by-parttypeuuid/4fbd7e29-9d25-41b8-afd0-062c0ceff05d.1a82050c-21e5-4c31-a707-d28dc064001a /dev/disk/by-partuuid/1a82050c-21e5-4c31-a707-d28dc064001a /dev/disk/by-path/pci-0000:00:10.0-scsi-0:0:0:0-part5 /dev/disk/by-uuid/13d92452-b0ad-4cf1-8301-f79eb585f521

May be we should also try by-partuuid?

[Stanislav Makar (smakar)]

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "472"
  build_id: "2015-05-26_23-21-01"
  nailgun_sha: "f737675091bd1903aace0e36812e855ce47dfec7"
  python-fuelclient_sha: "e19f1b65792f84c4a18b5a9473f85ef3ba172fce"
  astute_sha: "0bd72c72369e743376864e8e8dabfe873d40450a"
  fuel-library_sha: "8cfeca1a86179ebed1e4e03b2133b49c27350f6f"
  fuel-ostf_sha: "87819878bc0ca572900e1f6933d9b99e666d6f62"
  fuelmain_sha: "13b3e9cf074ba1cf1ae06509c55fbab613c73f4e"

[Vitaly Sedelnik (vsedelnik)]

Won't Fix for 6.0-updates as there is no delivery channel for Fuel fixes in 6.0 updates