| 2016-04-06 19:11:05 |
Ryan Beisner |
bug |
|
|
added bug |
| 2016-04-06 19:11:37 |
Ryan Beisner |
bug task added |
|
ceph-mon (Juju Charms Collection) |
|
| 2016-04-06 20:26:27 |
Ryan Beisner |
description |
When deploying ceph via charm to a system with multipath storage, the block device names and order may not be predictable. This essentially makes automated deployment of ceph via the charm onto these systems impossible.
Take:
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf
Those devices are not usable by ceph (errors out, indicating they are in use).
The corresponding /dev/dm-N devices would be usable, I believe, if they were predictable. But they are not. ie /dev/dm-3 might represent /dev/sdc on one deployment, but after a tear down and redeploy, /dev/dm-3 might represent the disk that contains the root linux filesystem (doh!).
ex.
root@gregory-ppc64:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 264.3G 0 disk
├─sda1 8:1 0 8M 0 part
├─sda2 8:2 0 264.3G 0 part
└─mpath0 (dm-0) 252:0 0 264.3G 0 mpath
├─mpath0-part1 (dm-2) 252:2 0 8M 0 part
└─mpath0-part2 (dm-3) 252:3 0 264.3G 0 part /
sdb 8:16 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sdc 8:32 0 264.3G 0 disk
└─mpath3 (dm-5) 252:5 0 264.3G 0 mpath
sdd 8:48 0 264.3G 0 disk
└─mpath4 (dm-4) 252:4 0 264.3G 0 mpath
sde 8:64 0 264.3G 0 disk
└─mpath5 (dm-6) 252:6 0 264.3G 0 mpath
sdf 8:80 0 264.3G 0 disk
└─mpath1 (dm-7) 252:7 0 264.3G 0 mpath
sdg 8:96 0 264.3G 0 disk
├─sdg1 8:97 0 8M 0 part
├─sdg2 8:98 0 264.3G 0 part
└─mpath0 (dm-0) 252:0 0 264.3G 0 mpath
├─mpath0-part1 (dm-2) 252:2 0 8M 0 part
└─mpath0-part2 (dm-3) 252:3 0 264.3G 0 part /
sdh 8:112 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sdi 8:128 0 264.3G 0 disk
└─mpath3 (dm-5) 252:5 0 264.3G 0 mpath
sdj 8:144 0 264.3G 0 disk
└─mpath4 (dm-4) 252:4 0 264.3G 0 mpath
sdk 8:160 0 264.3G 0 disk
└─mpath5 (dm-6) 252:6 0 264.3G 0 mpath
sdl 8:176 0 264.3G 0 disk
└─mpath1 (dm-7) 252:7 0 264.3G 0 mpath
sr0 11:0 1 521.9M 0 rom |
When deploying ceph via charm to a system with multipath storage, the block device names and order may not be predictable. This essentially makes automated deployment of ceph via the charm onto these systems impossible.
Take:
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd
/dev/sde
/dev/sdf
Those devices are not usable by ceph (errors out, indicating they are in use).
The corresponding /dev/dm-N devices would be usable, I believe, if they were predictable. But they are not. ie /dev/dm-3 might represent /dev/sdc on one deployment, but after a tear down and redeploy, /dev/dm-3 might represent the disk that contains the root linux filesystem (doh!).
ex.
### First Deploy:
root@gregory-ppc64:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 264.3G 0 disk
├─sda1 8:1 0 8M 0 part
├─sda2 8:2 0 264.3G 0 part
└─mpath0 (dm-0) 252:0 0 264.3G 0 mpath
├─mpath0-part1 (dm-2) 252:2 0 8M 0 part
└─mpath0-part2 (dm-3) 252:3 0 264.3G 0 part /
sdb 8:16 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sdc 8:32 0 264.3G 0 disk
└─mpath3 (dm-5) 252:5 0 264.3G 0 mpath
sdd 8:48 0 264.3G 0 disk
└─mpath4 (dm-4) 252:4 0 264.3G 0 mpath
sde 8:64 0 264.3G 0 disk
└─mpath5 (dm-6) 252:6 0 264.3G 0 mpath
sdf 8:80 0 264.3G 0 disk
└─mpath1 (dm-7) 252:7 0 264.3G 0 mpath
sdg 8:96 0 264.3G 0 disk
├─sdg1 8:97 0 8M 0 part
├─sdg2 8:98 0 264.3G 0 part
└─mpath0 (dm-0) 252:0 0 264.3G 0 mpath
├─mpath0-part1 (dm-2) 252:2 0 8M 0 part
└─mpath0-part2 (dm-3) 252:3 0 264.3G 0 part /
sdh 8:112 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sdi 8:128 0 264.3G 0 disk
└─mpath3 (dm-5) 252:5 0 264.3G 0 mpath
sdj 8:144 0 264.3G 0 disk
└─mpath4 (dm-4) 252:4 0 264.3G 0 mpath
sdk 8:160 0 264.3G 0 disk
└─mpath5 (dm-6) 252:6 0 264.3G 0 mpath
sdl 8:176 0 264.3G 0 disk
└─mpath1 (dm-7) 252:7 0 264.3G 0 mpath
sr0 11:0 1 521.9M 0 rom
...and...
### Second Deploy (same metal, no config changes):
ubuntu@gregory-ppc64:~$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 14.5G 0 disk
└─sda1 8:1 1 1.3G 0 part
sdb 8:16 0 264.3G 0 disk
├─sdb1 8:17 0 8M 0 part
├─sdb2 8:18 0 264.3G 0 part
└─mpath0 (dm-2) 252:2 0 264.3G 0 mpath
├─mpath0-part1 (dm-4) 252:4 0 8M 0 part
└─mpath0-part2 (dm-5) 252:5 0 264.3G 0 part /
sdc 8:32 0 264.3G 0 disk
└─mpath1 (dm-0) 252:0 0 264.3G 0 mpath
sdd 8:48 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sde 8:64 0 264.3G 0 disk
└─mpath3 (dm-3) 252:3 0 264.3G 0 mpath
sdf 8:80 0 264.3G 0 disk
└─mpath4 (dm-6) 252:6 0 264.3G 0 mpath
sdg 8:96 0 264.3G 0 disk
└─mpath5 (dm-7) 252:7 0 264.3G 0 mpath
sdh 8:112 0 264.3G 0 disk
├─sdh1 8:113 0 8M 0 part
├─sdh2 8:114 0 264.3G 0 part
└─mpath0 (dm-2) 252:2 0 264.3G 0 mpath
├─mpath0-part1 (dm-4) 252:4 0 8M 0 part
└─mpath0-part2 (dm-5) 252:5 0 264.3G 0 part /
sdi 8:128 0 264.3G 0 disk
└─mpath1 (dm-0) 252:0 0 264.3G 0 mpath
sdj 8:144 0 264.3G 0 disk
└─mpath2 (dm-1) 252:1 0 264.3G 0 mpath
sdk 8:160 0 264.3G 0 disk
└─mpath3 (dm-3) 252:3 0 264.3G 0 mpath
sdl 8:176 0 264.3G 0 disk
└─mpath4 (dm-6) 252:6 0 264.3G 0 mpath
sdm 8:192 0 264.3G 0 disk
└─mpath5 (dm-7) 252:7 0 264.3G 0 mpath
sr0 11:0 1 521.9M 0 rom |
|
| 2016-04-16 03:50:25 |
Nobuto Murata |
bug |
|
|
added subscriber Nobuto Murata |
| 2016-12-13 10:46:50 |
James Page |
ceph (Juju Charms Collection): status |
New |
Triaged |
|
| 2016-12-13 10:46:52 |
James Page |
ceph-mon (Juju Charms Collection): status |
New |
Triaged |
|
| 2016-12-13 10:46:55 |
James Page |
ceph (Juju Charms Collection): importance |
Undecided |
Low |
|
| 2016-12-13 10:46:57 |
James Page |
ceph-mon (Juju Charms Collection): importance |
Undecided |
Low |
|
| 2017-02-23 18:38:41 |
James Page |
charm-ceph: importance |
Undecided |
Low |
|
| 2017-02-23 18:38:41 |
James Page |
charm-ceph: status |
New |
Triaged |
|
| 2017-02-23 18:38:43 |
James Page |
ceph (Juju Charms Collection): status |
Triaged |
Invalid |
|
| 2017-02-23 18:39:56 |
James Page |
charm-ceph-mon: importance |
Undecided |
Low |
|
| 2017-02-23 18:39:56 |
James Page |
charm-ceph-mon: status |
New |
Triaged |
|
| 2017-02-23 18:39:57 |
James Page |
ceph-mon (Juju Charms Collection): status |
Triaged |
Invalid |
|
| 2018-06-26 02:33:40 |
Billy Olsen |
charm-ceph: status |
Triaged |
Won't Fix |
|
| 2018-06-26 02:34:01 |
Billy Olsen |
bug task added |
|
charm-ceph-osd |
|
| 2018-06-26 02:34:07 |
Billy Olsen |
charm-ceph-osd: status |
New |
Triaged |
|
| 2018-06-26 02:34:11 |
Billy Olsen |
charm-ceph-mon: status |
Triaged |
Invalid |
|
| 2018-06-26 02:34:17 |
Billy Olsen |
charm-ceph-osd: importance |
Undecided |
Low |
|
| 2022-05-06 07:03:00 |
nikhil kshirsagar |
charm-ceph-osd: assignee |
|
nikhil kshirsagar (nkshirsagar) |
|
