The disk drive for /var/lib/glance in not ready yet or not present

[Igor Shishkin]

Configuration: Ubuntu HA Neutrong GRE. @4.0-130
One some of nodes contains "The disk drive for /var/lib/glance in not ready yet or not present" message on the splash screen.

root@node-8:~# mount -a
mount: wrong fs type, bad option, bad superblock on /dev/mapper/image-glance,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so

root@node-8:~# fgrep glance /etc/fstab
/dev/mapper/image-glance /var/lib/glance xfs defaults 0 0

root@node-8:~# lvdisplay
  --- Logical volume ---
  LV Name /dev/image/glance
  VG Name image
  LV UUID TT0vIv-fheM-zBcN-Yzb1-1pKp-MI6L-2IedSP
  LV Write Access read/write
  LV Status available
  # open 0
  LV Size 16.19 GiB
  Current LE 518
  Segments 1
  Allocation inherit
  Read ahead sectors auto
  - currently set to 256
  Block device 252:0

root@node-8:~# ls -al /dev/image/glance
lrwxrwxrwx 1 root root 7 Dec 12 11:45 /dev/image/glance -> ../dm-0

root@node-8:~# blkid /dev/dm-0
root@node-8:~#

[Mike Scherbakov (mihgen)]

Was it installation with Ceph or Swift?

[Igor Shishkin (teran)]

ceph for both of images and volumes

[Mike Scherbakov (mihgen)]

Clearing up assignee for this. Please assign if only you started to work on this and expect the fix in a day or two.

[Vladimir Kozhukalov (kozhukalov)]

Partitioning data are correct. Pmanager treats those data correctly, it makes xfs file system on glance volume. The problem with mounting was likely to be connected with the fact that xfs is not power down tolerant. By the way I've added some logging lines into early and late commands in order to be able to check actual late commands. https://review.openstack.org/62654

[Vladimir Kozhukalov (kozhukalov)]

And now which of the nodes is failed to create glance partition?

[Vladimir Kozhukalov (kozhukalov)]

Сan we get logs which are collected with this patch https://review.openstack.org/62654?

[Mike Scherbakov (mihgen)]

Please take a look at this: https://bugs.launchpad.net/fuel/+bug/1262864
Looks very similar. Let's research if these issues have same root.

[Jason Venner (jvenner-e)]

I am having this problem in my bare metal cluster

1 node out of 9 had this problem
Install hung at installing ubuntu on this one node and 29%

[Igor Shishkin (teran)]

Can't reproduce nor this nor linked bug with Vladimir's patch. So I suggest to decrease importance of this bug.
Jason, what ISO did you tried to use?

[Dr_Drache (biomatrix)]

was told to post this information :
http://paste.openstack.org/show/62270/

and attached is my diag log

[Vladimir Kozhukalov (kozhukalov)]

Dr_Drache, who you was told by? Can you provide some context here about your attachment? As fa as I remember the problem was with smart array RAIDs. You provided us with logs which helped us to find out where the problem was. It appears because we have centos based discovery ramdisk which reports the names of hard drives on node which differ from those which appear under the debian-installer kernel. So we made a couple of dirty patches https://review.openstack.org/#/c/70316/1 and https://review.openstack.org/#/c/70323/ which we hope solve the problem with smart array devices.

[Vladimir Kozhukalov (kozhukalov)]

https://bugs.launchpad.net/fuel/+bug/1271549

[Andrew Woodward (xarses)]

This issue is most likely the same as https://bugs.launchpad.net/fuel/+bug/1297792. Since it's fixed on that bug, going to mark this as duplicate