Vm failed to boot after restart with message /var/lib/glance drive is not ready or not present

[Tatyanka]

Steps:
1. Deploy Ubuntu Ha cluster with
2. Force reboot primary controller

Expected:
Vm was successfully rebooted

Actual:
Vm hangs with message /var/lib/glance drive is not ready or not present (see attached image), as soon as we skip this action /dev/image/glance is not available on vm
root@node-1:/var/log# pvs
  Couldn't find device with uuid JodsJS-7RQv-VBF5-f0kh-x7ZG-tqbG-0QKFvk.
  Couldn't find device with uuid 6wvu6m-gcMG-lg12-dGTD-2art-EURp-mUmpRc.
  Couldn't find device with uuid KLVRec-zeqf-B6N6-nZKn-ZMXF-5TJa-SsZDMv.
  PV VG Fmt Attr PSize PFree
  /dev/vda4 os lvm2 a-- 17.50g 4.00m
  /dev/vda5 image lvm2 a-- 31.89g 12.00m
  /dev/vdb3 image lvm2 a-- 49.39g 0
  /dev/vdc3 image lvm2 a-- 49.39g 0
  unknown device image lvm2 a-m 31.89g 12.00m
  unknown device image lvm2 a-m 49.39g 0
  unknown device image lvm2 a-m 49.39g 0

http://paste.openstack.org/show/408259/

lvdisplay
http://paste.openstack.org/show/408261/

root@node-1:/var/log# ls -la /dev/image/glance
ls: cannot access /dev/image/glance: No such file or directory

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "129"
  build_id: "2015-08-03_09-24-46"
  nailgun_sha: "0261b86bd0615d15531e6c8413ad276c5df385bf"
  python-fuelclient_sha: "71bb8fa87ee25f0c1bb84317884da7c917902a63"
  fuel-agent_sha: "1fe47720ba554818a0be707f2e16281791492d50"
  fuel-nailgun-agent_sha: "1512b9af6b41cc95c4d891c593aeebe0faca5a63"
  astute_sha: "f05bf102bbf8038066280a465a19a4dd1f75192d"
  fuel-library_sha: "d254369406a59186e03515aa44572377daf0cdba"
  fuel-ostf_sha: "53109a99d923cccdf88c5cf5aba0af8050df47e3"
  fuelmain_sha: "7a374fbd1f5ebde943cb391a4f71b94888ce4a15"
[root@nailgun ~]#

[Alexander Gordeev (a-gordeev)]

reassigning to fuel-python under 'ibp' tag.

[Sergey Yudin (tsipa740)]

Have tried to reproduce on

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "150"
  build_id: "2015-08-09_09-17-24"
  nailgun_sha: "58c080206cc17137b124744a40218c89beb6bb28"
  python-fuelclient_sha: "e069d9fde92451ec7f555342951807c5528e96e5"
  fuel-agent_sha: "57145b1d8804389304cd04322ba0fb3dc9d30327"
  fuel-nailgun-agent_sha: "e01693992d7a0304d926b922b43f3b747c35964c"
  astute_sha: "e1d3a435e5df5b40cbfb1a3acf80b4176d15a2dc"
  fuel-library_sha: "1851b4dff75170dbd63f6e15cde734e348e86d27"
  fuel-ostf_sha: "c7f745431aa3c147f2491c865e029e0ffea91c47"
  fuelmain_sha: "bdca75d0256338519c7eddd8a840ee6ecba7f992"

The issue was not reproduced. Accodring to Tatiana this issue is reproduceable to her env, so i will wait for Tatiana's env.

[Alexander Gordeev (a-gordeev)]

Changing status to Incomplete as attached snapshot doesn't contain /var/log/docker-logs directory from fuel-master at all.

There's no logs about provisioning, so there's absolutely no clue about what has happened.

However, I was able to find few traces of accidental reboot.

http://paste.openstack.org/show/412566/

All complains about missing devices http://paste.openstack.org/show/412568/
Searching for few of them indicates that 1 out of 3 missing uuid from every node belonged to /dev/vdc3 in the past 2015-08-04_18-03-28

i think something bad happened to virtual image of /dev/vdc. I'm not sure.

Is it still reproducible?

[Alexander Gordeev (a-gordeev)]

Forwarded to reporter. Waiting for reply.

[Alexander Gordeev (a-gordeev)]

Tatyana was unable to reproduce bug yesterday too, so we came to agreement to close this bug as Invalid.