volume resize does not works as expected for Vertica

[Sushil Kumar]

while testing volume resize with vertica found that found that because of an incomplete unmount the resize fails

while debugging the situation found that mount-point never got unmount in gues

ubuntu@test-1:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 19G 1.6G 17G 9% /
udev 998M 12K 998M 1% /dev
tmpfs 201M 220K 201M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1002M 0 1002M 0% /run/shm
/dev/vdb 1008M 70M 887M 8% /var/lib/vertica

tried to do that manually and it suggested that device is busy

ubuntu@test-1:~$ sudo umount /dev/vdb
umount: /var/lib/vertica: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))

and on tracing the process occupying the disk found that guestagent itself is holding locks over the disk

 ubuntu@test-1:~$ fuser -vm /dev/vdb
                     USER PID ACCESS COMMAND
/dev/vdb: root kernel mount /dev
                     ubuntu 928 F.... trove-guestagen

after i stopped trove-guest service found that now kernel blocks probably because what all happened after incomplete unmount

ubuntu@test-1:~$ fuser -vm /dev/vdb
                     USER PID ACCESS COMMAND
/dev/vdb: root kernel mount /dev

probably failure was not detected by the pexpect process which originally tried to do it in actual place from volume.py, which is why next scenarios came in existence
Reverified the pexpect way of unmounting, and it does not show any error

>>> import pexpect
>>> cmd = "sudo umount %s" % "/dev/vdb"
>>> cmd
'sudo umount /dev/vdb'
>>> child = pexpect.spawn(cmd)
>>> child.expect(pexpect.EOF)
0
>>> quit()

which is why control went ahead.

[Sushil Kumar (sushil-kumar2)]

Do not want to block Vertica BP/patchset for this resize issue, but would look into this resize issue, why is this occurring.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/166382

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/166382
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=68c4246998eb13511d48f151b643be3911d321cf
Submitter: Jenkins
Branch: master

commit 68c4246998eb13511d48f151b643be3911d321cf
Author: Sushil Kumar <email address hidden>
Date: Fri Mar 20 20:54:51 2015 +0000

    Fixes the resize APIs for Vertica-guest

    Identified that there was a service vertica-agent still running
    after Vertica database was stopped, because of which Volume mount
    was not complete and hence resize-volume was failing.

    start_db_with_conf_changes implemented to start Vertica to complete
    the workflow of resize-instance.

    Also, updated the raise exception for bad-cluster-config to raise
    RuntimeError, and removed not needed raise from prepare exception
    handler.

    Change-Id: I8ccef3aef7725b0c7d46fd8851d69d3d80cfa995
    Closes-Bug: #1433852