resize-volume api is not working for mysql

Bug #1291896 reported by shivam shukla
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack DBaaS (Trove)
Fix Released
High
Sushil Kumar

Bug Description

trove api resize volume is not working .

trove resize-volume <instance ID> <size>

It is throwing below error in taskmanager logs:

 GuestError: An error occurred communicating with the guest: Remote error: GuestError Error resizing the filesystem: /dev/vdb
2014-03-12 17:49:30.757 TRACE trove.openstack.common.rpc.amqp [u'Traceback (most recent call last):\n', u' File "/home/ubuntu/trove/trove/openstack/common/rpc/amqp.py", line 440, in _process_data\n **args)\n', u' File "/home/ubuntu/trove/trove/openstack/common/rpc/dispatcher.py", line 172, in dispatch\n result = getattr(proxyobj, method)(ctxt, **kwargs)\n', u' File "/home/ubuntu/trove/trove/guestagent/datastore/mysql/manager.py", line 194, in resize_fs\n device.resize_fs(mount_point)\n', u' File "/home/ubuntu/trove/trove/guestagent/volume.py", line 110, in resize_fs\n self.device_path)\n', u'GuestError: Error resizing the filesystem: /dev/vdb\n']..
2014-03-12 17:49:30.757 TRACE trove.openstack.common.rpc.amqp

Changed in trove:
assignee: nobody → shivam shukla (shivam-shukla)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to trove (master)

Fix proposed to branch: master
Review: https://review.openstack.org/80315

Changed in trove:
assignee: shivam shukla (shivam-shukla) → Sushil Kumar (sushil-kumar2)
status: New → In Progress
Revision history for this message
Sushil Kumar (sushil-kumar2) wrote :

Tried the code few times, to dig into the issue.

Found that somehow the disk detach/attach from nova server instance is not functioning properly when using cinder APIs.

Need to update the code to use the Nova APIs instead of cinder APIs.

Revision history for this message
Denis M. (dmakogon) wrote :

I'd better ask cinder time why this API calls are not working.

Revision history for this message
Denis M. (dmakogon) wrote :

*cinder team

Revision history for this message
Mat Lowery (mat-lowery) wrote :

It's not clear to me how a volume detach from an instance would have ever worked by calling Cinder since Cinder does not control the actual attachment to Nova instances (however Cinder does store attachment metadata in its tables). While Cinder and python-cinderclient (Python bindings not CLI) contains methods/APIs that appear to be attach- and detach-related, those are metadata-related and do not perform the actual attach or detach. To prove that, consider that Cinder CLI does not have attach- or detach-related subcommands. To summarize, my belief is that nothing is wrong with Cinder and Nova volume attachment-related APIs should be called.

Revision history for this message
Viswa Vutharkar (vvutharkar) wrote :

Relevant code to prove Mat's point

 cinder client (v2) 'detach' API call is a legal call ( https://github.com/openstack/python-cinderclient/blob/master/cinderclient/v2/volumes.py#L260)

But the problem is that it only clear's the volume attachment metadata that Mat pointed out. You can see this from what call on the Cinder API server side get's executed ( https://github.com/openstack/cinder/blob/master/cinder/api/contrib/volume_actions.py#L122 )

Changed in trove:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to trove (master)

Reviewed: https://review.openstack.org/80315
Committed: https://git.openstack.org/cgit/openstack/trove/commit/?id=d62a1cf5585356a16bf08719155daeb170c21cfa
Submitter: Jenkins
Branch: master

commit d62a1cf5585356a16bf08719155daeb170c21cfa
Author: Sushil Kumar <email address hidden>
Date: Fri Mar 14 13:36:08 2014 +0000

    Resolves volume resize issue

    Reasons:
    - Cinder APIs are not able to complete the detach/attach properly with
      nova instance.
    - resize2fs is not proceeding with e2fsck having "-f" and "-n" options.

    Changes:
    - Used nova-volume attach-detach APIs instead of cinder APIs.
    - Updated e2fsck to use -p option to complete the e2fsck action so that
      resize2fs could proceed.
    - Updated tests as per replaced Nova APIs instead of Cinder APIs.

    Change-Id: I40f1dc8ca04b3bfcc4eae910515fcedf922c1a9e
    Closes-Bug: #1291896

Changed in trove:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in trove:
milestone: none → juno-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in trove:
milestone: juno-1 → 2014.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.