Attaching volume to instance failed on Hyper-V using IPv6

Bug #1148536 reported by ChangBo Guo(gcb)
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Undecided
Alessandro Pilotti

Bug Description

I want to attach iSCSI-based volume to a instance on Hyper-V . But the operation failed due to two issues .

1) Test environment
     control node : KVM running all nova services except nova-compute service , cinder-scheduler ,cinder-api services .
     compute node: Hyper-V (windows server 2012 ), running nova-compute service ,cinder-volume service

2) Test steps

    create a volume with 'cinder create 1' , boot a instance on Hyper-V ,then run the following command on control node
    'nova volume-attach 505d1eeb-4436-46dd-acb1-1ba50d2554e8 7b550385-a3e6-47f4-b7c8-acaee7d4316b /dev/sdh"

3) Nova Issues

   1. Parse IPv6 address and port wrong on Hper-V compute driver
      https://github.com/openstack/nova/blob/master/nova/virt/hyperv/volumeutilsv2.py#48
      separator = target_portal.find(':')
      this line can't handle IPv6 address/port like '2002:96f:2458::96f:2458:3260'
      Need replace with separator = target_portal.rfind(':')

Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :
Download full text (5.5 KiB)

This is the log :
 1.
2013-03-04 17:43:40.371 15412 ERROR cinder.openstack.common.rpc.amqp [-] Exception during message handling
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last):
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\cinder-2013.1-py2.7.egg\cinder\openstack\common\rpc\amqp.py", line 275, in _process_data
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args)
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\cinder-2013.1-py2.7.egg\cinder\openstack\common\rpc\dispatcher.py", line 145, in dispatch
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs)
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\cinder-2013.1-py2.7.egg\cinder\volume\manager.py", line 623, in initialize_connection
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp return self.driver.initialize_connection(volume_ref, connector)
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\cinder-2013.1-py2.7.egg\cinder\volume\drivers\windows.py", line 88, in initialize_connection
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp wt_idmethod.Value = initiator_name
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\wmi.py", line 572, in __setattr__
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp self._cached_properties (attribute).set (value)
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\wmi.py", line 446, in set
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp self.property.Value = value
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp File "C:\Program Files (x86)\Cloudbase Solutions\OpenStack\Python27\lib\site-packages\win32com\client\dynamic.py", line 544, in __setattr__
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp self._oleobj_.Invoke(entry.dispid, 0, invoke_type, 0, value)
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp TypeError: Objects of type 'dict' can not be converted to a COM VARIANT
2013-03-04 17:43:40.371 15412 TRACE cinder.openstack.common.rpc.amqp
2013-03-04 17:43:40.388 15412 ERROR cinder.openstack.common.rpc.common [-] Returning exception Objects of type 'dict' can not be converted to a COM VARIANT to caller

2.
2013-03-05 13:44:33.470 12332 TRACE nova.compute.manager [instance: 505d1eeb-4436-46dd-acb1-1ba50d2554e8] Traceback (most recent call last):
2013-03-05 13:44:33...

Read more...

Revision history for this message
Alessandro Pilotti (alexpilotti) wrote :

Good catch!

summary: - Attaching volume to instance failed on Hyper-V
+ Attaching volume to instance failed on Hyper-V using IPv6
Changed in nova:
assignee: nobody → Alessandro Pilotti (alexpilotti)
status: New → Incomplete
Revision history for this message
Alessandro Pilotti (alexpilotti) wrote :

This bug needs to be splitted in two separate ones:

One that affects nova (can be this one):
2. Parse IPv6 address and port wrong on Hper-V compute driver

One that affects Cinder
1. TypeError: Objects of type 'dict' can not be converted to a COM VARIANT

Changed in nova:
assignee: Alessandro Pilotti (alexpilotti) → nobody
Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

I have created another bug to track issue 'TypeError: Objects of type 'dict' can not be converted to a COM VARIANT'

https://bugs.launchpad.net/cinder/+bug/1149467

description: updated
Changed in nova:
status: Incomplete → New
assignee: nobody → Alessandro Pilotti (alexpilotti)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/23697

Changed in nova:
status: New → In Progress
Revision history for this message
Alessandro Pilotti (alexpilotti) wrote :

Can you please test it?

https://review.openstack.org/#/c/23697/

Thanks!

Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

I had tested it , this worked for IPv6 .
thanks

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/23697
Committed: http://github.com/openstack/nova/commit/958b68fb801b1c87b1e2878738c1d74f1aae7dac
Submitter: Jenkins
Branch: master

commit 958b68fb801b1c87b1e2878738c1d74f1aae7dac
Author: Alessandro Pilotti <email address hidden>
Date: Wed Mar 6 16:56:29 2013 +0200

    Fixes volume attach on Hyper-V with IPv6

    Fixes bug: 1148536

    This patch splits the target_portal address:port pair properly in case of IPv6
    addresses during login on the storage target.

    Change-Id: I87c1153dd0289ac73fb35c2cd57a27d128042b2c

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

 In Comment #7 , I tested patchset 1 using string .rfind() , not patchset2 which was not there by that time .

I found , the patchset 2 https://review.openstack.org/23697 can't handle the issue I found .
  u'target_portal': u'2002:96f:2458::96f:2458:3260'

[root@dirpc5 /]# python
Python 2.6.6 (r266:84292, May 1 2012, 13:52:17)
[GCC 4.4.6 20110731 (Red Hat 4.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import netaddr
>>> server_str="2002:96f:2458::96f:2458:3260"
>>> if netaddr.valid_ipv6(server_str):
... print(server_str)
... else:
... print("continue parsing...")
...
2002:96f:2458::96f:2458:3260
>>>
see https://github.com/openstack/nova/blob/master/nova/utils.py#888

Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

OR we need format the string '2002:96f:2458::96f:2458:3260' to [2002:96f:2458::96f:2458]:3260 when we get the address and port ?

Revision history for this message
Alessandro Pilotti (alexpilotti) wrote :

Can you please replace find() with rfind() at the following line?

https://github.com/openstack/nova/blob/master/nova/utils.py#L906

Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

the issue is target address portal string like '2002:96f:2458::96f:2458:3260' will be parsed as a valid IPv6 addresss [2002:96f:2458::96f:2458:3260] returned in https://github.com/openstack/nova/blob/master/nova/utils.py#898 ,
maybe we need deal with IPv6 at the place where It was got , format it as [2002:96f:2458::96f:2458:] .

Revision history for this message
Alessandro Pilotti (alexpilotti) wrote :

I did a quick bit of research on the subject:

http://tools.ietf.org/html/rfc5952

6. Notes on Combining IPv6 Addresses with Port Numbers

Excerpt: (referring to IPv6 addresses):

"The [] style as expressed in [RFC3986] SHOULD be employed, and is the default unless otherwise specified."

So the proposed approach in comment #10 is IMO the correct solution, please ignore comment #11.

Revision history for this message
ChangBo Guo(gcb) (glongwave) wrote :

Yes ,you are right , we need follow the standard , thanks for your research .

Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-rc1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-rc1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.