Bug #1833160 “VNX: driver cannot start under py37 due to wrap_so...” : Bugs : OpenStack Shared File Systems Service (Manila)

Tom Barron (tpb) on 2019-06-18

tags:

added: dell-emc driver python3 vnx

Revision history for this message

Jason Grosso (jgrosso) wrote on 2019-06-27:

#1

Hey Ryan is there dell-emc engineer we can assign this driver issues too? or are you taking it?

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-06-28:

#2

Actually, it isn't an issue of dellemc driver. It looks more like the eventlet issue: https://github.com/eventlet/eventlet/issues/526.

The reason why I filed this issue is to track the same issue which could meet by other users/in other drivers.

BTW, do you notice if there is any other driver meeting the same issue? What are their fixings or workarounds?

Revision history for this message

Tom Barron (tpb) wrote on 2019-06-28:

#3

Download full text (3.9 KiB)

Hmm, I see urllib used in the following:

tbarron@localhost manila (master)$ grep -r urllib manila/share/drivers/quobyte/share/drivers/huawei/share/drivers/huawei/share/drivers/huawei/share/drivers/huawei/share/drivers/nexenta/share/drivers/nexenta/share/drivers/nexenta/share/drivers/qnap/api.share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/qnap/api.py: manila/share/drivers/zfssa/restclienshare/drivers/zfssa/restclienshare/drivers/zfssa/restclient.py: manila/share/drivers/netapp/share/drivers/netapp/share/drivers/netapp/share/drivers
/>jsonrpc.py:import six.moves.urllib.parse as urlparse
/>v3/helper.py: requests.packages.urllib3.disable_warnings(
/>v3/helper.py: requests.packages.urllib3.exceptions.InsecureRequestWarning)
/>v3/helper.py: requests.packages.urllib3.disable_warnings(
/>v3/helper.py: requests.packages.urllib3.exceptions.InsecurePlatformWarning)
/>ns5/jsonrpc.py:from requests.packages.urllib3 import exceptions
/>ns5/jsonrpc.py:requests.packages.urllib3.disable_warnings(exceptions.InsecureRequestWarning)
/>ns5/jsonrpc.py:requests.packages.urllib3.disable_warnings(
/>py:from six.moves import urllib
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params, True)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
sanitized_params = urllib.parse.urlencode(sanitized_params)
/>t.py:from six.moves.urllib import error as urlerror
/>t.py:from six.moves.urllib import request as urlrequest
"""ZFSSA urllib client."""
/>dataontap/client/api.py:from six.moves import urllib
/>dataontap/client/api.py: except urllib.error.HTTPError as e:
/>dataontap/client/api.py: except urllib...

Hmm, I see urllib used in the following:

tbarron@localhost manila (master)$ grep -r urllib manila/share/drivers
manila/share/drivers/quobyte/jsonrpc.py:import six.moves.urllib.parse as urlparse
manila/share/drivers/huawei/v3/helper.py:        requests.packages.urllib3.disable_warnings(
manila/share/drivers/huawei/v3/helper.py:            requests.packages.urllib3.exceptions.InsecureRequestWarning)
manila/share/drivers/huawei/v3/helper.py:        requests.packages.urllib3.disable_warnings(
manila/share/drivers/huawei/v3/helper.py:            requests.packages.urllib3.exceptions.InsecurePlatformWarning)
manila/share/drivers/nexenta/ns5/jsonrpc.py:from requests.packages.urllib3 import exceptions
manila/share/drivers/nexenta/ns5/jsonrpc.py:requests.packages.urllib3.disable_warnings(exceptions.InsecureRequestWarning)
manila/share/drivers/nexenta/ns5/jsonrpc.py:requests.packages.urllib3.disable_warnings(
manila/share/drivers/qnap/api.py:from six.moves import urllib
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params, True)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/qnap/api.py:        sanitized_params = urllib.parse.urlencode(sanitized_params)
manila/share/drivers/zfssa/restclient.py:from six.moves.urllib import error as urlerror
manila/share/drivers/zfssa/restclient.py:from six.moves.urllib import request as urlrequest
manila/share/drivers/zfssa/restclient.py:    """ZFSSA urllib client."""
manila/share/drivers/netapp/dataontap/client/api.py:from six.moves import urllib
manila/share/drivers/netapp/dataontap/client/api.py:        except urllib.error.HTTPError as e:
manila/share/drivers/netapp/dataontap/client/api.py:        except urllib.error.URLError as e:
manila/share/drivers/netapp/dataontap/client/api.py:        request = urllib.request.Request(
manila/share/drivers/netapp/dataontap/client/api.py:        opener = urllib.request.build_opener(auth_handler)
manila/share/drivers/netapp/dataontap/client/api.py:        password_man = urllib.request.HTTPPasswordMgrWithDefaultRealm()
manila/share/drivers/netapp/dataontap/client/api.py:        auth_handler = urllib.request.HTTPBasicAuthHandler(password_man)
manila/share/drivers/dell_emc/common/enas/connector.py:from six.moves.urllib import error as url_error
manila/share/drivers/dell_emc/common/enas/connector.py:from six.moves.urllib import request as url_request
manila/share/drivers/hitachi/hsp/rest.py:requests.packages.urllib3.disable_warnings()  # pylint: disable=no-member

Revision history for this message

Tom Barron (tpb) wrote on 2019-06-28:

#4

Netapp folks report that they aren't yet running CI with python 3.7 but will check to see if they have the same issue under 3.7.

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-07-02:

#5

Hey Ryan, we also use the library urllib to communicate with the storage in the NetApp driver. I've created an environment that uses python 3.7 and tested the communication with our storage. Everything seems working fine.

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-07-02:

#6

Here are some command outputs:
pythirty-seven-test:/opt/stack/new/manila$ python3 --version
Python 3.7.3

Here is an example of a returned value from the storage:
Jul 02 12:40:39.549043 pythirty-seven-test manila-share[12663]: DEBUG manila.share.drivers.netapp.dataontap.client.api [None req-8eaf5be4-1a24-44f0-9ecb-31a381f41a4f None None] Request: b'<netapp xmlns="http://www.netapp.com/filer/admin" version="1.140">\n <aggr-get-iter>\n <query>\n <aggr-attributes>\n <aggregate-name>aggr1</aggregate-name>\n </aggr-attributes>\n </query>\n <desired-attributes>\n <aggr-attributes>\n <aggregate-name/>\n <aggr-ownership-attributes>\n <home-name/>\n </aggr-ownership-attributes>\n </aggr-attributes>\n </desired-attributes>\n <max-records>50</max-records>\n </aggr-get-iter>\n</netapp>\n' {{(pid=12685) invoke_elem /opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py:242}}

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-07-03:

#7

What version of eventlet and urllib are you using?

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-08:

#8

This issue occurs when below conditions meet together:
1. Python version is 3.7.3.
2. Using python/urllib (via six.moves.urllib or other ways).
3. Using https connection where ssl is involved.
4. Patching via eventlet.

The reason is that ssl.wrap_socket function prototype changes from python3.7.3, `_context` is not a parameter anymore. Check source codes /usr/local/lib/python3.7/ssl.py for detail.
However, eventlet doesn't change the prototype accordingly.

So, you'll see the error `TypeError: wrap_socket() got an unexpected keyword argument '_context'` above when eventlet does the patch.

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-12:

#9

Hi, Ryan! I'm so sorry I've missed the question you asked. Here are the versions of urllib, eventlet and six I used in the environment to test the NetApp driver against py37.

urllib:
Name: urllib3
Version: 1.25.3
Summary: HTTP library with thread-safe connection pooling, file post, and more.
Home-page: https://urllib3.readthedocs.io/
Author: Andrey Petrov
Author-email: <email address hidden>
License: MIT
Location: /usr/local/lib/python3.7/dist-packages

eventlet:
Name: eventlet
Version: 0.25.0
Summary: Highly concurrent networking library
Home-page: http://eventlet.net
Author: Linden Lab
Author-email: <email address hidden>
License: UNKNOWN
Location: /usr/local/lib/python3.7/dist-packages
Requires: greenlet, six, dnspython, monotonic

six:
Name: six
Version: 1.12.0
Summary: Python 2 and 3 compatibility utilities
Home-page: https://github.com/benjaminp/six
Author: Benjamin Peterson
Author-email: <email address hidden>
License: MIT
Location: /usr/local/lib/python3.7/dist-packages
Requires:

These outputs I received while running the command tox show `package_name`
I've realized that our versions (urllib and eventlet) are very different. I tried to check for upgrades for each package I've pasted here but pip says I'm already up to date.
Let me know if I can help you with something.
And sorry for the delay again

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-12:

#10

s/while running the command tox show/while running the command pip show

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-13:

#11

Hi Carlos, which version of python are you using? Could you give the full version number? python3.7.2 or python3.7.3 or others?

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-13:

#12

Download full text (3.4 KiB)

Oh, I see the version of python you are using. It's python3.7.3. Forget my last comment.
Are you using https connection? If yes, it doesn't make sense.

In the source of eventlet (0.25.0), /usr/local/lib/python3.7/site-packages/eventlet/green/ssl.py:

431 if hasattr(__ssl, 'SSLContext'):
432 _original_sslcontext = __ssl.SSLContext
433
434 class GreenSSLContext(_original_sslcontext):
435 __slots__ = ()
436
437 def wrap_socket(self, sock, *a, **kw):
438 return GreenSSLSocket(sock, *a, _context=self, **kw) # <<<<< _context passed in here

43 class GreenSSLSocket(_original_sslsocket):
57 def __new__(cls, sock=None, keyfile=None, certfile=None,
58 server_side=False, cert_reqs=CERT_NONE,
59 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
60 do_handshake_on_connect=True, *args, **kw): # <<<<< _context passed in via **kw
61 if _is_under_py_3_7:
62 return super(GreenSSLSocket, cls).__new__(cls)
63 else:
64 if not isinstance(sock, GreenSocket):
65 sock = GreenSocket(sock)
66 with _original_ssl_context():
67 ret = _original_wrap_socket(
68 sock=sock.fd,
69 keyfile=keyfile,
70 certfile=certfile,
71 server_side=server_side,
72 cert_reqs=cert_reqs,
73 ssl_version=ssl_version,
74 ca_certs=ca_certs,
75 do_handshake_on_connect=False,
76 *args, **kw # <<<<< _context passed in via **kw
77 )

1 __ssl = __import__('ssl')
25 _original_wrap_socket = __ssl.wrap_socket

But in python37's /usr/local/lib/python3.7/ssl.py

1215 def wrap_socket(sock, keyfile=None, certfile=None, # <<<<< no _context
1216 server_side=False, cert_reqs=CERT_NONE,
1217 ssl_version=PROTOCOL_TLS, ca_certs=None,
1218 do_handshake_on_connect=True,
1219 suppress_ragged_eofs=True,
1220 ciphers=None):

I made a quick workaround in eventlet, /usr/local/lib/python3.7/site-packages/eventlet/green/ssl.py.
Then it works.

57 def __new__(cls, sock=None, keyfile=None, certfile=None,
58 server_side=False, cert_reqs=CERT_NONE,
59 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
60 do_handshake_on_connect=True, *args, **kw):
61 if _is_under_py_3_7:
62 return super(GreenSSLSocket, cls).__new__(cls)
63 else:
64 if not isinstance(sock, GreenSocket):
65 sock = GreenSocket(sock)
66 with _original_ssl_context():
67 for ignore in ('_context', 'server_hostname'): # <<<<< remove _context and server_hostname
68 if ignore in kw:
69 kw.pop(ignore)
70 ret = _original_wrap_socket(
71 sock=sock.fd,
72 keyfile=keyfile,
73 certfile=certfile,
7...

Oh, I see the version of python you are using. It's python3.7.3. Forget my last comment.
Are you using https connection? If yes, it doesn't make sense.

In the source of eventlet (0.25.0), /usr/local/lib/python3.7/site-packages/eventlet/green/ssl.py:

431 if hasattr(__ssl, 'SSLContext'):
432     _original_sslcontext = __ssl.SSLContext
433 
434     class GreenSSLContext(_original_sslcontext):
435         __slots__ = ()
436 
437         def wrap_socket(self, sock, *a, **kw):
438             return GreenSSLSocket(sock, *a, _context=self, **kw)    # <<<<< _context passed in here

43 class GreenSSLSocket(_original_sslsocket):
 57     def __new__(cls, sock=None, keyfile=None, certfile=None,
 58                 server_side=False, cert_reqs=CERT_NONE,
 59                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
 60                 do_handshake_on_connect=True, *args, **kw):       # <<<<< _context passed in via **kw
 61         if _is_under_py_3_7:
 62             return super(GreenSSLSocket, cls).__new__(cls)
 63         else:
 64             if not isinstance(sock, GreenSocket):
 65                 sock = GreenSocket(sock)
 66             with _original_ssl_context():
 67                 ret = _original_wrap_socket(
 68                     sock=sock.fd,
 69                     keyfile=keyfile,
 70                     certfile=certfile,
 71                     server_side=server_side,
 72                     cert_reqs=cert_reqs,
 73                     ssl_version=ssl_version,
 74                     ca_certs=ca_certs,
 75                     do_handshake_on_connect=False,
 76                     *args, **kw                                   # <<<<< _context passed in via **kw
 77                 )

1 __ssl = __import__('ssl')
 25 _original_wrap_socket = __ssl.wrap_socket

But in python37's /usr/local/lib/python3.7/ssl.py

1215 def wrap_socket(sock, keyfile=None, certfile=None,              # <<<<< no _context
1216                 server_side=False, cert_reqs=CERT_NONE,
1217                 ssl_version=PROTOCOL_TLS, ca_certs=None,
1218                 do_handshake_on_connect=True,
1219                 suppress_ragged_eofs=True,
1220                 ciphers=None):

I made a quick workaround in eventlet, /usr/local/lib/python3.7/site-packages/eventlet/green/ssl.py.
Then it works.

57     def __new__(cls, sock=None, keyfile=None, certfile=None,
 58                 server_side=False, cert_reqs=CERT_NONE,
 59                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
 60                 do_handshake_on_connect=True, *args, **kw):
 61         if _is_under_py_3_7:
 62             return super(GreenSSLSocket, cls).__new__(cls)
 63         else:
 64             if not isinstance(sock, GreenSocket):
 65                 sock = GreenSocket(sock)
 66             with _original_ssl_context():
 67                 for ignore in ('_context', 'server_hostname'):   # <<<<< remove _context and server_hostname
 68                     if ignore in kw:
 69                         kw.pop(ignore)
 70                 ret = _original_wrap_socket(
 71                     sock=sock.fd,
 72                     keyfile=keyfile,
 73                     certfile=certfile,
 74                     server_side=server_side,
 75                     cert_reqs=cert_reqs,
 76                     ssl_version=ssl_version,
 77                     ca_certs=ca_certs,
 78                     do_handshake_on_connect=False,
 79                     *args, **kw
 80                 )

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-19:

#13

I tested with both HTTP and HTTPS. The first test I've performed was with HTTP, it worked perfectly. Then I've manually changed the driver to work with HTTPS and the communication is also working fine.

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-21:

#14

I wrote a sample code with `eventlet`, `urllib` and `python37`: https://gist.github.com/Murray-LIANG/9b506d98cf76d633a9d0e82f2bb5ece5

It raises the same exception as reported by this bug:
TypeError: wrap_socket() got an unexpected keyword argument '_context'

And the sample code is like the one in netapp dataontap driver (manila/share/drivers/netapp/dataontap/client/api.py).
Both of them use `urllib.request.HTTPBasicAuthHandler` to open requests.

I have no netapp device, so I can't figure out how netapp dataontap driver works around this issue.

The environment I am using:
Python 3.7.2, 3.7.3 and 3.7.4
eventlet==0.25.0
six==1.12.0

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-21:

#15

Hey, Ryan! I've tested this script you created in my environment and I had the same problem you're facing. The stacktrace I got while running the script is:

... resp = opener.open(req)
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/lib/python3.7/http/client.py", line 1392, in connect
    server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 438, in wrap_socket
    return GreenSSLSocket(sock, *a, _context=self, **kw)
  File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 76, in __new__
    *args, **kw
TypeError: wrap_socket() got an unexpected keyword argument '_context'
>>> print(resp)

I've looked how the NetApp driver works it around since I'm using the same library versions you're using and the driver is not breaking. I didn't found things that are strongly different.
I've added a log to show as a dict the created request object. Here is my output:

{'_full_url': 'http://x.x.x.x:80/servlets/netapp.servlets.admin.XMLrequest_filer', 'fragment': None, 'type': 'http', 'host': 'x.x.x.x:80', 'selector': '/servlets/netapp.servlets.admin.XMLrequest_filer', 'headers': {'Content-type': 'text/xml', 'Charset': 'utf-8'}, 'unredirected_hdrs': {}, '_data': b'<netapp xmlns="http://www.netapp.com/filer/admin" version="1.140"><perf-object-counter-list-info><objectname>processor</objectname></perf-object-counter-list-info></netapp>', '_tunnel_host': None, 'origin_req_host': 'x.x.x.x', 'unverifiable': False}

You can see if there is something different.
Also, does the VNX driver mounts the request url with a port?
You can take a look in the way NetApp driver does it [1]. There are few different things I've realized while comparing the way each driver create the requests. I think you're very close to figure out a solution to this problem.
[1] https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/client/api.py#L230

Hey, Ryan! I've tested this script you created in my environment and I had the same problem you're facing. The stacktrace I got while running the script is:

... resp = opener.open(req)
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/lib/python3.7/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/lib/python3.7/http/client.py", line 1392, in connect
    server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 438, in wrap_socket
    return GreenSSLSocket(sock, *a, _context=self, **kw)
  File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 76, in __new__
    *args, **kw
TypeError: wrap_socket() got an unexpected keyword argument '_context'
>>> print(resp)

I've looked how the NetApp driver works it around since I'm using the same library versions you're using and the driver is not breaking. I didn't found things that are strongly different.
I've added a log to show as a dict the created request object. Here is my output:

{'_full_url': 'http://x.x.x.x:80/servlets/netapp.servlets.admin.XMLrequest_filer', 'fragment': None, 'type': 'http', 'host': 'x.x.x.x:80', 'selector': '/servlets/netapp.servlets.admin.XMLrequest_filer', 'headers': {'Content-type': 'text/xml', 'Charset': 'utf-8'}, 'unredirected_hdrs': {}, '_data': b'<netapp xmlns="http://www.netapp.com/filer/admin" version="1.140"><perf-object-counter-list-info><objectname>processor</objectname></perf-object-counter-list-info></netapp>', '_tunnel_host': None, 'origin_req_host': 'x.x.x.x', 'unverifiable': False}

You can see if there is something different.
Also, does the VNX driver mounts the request url with a port?
You can take a look in the way NetApp driver does it [1]. There are few different things I've realized while comparing the way each driver create the requests. I think you're very close to figure out a solution to this problem.
[1] https://github.com/openstack/manila/blob/master/manila/share/drivers/netapp/dataontap/client/api.py#L230

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-22:

#16

So you're still using http instead of https to connect to netapp device.
'_full_url': 'http://x.x.x.x:80/servlets/netapp.servlets.admin.XMLrequest_filer'
'type': 'http'

Could you switch it to https and have another try?
urllib will use different ways to open the connection when the `type` is diffferent (http or https).

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-22:

#17

Download full text (10.6 KiB)

I haven't noticed that the driver was not using https to make its calls. Thanks for pointing it out. There are two points where we set http or https. Now I've forced it in both places and I had the same mistake you're having. If I use HTTP instead of HTTPS, it works correctly.

Aug 22 17:57:44.809258 37-carlos-pythirty-seven-test manila-share[1481]: WARNING manila.share.drivers.netapp.dataontap.client.api [None req-3970f319-d985-46ac-9d03-50211f861978 None None] {'_full_url': 'https://x.x.x.x:80/servlets/netapp.servlets.admin.XMLrequest_filer', 'fragment': None, 'type': 'https', 'host': 'x.x.x.x:80', 'selector': '/servlets/netapp.servlets.admin.XMLrequest_filer', 'headers': {'Content-type': 'text/xml', 'Charset': 'utf-8'}, 'unredirected_hdrs': {}, '_data': b'<netapp xmlns="https://www.netapp.com/filer/admin" version="1.15"><system-get-ontapi-version/></netapp>', '_tunnel_host': None, 'origin_req_host': 'x.x.x.x', 'unverifiable': False}
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager [None req-3970f319-d985-46ac-9d03-50211f861978 None None] Error encountered during initialization of driver NetAppCmodeMultiSvmShareDriver@37-carlos-pythirty-seven-test@netapp2: manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - unknown:wrap_socket() got an unexpected keyword argument '_context'
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager Traceback (most recent call last):
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py", line 254, in invoke_elem
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager response = self._opener.open(request)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/usr/lib/python3.7/urllib/request.py", line 525, in open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager response = self._open(req, data)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager '_open', req)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager result = func(*args)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager context=self._context, check_hostname=self._check_hostname)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager File "/u...

I haven't noticed that the driver was not using https to make its calls. Thanks for pointing it out. There are two points where we set http or https. Now I've forced it in both places and I had the same mistake you're having. If I use HTTP instead of HTTPS, it works correctly.

Aug 22 17:57:44.809258 37-carlos-pythirty-seven-test manila-share[1481]: WARNING manila.share.drivers.netapp.dataontap.client.api [None req-3970f319-d985-46ac-9d03-50211f861978 None None] {'_full_url': 'https://x.x.x.x:80/servlets/netapp.servlets.admin.XMLrequest_filer', 'fragment': None, 'type': 'https', 'host': 'x.x.x.x:80', 'selector': '/servlets/netapp.servlets.admin.XMLrequest_filer', 'headers': {'Content-type': 'text/xml', 'Charset': 'utf-8'}, 'unredirected_hdrs': {}, '_data': b'<netapp xmlns="https://www.netapp.com/filer/admin" version="1.15"><system-get-ontapi-version/></netapp>', '_tunnel_host': None, 'origin_req_host': 'x.x.x.x', 'unverifiable': False}
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager [None req-3970f319-d985-46ac-9d03-50211f861978 None None] Error encountered during initialization of driver NetAppCmodeMultiSvmShareDriver@37-carlos-pythirty-seven-test@netapp2: manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - unknown:wrap_socket() got an unexpected keyword argument '_context'
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager Traceback (most recent call last):
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py", line 254, in invoke_elem
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     response = self._opener.open(request)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/urllib/request.py", line 525, in open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     response = self._open(req, data)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     '_open', req)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     result = func(*args)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     context=self._context, check_hostname=self._check_hostname)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     encode_chunked=req.has_header('Transfer-encoding'))
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 1229, in request
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self._send_request(method, url, body, headers, encode_chunked)
Aug 22 17:57:44.815719 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 1275, in _send_request
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self.endheaders(body, encode_chunked=encode_chunked)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 1224, in endheaders
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self._send_output(message_body, encode_chunked=encode_chunked)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 1016, in _send_output
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self.send(msg)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 956, in send
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self.connect()
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/lib/python3.7/http/client.py", line 1392, in connect
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     server_hostname=server_hostname)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 438, in wrap_socket
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     return GreenSSLSocket(sock, *a, _context=self, **kw)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/usr/local/lib/python3.7/dist-packages/eventlet/green/ssl.py", line 76, in __new__
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     *args, **kw
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager TypeError: wrap_socket() got an unexpected keyword argument '_context'
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager During handling of the above exception, another exception occurred:
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager Traceback (most recent call last):
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/manager.py", line 311, in _driver_setup
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self.driver.do_setup(ctxt)
Aug 22 17:57:44.818107 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/drv_multi_svm.py", line 40, in do_setup
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self.library.do_setup(context)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/utils.py", line 96, in trace_wrapper
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     result = f(self, *args, **kwargs)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 138, in do_setup
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     self._client = self._get_api_client()
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/utils.py", line 96, in trace_wrapper
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     result = f(self, *args, **kwargs)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/cluster_mode/lib_base.py", line 175, in _get_api_client
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     api_trace_pattern=na_utils.API_TRACE_PATTERN)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/client_cmode.py", line 57, in __init__
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     (major, minor) = self.get_ontapi_version(cached=False)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/client_base.py", line 46, in get_ontapi_version
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     enable_tunneling=False)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/client_base.py", line 88, in send_request
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     return self.connection.invoke_successfully(request, enable_tunneling)
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py", line 278, in invoke_successfully
Aug 22 17:57:44.820323 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     result = self.invoke_elem(na_element, enable_tunneling)
Aug 22 17:57:44.822067 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager   File "/opt/stack/new/manila/manila/share/drivers/netapp/dataontap/client/api.py", line 260, in invoke_elem
Aug 22 17:57:44.822067 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager     raise NaApiError(message=e)
Aug 22 17:57:44.822067 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager manila.share.drivers.netapp.dataontap.client.api.NaApiError: NetApp API failed. Reason - unknown:wrap_socket() got an unexpected keyword argument '_context'
Aug 22 17:57:44.822067 37-carlos-pythirty-seven-test manila-share[1481]: ERROR manila.share.manager

Revision history for this message

Tom Barron (tpb) wrote on 2019-08-22:

#18

Do the ONTAP and VNX cinder drivers have the same issue?

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2019-08-23:

#19

VNX cinder driver doesn't have this issue. Because VNX cinder uses naviseccli via ssh to do VNX block related operations. And VNX manila uses XML api based on http/https.

It seems this issue happens when using urllib and eventlet. Our Unity Cinder and Manila drivers are using requests library and haven't met this issue.

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2019-08-23:

#20

Hey, Tom! As the NetApp Manila driver, NetApp Cinder driver also uses urllib to do its calls. I haven't tested it yet but I'm almost 100% sure that the same will happen with the NetApp Cinder driver.

Revision history for this message

Carlos Eduardo (silvacarlose) wrote on 2020-04-30:

#21

At that time (when we were discussing) we were able to identify the issue in our side. We have started looking for some possible fixes. Felipe Rodrigues will update this bug with his updates soon.

Revision history for this message

Felipe Rodrigues (felipefutty) wrote on 2020-05-04:

#22

I could reproduce the bug with netapp driver!

As it is state in the last line: the ssl.py patched by the eventlet is not expecting an argument called "_context" to wrap_socket(). This error happens because ssl library has changed from python 3.6 to python 3.7 as showed by [1], but the eventlet patching has not.

The eventlet bug with python3.7 as well-known by the issue 526 [2]. This issue was open more than one year ago. There is a long discussion about it! Summing up it: there is an attempt to fix it by a pull request [3], it was created may 2019 and it has not finished yet, though! Actually, since November 2019, no more updates on that. So, the eventlet bug will not be fixed early!
However, in the eventlet disscussion, it was pointed 2 possible solutions:

1. Replace the eventlet by gevent

2. Replace “ssl” by “pyopenssl” lib

The code [6] reproduces the bug in a small peace of code. It uses the same library and methods as netapp driver does. It can be used to test the solutions.

Just replacing the eventlet by gevent and running the code [4], the bug is solved, validating the solution “1”! For the manila scenario, it might not be feasible, because the patching is applied out of driver scope, affecting all vendors. So, this solution is skipped by now (maybe a discussion about eventlet vs gevent could be created).

The solution “2” is not applied for the code [6], because it uses urllib (from six.moves). The urllib is a library that comes with python and its secure connection is mandatory from ssl library. I haven’t found a way to replace the ssl by pyopenssl in urllib. Hence, for the manila driver we could not apply the “2”.

The possible solution would be change the request library: instead of using urllib, it uses a lib that can perform ssl from pyopenssl. The “requests” library does it! For this one, the default is “pyopenssl” if this library is installed, otherwise it would use the “ssl” lib.
To validate the “requests” solution, the following code is created:

import eventlet
requests = eventlet.import_patched(“requests”)
full_url = “https://www.wikipedia.org”
resp = requests.get(full_url)
print(rest)

Without installing the “pyopenssl”, running the above code with python3.7 the same reported bug happens. After installing the “pyopenssl” (with pip3.7), the bug is fixed. The reason is that the “requests” lib uses the patched “pyopenssl” instead of the broken patched “ssl”.

Summing up, the possible solution would be change the request library: instead of std python lib “urllib”, uses the third party lib “requests”!

[1] https://bugs.python.org/issue32951
[2] https://github.com/eventlet/eventlet/issues/526
[3] https://github.com/eventlet/eventlet/pull/565
[4] https://gist.github.com/Murray-LIANG/9b506d98cf76d633a9d0e82f2bb5ece5

I could reproduce the bug with netapp driver!

As it is state in the last line: the ssl.py patched by the eventlet is not expecting an argument called "_context" to wrap_socket(). This error happens because ssl library has changed from python 3.6 to python 3.7 as showed by [1], but the eventlet patching has not.

The eventlet bug with python3.7 as well-known by the issue 526 [2]. This issue was open more than one year ago. There is a long discussion about it! Summing up it: there is an attempt to fix it by a pull request [3], it was created may 2019 and it has not finished yet, though! Actually, since November 2019, no more updates on that. So, the eventlet bug will not be fixed early!
However, in the eventlet disscussion, it was pointed 2 possible solutions:

1. Replace the eventlet by gevent

2. Replace “ssl” by “pyopenssl”  lib

The code [6] reproduces the bug in a small peace of code. It uses the same library and methods as netapp driver does. It can be used to test the solutions.

Just replacing the eventlet by gevent and running the code [4], the bug is solved, validating the solution “1”! For the manila scenario, it might not be feasible, because the patching is applied out of driver scope, affecting all vendors. So, this solution is skipped by now (maybe a discussion about eventlet vs gevent could be created).

The solution “2” is not applied for the code [6], because it uses urllib (from six.moves). The urllib is a library that comes with python and its secure connection is mandatory from ssl library.  I haven’t found a way to replace the ssl by pyopenssl in urllib. Hence, for the manila driver we could not apply the “2”.

The possible solution would be change the request library: instead of using urllib, it uses a lib that can perform ssl from pyopenssl. The “requests” library does it! For this one, the default is “pyopenssl” if this library is installed, otherwise it would use the “ssl” lib.
To validate the “requests” solution, the following code is created:

import eventlet
requests = eventlet.import_patched(“requests”)
full_url = “https://www.wikipedia.org”
resp = requests.get(full_url)
print(rest)

Without installing the “pyopenssl”, running the above code with python3.7 the same reported bug happens. After installing the “pyopenssl” (with pip3.7), the bug is fixed. The reason is that the “requests” lib uses the patched “pyopenssl”  instead of the broken patched “ssl”.

Summing up, the possible solution would be change the request library: instead of std python lib “urllib”, uses the third party lib “requests”!

[1] https://bugs.python.org/issue32951
[2] https://github.com/eventlet/eventlet/issues/526
[3] https://github.com/eventlet/eventlet/pull/565
[4] https://gist.github.com/Murray-LIANG/9b506d98cf76d633a9d0e82f2bb5ece5

Revision history for this message

Ryan Liang (ryan-liang) wrote on 2020-07-02:

#23

The new version of requests 2.24.0 replaced `pyopenss` with `ssl`.
So, any vendor driver (no matter Cinder or Manila) which are using requests 2.24.0 will have the same issue.

Revision history for this message

Felipe Rodrigues (felipefutty) wrote on 2020-07-23:

#24

As the discussion of this new feature shows [1]: it can forces the use of `pyopenssl` by calling:

urllib3.contrib.pyopenssl.inject_into_urllib3()

[1] https://github.com/psf/requests/pull/5443#issuecomment-645740394

Vida Haririan (vhariria) on 2020-08-20

Changed in manila:
importance:	Undecided → Low

Revision history for this message

Vida Haririan (vhariria) wrote on 2020-08-20:

#25

Additional comments http://eavesdrop.openstack.org/meetings/manila/2020/manila.2020-08-20-15.00.log.html

Revision history for this message

Felipe Rodrigues (felipefutty) wrote on 2020-08-25:

#26

As the research showed, the source of the bug is the bug in the eventlet library, while patching the ssl library in py3.7 (or newer). The eventlet has released a new version (0.26) that fixes that bug, though!

The link of the new release: https://github.com/eventlet/eventlet/releases/tag/v0.26.0
It mentions: "Fix compatibility with SSLContext usage >= Python 3.7"

I tested the 0.26 eventlet with py3.7 for requesting a https content and it worked!

Revision history for this message

Sam Wan (sam-wan) wrote on 2020-10-28:

#27

Verified the issue has been fixed.
now powermax/vnx manila CI's can run under python3.7

Vida Haririan (vhariria) on 2023-05-18

Changed in manila:
status:	New → Triaged

OpenStack Shared File Systems Service (Manila)

VNX: driver cannot start under py37 due to wrap_socket() got an unexpected keyword argument '_context'

Bug Description

Other bug subscribers

Remote bug watches