requests to SSL wrapped sockets hang while reading using py3
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| | Glance |
Undecided
|
Unassigned | ||
| | OpenStack Compute (nova) |
Undecided
|
Unassigned | ||
| | OpenStack Shared File Systems Service (Manila) |
Low
|
Unassigned | ||
| | neutron |
High
|
Unassigned | ||
| | oslo.service |
High
|
Herve Beraud | ||
Bug Description
If we run unit tests using py3 then we get following errors:
=======
FAIL: manila.
tags: worker-0
-------
Empty attachments:
pythonlogging:''
stdout
stderr: {{{
Traceback (most recent call last):
File "/home/
timer()
File "/home/
cb(*args, **kw)
File "/home/
result = function(*args, **kwargs)
File "/home/
client_socket = sock.accept()
File "/home/
suppress_
File "/home/
self.
File "/home/
super(
File "/home/
return func(*a, **kw)
File "/usr/lib/
self.
ssl.SSLWantRead
}}}
Traceback (most recent call last):
File "/home/
'https:/
File "/usr/lib/
return opener.open(url, data, timeout)
File "/usr/lib/
response = self._open(req, data)
File "/usr/lib/
'_open', req)
File "/usr/lib/
result = func(*args)
File "/usr/lib/
context=
File "/usr/lib/
h.request(
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
self.
File "/usr/lib/
self.send(msg)
File "/usr/lib/
self.connect()
File "/usr/lib/
server_
File "/home/
return GreenSSLSocket(
File "/home/
self.
File "/home/
super(
File "/home/
timeout_
File "/home/
return hub.switch()
File "/home/
return self.greenlet.
File "/home/
self.
File "/home/
presult = self.do_
File "/home/
return self.poll.
File "/home/
raise TimeoutException()
fixtures.
Debugging by hands shown that service starting and closing OK, but when we request something from it using py3 and SSL then we do not get answer at all. In case of unit tests we face test timeout.
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 596b3405629f04f
Author: Valeriy Ponomaryov <email address hidden>
Date: Fri Aug 7 15:41:47 2015 +0300
Skip unit tests for SSL + py3
When we request something from SSL wrapped app using py3 and we do not get
answer at all. In case of unit tests we face test timeout.
So, skip these tests until related b_u_g is fixed.
Change-Id: I809668e5c04b7a
Related-Bug: #1482633
Partially-
| Changed in manila: | |
| importance: | Undecided → Low |
| Ihar Hrachyshka (ihar-hrachyshka) wrote : | #3 |
Neutron also hangs in py34 when running neutron.
| Changed in neutron: | |
| status: | New → Confirmed |
| importance: | Undecided → High |
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit ef409d9da2ecbf1
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Sep 1 21:45:55 2015 +0200
Enable most unit tests for py34 job
* Skip TestWSGIServerW
eventlet setup does not behave correctly now,
* Skip test_json_
* Fix some more tests to pass for py3,
* Replace print by print() in docs/docstrings.
[1] neutron.
[2] neutron.
Related-Bug: #1482633
Related-Bug: #1491824
Blueprint: neutron-python3
Co-Authored-By: Cyril Roelandt <email address hidden>
Co-Authored-By: Cedric Brandily <email address hidden>
Co-Authored-By: sonu.kumar <email address hidden>
Change-Id: I26e513d4dcf473
Related fix proposed to branch: feature/pecan
Review: https:/
| OpenStack Infra (hudson-openstack) wrote : | #6 |
Related fix proposed to branch: feature/pecan
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: feature/pecan
commit fe236bdaadb9496
Author: Miguel Angel Ajo <email address hidden>
Date: Thu Sep 3 15:40:12 2015 +0200
No network devices on network attached qos policies
Network devices, like internal router legs, or dhcp ports
should not be affected by bandwidth limiting rules.
This patch disables application of network attached policies
to network/neutron owned ports.
Closes-bug: #1486039
DocImpact
Change-Id: I75d80227f1e6c4
commit db4a06f7caa20a4
Author: Ken'ichi Ohmichi <email address hidden>
Date: Wed Sep 16 10:04:32 2015 +0000
Use tempest-lib's token_client
Now tempest-lib provides token_client modules as library and the
interface is stable. So neutron repogitory doesn't need to contain
these modules.
This patch makes neutron use tempest-lib's token_client and removes
the own modules for the maintenance.
Change-Id: Ieff7eb003f6e82
commit 78aed58edbe6eb8
Author: Jakub Libosvar <email address hidden>
Date: Thu Aug 13 09:08:20 2015 +0000
Fix establishing UDP connection
Previously, in establish_
but never read on peer socket. That lead to successful read on peer side
if this connection was filtered. Having constant testing string masked
this issue as we can't distinguish to which test of connectivity data
belong.
This patch makes unique data string per test_connectivity() and
also makes establish_
conntrack table. Finally, in last test after firewall filter was
removed, connection is re-established in order to avoid troubles with
terminated processes or TCP continuing sending packets which weren't
successfully delivered.
Closes-Bug: 1478847
Change-Id: I2920d587d8df8d
commit e6292fcdd626243
Author: YAMAMOTO Takashi <email address hidden>
Date: Wed Sep 16 13:20:51 2015 +0900
ovsdb: Fix a few docstring
Change-Id: I53e1e21655b28f
commit c22949a4449d96a
Author: fumihiko kakuma <email address hidden>
Date: Wed Sep 16 11:52:59 2015 +0900
Remove requirements.txt for the ofagent mechanism driver
It is no longer used.
Related-
https:/
Change-Id: Ib31fb3febf8968
commit d1d4de19d85f961
Author: Kevin Benton <email address hidden>
Date: Thu Sep 3 20:25:57 2015 -0700
Always return iterables in L3 get_candidates
The caller of this function expects iterables.
Closes-Bug: #1494996
Change-Id: I3d103e63f4e127
commit 1ad6ac448067306...
| tags: | added: in-feature-pecan |
Change abandoned by Doug Wiegley (<email address hidden>) on branch: feature/pecan
Review: https:/
| Elena Ezhova (eezhova) wrote : | #9 |
Neutron is going to consume ssl functionality fron oslo.service, so this bug is no longer valid for neutron.
| Changed in neutron: | |
| status: | Confirmed → Invalid |
| Cyril Roelandt (cyril-roelandt) wrote : | #10 |
@Elena: is there a blueprint/patch series about this upcoming change?
| Elena Ezhova (eezhova) wrote : | #11 |
@Cyril, Yes, here is a change: https:/
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 216d2d0b7594ad0
Author: Elena Ezhova <email address hidden>
Date: Fri Sep 25 16:07:51 2015 +0300
Consume sslutils and wsgi modules from oslo.service
sslutils and basic WSGI functionality have been moved to
oslo.service and now Neutron can reuse them.
Marked ssl options that were renamed in oslo.service as
deprecated.
Added a note about possible implications for out-of-tree plugins
to neutron_api.rst
Bumped oslo.service version to 0.9.0.
Related-Bug: #1482633
Depends-On: I0424a6c261fae4
Change-Id: Ibfdf07e665fcfc
| Sean Dague (sdague) wrote : | #13 |
Is there an upstream eventlet bug for this?
| Changed in nova: | |
| status: | New → Invalid |
| Sean Dague (sdague) wrote : | #14 |
Also on oslo.service now, so not actually a nova bug any more
| Changed in manila: | |
| milestone: | none → newton-1 |
| status: | New → Triaged |
| Elena Ezhova (eezhova) wrote : | #15 |
The related oslo.service tests no longer hang with python 3.4 but fail with http://
Related fix proposed to branch: master
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Jenkins
Branch: master
commit 1b7191f4f2cf322
Author: ChangBo Guo(gcb) <email address hidden>
Date: Sat Jan 28 21:33:45 2017 +0800
Skip unit tests for SSL + py3
When we request something from SSL wrapped app using py3 and we do
not get answer at all. In case of unit tests we face test timeout.
So, skip these tests until related bug is fixed. This also enables
other test cases in this module.
Related-Bug: #1482633
Partially-
Change-Id: I4933e47b075971
| Corey Bryant (corey.bryant) wrote : | #18 |
I appear to be hitting this same issue with novapackage builds on Ubuntu Artful with py2.
| Changed in manila: | |
| assignee: | nobody → Victoria Martinez de la Cruz (vkmc) |
I couldn't reproduce this with python3.5. I think it is safe to make python3 gate voting again.
| Ben Nemec (bnemec) wrote : | #20 |
Is there anything left to do on this from the oslo.service side? I think the python 3 unit tests are running almost everywhere now so I'm inclined to believe it was fixed.
| Changed in oslo.service: | |
| status: | New → Incomplete |
| Ben Nemec (bnemec) wrote : | #21 |
Oh, my mistake. I see the oslo.service tests are being skipped on py3. We need to stop doing that and figure out how to fix this soon. Without it we don't have proper python 3 support in oslo.service.
| Changed in oslo.service: | |
| status: | Incomplete → Triaged |
| importance: | Undecided → High |
| Zane Bitter (zaneb) wrote : | #22 |
This appears to be a series of long-standing bugs in eventlet:
Python 3.5 failure mode:
https:/
https:/
Python 3.4 failure mode:
https:/
https:/
There are also more problems coming down the pipeline in Python 3.6:
https:/
That one is resolved in eventlet 0.21, but we have that blocked by upper-constraints: http://
| Zane Bitter (zaneb) wrote : | #23 |
Almost certainly also affects Glance, since it appears to still use the old oslo-incubator version.
http://
| Brian Rosmaita (brian-rosmaita) wrote : | #24 |
This does indeed affect Glance. We were tracking this with https:/
| Changed in glance: | |
| status: | New → Triaged |
| Changed in manila: | |
| milestone: | newton-1 → rocky-3 |
| tags: | added: python3 |
| Changed in manila: | |
| milestone: | rocky-3 → stein-3 |
| Changed in manila: | |
| milestone: | stein-3 → stein-rc1 |
| Changed in manila: | |
| milestone: | stein-rc1 → train-1 |
| Goutham Pacha Ravi (gouthamr) wrote : | #25 |
In manila, we are no longer testing the eventlet server with SSL; we're rather testing running the eventlet server with a tls-proxy that terminates SSL requests. Distributions have been advised to switch to using manila-api behind web servers, such as apache, and most of them have done so for several releases now. I'm marking this bug to be low priority; testing and fixing it may be apt for those that really need the non-recommended
| summary: |
- requests to SSL wrapped sockets hang while reading using py3 + Xanax without Prescription at YourRxPills.Com |
| description: | updated |
| description: | updated |
| summary: |
- Xanax without Prescription at YourRxPills.Com + requests to SSL wrapped sockets hang while reading using py3 |
| Changed in manila: | |
| milestone: | train-1 → none |
| Zane Bitter (zaneb) wrote : | #26 |
Current py37 failure mode is:
https:/
But even when that is resolved there's no guarantee that this issue will be fixed.
| Changed in glance: | |
| importance: | Undecided → High |
| importance: | High → Undecided |
| Changed in oslo.service: | |
| assignee: | nobody → Herve Beraud (herveberaud) |
| status: | Triaged → In Progress |
Fix proposed to branch: master
Review: https:/
Fix proposed to branch: master
Review: https:/
Fix proposed to branch: master
Review: https:/
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit bfc817294297ddd
Author: Hervé Beraud <email address hidden>
Date: Tue May 26 18:39:29 2020 +0200
Fix some SSL tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).
Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:
```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```
This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.
We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.
In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.
These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).
[1] https:/
[2] https:/
Change-Id: Id44ad12a1cf3fd
Partial-Bug: #1482633
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit ebc2d8a42fd27f9
Author: Hervé Beraud <email address hidden>
Date: Tue Jun 16 13:34:33 2020 +0200
Fix wsgi/SSL/ipv6 tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).
Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:
```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```
This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.
We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.
In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.
These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).
[1] https:/
[2] https:/
Change-Id: Iacdde51d2d923b
Partial-Bug: #1482633
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit fb1ed4085b91303
Author: Hervé Beraud <email address hidden>
Date: Tue Jun 16 14:16:06 2020 +0200
Reactivate wsgi test related to socket option under python 3
Previously these tests was ignored surely because the WSGI tests
with SSL were broken due to some design changes introduced by python 3.7 [1]
in the SSL module of the stdlib.
However, unlike the other WSGI/SSL tests these changes don't use
the `requests` module or high level modules which are broken by
the monkey patching of the stdlib.
I suppose previous developers simply turned off all of tests related to
WSGI/SSL without much more details than "SSL tests with python 3 and eventlet
is broken" but some of them don't need much more work than just reactivate
them to be ran successfully.
This test seems to work without change, and it run successfully locally.
Let reactivate this test.
Change-Id: Ie0257af10a9439
Partial-Bug: #1482633
| Changed in oslo.service: | |
| status: | In Progress → Fix Released |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit caf66be763fd8f9
Author: Hervé Beraud <email address hidden>
Date: Tue Jun 16 14:42:03 2020 +0200
Fix wsgi SSL tests for wsgi module under python 3
Previously some tests were ignored under python 3 environment, this was
due to some design changes introduced by python 3.7 [1] in the SSL
module of the stdlib. These changes reactivate some of them (some other
are still skipped and needs further works).
Indeed, when we try to use requests with SSL in a monkey patched
environment we faced the following issue:
```
TypeError: wrap_socket() got an unexpected keyword argument '_context'
```
This is due to the fact that we are in a monkey patched environment
where `requests` is monkey patched too.
We don't need `request` for our needs. Indeed we can easily send
http requests through low level socket. Our main goal is to test
our wsgi server and not to test the `requests` library, and `requests`
was just used to make the code more simpler.
In our case we can implement a code dedicated to send request to our green
server, unlock our tests and move away from this bug/side effect.
Also this reactivated test will check WSGI server with and without SSL,
so these changes add changes that allow us to submit a request without
wrapping the socket with SSL.
These changes move away from `requests` which is badly monkey patched by
eventlet [1]. Now we use monkey patched socket and ssl to
request the green server which is executed in background. Low level
(monkey patched) modules could help us to skirt layers that are possibly
badly monkey patched on higher level modules (urllib, requests, etc...).
[1] https:/
[2] https:/
Change-Id: I3a018d507d1022
Closes-Bug: #1482633
| Changed in manila: | |
| status: | Triaged → Invalid |
| assignee: | Victoria Martinez de la Cruz (vkmc) → nobody |
| Goutham Pacha Ravi (gouthamr) wrote : | #34 |
This bug was discussed during the upstream openstack-manila meeting on Thu 25th June 2020:
http://
We recommend the use of uwsgi/mod_wsgi to frontend Manila API, so this has been marked Invalid.
Related fix proposed to branch: master
Review: https:/