neutron

python 3.9 failures

Bug #1923083 reported by Corey Bryant
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Medium
Brian Haley

Bug Description

It looks like all of the unit tests fail on the same error "TypeError: cannot pickle '_thread.RLock' object":

tox -e py39
...
ovn_octavia_provider.tests.unit.test_helper.TestOvnProviderHelper.test_vip_port_update_handler_lb_not_found
-----------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/ovn_octavia_provider/tests/unit/test_helper.py", line 34, in setUp
    super().setUp()

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/ovn_octavia_provider/tests/unit/base.py", line 24, in setUp
    super().setUp()

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/.tox/py39/lib/python3.9/site-packages/neutron/tests/base.py", line 397, in setUp
    policy.init()

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/.tox/py39/lib/python3.9/site-packages/neutron/policy.py", line 77, in init
    register_rules(_ENFORCER)

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/.tox/py39/lib/python3.9/site-packages/neutron/policy.py", line 68, in register_rules
    enforcer.register_defaults(itertools.chain(*policies))

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/.tox/py39/lib/python3.9/site-packages/oslo_policy/policy.py", line 1120, in register_defaults
    self.register_default(default)

      File "/home/corey/pkg/wallaby/upstream/ovn-octavia-provider/.tox/py39/lib/python3.9/site-packages/oslo_policy/policy.py", line 1109, in register_default
    self.registered_rules[default.name] = copy.deepcopy(default)

      File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)

      File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

      File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)

      File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)

      File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)

      File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

      File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)

      File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)

      File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)

      File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

      File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)

      File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)

      File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)

      File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

      File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)

      File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)

      File "/usr/lib/python3.9/copy.py", line 161, in deepcopy
    rv = reductor(4)

    TypeError: cannot pickle '_thread.RLock' object

======
Totals
======
Ran: 211 tests in 3.9547 sec.
 - Passed: 111
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 100
Sum of execute time for each test: 11.8598 sec.

==============
Worker Balance
==============
 - Worker 0 (54 tests) => 0:00:03.251029
 - Worker 1 (50 tests) => 0:00:02.942199
 - Worker 2 (53 tests) => 0:00:03.094253
 - Worker 3 (54 tests) => 0:00:03.180170

To recreate:

lxc launch ubuntu-daily:hirsute h1
lxc exec h1 /bin/bash
git clone https://opendev.org/openstack/ovn-octavia-provider
cd ovn-octavia-provider
tox -e py39

See original description
Corey Bryant (corey.bryant)
description: updated
Brian Haley (brian-haley)
tags: added: ovn-octavia-provider
Changed in neutron:
assignee: nobody → Brian Haley (brian-haley)
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Brian Haley (brian-haley) wrote :

I see this locally running unit tests as well, and the fix I've done is to update the neutron requirement in test-requirements.txt to at least 18.0.0 (there was a fix in neutron for this and we inherit it). I should be able to do that once Wallaby is released, it doesn't like any RC tag I tried in the gate, like 18.0.0.0rc1.

At least I think that's the fix...

Revision history for this message
Brian Haley (brian-haley) wrote :

Local run...

======
Totals
======
Ran: 211 tests in 15.4707 sec.
 - Passed: 211
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 42.8016 sec.

==============
Worker Balance
==============
 - Worker 0 (1 tests) => 0:00:15.402510
 - Worker 1 (39 tests) => 0:00:04.131763
 - Worker 2 (28 tests) => 0:00:03.696216
 - Worker 3 (31 tests) => 0:00:03.961309
 - Worker 4 (28 tests) => 0:00:03.843345
 - Worker 5 (28 tests) => 0:00:04.030647
 - Worker 6 (28 tests) => 0:00:04.055176
 - Worker 7 (28 tests) => 0:00:03.934892
___________________________________ summary ____________________________________
  py39: commands succeeded
  congratulations :)

bhaley@r ~/git/ovn-octavia-provider $
--> git diff
diff --git a/test-requirements.txt b/test-requirements.txt
index eb79de9..1400f42 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -11,10 +11,11 @@ python-subunit>=1.0.0 # Apache-2.0/BSD
 oslotest>=3.2.0 # Apache-2.0
 stestr>=1.0.0 # Apache-2.0
 pylint>=2.6.0 # GPLv2
+pycodestyle<2.7.0 # GPLv2
 testresources>=2.0.0 # Apache-2.0/BSD
 testscenarios>=0.4 # Apache-2.0/BSD
 WebTest>=2.0.27 # MIT
 testtools>=2.2.0 # MIT

 # NOTE: Precisely we need ussuri neutron
-neutron>=16.0.0 # Apache-2.0
+neutron>=18.0.0.0rc1 # Apache-2.0

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Hey Brian, Thanks a lot for the quick reply! The switch to neutron>=18.0.0.0rc1 results in 100% unit test success. \o/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ovn-octavia-provider (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/787460

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
Brian Haley (brian-haley) wrote :

https://review.opendev.org/c/openstack/ovn-octavia-provider/+/787460

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ovn-octavia-provider (master)

Reviewed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/787460
Committed: https://opendev.org/openstack/ovn-octavia-provider/commit/69b392de9c6d753262342b726c407abbb8b6b6bb
Submitter: "Zuul (22348)"
Branch: master

commit 69b392de9c6d753262342b726c407abbb8b6b6bb
Author: Brian Haley <email address hidden>
Date: Wed Apr 21 19:43:51 2021 -0400

    Fix python 3.9 unit test failures

    Running the python 3.9 unit tests locally results in failures
    like:

      TypeError: cannot pickle '_thread.RLock' object

    Because we are inheriting some of the neutron code, and this
    is fixed in that repository, just bump the minimum requirement
    to 18.0.0 (Wallaby) now that it is released. This does not
    happen in the gate since it is always using master.

    Raised minimum of other libraries as necessary.

    Change-Id: I820c924189d2973aefcb1452331c904f1ee7051f
    Closes-bug: #1923083

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ovn-octavia-provider 1.1.0

This issue was fixed in the openstack/ovn-octavia-provider 1.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.