test_reflection.CallbackEqualityTest.test_different_instance_callbacks fails on Python 3.8

Bug #1841072 reported by Javier Peña
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
oslo.utils
Undecided
Ghanshyam Mann
python-oslo.utils (Ubuntu)
Undecided
Unassigned
python3-defaults (Ubuntu)
Undecided
Unassigned

Bug Description

When running the unit test on Python 3.8, it fails with the following traceback:

oslo_utils.tests.test_reflection.CallbackEqualityTest.test_different_instance_callbacks
---------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    b'Traceback (most recent call last):'
    b' File "/tmp/oslo.utils/oslo_utils/tests/test_reflection.py", line 156, in test_different_instance_callbacks'
    b' self.assertTrue(reflection.is_same_callback(b.b, c.b, strict=False))'
    b' File "/tmp/oslo.utils/.tox/py38/lib/python3.8/site-packages/unittest2/case.py", line 702, in assertTrue'
    b' raise self.failureException(msg)'
    b'AssertionError: False is not true'
    b''

This is apparently caused by a behavior change in Python 3.8 due to [1]. I have confirmed the different behavior by running tests manually on 3.6, 3.7 (both return True) and 3.8.

According to [2], only taskflow seems to be using that method now, and it is not changing the default value for the "strict" parameter.

[1] - https://bugs.python.org/issue1617161
[2] - http://codesearch.openstack.org/?q=is_same_callback&i=nope&files=&repos=

Revision history for this message
Javier Peña (jpena-c) wrote :

Forget my comment about [2], I didn't realize strict was True by default.

Revision history for this message
Balint Reczey (rbalint) wrote :

This bug now blocks updating python-3-defaults.

tags: added: update-excuse
Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

I think actually strict can be dropped.

tags: added: patch
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-oslo.utils - 3.41.1-0ubuntu2

---------------
python-oslo.utils (3.41.1-0ubuntu2) focal; urgency=medium

  * Non-strict relfection comparison is not possible in py3.8+. LP: #1841072
  * Use python3 when detecting module name from setup.py in autopkgtest.

 -- Dimitri John Ledkov <email address hidden> Mon, 18 Nov 2019 11:23:18 +0000

Changed in python-oslo.utils (Ubuntu):
status: New → Fix Released
Steve Langasek (vorlon)
Changed in python3-defaults (Ubuntu):
status: New → Invalid
Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

This was considered as bugfix in python 3.8 because bound methods should only be equal if they are bound on the same instance, not if their instances are equal. In Python <= 3.7 the bound method equality calls the equivalent of instance1 == instance2 and return true based on that and while in Python 3.8 it checks if instance1 is instance2.

You can see details about this behavior change in py3.8
- https://bugs.python.org/issue1617161
- python-dev discussion: https://mail.python.org/pipermail/python-dev/2018-June/153959.html

With that and for cases of 'strict=False', oslo utils is_same_callback() method behaviour also changed. With 'strict=False', this method returned *True* for python <3.7 (because == comparison used to call the __eq__ method return value) and *False* for python3.8 onwards (because == never call __eq__ method and check the id of self).
-https://github.com/openstack/oslo.utils/blob/7c4a94c0c3fcbd8f05541944851728f30deadd9b/oslo_utils/reflection.py#L172-L174

For 'strict' is True (which is the default) there is no behavior change because is_same_callback() method checks if 'self' of both bound methods are equal or not
- https://github.com/openstack/oslo.utils/blob/7c4a94c0c3fcbd8f05541944851728f30deadd9b/oslo_utils/reflection.py#L183

We should modify the test to test the behaviors this way and in is_same_callback() method we should clearly document that 'strict' is no more an option or valid thing for python3.8 onwards and we should even deprecate this arg itself.

Changed in oslo.utils:
status: New → Confirmed
assignee: nobody → Ghanshyam Mann (ghanshyammann)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to oslo.utils (master)

Fix proposed to branch: master
Review: https://review.opendev.org/750216

Changed in oslo.utils:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to oslo.utils (master)

Reviewed: https://review.opendev.org/750216
Committed: https://git.openstack.org/cgit/openstack/oslo.utils/commit/?id=632f15515fb224863c1027c658858c595ad5d11b
Submitter: Zuul
Branch: master

commit 632f15515fb224863c1027c658858c595ad5d11b
Author: Ghanshyam Mann <email address hidden>
Date: Mon Sep 7 16:15:20 2020 -0500

    Fix is_same_callback() testing for python3.8

    Python3.8 onwards, comparision of bound methods is
    changed. It no longer decide the bound method's equality based
    on their bounded objects equality instead it checks the identity
    of their '__self__'.

    Details about this behavior change in python 3.8
    - https://bugs.python.org/issue1617161
    - python-dev discussion: https://mail.python.org/pipermail/python-dev/2018-June/153959.html

    So python3.8 onwards, 'strict' arg has no meaning. For backward compatibility
    for python <3.8, we can keep the 'strict' arg but with deprecation warning.

    Also modify the is_same_callback() unit tests to verify the 'strict'
    arg based on python version.

    Change-Id: I3e6d9bbceccacddd0e1a514bbe5d0722a3408ecb
    Closes-Bug: #1841072

Changed in oslo.utils:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.