python3.8+ test failures concerning the ipaddress module

Bug #1934345 reported by Goutham Pacha Ravi
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Critical
Goutham Pacha Ravi

Bug Description

Description
===========
Three unit tests started failing on 1st July 2021 across python3.8 and python3.9 test jobs in manila, openstack/requirements and stable branch bitrot jobs:

==============================
Failed 3 tests - output below:
==============================
manila.tests.share.drivers.infinidat.test_infinidat.InfiniboxDriverTestCase.test__get_ip_address_range_single_ip
----------------------------------------------------------------------------------------------------------------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
   Traceback (most recent call last):
     File "/home/zuul/src/opendev.org/openstack/manila/manila/tests/share/drivers/infinidat/test_infinidat.py", line 281, in test__get_ip_address_range_single_ip
   self.assertEqual('1.2.3.4', ip_address)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 415, in assertEqual
   self.assertThat(observed, matcher, message)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 502, in assertThat
   raise mismatch_error
   testtools.matchers._impl.MismatchError: '1.2.3.4' != '1.2.3.4-1.2.3.4'
manila.tests.share.drivers.infinidat.test_infinidat.InfiniboxDriverTestCase.test_update_access
----------------------------------------------------------------------------------------------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
   Traceback (most recent call last):
     File "/home/zuul/src/opendev.org/openstack/manila/manila/tests/share/drivers/infinidat/test_infinidat.py", line 639, in test_update_access
   self.assertEqual('1.2.3.4', permissions[0].client)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 415, in assertEqual
   self.assertThat(observed, matcher, message)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 502, in assertThat
   raise mismatch_error
   testtools.matchers._impl.MismatchError: '1.2.3.4' != '1.2.3.4-1.2.3.4'
manila.tests.share.drivers.infinidat.test_infinidat.InfiniboxDriverTestCase.test_snapshot_update_access
-------------------------------------------------------------------------------------------------------
Captured traceback:
~~~~~~~~~~~~~~~~~~~
   Traceback (most recent call last):
     File "/home/zuul/src/opendev.org/openstack/manila/manila/tests/share/drivers/infinidat/test_infinidat.py", line 726, in test_snapshot_update_access
   self.assertEqual('1.2.3.4', permissions[0].client)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 415, in assertEqual
   self.assertThat(observed, matcher, message)
     File "/home/zuul/src/opendev.org/openstack/manila/.tox/py38/lib/python3.8/site-packages/testtools/testcase.py", line 502, in assertThat
   raise mismatch_error
   testtools.matchers._impl.MismatchError: '1.2.3.4' != '1.2.3.4-1.2.3.4'

Steps to reproduce
==================
* Run "tox -epy3.9" (or 3.8) with the latest update of these python versions
* observed with python 3.8.11 and 3.9.6

Root Cause:
-----------

The code that's breaking is here: https://opendev.org/openstack/manila/src/commit/1beaf635791c56aafc6bf055e6a4d375197cd217/manila/share/drivers/infinidat/infinibox.py#L333-L334

The behavior of the ipaddress.ip_network.hosts() module seems to have changed in these versions. There is now a callout in the docs for /32 addresses:
"Networks with a mask of 32 will return a list containing the single host address."
https://docs.python.org/3.9/library/ipaddress.html#ipaddress.IPv4Network.hosts

This call out doesn't exist for python3.7 for instance:
https://docs.python.org/3.7/library/ipaddress.html#ipaddress.IPv4Network.hosts

Changed in manila:
milestone: none → xena-2
importance: Undecided → Critical
assignee: nobody → Goutham Pacha Ravi (gouthamr)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/manila/+/799107

Changed in manila:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/manila/+/799136

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.opendev.org/c/openstack/manila/+/799107
Committed: https://opendev.org/openstack/manila/commit/1161a56a74b0fd926c20a63242af9dc852f56401
Submitter: "Zuul (22348)"
Branch: master

commit 1161a56a74b0fd926c20a63242af9dc852f56401
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Jul 1 09:01:55 2021 -0700

    Fix ipaddress issues in the infinidat driver

    In python3.8.11 and python3.9.6, the behavior of the
    ipaddress.ip_network.hosts() method changed. Now /32
    addresses are returned by that method as a single
    element list. This was apparently done as a bugfix [1]

    Change-Id: Iab6d96351fa21131d834ccf07ffddd70555f25a7
    Closes-Bug: #1934345
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    [1] https://bugs.python.org/issue27683

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/wallaby)

Reviewed: https://review.opendev.org/c/openstack/manila/+/799136
Committed: https://opendev.org/openstack/manila/commit/d015399af51a677ae72e3b56399beaec0dac963d
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit d015399af51a677ae72e3b56399beaec0dac963d
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Jul 1 09:01:55 2021 -0700

    Fix ipaddress issues in the infinidat driver

    In python3.8.11 and python3.9.6, the behavior of the
    ipaddress.ip_network.hosts() method changed. Now /32
    addresses are returned by that method as a single
    element list. This was apparently done as a bugfix [1]

    Change-Id: Iab6d96351fa21131d834ccf07ffddd70555f25a7
    Closes-Bug: #1934345
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    [1] https://bugs.python.org/issue27683
    (cherry picked from commit 1161a56a74b0fd926c20a63242af9dc852f56401)

tags: added: in-stable-wallaby
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/manila/+/799249

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (stable/victoria)

Reviewed: https://review.opendev.org/c/openstack/manila/+/799249
Committed: https://opendev.org/openstack/manila/commit/783c0777e881446e1a49147b376a8ff25f907369
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit 783c0777e881446e1a49147b376a8ff25f907369
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Jul 1 09:01:55 2021 -0700

    Fix ipaddress issues in the infinidat driver

    In python3.8.11 and python3.9.6, the behavior of the
    ipaddress.ip_network.hosts() method changed. Now /32
    addresses are returned by that method as a single
    element list. This was apparently done as a bugfix [1]

    Change-Id: Iab6d96351fa21131d834ccf07ffddd70555f25a7
    Closes-Bug: #1934345
    Signed-off-by: Goutham Pacha Ravi <email address hidden>
    [1] https://bugs.python.org/issue27683
    (cherry picked from commit 1161a56a74b0fd926c20a63242af9dc852f56401)
    (cherry picked from commit d015399af51a677ae72e3b56399beaec0dac963d)

tags: added: in-stable-victoria
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 13.0.0.0rc1

This issue was fixed in the openstack/manila 13.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 12.1.0

This issue was fixed in the openstack/manila 12.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 11.1.0

This issue was fixed in the openstack/manila 11.1.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.