parameterized test coroutines not running in lunar

[Olivier Gayot]

I have upgraded my desktop to lunar recently, which pulls Python 3.11.
The subiquity unit tests look all green, but all tests that use coroutines + parameterized don't actually run the coroutine. The presence of warnings makes it more or less obvious.

excerpt
-------

subiquity/server/controllers/tests/test_filesystem.py: 12 warnings
  /usr/lib/python3.11/unittest/async_case.py:90: RuntimeWarning: coroutine 'TestGuidedV2.test_blank_disk' was never awaited
    if self._callMaybeAsync(method) is not None:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_00
  /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_00>>)
    return self.run(*args, **kwds)

subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_01
  /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_01>>)
    return self.run(*args, **kwds)

subiquity/server/controllers/tests/test_filesystem.py::TestGuidedV2::test_blank_disk_02
  /usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestGuidedV2.test_blank_disk of <subiquity.server.controllers.tests.test_filesystem.TestGuidedV2 testMethod=test_blank_disk_02>>)
    return self.run(*args, **kwds)
[...]

simple reproducer
-----------------

import unittest

from parameterized import parameterized

class TestParameterizedCoroutine(unittest.IsolatedAsyncioTestCase):

    def test_function_no_parameterized(self):
        self.fail("This test ran properly")

    async def test_coroutine_no_parameterized(self):
        self.fail("This test ran properly")

    @parameterized.expand([(1, 2)])
    def test_function_parameterized(self, x, y):
        self.fail("This test ran properly")

    @parameterized.expand([(1, 2)])
    async def test_coroutine_parameterized(self, x, y):
        self.fail("This test ran properly")

In Python3.10 and lower, all 4 tests "fail" with "This test ran properly".
In Python3.11, the last test "succeeds" and produces the following warnings:

/usr/lib/python3.11/unittest/async_case.py:90: RuntimeWarning: coroutine 'TestParameterizedCoroutine.test_coroutine_parameterized' was never awaited
  if self._callMaybeAsync(method) is not None:
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestParameterizedCoroutine.test_coroutine_parameterized of <test_parameterized_coroutines.TestParameterizedCoroutine testMethod=test_coroutine_parameterized_0>>)
  return self.run(*args, **kwds)

Tested with python3 -m unittest and python3 -m nose with the same result.

[Dan Bungert (dbungert)]

Likely related:
https://github.com/python/cpython/issues/101486
https://github.com/wolever/parameterized/issues/137

[Olivier Gayot (ogayot)]

Marking invalid for python3.11 and python-parameterized. Subiquity's test suite might have relied on undefined behavior.

https://github.com/canonical/subiquity/pull/1566 was merged.