TestTimer breaks VPNaaS functional tests

Bug #1912320 reported by Bodo Petermann
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fix Released

Bug Description

Some functional tests for neutron-vpnaas make use of the NamespaceFixture.
If the tests are run in combination with a recent neutron version some tests
fail because the TestTimer raises a TestTimerTimeout even if the namespace
cleanup finishes before the timeout.

In the test setup the tox env for dsvm-functional-sswan will normally
install neutron 17.0.0 (victoria), but for my tests I needed a recent
neutron, so I installed it as an additional step in the setup of the tox env.

The test setup steps are like these, on an Ubuntu 20.04 host:

git clone https://git.openstack.org/openstack-dev/devstack
git clone https://opendev.org/openstack/neutron
git clone https://opendev.org/openstack/neutron-vpnaas
cd neutron-vpnaas
VENV=dsvm-functional-sswan ./tools/configure_for_vpn_func_testing.sh ../devstack -i
tox -e dsvm-functional-sswan --notest
source .tox/dsvm-functional-sswan/bin/activate
python -m pip install ../neutron

Then run the neutron-vpnaas functional tests:

tox -e dsvm-functional-sswan

Some tests fail and you see the TestTimerTimeout exception.

The tests were fine with neutron 17.0.0.
The TestTimer was introduced later.
Change set https://review.opendev.org/c/openstack/neutron/+/754938/
Related bug https://bugs.launchpad.net/neutron/+bug/1838793

I could narrow the problem with the TestTimer down.
In at least one neutron-vpnaas test
the NamespaceFixture is used.
The TestTimer is set up, the test completes and the namespace is deleted
successfully before the 5 seconds of the timer are over. But shortly after
that the timer still fires.

The problem is the following: on timer start the old signal handler is
stored (Handler.SIG_DFL in my case) and the remaining time of any existing
alarm (in my case 0). On exit the signal handler is supposed to be reset
and the alarm too. But neither happens.
The signal handler is not set back, because Handler.SIG_DFL is falsy.
The alarm is not stopped because the old value was 0 (there was no ongoing
alarm). So in the end the alarm started by TestTimer will eventually be

Change set where the TestTimer was introduced:
That related to bug #1838793

Changed in neutron:
status: New → Triaged
Revision history for this message
Bodo Petermann (bpetermann) wrote :
Revision history for this message
Bence Romsics (bence-romsics) wrote :

I'm wondering if the root cause is not reinstalling the default signal handler as you write, or that TestTimer is used outside of a Timeout context.

Changed in neutron:
importance: Undecided → Medium
Revision history for this message
Bodo Petermann (bpetermann) wrote :

I see. Is there a way to run the tests using the Timeout fixture automatically? Or should the neutron-vpnaas tests be changed to use the Timeout fixture explicitly?

Revision history for this message
Bodo Petermann (bpetermann) wrote :

Setting OS_TEST_TIMEOUT in the tox env avoids the problem as TestTimer will find an existing alarm to re-arm in __exit__.
I will prepare a patch for neutron-vpnaas to set OS_TEST_TIMEOUT.

Maybe the patch for TestTimer is still acceptable though as it makes it more robust.

Changed in neutron:
status: Triaged → In Progress
Changed in neutron:
status: In Progress → Fix Committed
Revision history for this message
Bodo Petermann (bpetermann) wrote :

Since the neutron patch is merged and fixes the issue I did not file a patch for neutron-vpnaas to set OS_TEST_TIMEOUT.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron

This issue was fixed in the openstack/neutron release candidate.

Changed in neutron:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.