periodic tasks called twice in unit tests

[aeva black]

In the unit tests of ConductorManager's periodic tasks, eg,

    def test_periodic_keepalive(self):
        self.service.start()
        with mock.patch.object(self.dbapi, 'touch_conductor') as mock_touch:
            self.service.periodic_tasks(self.context)
            mock_touch.assert_called_once_with('test-host')

there seems to be an issue that, if more than one periodic_task is defined, this test will fail because dbapi.touch_conductor() will be called twice instead of once.

[Haomeng,Wang (whaom)]

Devananda, I try to recreate with my local env, but not reproduced, so can you show your steps/testcase/unittest code, and I will try to reproduce and debug the root cause?

[aeva black (tenbrae)]

This was caused by two things and the interplay between them:
- periodic tasks may be called when the Conductor service starts, if the periodic task is defined with run_immediately=True
- a periodic task may get called (again) if the service thinks NOW > LAST_RUN + INTERVAL, which could be triggered by timeutils.set_time_override
- conversely, a method decorated as a periodic task will _not_ get run again if the service thinks that not enough time has passed, even when called directly.

I have resolved these issues in the unit tests with a recent series of patches, which also added another periodic task. I'm marking this bug as Fix Committed. For reference, here are the relevant patches which affected this:
- https://review.openstack.org/#/c/58851/
- https://review.openstack.org/#/c/56530/