test_scheduler_multi_instance fails once in a while

Bug #1497269 reported by Renat Akhmerov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mistral
Fix Released
High
Nikolay Makhotkin
Liberty
Fix Released
High
Nikolay Makhotkin
Revision history for this message
Nikolay Makhotkin (nmakhotkin) wrote :

Can not reproduce this bug yet using Mistral cluster or using Mistral cluster with manual putting call to the scheduler.

I tend to think the problem is due to we use mocks in those tests and they are invoked simultenously more than 1 time.

Revision history for this message
Nikolay Makhotkin (nmakhotkin) wrote :

Not only the one test fails. test_scheduler_with_factory, test_scheduler_without_factory also fail once in a while. But these tests don't assume we have multi instance scheduler. Here is a concurrency problem.

======================================================================
FAIL: mistral.tests.unit.services.test_scheduler.SchedulerServiceTest.test_scheduler_without_factory
tags: worker-2
----------------------------------------------------------------------
Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/home/kolyan/projects/mistral/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "mistral/tests/unit/services/test_scheduler.py", line 119, in test_scheduler_without_factory
    method.assert_called_once_with(name='task', id='321')
  File "/home/kolyan/projects/mistral/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 947, in assert_called_once_with
    raise AssertionError(msg)
AssertionError: Expected 'factory_method' to be called once. Called 2 times.

Revision history for this message
Renat Akhmerov (rakhmerov) wrote : Re: [Bug 1497269] Re: test_scheduler_multi_instance fails once in a while

Может у нас по ошибке в тестах больше одного скедулера запускается?

Sent from Renat's iPad

> 29 сент. 2015 г., в 17:32, Nikolay Makhotkin <email address hidden> написал(а):
>
> Not only the one test fails. test_scheduler_with_factory,
> test_scheduler_without_factory also fail once in a while. But these
> tests don't assume we have multi instance scheduler. Here is a
> concurrency problem.
>
> ======================================================================
> FAIL: mistral.tests.unit.services.test_scheduler.SchedulerServiceTest.test_scheduler_without_factory
> tags: worker-2
> ----------------------------------------------------------------------
> Empty attachments:
> pythonlogging:''
> stderr
> stdout
>
> Traceback (most recent call last):
> File "/home/kolyan/projects/mistral/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
> return func(*args, **keywargs)
> File "mistral/tests/unit/services/test_scheduler.py", line 119, in test_scheduler_without_factory
> method.assert_called_once_with(name='task', id='321')
> File "/home/kolyan/projects/mistral/.tox/py27/local/lib/python2.7/site-packages/mock/mock.py", line 947, in assert_called_once_with
> raise AssertionError(msg)
> AssertionError: Expected 'factory_method' to be called once. Called 2 times.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1497269
>
> Title:
> test_scheduler_multi_instance fails once in a while
>
> Status in Mistral:
> Incomplete
> Status in Mistral liberty series:
> Incomplete
>
> Bug description:
> http://logs.openstack.org/59/222159/7/gate/gate-mistral-
> python27/6e11234/testr_results.html.gz
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mistral/+bug/1497269/+subscriptions

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to mistral (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/229845

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to mistral (master)

Reviewed: https://review.openstack.org/229845
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=0420bbd86fe53a66c9da8e8256bdd97817ffe894
Submitter: Jenkins
Branch: master

commit 0420bbd86fe53a66c9da8e8256bdd97817ffe894
Author: Nikolay Mahotkin <email address hidden>
Date: Mon Sep 28 16:31:23 2015 +0300

    Fixing scheduler tests

     * Don't instantinate more unneeded schedulers
     * Create separate target method

    Related-Bug: #1497269

    Change-Id: I52234f044111bd26eaa9217ea73dbe39c5227d94

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to mistral (stable/liberty)

Related fix proposed to branch: stable/liberty
Review: https://review.openstack.org/233965

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to mistral (stable/liberty)

Reviewed: https://review.openstack.org/233965
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=d8fe7927cb0416a4116487d388f53476550b33b0
Submitter: Jenkins
Branch: stable/liberty

commit d8fe7927cb0416a4116487d388f53476550b33b0
Author: Nikolay Mahotkin <email address hidden>
Date: Mon Sep 28 16:31:23 2015 +0300

    Fixing scheduler tests

     * Don't instantinate more unneeded schedulers
     * Create separate target method

    Related-Bug: #1497269

    Change-Id: I52234f044111bd26eaa9217ea73dbe39c5227d94
    (cherry picked from commit 0420bbd86fe53a66c9da8e8256bdd97817ffe894)

tags: added: in-stable-liberty
Changed in mistral:
milestone: 1.0.0 → none
Dougal Matthews (d0ugal)
Changed in mistral:
status: Incomplete → Fix Committed
Dougal Matthews (d0ugal)
Changed in mistral:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.