service record did not updated on time after restart

Bug #1226370 reported by Guangya Liu (Jay Lau)
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Michael H Wilson

Bug Description

When using db driver or mc driver for servicegroup, if the service was created before and restart, then the service record will not update on time but need to wait a delay_interval (10s by default) before update.

This will cause VM deploy failed when nova compute restart and user deploy VM immediately with ComputeFilter.

Tags: compute
Changed in nova:
assignee: nobody → Jay Lau (jay-lau-513)
tags: added: compute
Revision history for this message
Russell Bryant (russellb) wrote :

Since you assigned yourself to this bug, I take it that you're working on a fix? I took a quick look at the code and a quick guess is that the "initial_delay" needs to be set to 0 for updating the db.

diff --git a/nova/servicegroup/drivers/db.py b/nova/servicegroup/drivers/db.py
index 5a0869d..a25b525 100644
--- a/nova/servicegroup/drivers/db.py
+++ b/nova/servicegroup/drivers/db.py
@@ -49,7 +49,7 @@ class DbDriver(api.ServiceGroupDriver):
         report_interval = service.report_interval
         if report_interval:
             service.tg.add_timer(report_interval, self._report_state,
- report_interval, service)
+ 0, service)

     def is_up(self, service_ref):
         """Moved from nova.utils
diff --git a/nova/servicegroup/drivers/mc.py b/nova/servicegroup/drivers/mc.py
index a22aa7c..3df5e37 100644
--- a/nova/servicegroup/drivers/mc.py
+++ b/nova/servicegroup/drivers/mc.py
@@ -60,7 +60,7 @@ class MemcachedDriver(api.ServiceGroupDriver):
         report_interval = service.report_interval
         if report_interval:
             service.tg.add_timer(report_interval, self._report_state,
- report_interval, service)
+ 0, service)

     def is_up(self, service_ref):
         """Moved from nova.utils

Changed in nova:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Guangya Liu (Jay Lau) (jay-lau-513) wrote :

Thanks Russell, that's exactly my code diff. ;-)

But I did not find a good way to add unit test, and after some checking, seems we do not need to add unit test for this.

Will upload a patch soon.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/47051

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
John Garbutt (johngarbutt) wrote :
Changed in nova:
assignee: Jay Lau (jay-lau-513) → Michael H Wilson (geekinutah)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/71439
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=afb5cb64b59c982e541cf48de0f126f6e8e2400f
Submitter: Jenkins
Branch: master

commit afb5cb64b59c982e541cf48de0f126f6e8e2400f
Author: Michael Wilson <email address hidden>
Date: Wed Feb 5 18:41:42 2014 -0700

    Change initial delay for servicegroup api reporting

    Initial delay should not necessarily be equal to
    report_interval. Instead make it a reasonable value
    such as 5 seconds.

    This is so the surprising behavior of a service not
    reporting when you start it up doesn't happen if your
    report interval is configured to a larger value.

    For example, in an environment where there are many
    services it may not be optimal to have them all reporting
    every 10 seconds which is the default. You may have them
    report every 5 minutes. It is confusing to start up the
    service and not have it show as available until 5 minutes
    later.

    Fixes bug #1226370

    Change-Id: I85c4146bfa465fffaf89d4c2c74e6cb4ef84cee2

Changed in nova:
status: In Progress → Fix Committed
Changed in nova:
milestone: none → icehouse-rc1
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-rc1 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.