yep, sorry I miss it at the first time, I looked at snapshot on output from test, and next things I see here
packages/nova/servicegroup/drivers/db.py:75
nova-cert node-5.test.domain.local internal enabled XXX 2015-05-21 10:31:40
nova-consoleauth node-2.test.domain.local internal enabled :-) 2015-05-21 12:32:26
nova-scheduler node-2.test.domain.local internal enabled :-) 2015-05-21 12:32:29
nova-conductor node-2.test.domain.local internal enabled :-) 2015-05-21 12:32:31
nova-cert node-2.test.domain.local internal enabled :-) 2015-05-21 12:32:32
nova-consoleauth node-3.test.domain.local internal enabled :-) 2015-05-21 12:32:36
nova-scheduler node-3.test.domain.local internal enabled :-) 2015-05-21 12:32:39
nova-conductor node-3.test.domain.local internal enabled :-) 2015-05-21 12:32:41
nova-cert node-3.test.domain.local internal enabled :-) 2015-05-21 12:32:43
2015-05-21 12:32:48.538 23307 DEBUG nova.servicegroup.drivers.db [req-6525241a-228f-496b-a424-f8c5fb50160e None] Seems service is down. Last heartbeat was 2015-05-21 12:28:29. Elapsed time is 259.538842 is_up /usr/lib/python2.6/site-packages/nova/servicegroup/drivers/db.py:75
nova-compute node-7.test.domain.local nova enabled XXX 2015-05-21 12:28:29
2015-05-21 12:32:48.540 23307 DEBUG nova.servicegroup.drivers.db [req-6525241a-228f-496b-a424-f8c5fb50160e None] Seems service is down. Last heartbeat was 2015-05-21 12:29:21. Elapsed time is 207.540535 is_up /usr/lib/python2.6/site-packages/nova/servicegroup/drivers/db.py:75
nova-compute node-4.test.domain.local nova enabled XXX 2015-05-21 12:29:21
As you can see nodes where compute is running updated at 12:29:21 and controllers has 12:32:29, so it may related to the time sync issues
yep, sorry I miss it at the first time, I looked at snapshot on output from test, and next things I see here nova/servicegro up/drivers/ db.py:75 test.domain. local internal enabled XXX 2015-05-21 10:31:40 test.domain. local internal enabled :-) 2015-05-21 12:32:26 test.domain. local internal enabled :-) 2015-05-21 12:32:29 test.domain. local internal enabled :-) 2015-05-21 12:32:31 test.domain. local internal enabled :-) 2015-05-21 12:32:32 test.domain. local internal enabled :-) 2015-05-21 12:32:36 test.domain. local internal enabled :-) 2015-05-21 12:32:39 test.domain. local internal enabled :-) 2015-05-21 12:32:41 test.domain. local internal enabled :-) 2015-05-21 12:32:43 up.drivers. db [req-6525241a- 228f-496b- a424-f8c5fb5016 0e None] Seems service is down. Last heartbeat was 2015-05-21 12:28:29. Elapsed time is 259.538842 is_up /usr/lib/ python2. 6/site- packages/ nova/servicegro up/drivers/ db.py:75 test.domain. local nova enabled XXX 2015-05-21 12:28:29 up.drivers. db [req-6525241a- 228f-496b- a424-f8c5fb5016 0e None] Seems service is down. Last heartbeat was 2015-05-21 12:29:21. Elapsed time is 207.540535 is_up /usr/lib/ python2. 6/site- packages/ nova/servicegro up/drivers/ db.py:75 test.domain. local nova enabled XXX 2015-05-21 12:29:21
packages/
nova-cert node-5.
nova-consoleauth node-2.
nova-scheduler node-2.
nova-conductor node-2.
nova-cert node-2.
nova-consoleauth node-3.
nova-scheduler node-3.
nova-conductor node-3.
nova-cert node-3.
2015-05-21 12:32:48.538 23307 DEBUG nova.servicegro
nova-compute node-7.
2015-05-21 12:32:48.540 23307 DEBUG nova.servicegro
nova-compute node-4.
As you can see nodes where compute is running updated at 12:29:21 and controllers has 12:32:29, so it may related to the time sync issues