Deletion of cluster could result in ERROR logs
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Sahara |
Fix Released
|
Low
|
Andrew Lazarev |
Bug Description
Steps to repro:
1. Start cluster creation
2. Delete cluster before it is created
Expected behavior:
Cluster is deleted. No messages on error log level in logs.
Observed behavior:
Cluster is deleted. Errors (ERROR and WARNING) in logs.
Logs:
2014-01-30 14:46:04.162 52891 WARNING savanna.
2014-01-30 14:46:04.203 52891 ERROR root [-] Original exception being dropped: ['Traceback (most recent call last):\n', ' File "/Users/
2014-01-30 14:46:04.204 52891 ERROR savanna.context [-] Thread 'cluster-
2014-01-30 14:46:04.204 52891 TRACE savanna.context Traceback (most recent call last):
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context func(*args, **kwargs)
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context INFRA.create_
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context "status_
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context ret = func(*args, **kwargs)
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context values)
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context return self.db.
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context res = func(*args, **kwargs)
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context return IMPL.cluster_
2014-01-30 14:46:04.204 52891 TRACE savanna.context File "/Users/
2014-01-30 14:46:04.204 52891 TRACE savanna.context "Cluster id '%s' not found!")
2014-01-30 14:46:04.204 52891 TRACE savanna.context NotFoundException: Cluster id 'eeb64fd0-
2014-01-30 14:46:04.204 52891 TRACE savanna.context
summary: |
- [Vanilla] Deletion of cluster could result in ERROR logs + Deletion of cluster could result in ERROR logs |
Changed in sahara: | |
importance: | Undecided → Low |
milestone: | none → juno-1 |
Changed in sahara: | |
milestone: | juno-1 → juno-2 |
Changed in sahara: | |
assignee: | nobody → Andrew Lazarev (alazarev) |
Changed in sahara: | |
status: | Confirmed → In Progress |
Changed in sahara: | |
status: | Fix Committed → Fix Released |
Changed in sahara: | |
milestone: | juno-2 → 2014.2 |
I was able to reproduce this bug with a trunk version of Sahara (commit c1d6d02ab7b6c8e 50bbe637d9f6883 2f19dbcdc6) and a stable/icehouse devstack.
The error trace was very similar for me:
2014-05-27 11:04:24.944 WARNING sahara. service. engine [-] Can't start cluster '_unknown_' (reas service. engine [-] Presumably the operation failed becau mimccune/ venvs/sahara/ lib/python2. 7/site- packages/ sahara/ serv engine. py", line 65, in create_cluster\n volumes. attach( cluster) \n', ' File "/ho venvs/sahara/ lib/python2. 7/site- packages/ sahara/ service/ volumes. py", line 31, in at node_groups: \n', "AttributeError: 'NoneType' object has no creating- 5c8c707c- 88cd-4fd3- 8c mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ context. py", line 120, in _wrapper mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ service/ ops.py" , line 130, in _provision_cluster cluster( cluster) mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ service/ direct_ engine. py", line 80, in create_cluster description" : str(ex)}) mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ conductor/ resource. py", line 42, in handle mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ conductor/ api.py" , line 88, in cluster_update mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ conductor/ manager. py", line 165, in cluster_update cluster_ update( context, cluster mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ db/api. py", line 94, in decorator mimccune/ venvs/sahara/ lib/python2. 7/ sahara/ db/api. py", line 132, in cluster_update update( context, cluster, v
on: 'NoneType' object has no attribute 'node_groups')
2014-05-27 11:04:24.944 WARNING sahara.
se the cluster wasdeleted by a user during the process.
2014-05-27 11:04:24.963 ERROR root [-] Original exception being dropped: ['Traceback (most rece
nt call last):\n', ' File "/home/
ice/direct_
me/mimccune/
tach\n for node_group in cluster.
attribute 'node_groups'\n"]
2014-05-27 11:04:24.963 ERROR sahara.context [-] Thread 'cluster-
4b-bebdd25b9e0a' fails with exception: 'Cluster id 'None' not found!'
2014-05-27 11:04:24.963 TRACE sahara.context Traceback (most recent call last):
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context func(*args, **kwargs)
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context INFRA.create_
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context "status_
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context ret = func(*args, **kwargs)
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context values)
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context return self.db.
, values)
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context res = func(*args, **kwargs)
2014-05-27 11:04:24.963 TRACE sahara.context File "/home/
site-packages/
2014-05-27 11:04:24.963 TRACE sahara.context return IMPL.cluster_
alues)
2014-05-27 11:04...