Deleting of starting cluster could result in ERROR log (one more scenario)

Bug #1421032 reported by Andrew Lazarev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Sahara
Fix Released
Low
Andrew Lazarev

Bug Description

Steps to reproduce:
1. Start cluster
2. Delete it before it finished

Stacktrace:
2015-02-11 17:21:57.044 22422 ERROR sahara.context [-] Thread 'vanilla-configure-al-test-4-a-001' fails with exception: 'SSHException: Server connection dropped: '
2015-02-11 17:21:57.044 22422 TRACE sahara.context Traceback (most recent call last):
2015-02-11 17:21:57.044 22422 TRACE sahara.context File "/Users/alazarev/openstack/sahara/sahara/context.py", line 190, in _wrapper
2015-02-11 17:21:57.044 22422 TRACE sahara.context func(*args, **kwargs)
2015-02-11 17:21:57.044 22422 TRACE sahara.context File "/Users/alazarev/openstack/sahara/sahara/plugins/vanilla/v1_2_1/versionhandler.py", line 378, in _push_configs_to_new_node
2015-02-11 17:21:57.044 22422 TRACE sahara.context self._push_master_configs(r, cluster, extra, instance)
...
2015-02-11 17:21:57.044 22422 TRACE sahara.context File "/Users/alazarev/openstack/sahara/sahara/utils/ssh_remote.py", line 778, in _run
2015-02-11 17:21:57.044 22422 TRACE sahara.context return procutils.run_in_subprocess(self.proc, func, args, kwargs)
2015-02-11 17:21:57.044 22422 TRACE sahara.context File "/Users/alazarev/openstack/sahara/sahara/utils/procutils.py", line 52, in run_in_subprocess
2015-02-11 17:21:57.044 22422 TRACE sahara.context raise SubprocessException(result['exception'])
2015-02-11 17:21:57.044 22422 TRACE sahara.context SubprocessException: SSHException: Server connection dropped:
2015-02-11 17:21:57.044 22422 TRACE sahara.context
2015-02-11 17:21:57.089 22422 INFO sahara.service.ops [-] Cluster ca0f1e0c-7b9a-4230-a0e5-fa9666ff9c68 was deleted or marked for deletion. Canceling current operation.

Reason:
Exception is raised in separate thread and it is logged before ops.ops_error_handler catch it.

summary: - Deleting of starting cluster could result in ERROR log (one more time)
+ Deleting of starting cluster could result in ERROR log (one more
+ scenario)
Changed in sahara:
importance: Undecided → Low
status: New → Confirmed
Changed in sahara:
assignee: nobody → Andrew Lazarev (alazarev)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to sahara (master)

Fix proposed to branch: master
Review: https://review.openstack.org/155860

Changed in sahara:
status: Confirmed → In Progress
Changed in sahara:
milestone: none → kilo-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (master)

Reviewed: https://review.openstack.org/155860
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=d8c602cdf0c55247170b29a4a10c252b3356c011
Submitter: Jenkins
Branch: master

commit d8c602cdf0c55247170b29a4a10c252b3356c011
Author: Andrew Lazarev <email address hidden>
Date: Fri Feb 13 10:35:19 2015 -0800

    Removed error log for failure inside individual thread

    We don't want to have all errors from all threads logged on
    ERROR level. E.g. in case of network failure and 10 parallel VM
    tasks there will be 10 stacktraces in the logs. Parent thread
    will receive the first exception and in most cases it is enough
    to treat operation over all threads as errored.

    Change-Id: Idc93c33f41d502e5aa0149b93ce816379c1043b1
    Closes-Bug: #1421032

Changed in sahara:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in sahara:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in sahara:
milestone: kilo-3 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.