GEM Cluster worker-side connection problem

Bug #984731 reported by Laurentiu D.
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenQuake (deprecated)
Fix Released
Critical
Muharem Hrnjadovic

Bug Description

[2012-04-18 12:07:29,866 #586 - INFO MainProcess/6469 supervisor] Entering supervisor for job 586
[2012-04-18 12:07:29,880 #586 - DEBUG MainProcess/6469 amqplib] Start from server, version: 8.0, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {}, u'platform': u'Erlang/OTP', u'version': u'2.7.1'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2012-04-18 12:07:29,881 #586 - DEBUG MainProcess/6469 amqplib] Open OK! known_hosts []
[2012-04-18 12:07:29,881 #586 - DEBUG MainProcess/6469 amqplib] using channel_id: 1
[2012-04-18 12:07:29,882 #586 - DEBUG MainProcess/6469 amqplib] Channel open
[2012-04-18 12:07:30,344 #586 gemcontrol.ethz.ch INFO MainProcess/6468 root] Storing source model from job config
[2012-04-18 12:07:34,099 #586 gemcontrol.ethz.ch INFO MainProcess/6468 root] Storing GMPE map from job config
[2012-04-18 12:07:34,100 #586 gemcontrol.ethz.ch DEBUG MainProcess/6468 hazard] -data_length: 1
[2012-04-18 12:07:34,101 #586 gemcontrol.ethz.ch DEBUG MainProcess/6468 hazard] -#subtasks: 1
[2012-04-18 12:07:34,151 #586 gemcontrol.ethz.ch CRITICAL MainProcess/6468 root] Calculation failed with exception: 'connection already closed'
Traceback (most recent call last):
  File "/usr/bin/openquake", line 166, in <module>
    log_level=args.log_level)
  File "/usr/lib/pymodules/python2.7/openquake/engine.py", line 727, in run_calculation
    _launch_calculation(calc_proxy, sections)
  File "/usr/lib/pymodules/python2.7/openquake/engine.py", line 800, in _launch_calculation
    calculator.execute()
  File "/usr/lib/pymodules/python2.7/openquake/calculators/hazard/uhs/core.py", line 264, in execute
    ath=uhs_task_handler, ath_args=ath_args)
  File "/usr/lib/pymodules/python2.7/openquake/utils/tasks.py", line 82, in distribute
    _check_exception(results)
  File "/usr/lib/pymodules/python2.7/openquake/utils/tasks.py", line 94, in _check_exception
    raise result
psycopg2.InterfaceError: connection already closed
[2012-04-18 12:07:35,215 #586 - INFO MainProcess/6469 root] Recording stop time for job 586 to calc_stats
[2012-04-18 12:07:35,247 #586 - INFO MainProcess/6469 root] Cleaning up after job 586
[2012-04-18 12:07:35,249 #586 - INFO MainProcess/6469 root] KVS garbage collection removed 5 keys for job 586
[2012-04-18 12:07:35,253 #586 - DEBUG MainProcess/6469 amqplib] Closed channel #1
[2012-04-18 12:07:35,254 #586 - INFO MainProcess/6469 supervisor] Job 586 finished in 0:00:05.366533
[2012-04-18 12:07:35,254 #586 - INFO MainProcess/6469 supervisor] Exiting supervisor for job 586

Changed in openquake:
importance: Undecided → Critical
assignee: nobody → Muharem Hrnjadovic (al-maisan)
summary: - GEM Cluster Worker site connection problem
+ GEM Cluster worker-side connection problem
Changed in openquake:
milestone: none → 0.7.0
status: New → In Progress
tags: added: database enduser-visible mfcluster
Revision history for this message
Muharem Hrnjadovic (al-maisan) wrote :

Workers were running on the control node -- that may have caused the disruption

Changed in openquake:
status: In Progress → Fix Committed
Changed in openquake:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.