PowerVM drver uses stale ssh connection if hypervisor is rebooted
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Fix Released
|
Undecided
|
Matt Riedemann |
Bug Description
While trying to run this (on grizzly 2013.1):
tempest/
We get this:
2013-05-06 14:53:51.344 5682 DEBUG nova.utils [-] Running cmd (SSH): lshwres -r mem --level sys -F configurable_
2013-05-06 14:53:51.344 5682 ERROR nova.manager [-] Error during ComputeManager.
2013-05-06 14:53:51.344 5682 TRACE nova.manager Traceback (most recent call last):
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager task(self, context)
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager nodenames = set(self.
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager stats = self.get_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager return self._powervm.
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager self._update_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager memory_info = self._operator.
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager output = self.run_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager check_exit_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager stdin_stream, stdout_stream, stderr_stream = ssh.exec_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager chan = self._transport
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager return self.open_
2013-05-06 14:53:51.344 5682 TRACE nova.manager File "/usr/lib/
2013-05-06 14:53:51.344 5682 TRACE nova.manager raise SSHException('SSH session not active')
2013-05-06 14:53:51.344 5682 TRACE nova.manager SSHException: SSH session not active
This is because the test was ran earlier and the hypervisor was locked up. The tester rebooted the hypervisor but trying to run the test again results in the SSHException. This is due to the powervm operator.py code uses a cached paramiko ssh client connection:
https:/
The code should check the connection before assuming it's OK and then failing because it's bad.
Changed in nova: | |
assignee: | nobody → Matt Riedemann (mriedem) |
tags: | added: powervm |
Changed in nova: | |
milestone: | none → havana-1 |
status: | Fix Committed → Fix Released |
tags: | removed: grizzly-backport-potential |
Changed in nova: | |
milestone: | havana-1 → 2013.2 |
Fix proposed to branch: master /review. openstack. org/28451
Review: https:/