exec_command of ssh client errors when tests are run from OS X

[Daryl Walleck]

When running tests that perform remote validation (tempest.api.compute.servers.test_create_server.ServersTestJSON is used in the example below) from a machine running OS X, an exception is raised:

2015-07-31 15:00:06,131 4748 INFO [tempest_lib.common.ssh] Creating ssh connection to 'xx.xx.xx.xx' as 'root' with public key authentication
2015-07-31 15:00:06,225 4748 INFO [paramiko.transport] Connected (version 2.0, client OpenSSH_6.6.1p1)
2015-07-31 15:00:06,765 4748 INFO [paramiko.transport] Authentication (publickey) successful!
2015-07-31 15:00:06,783 4748 INFO [tempest_lib.common.ssh] ssh connection to <email address hidden> successfuly created
}}}

Traceback (most recent call last):
  File "/Users/dwalleck/new/tempest/tempest/api/compute/servers/test_create_server.py", line 125, in test_verify_created_server_vcpus
    self.assertEqual(flavor['vcpus'], linux_client.get_number_of_vcpus())
  File "/Users/dwalleck/new/tempest/tempest/common/utils/linux/remote_client.py", line 75, in get_number_of_vcpus
    output = self.exec_command('grep -c processor /proc/cpuinfo')
  File "/Users/dwalleck/new/tempest/tempest/common/utils/linux/remote_client.py", line 56, in exec_command
    return self.ssh_client.exec_command(cmd)
  File "/Users/dwalleck/new/tempest-env/lib/python2.7/site-packages/tempest_lib/common/ssh.py", line 119, in exec_command
    poll = select.poll()
AttributeError: 'module' object has no attribute 'poll'

This occurs because the poll method is not implemented for all operating systems (https://docs.python.org/2/library/select.html#select.poll). I see a few options:

- The client could raise a more clear exception that directly explains that this is a Linux only capability
- Provide a different implementation to be used when tests are not run on the Linux platform

It seems like there should be other implementations of this functionality that would not be platform specific, which would allow Tempest to be run by a larger audience. I've worked on other implementations of remote/ssh clients, so I wouldn't mind taking on this task if this was an acceptable option.

[Jordan Pittier (jordan-pittier)]

I agree that it would be nice to *not* rely on select.poll() especially because this is in Tempest-lib and we would like this library to be adopted broadly.

On the other hand, people running tempest from an OS X host seems in minority, so let's not add tons of code just to not use poll.

I guess it all depends on how 'complicated' the change would be.

[Daryl Walleck (dwalleck)]

I can see how from a gating/CI perspective it might be reasonable to say that the host is running Linux, but about half our team runs OS X on our personal hardware, and not being able to execute those tests locally would be a hinderance. The majority of this code (https://github.com/openstack/tempest-lib/blob/master/tempest_lib/common/ssh.py#L123) is a wrapper around the paramiko command execution (https://github.com/paramiko/paramiko/blob/54415207d186155afab6bd3a8571a70587a85fab/paramiko/client.py#L381). I'm assuming this implementation was added to address an issue or enhance the reliability of command execution. However, it does itself add complexity and break the cross-platform compatibility that paramiko natively has. I've been using essentially the default paramiko exec_command function in my remote client tests for some time now with no issues, so it's not clear to me what problem this extra code is trying to solve.

I was thinking of a basic patch to check if the poll capability was available, and using the enhanced exec if it is or the barebones exec from paramiko if it is not.

https://gist.github.com/dwalleck/75e19c02dbc92aa9642e

I've been using the standard exec_command from paramiko in tests for some time now, so I'm comfortable using just that when necessary,

[Jordan Pittier (jordan-pittier)]

I like the approach of defaulting to " _exec_command_safe" on Linux. I took a look at your change and it seems very reasonable. You should propose such a patch in Gerrit.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/212691

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/212691
Committed: https://git.openstack.org/cgit/openstack/tempest-lib/commit/?id=736de8113db02c7b47949d31729c53f885736bb4
Submitter: Jenkins
Branch: master

commit 736de8113db02c7b47949d31729c53f885736bb4
Author: Daryl Walleck <email address hidden>
Date: Thu Aug 13 13:24:42 2015 -0500

    Provide alternative ssh exec_command for non-linux environments

    When running tests that use remote validation on OSX (or any
    non-Linux OS), the exec_command method would raise an exception
    due to the "poll" method not existing for select. While this
    implementation is very safe and reliable, paramiko does provide
    other means for handling reading the output from commands. This
    patch provide a basic alternative to the existing method that is
    only used when the poll method is not available.

    Change-Id: Ic14b2331fd59a6311ebe51f598d2cc2320c4389e
    Closes-Bug: #1480983