Need fix issues due to no module available in XenServer dom0

[Jianghua Wang]

See the following error log in XenServer 7.0.

Apr 02 06:13:54.310455 DevStackOSDomU nova-compute[5735]: INFO nova.service [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Starting compute node (version 17.0.0)
Apr 02 06:13:54.555380 DevStackOSDomU nova-compute[5735]: WARNING nova.compute.manager [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] No compute node record found for host xrtmia-07-46. If this is the first time this service is starting on this host, then you can ignore this warning.: ComputeHostNotFound_Remote: Compute host xrtmia-07-46 could not be found.
Apr 02 06:13:54.555871 DevStackOSDomU nova-compute[5735]: DEBUG nova.virt.xenapi.host [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Updating host stats {{(pid=5735) update_status /opt/stack/nova/nova/virt/xenapi/host.py:382}}
Apr 02 06:13:54.611441 DevStackOSDomU nova-compute[5735]: DEBUG os_xenapi.client.session [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Got exception: ['XENAPI_PLUGIN_FAILURE', 'non-zero exit', '', 'Traceback (most recent call last):\n File "/etc/xapi.d/plugins/xenhost.py", line 38, in <module>\n import six\nImportError: No module named six\n'] {{(pid=5735) _unwrap_plugin_exceptions /usr/local/lib/python2.7/dist-packages/os_xenapi/client/session.py:329}}
Apr 02 06:13:54.611967 DevStackOSDomU nova-compute[5735]: ERROR nova.virt.xenapi.host [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] The call to get_host_data returned an error: ['XENAPI_PLUGIN_FAILURE', 'non-zero exit', '', 'Traceback (most recent call last):\n File "/etc/xapi.d/plugins/xenhost.py", line 38, in <module>\n import six\nImportError: No module named six\n'].: Failure: ['XENAPI_PLUGIN_FAILURE', 'non-zero exit', '', 'Traceback (most recent call last):\n File "/etc/xapi.d/plugins/xenhost.py", line 38, in <module>\n import six\nImportError: No module named six\n']
Apr 02 06:13:54.615846 DevStackOSDomU nova-compute[5735]: DEBUG oslo_concurrency.lockutils [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Lock "sr-scan-OpaqueRef:69f290c0-d19a-b4b4-b796-e897720cc37d" acquired by "nova.virt.xenapi.vm_utils.do_scan" :: waited 0.000s {{(pid=5735) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:273}}
Apr 02 06:13:54.616353 DevStackOSDomU nova-compute[5735]: DEBUG nova.virt.xenapi.vm_utils [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Scanning SR OpaqueRef:69f290c0-d19a-b4b4-b796-e897720cc37d {{(pid=5735) do_scan /opt/stack/nova/nova/virt/xenapi/vm_utils.py:1834}}
Apr 02 06:13:55.115989 DevStackOSDomU nova-compute[5735]: DEBUG oslo_concurrency.lockutils [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Lock "sr-scan-OpaqueRef:69f290c0-d19a-b4b4-b796-e897720cc37d" released by "nova.virt.xenapi.vm_utils.do_scan" :: held 0.499s {{(pid=5735) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:285}}
Apr 02 06:13:55.152925 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service [None req-8f40f509-84c4-4fcd-88c4-5021759381b8 None None] Error starting thread.: TypeError: 'str' object does not support item assignment
Apr 02 06:13:55.153375 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service Traceback (most recent call last):
Apr 02 06:13:55.153805 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py", line 729, in run_service
Apr 02 06:13:55.154293 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service service.start()
Apr 02 06:13:55.154722 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/service.py", line 180, in start
Apr 02 06:13:55.155140 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service self.manager.pre_start_hook()
Apr 02 06:13:55.155562 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/compute/manager.py", line 1175, in pre_start_hook
Apr 02 06:13:55.155978 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service startup=True)
Apr 02 06:13:55.156396 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/compute/manager.py", line 7330, in update_available_resource
Apr 02 06:13:55.156874 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service nodenames = set(self.driver.get_available_nodes())
Apr 02 06:13:55.157300 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 732, in get_available_nodes
Apr 02 06:13:55.157750 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service stats = self.host_state.get_host_stats(refresh=refresh)
Apr 02 06:13:55.158209 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/driver.py", line 104, in host_state
Apr 02 06:13:55.158633 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service self._host_state = host.HostState(self._session)
Apr 02 06:13:55.159053 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/host.py", line 152, in __init__
Apr 02 06:13:55.159494 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service self.update_status()
Apr 02 06:13:55.159915 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service File "/opt/stack/nova/nova/virt/xenapi/host.py", line 390, in update_status
Apr 02 06:13:55.160344 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service data["disk_total"] = total
Apr 02 06:13:55.160779 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service TypeError: 'str' object does not support item assignment
Apr 02 06:13:55.161252 DevStackOSDomU nova-compute[5735]: ERROR oslo_service.service

[Jianghua Wang (wjh-fresh)]

six module is installed via rpm python-six-1.9.0-2.el7.noarch; this RPM is installed in XS7.1. But unfortunately it's not included in XS7.0.

[Jianghua Wang (wjh-fresh)]

Even in XS7.1, there is also a dependence issue:
[root@xrtuk-11-13 plugins]# python -m agent.py
/usr/bin/python: No module named future

agent.py is invoked only when the instance agent is enabled (by default it's disabled), so it's not detected in CI job.

In XS7.0

[root@xrtmia-02-03 plugins]# for f in $files; do python -m $f 2>&1; done | grep -v "\.py"
/usr/bin/python: No module named future
/usr/bin/python: No module named future
/usr/bin/python: No module named six.moves
/usr/bin/python: No module named six
/usr/bin/python: No module named six

[root@xrtmia-02-03 plugins]# egrep 'six|future' *
agent.py:from future import standard_library
Binary file agent.pyc matches
glance.py: from six.moves import http_client as httplib
glance.py:from six.moves import urllib
glance.py:from six.moves.urllib.parse import urlparse
Binary file glance.pyc matches
utils.py: the future. This can occur during a migration if the clocks on the two
Binary file utils.pyc matches
xenhost.py:import six
xenhost.py: import six.moves.xmlrpc_client as xmlrpclib
xenhost.py: if not isinstance(cmd, six.string_types):

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/559559

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/559559
Committed: https://git.openstack.org/cgit/openstack/os-xenapi/commit/?id=6953134a1348e6cfce3166262ed5f4768ef37cf9
Submitter: Zuul
Branch: master

commit 6953134a1348e6cfce3166262ed5f4768ef37cf9
Author: Jianghua Wang <email address hidden>
Date: Sun Apr 8 09:21:55 2018 +0000

    Avoid using the module of future in xapi plugin

    In XenServer's dom0 (e.g. XS7.0/7.1), the module of future is
    not available. So this commit is to avoid using *future*. Instead
    it import check_output from different modules basing if *future*
    module is available of not.

    Change-Id: Ib75b2b7965e63033ea933f7cb4a4a54a2f48ef75
    Partial-Bug: #1762130