novaclient list servers by update attribute throws TypeError instead of CommandError

[Theodoros Tsioutsias]

Description
===========

nova list command fails with TypeError instead of CommandError when an existing but not valid attribute of the object is given as field.

$ /usr/bin/nova list --all --status ERROR --fields update
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()

Steps to reproduce
==================

At least one server has to exist so that the list is not empty.
In this case update was given as a field.
The Server object has an update method so the (not hasattr) check is False:

python-novaclient/novaclient/v2/shell.py:

1634 def _get_list_table_columns_and_formatters(fields, objs, exclude_fields=(),
1635 filters=None):
[...]
1680 for field in fields.split(','):
1681 if not hasattr(obj, field):
1682 non_existent_fields.append(field)
1683 continue
1684 if field in exclude_fields:
1685 continue
1686 field_title, formatter = utils.make_field_formatter(field,
1687 filters)
1688 columns.append(field_title)
1689 formatters[field_title] = formatter
1690 exclude_fields.add(field)

As a result of this check, all of the attributes of the server object can be used as fields.
Most of them cause a TypeError to be raised.

e.g:
[devstack-006 ~]$ /usr/bin/nova list --fields __dict__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __getattr__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __getattribute__
ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __hash__
ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __init__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 v2]$ /usr/bin/nova list --field unshelve
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields to_dict
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields revert_resize
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields _add_details
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[...]

Expected result
===============

A CommandError should be raised.

Actual result
=============

A TypeError is raised.

Environment
===========

$ rpm -qf /usr/bin/nova
python2-novaclient-9.1.1-1.el7.noarch

Reproduced in devstack:

[devstack-006 ~]$ /usr/bin/nova list --field update
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 python-novaclient]$ git log -1
commit c9e7a64ca83302bdeab2044c09f9063646cc59a3
Merge: dd520c7 bef6765
Author: Zuul <email address hidden>
Date: Tue Nov 21 19:51:32 2017 +0000

    Merge "Microversion 2.54 - Enable reset keypair while rebuild"

Logs & Configs
==============

Attaching the debug output of the command.

Comments
========

Attributes like __module__ or __class__, are not raising a TypeError. Should they be allowed?

[devstack-006 ~]$ /usr/bin/nova list --field __class__
+--------------------------------------+----------------------------------------+
| ID | Class |
+--------------------------------------+----------------------------------------+
| 353e6118-919e-4684-9e73-2441bbd8f0bd | <class 'novaclient.v2.servers.Server'> |
| b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | <class 'novaclient.v2.servers.Server'> |
| d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | <class 'novaclient.v2.servers.Server'> |
+--------------------------------------+----------------------------------------+
[devstack-006 ~]$ /usr/bin/nova list --field __module__
+--------------------------------------+-----------------------+
| ID | Module |
+--------------------------------------+-----------------------+
| 353e6118-919e-4684-9e73-2441bbd8f0bd | novaclient.v2.servers |
| b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | novaclient.v2.servers |
| d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | novaclient.v2.servers |
+--------------------------------------+-----------------------+

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/525110

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/525110
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=198c0d7327ca41167473f8aeb4c7bbdb4910cb5a
Submitter: Zuul
Branch: master

commit 198c0d7327ca41167473f8aeb4c7bbdb4910cb5a
Author: Theodoros Tsioutsias <email address hidden>
Date: Mon Dec 4 08:56:26 2017 +0000

    CommandError is raised for invalid server fields

    When listing servers with fields of invalid type, a TypeError was
    raised. With this change and in order to ensure the validity of the
    fields, we are whitelisting them using the keys of the dictionary
    Resource.to_dict(). For all fields not in the whitelist a CommandError
    is raised.

    Change-Id: I647fa611d29745f830daadac1c3f9c1c71c2733a
    Closes-Bug: #1733917

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-novaclient 10.0.0 release.