`openstack token issue` cannot print `expires` value when using JSON formatter

[Brian Hall]

Running the python-openstackclient version 3.2.0 on Ubuntu 16.04, I see the following failure:
$ openstack token issue --format json
{
  "project_id": "57b3240733b94ad7b99f52e4df8b1d65",
datetime.datetime(2016, 9, 4, 15, 3, 1, tzinfo=<iso8601.Utc>) is not JSON serializable
  "expires":

Version 2.6.0 of the python-openstackclient works properly in that environment:
$ openstack token issue --format json
{
  "project_id": "57b3240733b94ad7b99f52e4df8b1d65",
  "expires": "2016-09-04T15:09:39Z",
  "user_id": "535f4fbc9bea414fb07547844ed42337",
  "id": "45592ea61eab4b999562de95d9bdd433"

[Steve Martinelli (stevemar)]

Can you add --debug to the command?

I am unable to reproduce this:

$ openstack --version
openstack 3.2.0

$ env | grep OS
OS_PROJECT_NAME=admin
OS_IDENTITY_API_VERSION=2.0
OS_PASSWORD=openstack
OS_AUTH_URL=http://172.16.240.200:5000/v2.0
OS_USERNAME=admin
OS_TENANT_NAME=admin
OS_VOLUME_API_VERSION=2

$ openstack token issue
+------------+----------------------------------+
| Field | Value |
+------------+----------------------------------+
| expires | 2016-09-05 04:20:52+00:00 |
| id | a6366fb21b7d4fdea5f6031d3ee7f7f5 |
| project_id | 49f68521a7934de39d1cac59ff69f5b2 |
| user_id | aa11cee173d54672a5950a4b2c7bb0f4 |
+------------+----------------------------------+

[Brian Hall (n-brian-b)]

The problem only occurs when the '--format json' option is used.

$ openstack --version
openstack 3.2.0

$ env | grep OS
OS_PROJECT_NAME=client
OS_PASSWORD=XXXXXXXX
OS_AUTH_URL=http://X.XX.XX.XXX:5000
OS_USERNAME=brian

$ openstack token issue --format json --debug
START with options: [u'token', u'issue', u'--format', u'json', u'--debug']
options: Namespace(access_token='***', access_token_endpoint='', access_token_type='', auth_type='', auth_url='http://X.XX.XX.XXX:5000', authorization_code='', cacert=None, cert='', client_id='', client_secret='***', cloud='', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, openid_scope='', os_beta_command=False, os_compute_api_version='', os_identity_api_version='', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_volume_api_version='', passcode='', password='***', project_domain_id='', project_domain_name='', project_id='', project_name='client', protocol='', redirect_uri='', region_name='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='', user_id='', username='brian', verbose_level=3, verify=None)
Auth plugin password selected
auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', u'metering_api_version': u'2', 'auth_url': 'http://X.XX.XX.XXX:5000', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': u'2', 'verify': True, u'dns_api_version': u'2', u'object_store_api_version': u'1', u'status': u'active', 'verbose_level': 3, 'region_name': '', 'api_timeout': None, u'baremetal_api_version': u'1', 'auth': {'project_name': 'client'}, 'default_domain': 'default', 'debug': True, u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'timing': False, 'password': 'temp4now', 'cacert': None, u'key_manager_api_version': u'v1', 'deferred_help': False, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'username': 'brian', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'key_manager_api_version': u'v1', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}}
cloud cfg: ...

[Steve Martinelli (stevemar)]

Thanks for the clarification Brian!

Looks like it's an issue when writing passing datetime objets to the json formatter. We could check this for individual commands (like `openstack issue token`), but there are definitely others (like uptime in servers, or expires in oauth tokens). I'm wondering if theres a slicker way to handle this in cliff, where if the value is a datetime object, then automatically format it.

Either way, here's how I fixed it in openstackclient for the token issue commands. I'll post a patch soon.

+++ b/openstackclient/identity/v2_0/token.py
@@ -15,6 +15,8 @@

 """Identity v2 Token action implementations"""

+import datetime
+
 from osc_lib.command import command
 from osc_lib import exceptions
 import six
@@ -42,7 +44,9 @@ class IssueToken(command.ShowOne):
         if auth_ref.auth_token:
             data['id'] = auth_ref.auth_token
         if auth_ref.expires:
- data['expires'] = auth_ref.expires
+ datetime_obj = auth_ref.expires
+ expires_str = datetime_obj.strftime('%Y-%m-%d %H:%M:%S%z')
+ data['expires'] = expires_str
         if auth_ref.project_id:
             data['project_id'] = auth_ref.project_id
         if auth_ref.user_id:

(test_osc) stevemar@ubuntu:/opt/stack/python-openstackclient$ openstack token issue --format json
{
  "project_id": "257ba183e6434d81ae387fdb4d44a8df",
  "expires": "2016-09-05 18:04:41+0000",
  "user_id": "da1c702674d84d78a21fd6b86b3904c9",
  "id": "6176758a59f546b0868c7b905680a3bb"
}(test_osc) stevemar@ubuntu:/opt/stack/python-openstackclient$ openstack token issue
+------------+----------------------------------+
| Field | Value |
+------------+----------------------------------+
| expires | 2016-09-05 18:04:52+0000 |
| id | f1abddd1f4c04b06b31e3eccd0455e76 |
| project_id | 257ba183e6434d81ae387fdb4d44a8df |
| user_id | da1c702674d84d78a21fd6b86b3904c9 |
+------------+----------------------------------+

[Steve Martinelli (stevemar)]

Patch is here: https://review.openstack.org/#/c/365758/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/365758
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=24c8b94baf4c95cd3a76159009badbc53573be09
Submitter: Jenkins
Branch: master

commit 24c8b94baf4c95cd3a76159009badbc53573be09
Author: Steve Martinelli <email address hidden>
Date: Mon Sep 5 10:15:57 2016 -0700

    format token expires time to prevent json loading datetime data

    When output to shell, the token issue time is fine; however when
    selecting the json formatter (via the --format json) option, an
    exception is raised when formatting the dataetime data.

    Rather than pass in the datetime data, we should format the data
    with the ISO 8601 formatting.

    Closes-Bug: 1619937
    Change-Id: Iffebb2d5413fabfd283dfa94fc560fc37270f9dd

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-openstackclient 3.3.0 release.