tripleo

python-tripleoclient cannot handle multiple lines when conditionals in the upgrade tasks

Bug #1715871 reported by Marius Cornea on 2017-09-08

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	High	Emilien Macchi	tripleo queens-1

Bug Description

With the following patch applied:

https://review.openstack.org/#/c/501734/5/docker/services/pacemaker/haproxy.yaml

note lines 245-247

openstack --debug overcloud config download --config-dir compute-0/tripleo-JdgDtQ-config/
fails with the following error:

expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
clean_up DownloadConfig: expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

Adding a debug line to provide more info we can see that the when conditional containing multiple lines is not handled well by tripleoclient:

whenline=rpm_python_check.changed != false type=<type 'unicode'>
whenline=rpm_python_check.changed != false type=<type 'unicode'>
whenline=httpd_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_api_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_scheduler_enabled.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_server_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_dhcp_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_l3_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_metadata_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_ovs_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_apache.rc == 0 type=<type 'unicode'>
whenline=cinder_api_enabled.rc == 0 type=<type 'unicode'>
whenline=cinder_scheduler_enabled.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_apache.rc == 0 type=<type 'unicode'>
whenline=heat_api_cloudwatch_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_server_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_dhcp_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_l3_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_metadata_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=neutron_ovs_agent_enabled.rc == 0 type=<type 'unicode'>
whenline=is_bootstrap_node type=<type 'unicode'>
whenline=[u'is_bootstrap_node', u'haproxy_res is defined and haproxy_res|succeeded'] type=<type 'list'>
expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
clean_up DownloadConfig: expected string or buffer
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 134, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 279, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 169, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 400, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 137, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 129, in take_action
    role[config], role_name, filepath)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 83, in _write_playbook_get_tasks
    self._step_tags_to_when(sorted_tasks)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_config.py", line 72, in _step_tags_to_when
    when_exists = re.search('step.* == [0-9]', whenline)
  File "/usr/lib64/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

END return value: 1

Tags:

Revision history for this message

Steven Hardy (shardy) wrote on 2017-09-08:

I took a look at this, there's actually a comment about it in the code :(

I'm working on a patch

Changed in tripleo:
assignee:	nobody → Steven Hardy (shardy)
tags:	added: pike-backport-potential tripleoclient

Steven Hardy (shardy) on 2017-09-08

Changed in tripleo:
milestone:	none → queens-1

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-09-08: Fix proposed to python-tripleoclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/502076

Changed in tripleo:
status:	Confirmed → In Progress

OpenStack Infra (hudson-openstack) on 2017-09-09

Changed in tripleo:
assignee:	Steven Hardy (shardy) → Emilien Macchi (emilienm)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-09-10: Fix merged to python-tripleoclient (master)

Reviewed: https://review.openstack.org/502076
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95
Submitter: Jenkins
Branch: master

commit fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95
Author: Steven Hardy <email address hidden>
Date: Fri Sep 8 15:46:13 2017 +0100

Handle when lists not only string in config download

There was a comment about this in the code, we didn't handle
when lists properly, only strings.

Also fixed the test to avoid leaking temporary files every tox run.

Change-Id: I24f80199ba430870af2b4aab9453387304403215
Closes-Bug: #1715871

Changed in tripleo:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-09-10: Fix proposed to python-tripleoclient (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/502291

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2017-09-12: Fix merged to python-tripleoclient (stable/pike)

Reviewed: https://review.openstack.org/502291
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=e786a130fa699f0137bad186942bab47ed220786
Submitter: Jenkins
Branch: stable/pike

commit e786a130fa699f0137bad186942bab47ed220786
Author: Steven Hardy <email address hidden>
Date: Fri Sep 8 15:46:13 2017 +0100

Handle when lists not only string in config download

There was a comment about this in the code, we didn't handle
when lists properly, only strings.

Also fixed the test to avoid leaking temporary files every tox run.

    Change-Id: I24f80199ba430870af2b4aab9453387304403215
    Closes-Bug: #1715871
    (cherry picked from commit fa4b8fb00ce199fa767e0b0b0349e6fc3eaf4b95)