| 2016-11-10 14:44:34 |
Julie Pichon |
description |
I'm using the UI to configure a deployment. I selected a bunch of environments (4 in this specific case: network isolation v6, storage environment, storage environment in order to deploy Ceph, enable tls, tls endpoints public IPs). I have 1 Controller, 1 Compute, 1 Ceph Node assigned.
From them on it because impossible to use the UI to configure any parameters:
- If I click on Parameters in "Edit Configuration", I get the message "There are currently no parameters to configure in this section"
From the browser console:
action @ 13:48:38.418 FETCH_PARAMETERS_PENDING
reqwest.js:240 POST http://192.0.2.1:8989/v2/action_executions 500 (Internal Server Error)
logger.js:66 action @ 13:48:49.833 FETCH_PARAMETERS_FAILED
logger.js:58 Error in ParametersActions.fetchParameters
XMLHttpRequest {readyState: 4, timeout: 0, withCredentials: false, upload: XMLHttpRequestUpload, responseURL: "http://192.0.2.1:8989/v2/action_executions"…}
- If I click on any edit "Pencil" button for any card, all 3 tabs - Parameters, Services, Network Configuration - shows the same "no parameters" error message.
- If I click on "Edit Configuration" and wait for the "Overall settings to load", the window closes on its own with the message "{"debuginfo": null, "faultcode": "Server", "faultstring": "__init__() takes exactly 4 arguments (2 given)"}"
This error appears to be coming from Mistral directly.
Looking at the api.log:
2016-11-10 13:51:57.757 11167 INFO mistral.api.controllers.v2.action_execution [-] Create action_execution [action_execution=ActionExecution [id='Unset', workflow_name='Unset', task_name='Unset', task_execution_id='Unset', state='Unset', state_info='Unset', tags='Unset', name='tripleo.heat_capabilities.get', description='Unset', accepted='Unset', input='{'container': 'test'}', output='Unset', created_at='Unset', updated_at='Unset', params='{'save_result': True, 'run_sync': True}']]
2016-11-10 13:52:03.057 11167 INFO mistral.api.controllers.v2.environment [-] Fetch environment [name=test]
2016-11-10 13:52:07.666 11167 ERROR wsme.api [-] Server-side error: "__init__() takes exactly 4 arguments (2 given)". Detail:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/wsmeext/pecan.py", line 84, in callfunction
result = f(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/mistral/api/controllers/v2/action_execution.py", line 141, in post
**params
File "/usr/lib/python2.7/site-packages/mistral/engine/rpc_backend/rpc.py", line 317, in decorator
raise exc_cls(e.value)
TypeError: __init__() takes exactly 4 arguments (2 given)
That looks like it may be a bug on the Mistral side?
The engine logs show the following:
2016-11-10 13:52:07.657 11554 ERROR mistral.db.v2.sqlalchemy.models [-] Size limit 1024KB exceed for class [<class 'mistral.db.v2.sqlalchemy.models.TaskExecution'>], field output of size 1148KB.
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server [-] Exception during message handling
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 225, in dispatch
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 195, in _do_dispatch
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/rpc_backend/rpc.py", line 177, in start_action
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server **params
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/action_queue.py", line 70, in decorate
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server res = func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/utils/__init__.py", line 123, in _logged
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/default_engine.py", line 110, in start_action
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return db_api.create_action_execution(values)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/api.py", line 224, in create_action_execution
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return IMPL.create_action_execution(values)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/sqlalchemy/base.py", line 118, in _within_session
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server result = func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/api.py", line 662, in create_action_execution
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server a_ex.update(values.copy())
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 94, in update
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server setattr(self, k, v)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 224, in __set__
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server instance_dict(instance), value, None)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 701, in set
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server value, old, initiator)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 710, in fire_replace_event
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server self._init_append_or_replace_token())
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/events.py", line 1606, in wrap
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server fn(target, value, *arg)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/models.py", line 92, in <lambda>
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server lambda t, v, o, i: validate_long_type_length(cls, attr_name, v)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/models.py", line 77, in validate_long_type_length
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server size_limit_kb
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server SizeLimitExceededException: Size of 'output' is 1148KB which exceeds the limit of 1024KB
It looks like we may need to adjust the Mistral configuration on the undercloud.
Additional information:
$ mistral environment-get test
+-------------+-----------------------------------------------------------------------------------+
| Field | Value |
+-------------+-----------------------------------------------------------------------------------+
| Name | test |
| Description | <none> |
| Variables | { |
| | "passwords": { |
| | "HAProxyStatsPassword": "eRH2Te3TRNHhqc3vhVX6Jjbq7", |
| | "CephMonKey": "AQCJ0BlYAAAAABAAnjgsMS0yUThTokp644rxfg==", |
| | "NeutronPassword": "Jaqwtm6Wn32rCvbdR3XchhNPU", |
| | "HeatPassword": "Cn9WHhYZt4xGAzGVm3ttAEgCw", |
| | "SnmpdReadonlyUserPassword": "5a81d2d83ee4b69b33587249abf49cd672d08541", |
| | "GlancePassword": "QbaKFyataq9g8wQuwCaTbQw6g", |
| | "IronicPassword": "YNhmNMYZ3d3HCGQBCzEZcT6Fv", |
| | "HeatStackDomainAdminPassword": "UnF3dXY3rHJrfBk8Yudmq2bXE", |
| | "ZaqarPassword": "zUxDHXD9YtWg6btHUWnuAA63s", |
| | "MysqlClustercheckPassword": "6qzKuefZfcZrthBEbPvNzj9RU", |
| | "GnocchiPassword": "FXBQe3uErWZJeBQyuyvBDcwyv", |
| | "CephAdminKey": "AQCJ0BlYAAAAABAAzV243FBkxMCnC3FdmJp9OA==", |
| | "CeilometerPassword": "K49rJbCQTU7vJxHFbsgZb6BNy", |
| | "CinderPassword": "249BAzhaNaKMFDdKhjATDp98m", |
| | "RabbitPassword": "Whhc9QtuFbCKTmHTkYXzeBHv8", |
| | "CephRgwKey": "AQCJ0BlYAAAAABAAgkwZJoTLuprZxIRBqqHiRA==", |
| | "TrovePassword": "rzefeQV6qGZc7jUnEa4J4pPP3", |
| | "KeystoneCredential0": "jmqQdybmudfLd-gftwF9dDQsG5XtTTm4hXqH4dzEqno=", |
| | "KeystoneCredential1": "_ulAQEXIqyK7CEdFYWUc11JNAgWHco3MK58G8VK6f_s=", |
| | "CephClientKey": "AQCJ0BlYAAAAABAA4bxmTBjjBvgoZP6dHrKXjg==", |
| | "NovaPassword": "WCCKzWeP7RsHte7uzRpuJGnJf", |
| | "AdminToken": "xNWwyzTNBCfBNEyBkGgc2Pp3H", |
| | "RedisPassword": "TZq9xQJFVt9TZ7dVXfmGuakE7", |
| | "MistralPassword": "ChthnfF6hgWeddC3EaA23BURT", |
| | "SwiftHashSuffix": "jTQ9jqkQsJgxGbtRXGVPvFqmM", |
| | "AdminPassword": "BPWshyAkWNbWDA7HVhm9H3E8z", |
| | "AodhPassword": "QrQ48ebuqfd2ezWUZJ4xgvsPw", |
| | "ManilaPassword": "amtXbYqWDAn8pWtWAarVpWRaV", |
| | "SwiftPassword": "qavgpKwzVs2BJfh44WmzN8Jbr", |
| | "CeilometerMeteringSecret": "HyH3C9hgbXvG6DgrCPTaUmXG3", |
| | "NeutronMetadataProxySharedSecret": "nW6KVpRwzFApGr22pamVuqaUw", |
| | "BarbicanPassword": "CWhus8tB66n4yRY69VCNRfXsf", |
| | "SaharaPassword": "aduhp3qErpb4eeURkNcMt4MAn" |
| | }, |
| | "parameter_defaults": { |
| | "CloudDomain": "", |
| | "CephStorageCount": "1" |
| | }, |
| | "template": "overcloud.yaml", |
| | "environments": [ |
| | { |
| | "path": "overcloud-resource-registry-puppet.yaml" |
| | }, |
| | { |
| | "path": "environments/network-isolation-v6.yaml" |
| | }, |
| | { |
| | "path": "environments/storage-environment.yaml" |
| | }, |
| | { |
| | "path": "environments/enable-tls.yaml" |
| | }, |
| | { |
| | "path": "environments/tls-endpoints-public-ip.yaml" |
| | } |
| | ] |
| | } |
| Scope | private |
| Created at | 2016-11-02 11:39:53 |
| Updated at | 2016-11-10 13:43:13 |
This reproduced on both a Newton and Ocata environment. |
I'm using the UI to configure a deployment.
Steps to reproduce:
-------------------
1. Select a bunch of environments. I tried with 4 in this case: network isolation v6, storage environment, storage environment in order to deploy Ceph, enable tls, tls endpoints public IPs. I have 1 Controller, 1 Compute, 1 Ceph Node assigned.
Problem description:
--------------------
After saving the new environments, it becomes impossible to use the UI to configure any parameters:
- If I click on Parameters in "Edit Configuration", I get the message "There are currently no parameters to configure in this section"
From the browser console:
action @ 13:48:38.418 FETCH_PARAMETERS_PENDING
reqwest.js:240 POST http://192.0.2.1:8989/v2/action_executions 500 (Internal Server Error)
logger.js:66 action @ 13:48:49.833 FETCH_PARAMETERS_FAILED
logger.js:58 Error in ParametersActions.fetchParameters
XMLHttpRequest {readyState: 4, timeout: 0, withCredentials: false, upload: XMLHttpRequestUpload, responseURL: "http://192.0.2.1:8989/v2/action_executions"…}
- If I click on any edit "Pencil" button for any card, all 3 tabs - Parameters, Services, Network Configuration - shows the same "no parameters" error message.
- If I click on "Edit Configuration" and wait for the "Overall settings to load", the window closes on its own with the message "{"debuginfo": null, "faultcode": "Server", "faultstring": "__init__() takes exactly 4 arguments (2 given)"}"
Mistral logs
------------
This error appears to be coming from Mistral directly.
Looking at the api.log:
2016-11-10 13:51:57.757 11167 INFO mistral.api.controllers.v2.action_execution [-] Create action_execution [action_execution=ActionExecution [id='Unset', workflow_name='Unset', task_name='Unset', task_execution_id='Unset', state='Unset', state_info='Unset', tags='Unset', name='tripleo.heat_capabilities.get', description='Unset', accepted='Unset', input='{'container': 'test'}', output='Unset', created_at='Unset', updated_at='Unset', params='{'save_result': True, 'run_sync': True}']]
2016-11-10 13:52:03.057 11167 INFO mistral.api.controllers.v2.environment [-] Fetch environment [name=test]
2016-11-10 13:52:07.666 11167 ERROR wsme.api [-] Server-side error: "__init__() takes exactly 4 arguments (2 given)". Detail:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/wsmeext/pecan.py", line 84, in callfunction
result = f(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/mistral/api/controllers/v2/action_execution.py", line 141, in post
**params
File "/usr/lib/python2.7/site-packages/mistral/engine/rpc_backend/rpc.py", line 317, in decorator
raise exc_cls(e.value)
TypeError: __init__() takes exactly 4 arguments (2 given)
That one looks like a bug on the Mistral side?
The engine logs show the following:
2016-11-10 13:52:07.657 11554 ERROR mistral.db.v2.sqlalchemy.models [-] Size limit 1024KB exceed for class [<class 'mistral.db.v2.sqlalchemy.models.TaskExecution'>], field output of size 1148KB.
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server [-] Exception during message handling
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 225, in dispatch
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 195, in _do_dispatch
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/rpc_backend/rpc.py", line 177, in start_action
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server **params
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/action_queue.py", line 70, in decorate
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server res = func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/utils/__init__.py", line 123, in _logged
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/engine/default_engine.py", line 110, in start_action
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return db_api.create_action_execution(values)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/api.py", line 224, in create_action_execution
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server return IMPL.create_action_execution(values)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/sqlalchemy/base.py", line 118, in _within_session
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server result = func(*args, **kw)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/api.py", line 662, in create_action_execution
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server a_ex.update(values.copy())
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/models.py", line 94, in update
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server setattr(self, k, v)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 224, in __set__
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server instance_dict(instance), value, None)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 701, in set
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server value, old, initiator)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 710, in fire_replace_event
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server self._init_append_or_replace_token())
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/events.py", line 1606, in wrap
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server fn(target, value, *arg)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/models.py", line 92, in <lambda>
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server lambda t, v, o, i: validate_long_type_length(cls, attr_name, v)
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/mistral/db/v2/sqlalchemy/models.py", line 77, in validate_long_type_length
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server size_limit_kb
2016-11-10 13:52:07.659 11554 ERROR oslo_messaging.rpc.server SizeLimitExceededException: Size of 'output' is 1148KB which exceeds the limit of 1024KB
It looks like we may need to adjust the Mistral configuration on the undercloud.
Additional information:
------------------------
$ mistral environment-get test
+-------------+-----------------------------------------------------------------------------------+
| Field | Value |
+-------------+-----------------------------------------------------------------------------------+
| Name | test |
| Description | <none> |
| Variables | { |
| | "passwords": { |
(snip)
|
| | "parameter_defaults": { |
| | "CloudDomain": "", |
| | "CephStorageCount": "1" |
| | }, |
| | "template": "overcloud.yaml", |
| | "environments": [ |
| | { |
| | "path": "overcloud-resource-registry-puppet.yaml" |
| | }, |
| | { |
| | "path": "environments/network-isolation-v6.yaml" |
| | }, |
| | { |
| | "path": "environments/storage-environment.yaml" |
| | }, |
| | { |
| | "path": "environments/enable-tls.yaml" |
| | }, |
| | { |
| | "path": "environments/tls-endpoints-public-ip.yaml" |
| | } |
| | ] |
| | } |
| Scope | private |
| Created at | 2016-11-02 11:39:53 |
| Updated at | 2016-11-10 13:43:13 |
This reproduced on both a Newton and Ocata environment. |
|
