member-create will raise 500 error if member-id is greater than 255 characters

[Abhishek Kekane]

While adding members to image (image sharing) if member-id is specified above 255 characters then it will raise 500 Internal server error.

Steps to reproduce:

:~$ glance member-create 3174767b-7ccc-4691-a3ca-6884d04c0330 2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
500 Internal Server Error
The server has either erred or is incapable of performing the requested operation.
    (HTTP 500)

glance-api logs:

2016-05-26 12:23:43.186 DEBUG oslo_policy.policy [req-ca9b4266-b1a9-406a-9b77-6905ce3bf658 9448716ab50145dcbdb7d5653ecdc2dc 9f8383968a0a4d0bb303500cd5c49f51] Reloaded pol
icy file: /etc/glance/policy.json from (pid=26413) _load_policy_file /usr/local/lib/python2.7/dist-packages/oslo_policy/policy.py:520
2016-05-26 12:23:43.201 ERROR oslo_db.sqlalchemy.exc_filters [req-ca9b4266-b1a9-406a-9b77-6905ce3bf658 9448716ab50145dcbdb7d5653ecdc2dc 9f8383968a0a4d0bb303500cd5c49f51]
DBAPIError exception wrapped from (pymysql.err.DataError) (1406, u"Data too long for column 'member' at row 1") [SQL: u'INSERT INTO image_members (created_at, updated_at,
deleted_at, deleted, image_id, member, can_share, status) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(image_id)s, %(member)s, %(can_share)s, %
(status)s)'] [parameters: {'status': 'pending', 'deleted': 0, 'created_at': datetime.datetime(2016, 5, 26, 6, 53, 43, 201107), 'updated_at': datetime.datetime(2016, 5, 26
, 6, 53, 43, 201144), 'member': u'2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd', '
image_id': u'3174767b-7ccc-4691-a3ca-6884d04c0330', 'can_share': 0, 'deleted_at': None}]
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters context)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 161, in execute
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters result = self._query(query)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 317, in _query
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 835, in query
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1019, in _read_query_result
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters result.read()
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1302, in read
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet()
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 981, in _read_packet
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters packet.check_error()
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters err.raise_mysql_exception(self._data)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters _check_mysql_exception(errinfo)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters raise errorclass(errno, errorvalue)
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters DataError: (1406, u"Data too long for column 'member' at row 1")
2016-05-26 12:23:43.201 TRACE oslo_db.sqlalchemy.exc_filters
2016-05-26 12:23:43.234 ERROR glance.common.wsgi [req-ca9b4266-b1a9-406a-9b77-6905ce3bf658 9448716ab50145dcbdb7d5653ecdc2dc 9f8383968a0a4d0bb303500cd5c49f51] Caught error: (pymysql.err.DataError) (1406, u"Data too long for column 'member' at row 1") [SQL: u'INSERT INTO image_members (created_at, updated_at, deleted_at, deleted, image_id, member, can_share, status) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(image_id)s, %(member)s, %(can_share)s, %(status)s)'] [parameters: {'status': 'pending', 'deleted': 0, 'created_at': datetime.datetime(2016, 5, 26, 6, 53, 43, 201107), 'updated_at': datetime.datetime(2016, 5, 26, 6, 53, 43, 201144), 'member': u'2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd', 'image_id': u'3174767b-7ccc-4691-a3ca-6884d04c0330', 'can_share': 0, 'deleted_at': None}]
2016-05-26 12:23:43.234 TRACE glance.common.wsgi Traceback (most recent call last):
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 918, in __call__
2016-05-26 12:23:43.234 TRACE glance.common.wsgi request, **action_args)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/common/wsgi.py", line 957, in dispatch
2016-05-26 12:23:43.234 TRACE glance.common.wsgi return method(*args, **kwargs)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/common/utils.py", line 362, in wrapped
2016-05-26 12:23:43.234 TRACE glance.common.wsgi return func(self, req, *args, **kwargs)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/api/v2/image_members.py", line 116, in create
2016-05-26 12:23:43.234 TRACE glance.common.wsgi member_repo.add(new_member)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/api/authorization.py", line 166, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi self.member_repo.add(image_member)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/notifier.py", line 520, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi super(ImageMemberRepoProxy, self).add(member)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 121, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi self.base.add(self.member_proxy_helper.unproxy(member))
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/api/policy.py", line 241, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi self.member_repo.add(member)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/location.py", line 494, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi super(ImageMemberRepoProxy, self).add(member)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/domain/proxy.py", line 94, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi result = self.base.add(base_item)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/db/__init__.py", line 367, in add
2016-05-26 12:23:43.234 TRACE glance.common.wsgi image_member_values)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 1066, in image_member_create
2016-05-26 12:23:43.234 TRACE glance.common.wsgi _image_member_update(context, memb_ref, values, session=session)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 1098, in _image_member_update
2016-05-26 12:23:43.234 TRACE glance.common.wsgi memb_ref.save(session=session)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/opt/stack/glance/glance/db/sqlalchemy/models.py", line 72, in save
2016-05-26 12:23:43.234 TRACE glance.common.wsgi super(GlanceBase, self).save(session or db_api.get_session())
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/models.py", line 48, in save
2016-05-26 12:23:43.234 TRACE glance.common.wsgi session.flush()
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
2016-05-26 12:23:43.234 TRACE glance.common.wsgi self._flush(objects)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush
2016-05-26 12:23:43.234 TRACE glance.common.wsgi transaction.rollback(_capture_exception=True)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2016-05-26 12:23:43.234 TRACE glance.common.wsgi compat.reraise(exc_type, exc_value, exc_tb)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush
2016-05-26 12:23:43.234 TRACE glance.common.wsgi flush_context.execute()
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
2016-05-26 12:23:43.234 TRACE glance.common.wsgi rec.execute(self)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
2016-05-26 12:23:43.234 TRACE glance.common.wsgi uow
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 174, in save_obj
2016-05-26 12:23:43.234 TRACE glance.common.wsgi mapper, table, insert)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 800, in _emit_insert_statements
2016-05-26 12:23:43.234 TRACE glance.common.wsgi execute(statement, params)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-05-26 12:23:43.234 TRACE glance.common.wsgi return meth(self, multiparams, params)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2016-05-26 12:23:43.234 TRACE glance.common.wsgi return connection._execute_clauseelement(self, multiparams, params)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2016-05-26 12:23:43.234 TRACE glance.common.wsgi compiled_sql, distilled_params
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2016-05-26 12:23:43.234 TRACE glance.common.wsgi context)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
2016-05-26 12:23:43.234 TRACE glance.common.wsgi util.raise_from_cause(newraise, exc_info)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
2016-05-26 12:23:43.234 TRACE glance.common.wsgi reraise(type(exception), exception, tb=exc_tb, cause=cause)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-05-26 12:23:43.234 TRACE glance.common.wsgi context)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-05-26 12:23:43.234 TRACE glance.common.wsgi cursor.execute(statement, parameters)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 161, in execute
2016-05-26 12:23:43.234 TRACE glance.common.wsgi result = self._query(query)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 317, in _query
2016-05-26 12:23:43.234 TRACE glance.common.wsgi conn.query(q)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 835, in query
2016-05-26 12:23:43.234 TRACE glance.common.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1019, in _read_query_result
2016-05-26 12:23:43.234 TRACE glance.common.wsgi result.read()
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1302, in read
2016-05-26 12:23:43.234 TRACE glance.common.wsgi first_packet = self.connection._read_packet()
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 981, in _read_packet
2016-05-26 12:23:43.234 TRACE glance.common.wsgi packet.check_error()
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
2016-05-26 12:23:43.234 TRACE glance.common.wsgi err.raise_mysql_exception(self._data)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2016-05-26 12:23:43.234 TRACE glance.common.wsgi _check_mysql_exception(errinfo)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2016-05-26 12:23:43.234 TRACE glance.common.wsgi raise errorclass(errno, errorvalue)
2016-05-26 12:23:43.234 TRACE glance.common.wsgi DBError: (pymysql.err.DataError) (1406, u"Data too long for column 'member' at row 1") [SQL: u'INSERT INTO image_members (created_at, updated_at, deleted_at, deleted, image_id, member, can_share, status) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(image_id)s, %(member)s, %(can_share)s, %(status)s)'] [parameters: {'status': 'pending', 'deleted': 0, 'created_at': datetime.datetime(2016, 5, 26, 6, 53, 43, 201107), 'updated_at': datetime.datetime(2016, 5, 26, 6, 53, 43, 201144), 'member': u'2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd', 'image_id': u'3174767b-7ccc-4691-a3ca-6884d04c0330', 'can_share': 0, 'deleted_at': None}]

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi all,

There is also an issue with APIs listed below are returning 500 error if you pass "name" more than
255 characters.
1. metadef tag create
2. metadef property create
3. metadef object create

Steps to reproduce:

1. $ glance md-object-create OS::Compute::HostCapabilities --name 2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd --schema '{}'

$ glance md-tag-create OS::Compute::HostCapabilities --name 2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd

$ glance md-property-create OS::Compute::HostCapabilities --name nvn --title abcd --schema '{"type": "string", "title": "vbfh", "name":"2222222222222222222234244444444424444444444gdegddddddddddfgdffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffdddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"}'

Output: 500 Internal Server Error The server has either erred or is incapable of performing the requested operation. (HTTP 500)

[Abhishek Kekane (abhishek-kekane)]

As of now while creating a member for image "member" is expected in request body where as "member_id" is returned in response body. This behavior is causing problem to add schema validation for member because in schema "member_id" is defined. If we change this to "member" then we are able to perform the validation but it is eliminating "member_id" from the response as it is not present in the schema. In order to solve this issue we are having two solutions:

Solution 1: Add "member" attribute in schema

New schema will look like:
_MEMBER_SCHEMA = {
    'member_id': {
        'type': 'string',
        'description': _('An identifier for the image member (tenantId)')
    },
    'member': {
        'type': 'string',
        'maxLength': 255,
        'description': _('An identifier for the image member (tenantId)')
    },
    'image_id': {
        'type': 'string',
        'description': _('An identifier for the image'),
        'pattern': ('^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}'
                    '-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$'),
    },
    'created_at': {
        'type': 'string',
        'description': _('Date and time of image member creation'),
        # TODO(brian-rosmaita): our jsonschema library doesn't seem to like the
        # format attribute, figure out why (and also fix in images.py)
        # 'format': 'date-time',
    },
    'updated_at': {
        'type': 'string',
        'description': _('Date and time of last modification of image member'),
        # 'format': 'date-time',
    },
    'status': {
        'type': 'string',
        'description': _('The status of this image member'),
        'enum': [
            'pending',
            'accepted',
            'rejected'
        ]
    },
    'schema': {
        'readOnly': True,
        'type': 'string'
    }
}

pros:
1. Present behavior of api will be maintianed
2. Less changes in code

cons:
1. Two attributes for member in schema

Solution 2: Replace "member_id" attribute with "member" in schema

pros:
1. No need to add duplicate eliments in schema

cons:
1. Will change api behavior as "member" instead of "member_id" will be returned in response body
2. Requires lot of code chagne in glance and also needs changes in python-glanceclient

Please give us your opinion about the same.

NOTE:
As of now for member creation schema is not used for validation purpose.

[Brian Rosmaita (brian-rosmaita)]

Hopefully this suggestion isn't completely insane. What if we update the schema to accommodate *all* the member related calls, that is, member-create, member-show, and member-update? I think the following will work. I tested it out quickly in the online json-validator, and it does seem to handle the current request bodies and request response.

http://jsonschemalint.com/draft4/#

Here's the schema:

{
    "id": "http://openstack.org/glance/member/schema#",
    "$schema": "http://json-schema.org/draft-04/schema#",
    "description": "OpenStack Glance image member schema for member-create, member-detail, and member-update requests.",
    "type": "object",
    "oneOf": [
        { "$ref": "#/definitions/image-member-create-request" },
        { "$ref": "#/definitions/image-member-detail-response" },
        { "$ref": "#/definitions/image-member-update-request" }
    ],
    "definitions": {
        "image-member-create-request": {
            "additionalProperties": false,
            "properties": {
                "member": {
                    "description": "An identifier for the image member (tenantId). (The image_id of the image to which this request applies is specified in the path of the request.)",
                    "maxLength": 255,
                    "type": "string"
                }
            },
            "required": [
                "member"
            ]
        },
        "image-member-detail-response": {
            "additionalProperties": false,
            "properties": {
                "created_at": {
                    "description": "Date and time of image member creation",
                    "readOnly": true,
                    "type": "string"
                },
                "image_id": {
                    "description": "An identifier for the image",
                    "pattern": "^([0-9a-fA-F]){8}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){4}-([0-9a-fA-F]){12}$",
                    "readOnly": true,
                    "type": "string"
                },
                "member_id": {
                    "description": "An identifier for the image member (tenantId)",
                    "maxLength": 255,
                    "readOnly": true,
                    "type": "string"
                },
                "schema": {
                    "readOnly": true,
                    "type": "string"
                },
                "status": {
                    "description": "The status of this image member",
                    "enum": [
                        "pending",
                        "accepted",
                        "rejected"
                    ],
                    "readOnly": true,
                    "type": "string"
                },
                "updated_at": {
                    "description": "Date and time of last modification of image member",
                    "readOnly": true,
                    "type": "string"
                }
            },
            "required": [
                "created_at",
                "image_id",
                "member_id",
                "schema",
                "status",
                "updated_at"
            ]
    ...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/329288
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=9eed745871814d611788f560965f78be8e7b5b0d
Submitter: Jenkins
Branch: master

commit 9eed745871814d611788f560965f78be8e7b5b0d
Author: bhagyashris <email address hidden>
Date: Fri Jun 3 09:16:44 2016 +0530

    Return 400 when name is more than 255 characters

    APIs listed below are returning 500 error if you pass name more than
    255 characters.
    1. metadef tag create
    2. metadef property create
    3. metadef object create

    Added maxLength check in schema to ensure name should not be more than
    255 characters.

    DocImpact: Earlier for all of the above APIs, maxLength check
    for "name" paramerter was not present in schema.
    Now all of the above APIs will return 400 HTTPBadRequest
    if user passes "name" more than 255 characters.

    Partial-Bug: #1585917
    Change-Id: Ib85d703442d39a1f5b79b383f6b2dd800b9c7b20

[Brian Rosmaita (brian-rosmaita)]

Discussed the schema change suggested in comment #3 at the API WG meeting today. General consensus is that it's an OK approach to this problem.

http://eavesdrop.openstack.org/meetings/api_wg/2016/api_wg.2016-07-07-16.00.log.html

[Bhagyashri Shewale (bhagyashri-shewale)]

In case of member api schema is used to check required properties are present in response and not for the validation purpose. To adopt new schema suggested in comment #3,
I have done some performance testing to ensure performance is not degraded by using "oneOf" in schema as it will try to find the exact match for the schema from "oneOf" definitions.
As per results, there is not much impact on performance if we go for adopting schema suggested in comment #3.

I will identify the changes required and will propose the new patch ASAP.

[Brian Rosmaita (brian-rosmaita)]

Bhagyashri: thanks for following up on this. If you do put up the schema change patch, please include "Closes-bug: #1526991" in your commit message.

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi Brian,

I have explored few things to find whether we can solve this issue without introducing new schema.

My observation so far is:

1. for member-update shcema is not used for validation and status is validated at [1]. So if we used "oneOf" shcema and added schema definition for update then we need to get rid of this existing validation as well.

2. We can use existing schema for member-create api with minimal changes in create method of RequestDeserializer.

Please let me know your opinion about the same.

[1] https://github.com/openstack/glance/blob/master/glance/domain/__init__.py#L322

[Brian Rosmaita (brian-rosmaita)]

Hi Bhagyashri,

We'll have to get another Glance core to look at this because my opinion may be non-standard.

With respect to 1: I see the schema as primarily a client-side thing that a client (software or person) can use to determine the correct body for making a call. Thus the only problem I see with the API-side validation in your link [1] is that if more changes are made, it could drift out of sync with the JSON schema. But that's pretty unlikely because we should have tests around that.

With respect to 2: I don't like having a published schema that is inaccurate, as it would be if we use the existing schema and instead modify the RequestDeserializer to accept either the published "member_id" or the backwards-compatible (and previously documented) "member" field.

Thus, I prefer the "oneOf" schema ... but of course I would because I wrote it! So let's get some more opinions on this issue.

cheers,
brian

[Ian Cordasco (icordasc)]

I don't understand the opposition to adding a new schema that is *accurate* and reflects reality while simplifying things and making them more consistent.

I have to agree with Brian's proposed solution for this. I don't think updating the request deserializer or any other minimal hack is the right approach.

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi Brian,

Current implementation of schema validation in glance is not compatible to use "oneOf". This will require substantial changes to incorporate "oneOf" implementation which might lead to regression. I have pointed out two issues to use schema suggested by you.

1. Request validation:
As per current implementation we define only properties of schema in the controller and schema is generated at runtime in raw() [2] call. To use "oneOf" we only need "oneOf" and "definitions" while creating schema [3] and properties will be empty {} in this case. So we need to modify Schema class to accept "oneOf" as well as need to make changes in raw method [4] to add "oneOf" to schema if it is present. This way we will able to validate the input request.

[1] https://github.com/openstack/glance/blob/master/glance/api/v2/image_members.py#L335
[2] https://github.com/openstack/glance/blob/master/glance/schema.py#L86
[3] https://github.com/openstack/glance/blob/master/glance/schema.py#L26
[4] https://github.com/openstack/glance/blob/master/glance/schema.py#L85

2. Response will be blank:
If we made changes as suggested in above #1 then empty response will be returned to user because while sending response back it is formatted and filtered with properties [5]. As mentioned in "oneOf" case properties will be empty {}, it will not add any output to the filter response and empty {} will be returned. To avoid this we need to make changes in filter method [6] such that not to use properties for filtering if "oneOf" is used.

[5] https://github.com/openstack/glance/blob/master/glance/api/v2/image_members.py#L301
[6] https://github.com/openstack/glance/blob/master/glance/schema.py#L46

[Erno Kuvaja (jokke)]

Bhagyashri & Brian,

Thanks for looking into this and doing the ground work what needs to be done to fix the issue.

I'd like to see our schema reflecting the reality as well and if we have identified a way to make that happening, I'd say that's what we need to do then. What comes to the timeline to make this happen, we should have time to get this done. The bug is not new per se and it does not crash the service or anything. So lets do this right rather than hack another thing that might regress easily or miss some cases we are having there.

[Brian Rosmaita (brian-rosmaita)]

Erno, thanks for your comment. I think you're right that a quick fix isn't the way to go. I had been hoping that the length issue could be fixed independently of the schema change, but they're more related than I had thought at first.

Bhagyashri, if you'd like to continue working on this, can you provide us with an assessment of what exactly will have to happen in Glance to get the appropriate schema support? Thanks!

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422603

[Bhagyashri Shewale (bhagyashri-shewale)]

Hi All,

Current implementation of schema validation in glance is not compatible to use "oneOf". This will require substantial changes to incorporate "oneOf" implementation (Please refer patch [1])and that has some cons which are mentioned below:

1. Additional GenericJSONSchema class needs to write in galnce.schema module to support the complex schama for request and response validation which basically overrides the glance.schema.Schema class and validate the request object with the whole schema (without extracting the properties only).

2. For the validation of response at glanceclient side the minimal method in glance.schema.GenericJSONSchema class [1] is used to get the schema from glance in that we are passing whole schema and "properties" parameter as "None", that doesn't make any sense and that will cause problem when other services will call the member api because is_base_property() method in [2] will be call to use the member api in other service.(like cinder is using the glance service where is_base_property() method is used to get the image list)

3. At the glance side glance.api.v2.image_member.get_collection_schema() method [3] is used for the member list api so if we passed the whole schema to the glance.schema.CollectionSchema() class then it will degrades the performance when there will be maximum records, so passed the “image-member-detail-response “ definitions properties from member schema in [1] and this seems hackish.

So to overcome all these cons, the additional maxLength check for member_id (using the strutils.check_string_length() method of oslo_utils) in glance.api.v2.RequestDeserializer.create() method will solve all the problem.

[1] https://review.openstack.org/#/c/422603/1
[2] https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v2/schemas.py#L100
[3] https://github.com/openstack/glance/blob/master/glance/api/v2/image_members.py#L383

[Brian Rosmaita (brian-rosmaita)]

Bhagyashri,

If I understand correctly, your proposal is:

(1) the 'member' (request) and 'member_id' (response) mismatch should be considered a separate bug, which can be fixed when there is another reason to make the JSON schemas more complicated

(2) the subject of this bug is that the API is returning a 500 when the id passed in the request for member-create is > 255 chars. That can be fixed if the length of the passed-in id is checked when the member-create request is deserialized.

I'm hesitating to endorse this approach, because it will leave us with a weird length check in the RequestDeserializer.create() that may cause problems later if the complicated schema is implemented and the maxLength is changed in the schema. On the other hand, if this bug is causing problems now, then that would be a good reason to fix it. I imagine that most deployments use UUIDs for user or project identifiers, so this problem is unlikely to come up in practice. Is this bug currently causing a problem for your customers?

[Dharini Chandrasekar (dharini-chandrasekar)]

Hi Bhagyashree,

Any reason you have not used a direct jsonchema.Draft4Validator method for validation in the class you have introduced?
I was thinking if we could have a separate 'validate()' in your GenericJSONSchema class that would use something like : "Draft4Validator(schema).validate(obj)" instead of calling schema's validate(). That way we need not worry about overriding the existing functionality of glance.schema.Schema (properties being None, and that causing a problem) or modifying raw(). Please let me know if I am missing something here.

Thanks.

[Dharini Chandrasekar (dharini-chandrasekar)]

Alternatively, we can do some slight tinkering on Brian's schema in comment #3 to include properties that will contain our "OneOf". Something like in this example [1]

[1] http://json-schema.org/example2.html

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bhagyashri Shewale (<email address hidden>) on branch: master
Review: https://review.opendev.org/422603