Glance

glance 500's when passed image name with a 4-byte utf-8 character

Bug #1370954 reported by Christopher Yeoh on 2014-09-18

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Glance	Fix Released	Medium	Inessa Vasilevskaya	Glance 2015.1.0 "kilo"

Bug Description

Glance currently 500's when passed an image name with a 4-byte utf-8 character in the name. This is because the mysql utf-8 type only handles up to 3-byte utf-8 characters:

See http://stackoverflow.com/questions/10957238/incorrect-string-value-when-trying-to-insert-utf-8-into-mysql-via-jdbc

You can replicate this by using the tempest test:
test_create_image_specify_multibyte_character_image_name (the positive not negative one)
You'll need to change utf8_name to
utf8_name = data_utils.rand_name('\xF0\x9F\x92\xA9')

(note removal of unicode prefix)

Backtrace from nova request to glance that triggers this:

2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Route path: '/images', defaults: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c
47a6d10>} __call__ /usr/local/lib/python2.7/dist-packages/routes/middleware.py:102
2014-09-18 07:35:47.343 843 DEBUG routes.middleware [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644
a9ee2a0433f45d0b - - -] Match dict: {'action': u'create', 'controller': <glance.common.wsgi.Resource object at 0x7f90c47a6d10>} __call__ /u
sr/local/lib/python2.7/dist-packages/routes/middleware.py:103
2014-09-18 07:35:47.348 843 ERROR glance.registry.api.v1.images [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 3376
0203c2944644a9ee2a0433f45d0b - - -] Unable to create image None
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images Traceback (most recent call last):
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/registry/api/v1/images.py", line 424, in c
reate
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_data = self.db_api.image_create(req.context, image_data)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 124, in image_
create
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return _image_update(context, values, None, purge_props=False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 92, in wr
apped_f
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return Retrying(*dargs, **dkw).call(f, *args, **kw)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 239, in c
all
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images return attempt.get(self._wrap_exception)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images reraise(self.value[0], self.value[1], self.value[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 233, in call
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/api.py", line 759, in _image_update
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images image_ref.save(session=session)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/opt/stack/glance/glance/db/sqlalchemy/models.py", line 77, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images super(GlanceBase, self).save(session or db_api.get_session())
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/models.py", line 48, in save
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images session.flush()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1818, in flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images self._flush(objects)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1936, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images transaction.rollback(_capture_exception=True)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compat.reraise(exc_type, exc_value, exc_tb)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1900, in _flush
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images flush_context.execute()
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images rec.execute(self)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", lin
e 525, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images uow
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images table, insert)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images execute(statement, params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 662, in execute
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images params)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images compiled_sql, distilled_params
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 125, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images six.reraise(type(newraise), newraise, sys.exc_info()[2])
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 102, in _handle_dbapi_exception
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images per_fn = fn(ctx)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 323, in handler
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images context.is_disconnect)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/exc_filters.py", line 254, in _raise_operational_errors_directly_filter
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images raise operational_error
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images OperationalError: (OperationalError) (1366, "Incorrect string value: '\\xF0\\x9F\\x92\\xA9-1...' for column 'name' at row 1") 'INSERT INTO images (created_at, updated_at, deleted_at, deleted, id, name, disk_format, container_format, size, virtual_size, status, is_public, checksum, min_disk, min_ram, owner, protected) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2014, 9, 18, 7, 35, 47, 347058), datetime.datetime(2014, 9, 18, 7, 35, 47, 347072), None, 0, '5ff754de-4b8c-4cca-818c-d235dda03b83', '\xf0\x9f\x92\xa9-1786046285', 'ami', 'ami', 0, None, 'queued', 0, None, 0, 0, '33760203c2944644a9ee2a0433f45d0b', 0)
2014-09-18 07:35:47.348 843 TRACE glance.registry.api.v1.images
2014-09-18 07:35:47.356 843 INFO glance.wsgi.server [797abe72-9a70-488e-9254-c71888536278 a2bb050fa6e647398991ebc635741cb1 33760203c2944644a9ee2a0433f45d0b - - -] Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py", line 433, in handle_one_response
result = self.application(self.environ, start_response)

Tags:

Mark Washenberger (markwash) on 2014-09-23

Changed in glance:
importance:	Undecided → Medium
status:	New → Confirmed

Roman Vasilets (rvasilets) on 2014-10-04

Changed in glance:
assignee:	nobody → Roman Vasilets (rvasilets)

Alexander Tivelkov (ativelkov) on 2014-10-23

Changed in glance:
assignee:	Roman Vasilets (rvasilets) → nobody

Inessa Vasilevskaya (ivasilevskaya) on 2014-10-23

Changed in glance:
assignee:	nobody → Inessa Vasilevskaya (ivasilevskaya)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-10-27: Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.openstack.org/131148

Changed in glance:
status:	Confirmed → In Progress

Erno Kuvaja (jokke) on 2014-12-03

tags:

added: juno-backport-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-12-03: Fix merged to glance (master)

Reviewed: https://review.openstack.org/131148
Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=e5cba08b08a2174aa4496406bcc84e94b6907ff4
Submitter: Jenkins
Branch: master

commit e5cba08b08a2174aa4496406bcc84e94b6907ff4
Author: Inessa Vasilevskaya <email address hidden>
Date: Mon Oct 27 17:24:47 2014 +0400

No 4 byte unicode allowed in image parameters

    Image create or update with params containing 4 byte unicode characters
    results in 400 server error code (instead of former 500).
    Image name/location/tags/properties are validated this way

Change-Id: Ib0fcf2456f0996e8235983b361d1ee5f66136375
Closes-bug: #1370954

Changed in glance:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2014-12-19

Changed in glance:
milestone:	none → kilo-1
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2015-04-30

Changed in glance:
milestone:	kilo-1 → 2015.1.0

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.