Nova allows creating an Image name with invalid utf8 (thus breaking tools further down the pipeline)

[Rohit Karajgi]

Our tempest tests that checks for 400 Bad Request return code fails with a ComputeFault instead.

Pass multi-byte character image name during Create Image
Actual Response Code: ComputeFault, 500
Expected Response Code: 400 Bad Request

Return an error if the server name has a multi-byte character ... FAIL

======================================================================
FAIL: Return an error if the server name has a multi-byte character
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/stack/tempest/tests/test_images.py", line 251, in test_create_image_specify_multibyte_character_server_name
    self.fail("Should return 400 Bad Request if multi byte characters"
AssertionError: Should return 400 Bad Request if multi byte characters are used for image name
-------------------- >> begin captured logging << --------------------
tempest.config: INFO: Using tempest config file /opt/stack/tempest/etc/tempest.conf
tempest.common.rest_client: ERROR: Request URL: http://10.2.3.164:8774/v2/1aeac1cfbfdd43c2845b2cb3a4f15790/images/24ceff93-1af3-41ab-802f-9fc4d8b90b69
tempest.common.rest_client: ERROR: Request Body: None
tempest.common.rest_client: ERROR: Response Headers: {'date': 'Thu, 31 May 2012 06:02:33 GMT', 'status': '404', 'content-length': '62', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-7a15d284-e934-47a1-87f4-7746e949c7a2'}
tempest.common.rest_client: ERROR: Response Body: {"itemNotFound": {"message": "Image not found.", "code": 404}}
tempest.common.rest_client: ERROR: Request URL: http://10.2.3.164:8774/v2/1aeac1cfbfdd43c2845b2cb3a4f15790/servers/ecb51dfb-493d-4ef8-9178-1adc3d96a04d/action
tempest.common.rest_client: ERROR: Request Body: {"createImage": {"name": "\ufeff43802479847"}}
tempest.common.rest_client: ERROR: Response Headers: {'date': 'Thu, 31 May 2012 06:02:44 GMT', 'status': '500', 'content-length': '128', 'content-type': 'application/json; charset=UTF-8', 'x-compute-request-id': 'req-1a9505f5-4dfc-44e7-b04a-f8daec0f956e'}
tempest.common.rest_client: ERROR: Response Body: {u'computeFault': {u'message': u'The server has either erred or is incapable of performing the requested operation.', u'code': 500}}
--------------------- >> end captured logging << ---------------------

[Prem Karat (prem-karat)]

This bug doesn't make sense as this test case is invalid @ the first place.

We can always create a snapshot (image-create) with any string value. So this specific test case is invalid here. As long as mulitbye characters are included in a string it would create the images as shown below.

xxxxx@xxxxx:/opt/stack/tempest$ nova image-list
+--------------------------------------+---------------------------------+--------+--------------------------------------+
| ID | Name | Status | Server |
+--------------------------------------+---------------------------------+--------+--------------------------------------+
| 8163d097-d05b-44fc-9628-f33d0f22a6da | check-this-@-out\xef\xbb\xbf | ACTIVE | 21274a7e-df35-4698-87e9-4ee779a89098 |
| 3afd61b7-117b-4ca8-9cbb-19d7e02e22fc | test-@@@11122nova image-list | ACTIVE | 21274a7e-df35-4698-87e9-4ee779a89098 |

Submitting a patch to remove this test case from tempest.

--prem

[Prem Karat (prem-karat)]

The error code now returned is MIsmatch error from assertRaises which when debugged showed that the snapshot was created successfully and the test case was expecting it to fail (its a negative test)

--prem

[Jordan Pittier (jordan-pittier)]

If a try to run this test on a up o date ubuntu 12.10 I got

ERROR: tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestXML.test_create_image_specify_multibyte_character_image_name[negative]
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/api/compute/images/test_images_oneserver.py", line 72, in test_create_image_specify_multibyte_character_image_name
    snapshot_name)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 394, in assertRaises
    self.assertThat(our_callable, matcher)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 406, in assertThat
    mismatch = matcher.match(matchee)
  File "/usr/local/lib/python2.7/dist-packages/testtools/matchers/_exception.py", line 99, in match
    mismatch = self.exception_matcher.match(exc_info)
  File "/usr/local/lib/python2.7/dist-packages/testtools/matchers/_higherorder.py", line 61, in match
    mismatch = matcher.match(matchee)
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 386, in match
    reraise(*matchee)
  File "/usr/local/lib/python2.7/dist-packages/testtools/matchers/_exception.py", line 92, in match
    result = matchee()
  File "/usr/local/lib/python2.7/dist-packages/testtools/testcase.py", line 867, in __call__
    return self._callable_object(*self._args, **self._kwargs)
  File "/opt/stack/tempest/tempest/services/compute/xml/images_client.py", line 105, in create_image
    str(Document(post_body)), self.headers)
  File "/opt/stack/tempest/tempest/common/rest_client.py", line 174, in post
    return self.request('POST', url, headers, body)
  File "/opt/stack/tempest/tempest/common/rest_client.py", line 289, in request
    headers=headers, body=body)
  File "/opt/stack/tempest/tempest/common/rest_client.py", line 272, in _request
    headers=headers, body=body)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1543, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1293, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1230, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 962, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 996, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 958, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 816, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 58: ordinal not in range(128)

[Attila Fazekas (afazekas)]

I do not think it is good practice to allow invalid utf-8 sequences in any name.
Most people expects valid UTF-8 names in xml and json rest API-s.
IMHO the invalid UTF-8 names should be rejected by 400.

(The test case in tempest is wrong now, but it can be fixed.)

Now, I can can cause an internal server error in glance by an invalid name.

[Russell Bryant (russellb)]

Seems to work in the latest code in master. Feel free to reopen if you think further changes are needed.

[rbryant@devstack devstack(master)]$ nova image-create test "ಠ_ರೃ"
[rbryant@devstack devstack(master)]$ nova image-list
+--------------------------------------+---------------------------------+--------+--------------------------------------+
| ID | Name | Status | Server |
+--------------------------------------+---------------------------------+--------+--------------------------------------+
| d7e2a05c-3dd9-49b1-9e35-e31805280ff4 | cirros-0.3.1-x86_64-uec | ACTIVE | |
| 83cb17cc-5a7f-42bc-87c1-8c49100b2c93 | cirros-0.3.1-x86_64-uec-kernel | ACTIVE | |
| db525ca4-0bc3-4229-8c4a-b23799475ead | cirros-0.3.1-x86_64-uec-ramdisk | ACTIVE | |
| fb8fe8d6-9934-474e-8ad6-61bc83d7e39f | ಠ_ರೃ | ACTIVE | 70dabb27-e733-46ab-890a-f714d3bb504d |
+--------------------------------------+---------------------------------+--------+--------------------------------------+

[rbryant@devstack tempest(master)]$ nosetests -sv tempest.api.compute.images.test_images_oneserver.py:ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_image_name[gate,negative] ... ok

----------------------------------------------------------------------
Ran 1 test in 53.895s

OK

[rbryant@devstack tempest(master)]$ git diff tempest/api/compute/
diff --git a/tempest/api/compute/images/test_images_oneserver.py b/tempest/api/compute/images/test_images_onese
index 06e9ab2..7d225e2 100644
--- a/tempest/api/compute/images/test_images_oneserver.py
+++ b/tempest/api/compute/images/test_images_oneserver.py
@@ -64,14 +64,12 @@ class ImagesOneServerTestJSON(base.BaseComputeTest):
                 cls.alt_manager = clients.AltManager()
             cls.alt_client = cls.alt_manager.images_client

- @testtools.skip("Skipped until the Bug #1006725 is resolved.")
     @attr(type=['negative', 'gate'])
     def test_create_image_specify_multibyte_character_image_name(self):
         # Return an error if the image name has multi-byte characters
         snapshot_name = rand_name('\xef\xbb\xbf')
- self.assertRaises(exceptions.BadRequest,
- self.client.create_image, self.server['id'],
- snapshot_name)
+ resp, body = self.client.create_image(self.server['id'], snapshot_name)
+ self.assertEqual(202, resp.status)

     @attr(type=['negative', 'gate'])
     def test_create_image_specify_invalid_metadata(self):

[Matthew Treinish (treinish)]

Russell, if you've got the change to unskip the bug how about you push it out for review to close the tempest side.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/61747

[Sean Dague (sdague)]

Definitely not fixed on the nova side. Attempts to unskip the bug generated issues.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/62161

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/61747
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=7eb1077ee7557bad88a4b2316eb8e8ef16cd328f
Submitter: Jenkins
Branch: master

commit 7eb1077ee7557bad88a4b2316eb8e8ef16cd328f
Author: Sean Dague <email address hidden>
Date: Sat Dec 14 12:50:02 2013 +0000

    provide a valid utf8 multibyte test for nova images

    this only works for json because getting our xml client to be
    fully utf8 compliant would be *a ton* of work.

    Change-Id: I5e2f05274aaa6a2eb97b6206240ffd44e9f12526
    Related-Bug: #1006725

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/62161
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=4ff69bb44fbac177b4ed2649a3f045a81ff4b427
Submitter: Jenkins
Branch: master

commit 4ff69bb44fbac177b4ed2649a3f045a81ff4b427
Author: Sean Dague <email address hidden>
Date: Sat Dec 14 12:52:24 2013 +0000

    clean up invalid_multibyte test

    This cleans up the invalid_multibyte test to make it more clear
    what it is trying to do, which is send an invalid utf8 sequence as
    an image name (it should fail with a 400, it currently passes with
    a 202).

    Change-Id: I58ad059a138ae96201bbf50df0c22bf259ae9716
    Related-Bug: #1006725

[Tracy Jones (tjones-i)]

prem - are you still working this actively? If not can you remove your name so others can pick it up?

[Mike Spreitzer (mike-spreitzer)]

I am still getting Tempest failures at this test. It looks like this:

FAIL: tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestJSON.test_create_image_specify_multibyte_character_ima\
ge_name[gate]
tags: worker-0
----------------------------------------------------------------------
Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2014-06-24 13:38:04,570 15948 INFO [tempest.common.rest_client] Request (ImagesOneServerTestJSON:setUp): 200 GET http://10.1\
0.0.123:8774/v2/12ab086fc62d4e19bedabaf3039a18bc/servers/a75d561f-e527-4fe7-a9f7-c332274a00c3 0.215s
2014-06-24 13:38:04,820 15948 INFO [tempest.common.rest_client] Request (ImagesOneServerTestJSON:test_create_image_specify_m\
ultibyte_character_image_name): 500 POST http://10.10.0.123:8774/v2/12ab086fc62d4e19bedabaf3039a18bc/servers/a75d561f-e527-4fe7-\
a9f7-c332274a00c3/action 0.249s
2014-06-24 13:38:05,054 15948 INFO [tempest.common.rest_client] Request (ImagesOneServerTestJSON:tearDown): 200 GET http://1\
0.10.0.123:8774/v2/12ab086fc62d4e19bedabaf3039a18bc/servers/a75d561f-e527-4fe7-a9f7-c332274a00c3 0.232s
}}}

Traceback (most recent call last):
  File "tempest/api/compute/images/test_images_oneserver.py", line 108, in test_create_image_specify_multibyte_character_image_n\
ame
    resp, body = self.client.create_image(self.server_id, utf8_name)
  File "tempest/services/compute/json/images_client.py", line 50, in create_image
    post_body)
  File "tempest/common/rest_client.py", line 218, in post
    return self.request('POST', url, extra_headers, headers, body)
  File "tempest/common/rest_client.py", line 430, in request
    resp, resp_body)
  File "tempest/common/rest_client.py", line 526, in _error_checker
    raise exceptions.ServerFault(message)
ServerFault: Got server fault
Details: The server has either erred or is incapable of performing the requested operation.

[Sean Dague (sdague)]

We are running this test in the gate, and not seeing it. Can you provide links to complete logs somewhere so we can figure out what's going on here?

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/121524

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/121524
Reason: still an issue

[Christopher Yeoh (cyeoh-0)]

I think the tempest test is bogus.

For starters the test name:

invalid_name = data_utils.rand_name(u'\xc3\x28')

is a unicode string, not utf-8.

"\xc3\x28" is indeed an invalid utf-8 string but the tempest infrastructure prevents us from sending invalid utf-8 strings. At the lower levels a json.dumps is done on the post body. JSON represents strings as a sequence of unicode characters (http://tools.ietf.org/html/rfc4627.html). Therefore it attempts to convert utf-8 (which is the default encoding for dumps) to unicode characters and fails if it seens an invalid utf-8 string.

You can see this if you remove the u prefix from the tempest test which then makes it correctly pass an invalid utf-8 string.

The reason I believe the fundamental issue of sending an invalid utf-8 string is not actually valid is that we accept JSON on the nova API side. Again, JSON strings are just a sequence of unicode characters and we can always convert that to valid UTF-8. If the JSON string itself is not a sequence of unicode characters then the json decoding will fail in wsgi json deserialization and the user will be returned a 400. It is possible to directly inject an invalid utf-8 string in a unittest but that won't happen in real life with the json requests. Now it is probably possible to send an invalid utf-8 string via XML but we're not supporting that for long anyway.

I'll propose a couple of patches:
- remove the test_create_image_specify_multibyte_character_image_name tempest test
- add a check in Nova so if a non valid multibyte utf-8 string is passed it rejects it.

I did discover glance does 500 if you send it a unicode string which request a 4 byte utf-8 because thats all the mysql's utf8 supports (you need to use utf8mb4 instead). But I think that should be addressed separately. And the nova side fix will depend on what the glance team decides to do.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/122361

[OpenStack Infra (hudson-openstack)]

Change abandoned by Christopher Yeoh (<email address hidden>) on branch: master
Review: https://review.openstack.org/122361
Reason: Turns out the XML deserializer already looks for invalid utf8 sequences. So no work is required here.

[Christopher Yeoh (cyeoh-0)]

It turns out that both the JSON and XML deserializers in nova will detect invalid utf8 sequences so this bug is invalid. The only issue is on the tempest test side

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/122354
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=7a78cc8ecdade776bdb18f61b3fda8d9c4ae8c26
Submitter: Jenkins
Branch: master

commit 7a78cc8ecdade776bdb18f61b3fda8d9c4ae8c26
Author: Chris Yeoh <email address hidden>
Date: Thu Sep 18 17:51:30 2014 +0930

    Fixes image create utf-8 multibyte test

    Fixes test_create_image_specify_multibyte_character_image_name
    so that it actually sends a UTF-8 multibyte character rather
    than unicode. Also changes it to a 3 byte multibyte character
    rather than a 4 byte multibyte character because glance currently
    doesn't handle 4 byte utf-8 and will 500.

    Change-Id: I064b563024a090899a4adcbbd462a18c5450830c
    Related-Bug: 1006725