index when add is occurring is slow

[smallma]

I use two consoles to test glance server, one add image (500mb upper) to glance and the other execute 'glance index'. The performance is too low when I executing 'index' and 'add image' at the same time. the index needs 3~10sec to display image list when another one is adding image to glance. I already have checked the execution time of SQL statement is normal and DNS normal. and I also traced code and found that it's waiting executing before into "Resource function in /glance/common/wsgi.py" when I'm executing 'index'. Is there any problem? Thanks.

[Jay Pipes (jaypipes)]

Hi! This performance issue is being addressed in the work here:

https://review.openstack.org/#change,2486

If you could, please pull the code in that branch and check to see if the problem persists...

Thanks!

-jay

[smallma (s-rain)]

Hello Jay,
I have download and roughly test the modified code. It seems little delay(0.3~1) when add image and execute "index" at the same time, but it's look fine. Thank you.

[Jay Pipes (jaypipes)]

Excellent news, Rain, thank you!

[smallma (s-rain)]

Hello Jay,
I pull these files and test it and I got a problem when I curl a image from glance below. Thank you.

curl http://x.x.x.x:9292/v1/images/dd89d8c4-7261-40a6-bd61-45e30ca523bc >1
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (52) Empty reply from server

[Jay Pipes (jaypipes)]

Hi again, Rain!

Is this something that is repeatable? Is this something you can provide a test case for that shows the issue?

Thanks,
-jay

[smallma (s-rain)]

Jay, I upload a image and use "curl" to download this image from glance. After I try several times, this problem is still there. Is there any method to download image? the following section is how I test glance.

root@ubuntu:~# glance add name="test2" is_public=true < /mnt/500M.img --host=x.x.x.x

root@ubuntu:~# glance details --host x.x.x.x
================================================================================
URI: http://x.x.x.x:9292/v1/images/741946f9-ba77-4294-a4d7-9c124762673a
Id: 741946f9-ba77-4294-a4d7-9c124762673a
Public: Yes
Protected: No
Name: test2
Status: active
Size: 524390194
Disk format: raw
Container format: ovf
Minimum Ram Required (MB): 0
Minimum Disk Required (GB): 0
================================================================================

root@ubuntu:~# curl -O http://x.x.x.x:9292/v1/images/741946f9-ba77-4294-a4d7-9c124762673a
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (52) Empty reply from server

Thanks,
Rain

[Jay Pipes (jaypipes)]

On Mon, Jan 16, 2012 at 7:23 PM, smallma <email address hidden> wrote:
> root@ubuntu:~# curl -O http://x.x.x.x:9292/v1/images/741946f9-ba77-4294-a4d7-9c124762673a
>  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                 Dload  Upload   Total   Spent    Left  Speed
>  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
> curl: (52) Empty reply from server

Hi Rain! Is authentication enabled on the server? If so, you will need
to supply an authentication token.

As you can see below, this is working fine for me locally:

jpipes@librebox:~/repos/devstack$ glance -A servicetoken index
ID Name
Disk Format Container Format Size
------------------------------------ ------------------------------
-------------------- -------------------- --------------
f84efa21-400c-4d72-acc5-bc3bb857b7a4 cirros-0.3.0-x86_64-blank
ami ami 25165824
efe53cc5-4205-4165-b8cb-e654ae93de79 cirros-0.3.0-x86_64-blank-ramd
ari ari 2254249
0f4704ae-93ef-44f2-8a63-302c66ec6b9e cirros-0.3.0-x86_64-blank-kern
aki aki 4731440
jpipes@librebox:~/repos/devstack$ curl -v -O -H "X-Auth-Token:
servicetoken" http://127.0.0.1:9292/v1/images/f84efa21-400c-4d72-acc5-bc3bb857b7a4
* About to connect() to 127.0.0.1 port 9292 (#0)
* Trying 127.0.0.1... % Total % Received % Xferd Average Speed
  Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:--
--:--:-- 0connected
* Connected to 127.0.0.1 (127.0.0.1) port 9292 (#0)
> GET /v1/images/f84efa21-400c-4d72-acc5-bc3bb857b7a4 HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: 127.0.0.1:9292
> Accept: */*
> X-Auth-Token: servicetoken
>
< HTTP/1.1 200 OK
< Content-Length: 25165824
< Content-Type: application/octet-stream
< X-Image-Meta-Property-Ramdisk_id: efe53cc5-4205-4165-b8cb-e654ae93de79
< X-Image-Meta-Id: f84efa21-400c-4d72-acc5-bc3bb857b7a4
< X-Image-Meta-Deleted: False
< X-Image-Meta-Container_format: ami
< X-Image-Meta-Checksum: 2f81976cae15c16ef0010c51e3a6c163
< X-Image-Meta-Protected: False
< X-Image-Meta-Min_disk: 0
< X-Image-Meta-Deleted_at:
< X-Image-Meta-Created_at: 2012-01-16T18:30:44
< X-Image-Meta-Size: 25165824
< X-Image-Meta-Status: active
< X-Image-Meta-Is_public: True
< X-Image-Meta-Min_ram: 0
< X-Image-Meta-Property-Kernel_id: 0f4704ae-93ef-44f2-8a63-302c66ec6b9e
< X-Image-Meta-Owner: 1
< X-Image-Meta-Updated_at: 2012-01-16T18:30:44
< X-Image-Meta-Disk_format: ami
< X-Image-Meta-Name: cirros-0.3.0-x86_64-blank
< Location: http://127.0.0.1:9292/v1/images/f84efa21-400c-4d72-acc5-bc3bb857b7a4
< Etag: 2f81976cae15c16ef0010c51e3a6c163
< Date: Tue, 17 Jan 2012 02:05:58 GMT
<
{ [data not shown]
100 24.0M 100 24.0M 0 0 175M 0 --:--:-- --:--:-- --:--:-- 176M
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
jpipes@librebox:~/repos/devstack$ l...

[smallma (s-rain)]

I'm not enable authentication. It's strange. There is a problem below when I execute "glance -A servicetoken index --host x.x.x.x". I upload source code to the sharing site. ( http://FileDeck.net/zh-tw/files/XX9LKUGU/openstack-glance-010117.rar ) Is it different?

root@ubuntu:/etc# glance -A servicetoken index --host 172.16.71.94
Failed to show index. Got error:
Internal Server error: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/glance/common/wsgi.py", line 274, in __call__
    response = req.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1053, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1022, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 208, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/pymodules/python2.7/glance/common/wsgi.py", line 271, in __call__
    response = self.process_request(req)
  File "/usr/lib/pymodules/python2.7/glance/common/context.py", line 115, in process_request
    raise exception.NotAuthorized()
NotAuthorized: You are not authorized to complete this action.

Thanks,
Rain

[smallma (s-rain)]

Jay,
This problem has solved after git clone and git fetch again. Thanks. It seems main code need to update to the latest version :)

Thanks,
Rain