swift glance_store not using modern swiftclient

Bug #1921953 reported by Mike Durnosvistov
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Glance
New
Undecided
Unassigned

Bug Description

The original code of the `add`[1] method (upload) is inefficient and takes more time to upload images.
It's significantly improving after using the `python-swiftclient` library API.
Additionally `add` method became simpler.

[1] https://github.com/openstack/glance_store/blob/stable/victoria/glance_store/_drivers/swift/store.py#L915

# The original implementation
> time glance image-create --protected=False --name=some-large-image --progress --disk-format=vmdk --visibility=private --file some-large-image --container-format=bare
[=============================>] 100%
+------------------+----------------------------------------------------------------------------------+
| Property | Value |
+------------------+----------------------------------------------------------------------------------+
| checksum | aa7399679400faa31fb9f01bbae6758f |
| container_format | bare |
| created_at | 2021-03-30T20:00:26Z |
| direct_url | swift+https://objectstore-3.cloud:443/v1/AUTH_e9141fb24eee4b3e9f27ae |
| | 69cda31132/glance_7d0607a8-80c1-465f-aecc-0c9129366a0e/7d0607a8-80c1-465f-aecc-0 |
| | c9129366a0e |
| disk_format | vmdk |
| id | 7d0607a8-80c1-465f-aecc-0c9129366a0e |
| min_disk | 0 |
| min_ram | 0 |
| name | some-large-image |
| os_hash_algo | sha512 |
| os_hash_value | e9d5babf23f24643f06a2bbd2e48f7a9a04109e176d46c6930ae0a3f76c571440367d6a400d45fd4 |
| | f07f0ed5e36d422825b33ae00d287ffad07a2ea16800f6cf |
| os_hidden | False |
| owner | e9141fb24eee4b3e9f25ae69cda31132 |
| protected | False |
| size | 7980414976 |
| status | active |
| tags | [] |
| updated_at | 2021-03-30T20:04:33Z |
| virtual_size | 64424509440 |
| visibility | private |
+------------------+----------------------------------------------------------------------------------+
real 4m9.821s
user 0m18.227s
sys 0m15.272s

# The improved version using `SwiftService` from `python-swiftclient`
> time glance image-create --protected=False --name=some-large-image --progress --disk-format=vmdk --visibility=private --file some-large-image -container-format=bare
[=============================>] 100%
+------------------+----------------------------------------------------------------------------------+
| Property | Value |
+------------------+----------------------------------------------------------------------------------+
| checksum | 818971a4539213d56f1d5b5f37efeab6 |
| container_format | bare |
| created_at | 2021-03-30T19:53:27Z |
| direct_url | swift+https://objectstore-3.cloud:443/v1/AUTH_e9141fb24eee4b3e9f27ae |
| | 69cda31132/glance_b6f49875-7769-45d3-8b07-e7c0118f4a02/b6f49875-7769-45d3-8b07-e |
| | 7c0118f4a02 |
| disk_format | vmdk |
| id | b6f49875-7769-45d3-8b07-e7c0118f4a02 |
| min_disk | 0 |
| min_ram | 0 |
| name | some-large-image |
| os_hash_algo | sha512 |
| os_hash_value | 7fa67ff22a1bc180b5821f86a78370f4 |
| os_hidden | False |
| owner | e9141fb24eee4b3e9f25ae69cda31132 |
| protected | False |
| size | 0 |
| status | active |
| tags | [] |
| updated_at | 2021-03-30T19:56:13Z |
| virtual_size | 64424509440 |
| visibility | private |
+------------------+----------------------------------------------------------------------------------+
real 2m48.374s
user 0m19.205s
sys 0m16.002s

Release: Victoria
glance-21.0.0
glance_store-2.3.0

Revision history for this message
Mike Durnosvistov (glacierrdev) wrote :

There is an even better result without using `swiftclient`, just using ThreadPoolExecutor I've got such results:

real 2m29.227s
user 0m18.923s
sys 0m17.172s

summary: - The upload of image is slow
+ swift glance_store not using modern swiftclient
Revision history for this message
Cyril Roelandt (cyril-roelandt) wrote :

So this is related to https://review.opendev.org/c/openstack/glance-specs/+/787179 , which was abandoned a few months ago. Nothing is going to happen unless that spec is reopened. Also, please show us the code :)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.