Comment 8 for bug 1934849

I think the swift driver uses 200M:

DEFAULT_LARGE_OBJECT_CHUNK_SIZE = 200 # 200M

On Wed, Jul 7, 2021 at 1:50 PM Nobuto Murata <email address hidden>
wrote:

> And by using "4 * units.Mi" it can be 20s.
>
> --
> You received this bug notification because you are a member of Ubuntu
> OpenStack, which is subscribed to python-glance-store in Ubuntu.
> https://bugs.launchpad.net/bugs/1934849
>
> Title:
> s3 backend takes time exponentially
>
> Status in glance_store:
> New
> Status in python-glance-store package in Ubuntu:
> New
>
> Bug description:
> I have a test Ceph cluster as an object storage with both Swift and S3
> protocols enabled for Glance (Ussuri). When I use Swift backend with
> Glance, an image upload completes quickly enough. But with S3 backend
> Glance, it takes much more time to upload an image and it seems to
> rise exponentially.
>
> It's worth noting that when uploading an image with S3 backend, a
> single core is consumed 100% by glance-api process.
>
> for backend in swift s3; do
> for i in {8,16,32,64,128,512}; do
> dd if=/dev/zero of=my-image.img bs=1M count=$i oflag=sync
> time glance image-create \
> --store $backend \
> --file my-image.img --name my-image \
> --disk-format raw --container-format bare \
> --progress
> done
> done
>
> [swift]
> 8MB - 2.4s
> 16MB - 2.8s
> 32MB - 2.6s
> 64MB - 2.7s
> 128MB - 3.1s
> ...
> 512MB - 5.9s
>
> [s3]
> 8MB - 2.2s
> 16MB - 2.9s
> 32MB - 5.5s
> 64MB - 16.3s
> 128MB - 54.9s
> ...
> 512MB - 14m26s
>
> Btw, downloading of 512MB image with S3 backend can complete with less
> than 10 seconds.
>
> $ time openstack image save --file downloaded.img
> 917c5424-4350-4bc5-98ca-66d40e101843
> real 0m5.673s
>
> $ du -h downloaded.img
> 512M downloaded.img
>
>
> [/etc/glance/glance-api.conf]
>
> enabled_backends = local:file, ceph:rbd, swift:swift, s3:s3
>
> [swift]
> auth_version = 3
> auth_address = http://192.168.151.131:5000/v3
> ...
> container = glance
> large_object_size = 5120
> large_object_chunk_size = 200
>
>
> [s3]
> s3_store_host = http://192.168.151.137:80/
> ...
> s3_store_bucket = zaza-glance-s3-test
> s3_store_large_object_size = 5120
> s3_store_large_object_chunk_size = 200
>
> ProblemType: Bug
> DistroRelease: Ubuntu 20.04
> Package: python3-glance-store 2.0.0-0ubuntu1
> ProcVersionSignature: Ubuntu 5.4.0-77.86-generic 5.4.119
> Uname: Linux 5.4.0-77-generic x86_64
> NonfreeKernelModules: bluetooth ecdh_generic ecc tcp_diag inet_diag
> binfmt_misc veth zfs zunicode zlua zavl icp zcommon znvpair spl unix_diag
> nft_masq nft_chain_nat bridge stp llc vhost_vsock
> vmw_vsock_virtio_transport_common vhost vsock ebtable_filter ebtables
> ip6table_raw ip6table_mangle ip6table_nat ip6table_filter ip6_tables
> iptable_raw iptable_mangle iptable_nat nf_nat nf_conntrack nf_defrag_ipv6
> nf_defrag_ipv4 iptable_filter bpfilter nf_tables nfnetlink dm_multipath
> scsi_dh_rdac scsi_dh_emc scsi_dh_alua kvm_amd ccp input_leds kvm joydev
> mac_hid serio_raw qemu_fw_cfg sch_fq_codel ip_tables x_tables autofs4 btrfs
> zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq
> async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear
> crct10dif_pclmul crc32_pclmul cirrus ghash_clmulni_intel drm_kms_helper
> virtio_net syscopyarea aesni_intel sysfillrect sysimgblt fb_sys_fops
> crypto_simd cryptd drm virtio_blk glue_helper net_failover psmouse failover
> floppy i2c_piix4 pata_acpi
> ApportVersion: 2.20.11-0ubuntu27.18
> Architecture: amd64
> CasperMD5CheckResult: skip
> Date: Wed Jul 7 04:46:05 2021
> PackageArchitecture: all
> ProcEnviron:
> TERM=screen-256color
> PATH=(custom, no user)
> LANG=C.UTF-8
> SHELL=/bin/bash
> SourcePackage: python-glance-store
> UpgradeStatus: No upgrade log present (probably fresh install)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/glance-store/+bug/1934849/+subscriptions
>
> --
> Ubuntu-openstack-bugs mailing list
> <email address hidden>
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-bugs
>