Comment 10 for bug 1576669

Revision history for this message
Alexei Sheplyakov (asheplyakov) wrote : Re: Glance performance degradation in 'list_images' Rally test scenario

ceph cluster is badly misconfigured:

- OSD journals reside on rotating hard drives within the filesystem which stores the actual data [1],
  which makes bandwidth at least 2x smaller (due to lots of fsync's) and latency 4x -- 10x bigger
  due to numerious seeks,
- OSDs are co-hosted with hypervisors which is a bad idea (both need quite a lot of RAM and CPU)

In order to avoid performance problems please

- Put OSD journals on SSD, use a dedicated partition/logical volume instead of a file
- Allocate nodes specifically for OSDs, that is, don't co-host hypervisors/databases/etc there

[1] From node-51/var/log/ceph/ceph-osd.0.log

2016-04-25 06:49:10.141975 7f0e6532a800 1 journal _open /var/lib/ceph/osd/ceph-0/journal fd 20: 2147483648 bytes, block size 4096 bytes, directio = 1, aio = 0
2016-04-25 06:49:10.142203 7f0e6532a800 1 journal _open /var/lib/ceph/osd/ceph-0/journal fd 20: 2147483648 bytes, block size 4096 bytes, directio = 1, aio = 0