Forgot to mention that the Ceph cluster has to be under write load in order to reproduce, i.e. running something like
rados -p rbd bench 600 write -t 1 --show-time --run-length 60
There is no effect of running os-prober if the cluster is idle. Based with that information, though, I can also reproduce the issue by running fio on some partition and os-prober in parallel, getting:
# fio --ioengine=libaio --filename=/dev/sdc4 --bs=64k --rw=randwrite --runtime=300 --size=1G --direct=1 --iodepth=8 --name=a a: (g=0): rw=randwrite, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=8 fio-2.2.13 Starting 1 process fio: io_u error on file /dev/sdc4: Operation not permitted: write offset=531300352, buflen=65536 fio: pid=17543, err=1/file:io_u.c:1596, func=io_u error, error=Operation not permitted
So I think the error has nothing to do with Ceph in particular, but really os-prober should be made more conservative when trying to probe partitions.
Forgot to mention that the Ceph cluster has to be under write load in order to reproduce, i.e. running something like
rados -p rbd bench 600 write -t 1 --show-time --run-length 60
There is no effect of running os-prober if the cluster is idle. Based with that information, though, I can also reproduce the issue by running fio on some partition and os-prober in parallel, getting:
# fio --ioengine=libaio --filename= /dev/sdc4 --bs=64k --rw=randwrite --runtime=300 --size=1G --direct=1 --iodepth=8 --name=a 64K/64K- 64K/64K- 64K, ioengine=libaio, iodepth=8 io_u.c: 1596, func=io_u error, error=Operation not permitted
a: (g=0): rw=randwrite, bs=64K-
fio-2.2.13
Starting 1 process
fio: io_u error on file /dev/sdc4: Operation not permitted: write offset=531300352, buflen=65536
fio: pid=17543, err=1/file:
So I think the error has nothing to do with Ceph in particular, but really os-prober should be made more conservative when trying to probe partitions.