[2022-10-17 20:55:47,285][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid 1fcf7d20-a470-4cba-8b76-fca489b0aee9 --bluestore --data ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,286][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:55:47,290][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 lvm [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:55:47,291][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:55:47,297][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:55:47,310][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:47,311][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,vg_name=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:47,442][ceph_volume.process][INFO ] stdout ";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:47,442][ceph_volume.util.disk][INFO ] opening device /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 to check for BlueStore label [2022-10-17 20:55:47,443][ceph_volume.util.disk][INFO ] opening device /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 to check for BlueStore label [2022-10-17 20:55:47,443][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-7 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-7 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DISKSEQ=38 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout MINOR=7 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13057430034 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_NAME=ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_UUID=LVM-ffbCrwQlrG5O8xeAJy2k5Dq6nBJWiGJU6xfdJ43lHBH4kAeguQk02Ar0Mm7tOcpJ [2022-10-17 20:55:47,446][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/disk/by-id/dm-name-ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 /dev/mapper/ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 /dev/disk/by-id/dm-uuid-LVM-ffbCrwQlrG5O8xeAJy2k5Dq6nBJWiGJU6xfdJ43lHBH4kAeguQk02Ar0Mm7tOcpJ /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,447][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:55:47,448][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:47,449][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,vg_name=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:47,525][ceph_volume.process][INFO ] stdout ";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:47,525][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:55:47,526][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:55:47,536][ceph_volume.process][INFO ] stdout AQBTwU1jYaTvHxAARwSSn1ZeQmGSh1AHN1yoKg== [2022-10-17 20:55:47,536][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,747][ceph_volume.process][INFO ] stdout 1 [2022-10-17 20:55:47,748][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:47.589+0000 7f35bba70700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:55:47,748][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:47.589+0000 7f35bba70700 -1 AuthRegistry(0x7f35b405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:55:47,749][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:47,750][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,vg_name=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:47,830][ceph_volume.process][INFO ] stdout ";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:47,831][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:55:47,832][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=1fcf7d20-a470-4cba-8b76-fca489b0aee9 --addtag ceph.osd_id=1 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 --addtag ceph.block_uuid=6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,929][ceph_volume.process][INFO ] stdout Logical volume ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 changed. [2022-10-17 20:55:47,930][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:55:47,940][ceph_volume.process][INFO ] stdout AQBTwU1j1G8AOBAAHUG+mOhE13pcKH0wYsx5Hg== [2022-10-17 20:55:47,940][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1 [2022-10-17 20:55:47,942][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:55:47,942][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:55:47,943][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:55:47,943][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:47,945][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-7 [2022-10-17 20:55:47,946][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 /var/lib/ceph/osd/ceph-1/block [2022-10-17 20:55:47,948][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap [2022-10-17 20:55:48,007][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.001+0000 7fc7a17bf700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:55:48.001+0000 7fc7a17bf700 -1 AuthRegistry(0x7fc79c05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:55:48,176][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:55:48,183][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-1/keyring --create-keyring --name osd.1 --add-key AQBTwU1jYaTvHxAARwSSn1ZeQmGSh1AHN1yoKg== [2022-10-17 20:55:48,195][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-1/keyring added entity osd.1 auth(key=AQBTwU1jYaTvHxAARwSSn1ZeQmGSh1AHN1yoKg==) [2022-10-17 20:55:48,196][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring [2022-10-17 20:55:48,198][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/ [2022-10-17 20:55:48,199][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 1fcf7d20-a470-4cba-8b76-fca489b0aee9 --setuser ceph --setgroup ceph [2022-10-17 20:55:48,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.213+0000 7f721ae575c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid [2022-10-17 20:55:48,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.213+0000 7f721ae575c0 -1 bdev(0x55e41fc09d60 /var/lib/ceph/osd/ceph-1//block) open failed to lock /var/lib/ceph/osd/ceph-1//block: (13) Permission denied [2022-10-17 20:55:48,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.213+0000 7f721ae575c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/) mkfs failed, (13) Permission denied [2022-10-17 20:55:48,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.213+0000 7f721ae575c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:55:48,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.213+0000 7f721ae575c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-1/: (13) Permission denied [2022-10-17 20:55:48,223][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 1fcf7d20-a470-4cba-8b76-fca489b0aee9 --setuser ceph --setgroup ceph [2022-10-17 20:55:48,224][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:55:48,224][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.1 --yes-i-really-mean-it [2022-10-17 20:55:48,291][ceph_volume.process][INFO ] stderr 2022-10-17T20:55:48.281+0000 7f1559598700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:55:48.281+0000 7f1559598700 -1 AuthRegistry(0x7f155405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:55:48,433][ceph_volume.process][INFO ] stderr purged osd.1 [2022-10-17 20:55:48,442][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@1 [2022-10-17 20:55:48,447][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:55:48,448][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,448][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,525][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.block_uuid=6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:48,527][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,527][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,631][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,631][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,715][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,715][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,783][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,783][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,858][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.block_uuid=6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:48,858][ceph_volume.util.disk][INFO ] opening device /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 to check for BlueStore label [2022-10-17 20:55:48,859][ceph_volume.util.disk][INFO ] opening device /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 to check for BlueStore label [2022-10-17 20:55:48,859][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-7 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-7 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DISKSEQ=38 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout MINOR=7 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13057430034 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_NAME=ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_UUID=LVM-ffbCrwQlrG5O8xeAJy2k5Dq6nBJWiGJU6xfdJ43lHBH4kAeguQk02Ar0Mm7tOcpJ [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 /dev/disk/by-id/dm-name-ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 /dev/mapper/ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 /dev/disk/by-id/dm-uuid-LVM-ffbCrwQlrG5O8xeAJy2k5Dq6nBJWiGJU6xfdJ43lHBH4kAeguQk02Ar0Mm7tOcpJ [2022-10-17 20:55:48,862][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:55:48,863][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:48,864][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:55:48,864][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:48,925][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9". [2022-10-17 20:55:48,927][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:48,927][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,vg_name=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:48,994][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.block_uuid=6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:48,995][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-1 [2022-10-17 20:55:48,995][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-1 [2022-10-17 20:55:48,997][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-1 unmounted [2022-10-17 20:55:48,997][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:49,002][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9 bs=1M count=10 conv=fsync [2022-10-17 20:55:49,019][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 20:55:49,019][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0148679 s, 705 MB/s [2022-10-17 20:55:49,021][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:55:49,021][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:55:49,126][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.block_uuid=6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=1fcf7d20-a470-4cba-8b76-fca489b0aee9,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9/osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9";"6xfdJ4-3lHB-H4kA-eguQ-k02A-r0Mm-7tOcpJ";"6401233518592 [2022-10-17 20:55:49,126][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:49,127][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:55:49,128][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9 [2022-10-17 20:55:49,154][ceph_volume.process][INFO ] stderr Removing ceph--1fcf7d20--a470--4cba--8b76--fca489b0aee9-osd--block--1fcf7d20--a470--4cba--8b76--fca489b0aee9 (253:7) [2022-10-17 20:55:49,185][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9" metadata (seqno 3). [2022-10-17 20:55:49,185][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9" [2022-10-17 20:55:49,186][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9" (seqno 4). [2022-10-17 20:55:49,187][ceph_volume.process][INFO ] stdout Logical volume "osd-block-1fcf7d20-a470-4cba-8b76-fca489b0aee9" successfully removed [2022-10-17 20:55:49,187][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9" from volume group "ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9" [2022-10-17 20:55:49,187][ceph_volume.process][INFO ] stdout Volume group "ceph-1fcf7d20-a470-4cba-8b76-fca489b0aee9" successfully removed [2022-10-17 20:55:49,254][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 1fcf7d20-a470-4cba-8b76-fca489b0aee9 --setuser ceph --setgroup ceph [2022-10-17 20:56:04,918][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid 3d293c18-f62f-4f92-a77d-69b681841c34 --bluestore --data ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:04,919][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 lvm [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:56:04,924][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:56:04,930][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:56:04,941][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:04,941][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,vg_name=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:05,074][ceph_volume.process][INFO ] stdout ";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:05,074][ceph_volume.util.disk][INFO ] opening device /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 to check for BlueStore label [2022-10-17 20:56:05,074][ceph_volume.util.disk][INFO ] opening device /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 to check for BlueStore label [2022-10-17 20:56:05,075][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-8 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-8 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DISKSEQ=40 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout MINOR=8 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13075051122 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_NAME=ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_UUID=LVM-92GJtRp84xKMP0VOgcPS9G1qe2VnMHdhXODtv6nnC2jh8UcXVdcgkzz1IWN935Qw [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 /dev/disk/by-id/dm-uuid-LVM-92GJtRp84xKMP0VOgcPS9G1qe2VnMHdhXODtv6nnC2jh8UcXVdcgkzz1IWN935Qw /dev/disk/by-id/dm-name-ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 /dev/mapper/ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 [2022-10-17 20:56:05,078][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:56:05,080][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:05,080][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,vg_name=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:05,201][ceph_volume.process][INFO ] stdout ";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:05,202][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:56:05,202][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:56:05,212][ceph_volume.process][INFO ] stdout AQBlwU1jHTalDBAA6/UvRkpnmOK04NRqTWsZjg== [2022-10-17 20:56:05,213][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,425][ceph_volume.process][INFO ] stdout 2 [2022-10-17 20:56:05,425][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.265+0000 7fef3d8cd700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:56:05,425][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.265+0000 7fef3d8cd700 -1 AuthRegistry(0x7fef3805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:05,426][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:05,427][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,vg_name=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:05,497][ceph_volume.process][INFO ] stdout ";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:05,499][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:56:05,499][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=3d293c18-f62f-4f92-a77d-69b681841c34 --addtag ceph.osd_id=2 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 --addtag ceph.block_uuid=XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,586][ceph_volume.process][INFO ] stdout Logical volume ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 changed. [2022-10-17 20:56:05,586][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:56:05,596][ceph_volume.process][INFO ] stdout AQBlwU1j+/6FIxAA7//Q0QSIvj0ZSVpyxnI2Yw== [2022-10-17 20:56:05,597][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-2 [2022-10-17 20:56:05,598][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:56:05,599][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:56:05,600][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:56:05,600][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:05,602][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-8 [2022-10-17 20:56:05,603][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 /var/lib/ceph/osd/ceph-2/block [2022-10-17 20:56:05,604][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap [2022-10-17 20:56:05,666][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.657+0000 7fe5dc127700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:56:05.657+0000 7fe5dc127700 -1 AuthRegistry(0x7fe5d405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:05,832][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:56:05,840][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-2/keyring --create-keyring --name osd.2 --add-key AQBlwU1jHTalDBAA6/UvRkpnmOK04NRqTWsZjg== [2022-10-17 20:56:05,853][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-2/keyring added entity osd.2 auth(key=AQBlwU1jHTalDBAA6/UvRkpnmOK04NRqTWsZjg==) [2022-10-17 20:56:05,854][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/keyring [2022-10-17 20:56:05,855][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/ [2022-10-17 20:56:05,857][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-2/ --osd-uuid 3d293c18-f62f-4f92-a77d-69b681841c34 --setuser ceph --setgroup ceph [2022-10-17 20:56:05,880][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.869+0000 7f760a0895c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/) _read_fsid unparsable uuid [2022-10-17 20:56:05,880][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.873+0000 7f760a0895c0 -1 bdev(0x5641e8541520 /var/lib/ceph/osd/ceph-2//block) open failed to lock /var/lib/ceph/osd/ceph-2//block: (13) Permission denied [2022-10-17 20:56:05,880][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.873+0000 7f760a0895c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/) mkfs failed, (13) Permission denied [2022-10-17 20:56:05,880][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.873+0000 7f760a0895c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:56:05,880][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.873+0000 7f760a0895c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-2/: (13) Permission denied [2022-10-17 20:56:05,880][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-2/ --osd-uuid 3d293c18-f62f-4f92-a77d-69b681841c34 --setuser ceph --setgroup ceph [2022-10-17 20:56:05,881][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:56:05,881][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.2 --yes-i-really-mean-it [2022-10-17 20:56:05,951][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:05.941+0000 7f9f43513700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:56:05.941+0000 7f9f43513700 -1 AuthRegistry(0x7f9f3c05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:06,096][ceph_volume.process][INFO ] stderr purged osd.2 [2022-10-17 20:56:06,105][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@2 [2022-10-17 20:56:06,110][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:56:06,111][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,112][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=2} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,205][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,ceph.block_uuid=XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=3d293c18-f62f-4f92-a77d-69b681841c34,ceph.osd_id=2,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:06,207][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,207][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=2,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,291][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,291][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=2,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,359][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,359][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=2,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,447][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,447][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,514][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,ceph.block_uuid=XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=3d293c18-f62f-4f92-a77d-69b681841c34,ceph.osd_id=2,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:06,514][ceph_volume.util.disk][INFO ] opening device /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 to check for BlueStore label [2022-10-17 20:56:06,514][ceph_volume.util.disk][INFO ] opening device /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 to check for BlueStore label [2022-10-17 20:56:06,515][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-8 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-8 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DISKSEQ=40 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout MINOR=8 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13075051122 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_NAME=ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_UUID=LVM-92GJtRp84xKMP0VOgcPS9G1qe2VnMHdhXODtv6nnC2jh8UcXVdcgkzz1IWN935Qw [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 /dev/mapper/ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 /dev/disk/by-id/dm-uuid-LVM-92GJtRp84xKMP0VOgcPS9G1qe2VnMHdhXODtv6nnC2jh8UcXVdcgkzz1IWN935Qw /dev/disk/by-id/dm-name-ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 [2022-10-17 20:56:06,518][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:56:06,519][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,520][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:56:06,520][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,609][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34". [2022-10-17 20:56:06,611][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,611][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,vg_name=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,694][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,ceph.block_uuid=XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=3d293c18-f62f-4f92-a77d-69b681841c34,ceph.osd_id=2,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:06,695][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-2 [2022-10-17 20:56:06,695][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-2 [2022-10-17 20:56:06,697][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-2 unmounted [2022-10-17 20:56:06,697][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,703][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34 bs=1M count=10 conv=fsync [2022-10-17 20:56:06,723][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 20:56:06,723][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0190878 s, 549 MB/s [2022-10-17 20:56:06,725][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:06,725][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-3d293c18-f62f-4f92-a77d-69b681841c34 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:06,842][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34,ceph.block_uuid=XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=3d293c18-f62f-4f92-a77d-69b681841c34,ceph.osd_id=2,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-3d293c18-f62f-4f92-a77d-69b681841c34/osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"osd-block-3d293c18-f62f-4f92-a77d-69b681841c34";"ceph-3d293c18-f62f-4f92-a77d-69b681841c34";"XODtv6-nnC2-jh8U-cXVd-cgkz-z1IW-N935Qw";"6401233518592 [2022-10-17 20:56:06,842][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,843][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:56:06,844][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-3d293c18-f62f-4f92-a77d-69b681841c34 [2022-10-17 20:56:06,870][ceph_volume.process][INFO ] stderr Removing ceph--3d293c18--f62f--4f92--a77d--69b681841c34-osd--block--3d293c18--f62f--4f92--a77d--69b681841c34 (253:8) [2022-10-17 20:56:06,913][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-3d293c18-f62f-4f92-a77d-69b681841c34" metadata (seqno 3). [2022-10-17 20:56:06,913][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-3d293c18-f62f-4f92-a77d-69b681841c34" [2022-10-17 20:56:06,914][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-3d293c18-f62f-4f92-a77d-69b681841c34" (seqno 4). [2022-10-17 20:56:06,915][ceph_volume.process][INFO ] stdout Logical volume "osd-block-3d293c18-f62f-4f92-a77d-69b681841c34" successfully removed [2022-10-17 20:56:06,915][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34" from volume group "ceph-3d293c18-f62f-4f92-a77d-69b681841c34" [2022-10-17 20:56:06,915][ceph_volume.process][INFO ] stdout Volume group "ceph-3d293c18-f62f-4f92-a77d-69b681841c34" successfully removed [2022-10-17 20:56:06,998][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-2/ --osd-uuid 3d293c18-f62f-4f92-a77d-69b681841c34 --setuser ceph --setgroup ceph [2022-10-17 20:56:27,705][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --bluestore --data ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:27,706][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 crypt [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/dm-9 /dev/mapper/ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 lvm [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:56:27,711][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:56:27,712][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:56:27,717][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:56:27,727][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:27,727][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,vg_name=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:27,854][ceph_volume.process][INFO ] stdout ";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:27,854][ceph_volume.util.disk][INFO ] opening device /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 to check for BlueStore label [2022-10-17 20:56:27,854][ceph_volume.util.disk][INFO ] opening device /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 to check for BlueStore label [2022-10-17 20:56:27,855][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-9 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-9 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DISKSEQ=42 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout MINOR=9 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13097854420 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_NAME=ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_UUID=LVM-xrVpkgdtWA1dUQbcO75wZTwlAwe75K35CAMDgWeh2nrVLije9sNgQM6jilLAeQD0 [2022-10-17 20:56:27,858][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/disk/by-id/dm-uuid-LVM-xrVpkgdtWA1dUQbcO75wZTwlAwe75K35CAMDgWeh2nrVLije9sNgQM6jilLAeQD0 /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 /dev/disk/by-id/dm-name-ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 /dev/mapper/ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 [2022-10-17 20:56:27,859][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:56:27,860][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:27,860][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,vg_name=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:27,937][ceph_volume.process][INFO ] stdout ";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:27,937][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:56:27,938][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:56:27,948][ceph_volume.process][INFO ] stdout AQB7wU1jLK18OBAAzSbarfFCKLJ67dBZ2wDjdg== [2022-10-17 20:56:27,948][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:28,161][ceph_volume.process][INFO ] stdout 0 [2022-10-17 20:56:28,161][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.001+0000 7f0d9750f700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:56:28,161][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.001+0000 7f0d9750f700 -1 AuthRegistry(0x7f0d9005f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:28,163][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:28,163][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,vg_name=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:28,249][ceph_volume.process][INFO ] stdout ";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:28,251][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:56:28,251][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --addtag ceph.osd_id=0 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --addtag ceph.block_uuid=CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0 --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:28,341][ceph_volume.process][INFO ] stdout Logical volume ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 changed. [2022-10-17 20:56:28,342][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:56:28,352][ceph_volume.process][INFO ] stdout AQB8wU1jsfD3FBAA7kqtHfzj1lPakfWGzcNosA== [2022-10-17 20:56:28,352][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 [2022-10-17 20:56:28,354][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:56:28,354][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:56:28,355][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:56:28,356][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:28,357][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-9 [2022-10-17 20:56:28,359][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 /var/lib/ceph/osd/ceph-0/block [2022-10-17 20:56:28,360][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap [2022-10-17 20:56:28,420][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.413+0000 7f0893eb4700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:56:28.413+0000 7f0893eb4700 -1 AuthRegistry(0x7f088c05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:28,591][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:56:28,599][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQB7wU1jLK18OBAAzSbarfFCKLJ67dBZ2wDjdg== [2022-10-17 20:56:28,611][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-0/keyring added entity osd.0 auth(key=AQB7wU1jLK18OBAAzSbarfFCKLJ67dBZ2wDjdg==) [2022-10-17 20:56:28,612][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring [2022-10-17 20:56:28,614][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ [2022-10-17 20:56:28,615][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --setuser ceph --setgroup ceph [2022-10-17 20:56:28,639][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.629+0000 7f363b3255c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [2022-10-17 20:56:28,639][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.629+0000 7f363b3255c0 -1 bdev(0x56406368de30 /var/lib/ceph/osd/ceph-0//block) open failed to lock /var/lib/ceph/osd/ceph-0//block: (13) Permission denied [2022-10-17 20:56:28,639][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.629+0000 7f363b3255c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (13) Permission denied [2022-10-17 20:56:28,639][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.629+0000 7f363b3255c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:56:28,639][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.629+0000 7f363b3255c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0/: (13) Permission denied [2022-10-17 20:56:28,639][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --setuser ceph --setgroup ceph [2022-10-17 20:56:28,640][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:56:28,640][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it [2022-10-17 20:56:28,711][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:28.701+0000 7fdbcaf62700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:56:28.701+0000 7fdbcaf62700 -1 AuthRegistry(0x7fdbc405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:56:28,857][ceph_volume.process][INFO ] stderr purged osd.0 [2022-10-17 20:56:28,866][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@0 [2022-10-17 20:56:28,871][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:56:28,872][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:28,872][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:28,934][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.block_uuid=CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:28,935][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:28,936][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,007][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:29,007][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,075][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:29,076][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,155][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:29,155][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,222][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.block_uuid=CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:29,222][ceph_volume.util.disk][INFO ] opening device /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 to check for BlueStore label [2022-10-17 20:56:29,222][ceph_volume.util.disk][INFO ] opening device /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 to check for BlueStore label [2022-10-17 20:56:29,223][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-9 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-9 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DISKSEQ=42 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout MINOR=9 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13097854420 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_NAME=ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_UUID=LVM-xrVpkgdtWA1dUQbcO75wZTwlAwe75K35CAMDgWeh2nrVLije9sNgQM6jilLAeQD0 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/disk/by-id/dm-uuid-LVM-xrVpkgdtWA1dUQbcO75wZTwlAwe75K35CAMDgWeh2nrVLije9sNgQM6jilLAeQD0 /dev/disk/by-id/dm-name-ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 /dev/mapper/ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,226][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:56:29,226][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,228][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:56:29,228][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,314][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2". [2022-10-17 20:56:29,315][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:29,315][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,vg_name=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,390][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.block_uuid=CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:29,391][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-0 [2022-10-17 20:56:29,391][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-0 [2022-10-17 20:56:29,393][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-0 unmounted [2022-10-17 20:56:29,393][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,399][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 bs=1M count=10 conv=fsync [2022-10-17 20:56:29,417][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 20:56:29,418][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0173944 s, 603 MB/s [2022-10-17 20:56:29,419][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:29,420][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:29,541][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.block_uuid=CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=fe2f7bfd-5034-443e-9ecc-a28b588f14b2,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2/osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2";"CAMDgW-eh2n-rVLi-je9s-NgQM-6jil-LAeQD0";"6401233518592 [2022-10-17 20:56:29,542][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,543][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:56:29,543][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 [2022-10-17 20:56:29,571][ceph_volume.process][INFO ] stderr Removing ceph--fe2f7bfd--5034--443e--9ecc--a28b588f14b2-osd--block--fe2f7bfd--5034--443e--9ecc--a28b588f14b2 (253:9) [2022-10-17 20:56:29,617][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" metadata (seqno 3). [2022-10-17 20:56:29,617][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" [2022-10-17 20:56:29,618][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" (seqno 4). [2022-10-17 20:56:29,618][ceph_volume.process][INFO ] stdout Logical volume "osd-block-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" successfully removed [2022-10-17 20:56:29,618][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" from volume group "ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" [2022-10-17 20:56:29,619][ceph_volume.process][INFO ] stdout Volume group "ceph-fe2f7bfd-5034-443e-9ecc-a28b588f14b2" successfully removed [2022-10-17 20:56:29,710][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid fe2f7bfd-5034-443e-9ecc-a28b588f14b2 --setuser ceph --setgroup ceph [2022-10-17 20:56:59,719][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid 5a6d2803-21f9-41d2-a134-decdb9ad2c90 --bluestore --data ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:56:59,720][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:56:59,724][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 crypt [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-9 /dev/mapper/crypt-5a6d2803-21f9-41d2-a134-decdb9ad2c90 crypt [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/dm-10 /dev/mapper/ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 lvm [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:56:59,725][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:56:59,731][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:56:59,738][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:59,738][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,vg_name=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:59,858][ceph_volume.process][INFO ] stdout ";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:56:59,858][ceph_volume.util.disk][INFO ] opening device /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 to check for BlueStore label [2022-10-17 20:56:59,859][ceph_volume.util.disk][INFO ] opening device /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 to check for BlueStore label [2022-10-17 20:56:59,859][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-10 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-10 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DISKSEQ=44 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout MINOR=10 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13129871306 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_NAME=ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_UUID=LVM-I4zIt7ldtgKB61FWvHwMIYNTxex1yGMhggM36qTea74wIq6HXCkZk0ZmaJ4tJ3bH [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/disk/by-id/dm-name-ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 /dev/mapper/ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 /dev/disk/by-id/dm-uuid-LVM-I4zIt7ldtgKB61FWvHwMIYNTxex1yGMhggM36qTea74wIq6HXCkZk0ZmaJ4tJ3bH [2022-10-17 20:56:59,863][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:56:59,865][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:56:59,865][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,vg_name=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:56:59,925][ceph_volume.process][INFO ] stdout ";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:56:59,926][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:56:59,926][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:56:59,936][ceph_volume.process][INFO ] stdout AQCbwU1j4DDJNxAAnhc/RXsxOkEY6mCqKAnzEQ== [2022-10-17 20:56:59,937][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:00,150][ceph_volume.process][INFO ] stdout 1 [2022-10-17 20:57:00,150][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:59.990+0000 7f1ecd8ae700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:57:00,150][ceph_volume.process][INFO ] stderr 2022-10-17T20:56:59.990+0000 7f1ecd8ae700 -1 AuthRegistry(0x7f1ec805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:00,152][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:00,152][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,vg_name=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:00,229][ceph_volume.process][INFO ] stdout ";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:57:00,231][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:57:00,232][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=5a6d2803-21f9-41d2-a134-decdb9ad2c90 --addtag ceph.osd_id=1 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 --addtag ceph.block_uuid=ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:00,330][ceph_volume.process][INFO ] stdout Logical volume ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 changed. [2022-10-17 20:57:00,330][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:57:00,341][ceph_volume.process][INFO ] stdout AQCcwU1jqr9RFBAASI54vvMU079J6Y8h9I4l+g== [2022-10-17 20:57:00,342][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1 [2022-10-17 20:57:00,343][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:57:00,344][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:57:00,345][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:57:00,345][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:00,346][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-10 [2022-10-17 20:57:00,348][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 /var/lib/ceph/osd/ceph-1/block [2022-10-17 20:57:00,349][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap [2022-10-17 20:57:00,410][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.402+0000 7fb0c1a04700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:57:00.402+0000 7fb0c1a04700 -1 AuthRegistry(0x7fb0bc05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:00,578][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:57:00,586][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-1/keyring --create-keyring --name osd.1 --add-key AQCbwU1j4DDJNxAAnhc/RXsxOkEY6mCqKAnzEQ== [2022-10-17 20:57:00,599][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-1/keyring added entity osd.1 auth(key=AQCbwU1j4DDJNxAAnhc/RXsxOkEY6mCqKAnzEQ==) [2022-10-17 20:57:00,599][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring [2022-10-17 20:57:00,601][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/ [2022-10-17 20:57:00,603][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 5a6d2803-21f9-41d2-a134-decdb9ad2c90 --setuser ceph --setgroup ceph [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.618+0000 7f01bdb105c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.618+0000 7f01bdb105c0 -1 bdev(0x564212544cd0 /var/lib/ceph/osd/ceph-1//block) open failed to lock /var/lib/ceph/osd/ceph-1//block: (13) Permission denied [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.618+0000 7f01bdb105c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/) mkfs failed, (13) Permission denied [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.618+0000 7f01bdb105c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.618+0000 7f01bdb105c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-1/: (13) Permission denied [2022-10-17 20:57:00,627][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 5a6d2803-21f9-41d2-a134-decdb9ad2c90 --setuser ceph --setgroup ceph [2022-10-17 20:57:00,627][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:57:00,627][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.1 --yes-i-really-mean-it [2022-10-17 20:57:00,698][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:00.690+0000 7fea041cf700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:57:00.690+0000 7fea041cf700 -1 AuthRegistry(0x7fe9fc05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:00,847][ceph_volume.process][INFO ] stderr purged osd.1 [2022-10-17 20:57:00,856][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@1 [2022-10-17 20:57:00,862][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:57:00,863][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:00,864][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:00,929][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.block_uuid=ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:57:00,931][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:00,931][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,011][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:01,011][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,087][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:01,087][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=1,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,155][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:01,156][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,221][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.block_uuid=ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:57:01,222][ceph_volume.util.disk][INFO ] opening device /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 to check for BlueStore label [2022-10-17 20:57:01,222][ceph_volume.util.disk][INFO ] opening device /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 to check for BlueStore label [2022-10-17 20:57:01,223][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-10 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-10 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DISKSEQ=44 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout MINOR=10 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13129871306 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_NAME=ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_UUID=LVM-I4zIt7ldtgKB61FWvHwMIYNTxex1yGMhggM36qTea74wIq6HXCkZk0ZmaJ4tJ3bH [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/disk/by-id/dm-uuid-LVM-I4zIt7ldtgKB61FWvHwMIYNTxex1yGMhggM36qTea74wIq6HXCkZk0ZmaJ4tJ3bH /dev/mapper/ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 /dev/disk/by-id/dm-name-ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 [2022-10-17 20:57:01,226][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:57:01,226][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,228][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:57:01,228][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,297][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90". [2022-10-17 20:57:01,299][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:01,299][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,vg_name=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,362][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.block_uuid=ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:57:01,363][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-1 [2022-10-17 20:57:01,363][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-1 [2022-10-17 20:57:01,365][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-1 unmounted [2022-10-17 20:57:01,365][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,370][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90 bs=1M count=10 conv=fsync [2022-10-17 20:57:01,388][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0164301 s, 638 MB/s [2022-10-17 20:57:01,390][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:01,390][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:01,478][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.block_uuid=ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=5a6d2803-21f9-41d2-a134-decdb9ad2c90,ceph.osd_id=1,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90/osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90";"ggM36q-Tea7-4wIq-6HXC-kZk0-ZmaJ-4tJ3bH";"6401233518592 [2022-10-17 20:57:01,478][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,479][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:57:01,479][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90 [2022-10-17 20:57:01,507][ceph_volume.process][INFO ] stderr Removing ceph--5a6d2803--21f9--41d2--a134--decdb9ad2c90-osd--block--5a6d2803--21f9--41d2--a134--decdb9ad2c90 (253:10) [2022-10-17 20:57:01,549][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90" metadata (seqno 3). [2022-10-17 20:57:01,549][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90" [2022-10-17 20:57:01,550][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90" (seqno 4). [2022-10-17 20:57:01,550][ceph_volume.process][INFO ] stdout Logical volume "osd-block-5a6d2803-21f9-41d2-a134-decdb9ad2c90" successfully removed [2022-10-17 20:57:01,550][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-5a6d2803-21f9-41d2-a134-decdb9ad2c90" from volume group "ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90" [2022-10-17 20:57:01,550][ceph_volume.process][INFO ] stdout Volume group "ceph-5a6d2803-21f9-41d2-a134-decdb9ad2c90" successfully removed [2022-10-17 20:57:01,610][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid 5a6d2803-21f9-41d2-a134-decdb9ad2c90 --setuser ceph --setgroup ceph [2022-10-17 20:57:50,226][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid 90961911-85ba-445d-abb9-548dd4047e1e --bluestore --data ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,227][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:57:50,231][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 crypt [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-9 /dev/mapper/crypt-5a6d2803-21f9-41d2-a134-decdb9ad2c90 crypt [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-10 /dev/mapper/crypt-90961911-85ba-445d-abb9-548dd4047e1e crypt [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/dm-11 /dev/mapper/ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e lvm [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:57:50,232][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:57:50,238][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:57:50,245][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:50,245][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-90961911-85ba-445d-abb9-548dd4047e1e,vg_name=ceph-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:50,366][ceph_volume.process][INFO ] stdout ";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:50,366][ceph_volume.util.disk][INFO ] opening device /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e to check for BlueStore label [2022-10-17 20:57:50,366][ceph_volume.util.disk][INFO ] opening device /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e to check for BlueStore label [2022-10-17 20:57:50,367][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-11 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-11 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DISKSEQ=46 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout MINOR=11 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13180364388 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_NAME=ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_UUID=LVM-gpXR5e5ZcqStVWYX4nqh8rzyajnHEfA7IVoeHLMm0YzrCXYYC8J3rTCkIsxwbFkK [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/mapper/ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e /dev/disk/by-id/dm-name-ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e /dev/disk/by-id/dm-uuid-LVM-gpXR5e5ZcqStVWYX4nqh8rzyajnHEfA7IVoeHLMm0YzrCXYYC8J3rTCkIsxwbFkK [2022-10-17 20:57:50,370][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:57:50,372][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:50,372][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-90961911-85ba-445d-abb9-548dd4047e1e,vg_name=ceph-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:50,446][ceph_volume.process][INFO ] stdout ";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:50,446][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:57:50,446][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:57:50,457][ceph_volume.process][INFO ] stdout AQDOwU1j2KU4GxAAvqOc3tNQsUAsWzpJhI08XQ== [2022-10-17 20:57:50,457][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,671][ceph_volume.process][INFO ] stdout 0 [2022-10-17 20:57:50,672][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:50.511+0000 7f32cddc9700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:57:50,672][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:50.511+0000 7f32cddc9700 -1 AuthRegistry(0x7f32c805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:50,673][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:50,674][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-90961911-85ba-445d-abb9-548dd4047e1e,vg_name=ceph-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:50,750][ceph_volume.process][INFO ] stdout ";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:50,751][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:57:50,752][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=90961911-85ba-445d-abb9-548dd4047e1e --addtag ceph.osd_id=0 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e --addtag ceph.block_uuid=IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,849][ceph_volume.process][INFO ] stdout Logical volume ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e changed. [2022-10-17 20:57:50,850][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:57:50,860][ceph_volume.process][INFO ] stdout AQDOwU1jR0VAMxAAs9sN7aOE6m/4bTS+M5aqcA== [2022-10-17 20:57:50,860][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 [2022-10-17 20:57:50,862][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:57:50,862][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:57:50,863][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:57:50,864][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:50,865][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-11 [2022-10-17 20:57:50,867][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e /var/lib/ceph/osd/ceph-0/block [2022-10-17 20:57:50,868][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap [2022-10-17 20:57:50,929][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:50.923+0000 7f36cd8c1700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:57:50.923+0000 7f36cd8c1700 -1 AuthRegistry(0x7f36c805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:51,100][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:57:51,108][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQDOwU1j2KU4GxAAvqOc3tNQsUAsWzpJhI08XQ== [2022-10-17 20:57:51,120][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-0/keyring added entity osd.0 auth(key=AQDOwU1j2KU4GxAAvqOc3tNQsUAsWzpJhI08XQ==) [2022-10-17 20:57:51,121][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring [2022-10-17 20:57:51,122][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ [2022-10-17 20:57:51,124][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 90961911-85ba-445d-abb9-548dd4047e1e --setuser ceph --setgroup ceph [2022-10-17 20:57:51,147][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.139+0000 7f9b490265c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [2022-10-17 20:57:51,147][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.139+0000 7f9b490265c0 -1 bdev(0x55ba56290450 /var/lib/ceph/osd/ceph-0//block) open failed to lock /var/lib/ceph/osd/ceph-0//block: (13) Permission denied [2022-10-17 20:57:51,147][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.139+0000 7f9b490265c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (13) Permission denied [2022-10-17 20:57:51,147][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.139+0000 7f9b490265c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:57:51,147][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.139+0000 7f9b490265c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0/: (13) Permission denied [2022-10-17 20:57:51,147][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 90961911-85ba-445d-abb9-548dd4047e1e --setuser ceph --setgroup ceph [2022-10-17 20:57:51,148][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:57:51,148][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it [2022-10-17 20:57:51,223][ceph_volume.process][INFO ] stderr 2022-10-17T20:57:51.211+0000 7fa599711700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:57:51.211+0000 7fa599711700 -1 AuthRegistry(0x7fa59405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:57:51,407][ceph_volume.process][INFO ] stderr purged osd.0 [2022-10-17 20:57:51,416][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@0 [2022-10-17 20:57:51,421][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:57:51,422][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,423][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,490][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e,ceph.block_uuid=IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=90961911-85ba-445d-abb9-548dd4047e1e,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:51,491][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,491][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,563][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,563][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,651][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,651][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,723][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,723][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,794][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e,ceph.block_uuid=IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=90961911-85ba-445d-abb9-548dd4047e1e,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:51,794][ceph_volume.util.disk][INFO ] opening device /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e to check for BlueStore label [2022-10-17 20:57:51,794][ceph_volume.util.disk][INFO ] opening device /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e to check for BlueStore label [2022-10-17 20:57:51,795][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-11 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-11 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DISKSEQ=46 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout MINOR=11 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13180364388 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_NAME=ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_UUID=LVM-gpXR5e5ZcqStVWYX4nqh8rzyajnHEfA7IVoeHLMm0YzrCXYYC8J3rTCkIsxwbFkK [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e /dev/disk/by-id/dm-name-ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e /dev/mapper/ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e /dev/disk/by-id/dm-uuid-LVM-gpXR5e5ZcqStVWYX4nqh8rzyajnHEfA7IVoeHLMm0YzrCXYYC8J3rTCkIsxwbFkK [2022-10-17 20:57:51,798][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:57:51,799][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,800][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:57:51,800][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,890][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e". [2022-10-17 20:57:51,891][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:51,891][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-90961911-85ba-445d-abb9-548dd4047e1e,vg_name=ceph-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:51,974][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e,ceph.block_uuid=IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=90961911-85ba-445d-abb9-548dd4047e1e,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:51,975][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-0 [2022-10-17 20:57:51,975][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-0 [2022-10-17 20:57:51,977][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-0 unmounted [2022-10-17 20:57:51,977][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:51,983][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e bs=1M count=10 conv=fsync [2022-10-17 20:57:52,001][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 20:57:52,001][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0167729 s, 625 MB/s [2022-10-17 20:57:52,003][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:57:52,003][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-90961911-85ba-445d-abb9-548dd4047e1e -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:57:52,118][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e,ceph.block_uuid=IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=90961911-85ba-445d-abb9-548dd4047e1e,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-90961911-85ba-445d-abb9-548dd4047e1e/osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"osd-block-90961911-85ba-445d-abb9-548dd4047e1e";"ceph-90961911-85ba-445d-abb9-548dd4047e1e";"IVoeHL-Mm0Y-zrCX-YYC8-J3rT-CkIs-xwbFkK";"6401233518592 [2022-10-17 20:57:52,118][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:52,120][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:57:52,120][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-90961911-85ba-445d-abb9-548dd4047e1e [2022-10-17 20:57:52,148][ceph_volume.process][INFO ] stderr Removing ceph--90961911--85ba--445d--abb9--548dd4047e1e-osd--block--90961911--85ba--445d--abb9--548dd4047e1e (253:11) [2022-10-17 20:57:52,201][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-90961911-85ba-445d-abb9-548dd4047e1e" metadata (seqno 3). [2022-10-17 20:57:52,201][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-90961911-85ba-445d-abb9-548dd4047e1e" [2022-10-17 20:57:52,202][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-90961911-85ba-445d-abb9-548dd4047e1e" (seqno 4). [2022-10-17 20:57:52,203][ceph_volume.process][INFO ] stdout Logical volume "osd-block-90961911-85ba-445d-abb9-548dd4047e1e" successfully removed [2022-10-17 20:57:52,203][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-90961911-85ba-445d-abb9-548dd4047e1e" from volume group "ceph-90961911-85ba-445d-abb9-548dd4047e1e" [2022-10-17 20:57:52,203][ceph_volume.process][INFO ] stdout Volume group "ceph-90961911-85ba-445d-abb9-548dd4047e1e" successfully removed [2022-10-17 20:57:52,286][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 90961911-85ba-445d-abb9-548dd4047e1e --setuser ceph --setgroup ceph [2022-10-17 20:59:18,544][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid b30d5f86-ec7e-4915-81f3-bd5cb42a645b --bluestore --data ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:18,545][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 20:59:18,550][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 20:59:18,550][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 20:59:18,550][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-9 /dev/mapper/crypt-5a6d2803-21f9-41d2-a134-decdb9ad2c90 crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-10 /dev/mapper/crypt-90961911-85ba-445d-abb9-548dd4047e1e crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-11 /dev/mapper/crypt-b30d5f86-ec7e-4915-81f3-bd5cb42a645b crypt [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/dm-12 /dev/mapper/ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b lvm [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 20:59:18,551][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 20:59:18,557][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 20:59:18,564][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:18,564][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,vg_name=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:18,678][ceph_volume.process][INFO ] stdout ";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:18,678][ceph_volume.util.disk][INFO ] opening device /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b to check for BlueStore label [2022-10-17 20:59:18,678][ceph_volume.util.disk][INFO ] opening device /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b to check for BlueStore label [2022-10-17 20:59:18,679][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-12 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-12 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DISKSEQ=48 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout MINOR=12 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13268707297 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_NAME=ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_UUID=LVM-ek9Pi62N1n3353csL8gD95jBmIZrcXjFceWlWNVM5JZmi0ZYMr2mRLBf3MOsgff2 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/mapper/ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b /dev/disk/by-id/dm-name-ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b /dev/disk/by-id/dm-uuid-LVM-ek9Pi62N1n3353csL8gD95jBmIZrcXjFceWlWNVM5JZmi0ZYMr2mRLBf3MOsgff2 /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:18,682][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:59:18,684][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:18,684][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,vg_name=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:18,778][ceph_volume.process][INFO ] stdout ";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:18,778][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 20:59:18,778][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:59:18,789][ceph_volume.process][INFO ] stdout AQAmwk1jYmr+LhAAK7ZSMNySPGTe27LzYqP+GQ== [2022-10-17 20:59:18,789][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:19,007][ceph_volume.process][INFO ] stdout 0 [2022-10-17 20:59:19,008][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:18.844+0000 7f7b5011e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 20:59:19,008][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:18.844+0000 7f7b5011e700 -1 AuthRegistry(0x7f7b4805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:59:19,009][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:19,010][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,vg_name=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:19,097][ceph_volume.process][INFO ] stdout ";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:19,099][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 20:59:19,099][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=b30d5f86-ec7e-4915-81f3-bd5cb42a645b --addtag ceph.osd_id=0 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b --addtag ceph.block_uuid=ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2 --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:19,230][ceph_volume.process][INFO ] stdout Logical volume ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b changed. [2022-10-17 20:59:19,230][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 20:59:19,241][ceph_volume.process][INFO ] stdout AQAnwk1jtbdRDhAAplQyNv5f6V41VVzDLpWo3A== [2022-10-17 20:59:19,241][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 [2022-10-17 20:59:19,243][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 20:59:19,243][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 20:59:19,244][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 20:59:19,244][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:19,246][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-12 [2022-10-17 20:59:19,247][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b /var/lib/ceph/osd/ceph-0/block [2022-10-17 20:59:19,248][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap [2022-10-17 20:59:19,308][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.304+0000 7f04f235e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:59:19.304+0000 7f04f235e700 -1 AuthRegistry(0x7f04ec05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:59:19,476][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 20:59:19,484][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQAmwk1jYmr+LhAAK7ZSMNySPGTe27LzYqP+GQ== [2022-10-17 20:59:19,496][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-0/keyring added entity osd.0 auth(key=AQAmwk1jYmr+LhAAK7ZSMNySPGTe27LzYqP+GQ==) [2022-10-17 20:59:19,497][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring [2022-10-17 20:59:19,499][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ [2022-10-17 20:59:19,500][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid b30d5f86-ec7e-4915-81f3-bd5cb42a645b --setuser ceph --setgroup ceph [2022-10-17 20:59:19,523][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.516+0000 7f521a84c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [2022-10-17 20:59:19,523][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.516+0000 7f521a84c5c0 -1 bdev(0x55f4e2527d50 /var/lib/ceph/osd/ceph-0//block) open failed to lock /var/lib/ceph/osd/ceph-0//block: (13) Permission denied [2022-10-17 20:59:19,523][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.516+0000 7f521a84c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (13) Permission denied [2022-10-17 20:59:19,523][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.516+0000 7f521a84c5c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 20:59:19,523][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.516+0000 7f521a84c5c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0/: (13) Permission denied [2022-10-17 20:59:19,523][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid b30d5f86-ec7e-4915-81f3-bd5cb42a645b --setuser ceph --setgroup ceph [2022-10-17 20:59:19,524][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 20:59:19,524][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it [2022-10-17 20:59:19,593][ceph_volume.process][INFO ] stderr 2022-10-17T20:59:19.588+0000 7f53405ce700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T20:59:19.588+0000 7f53405ce700 -1 AuthRegistry(0x7f533805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 20:59:19,737][ceph_volume.process][INFO ] stderr purged osd.0 [2022-10-17 20:59:19,746][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@0 [2022-10-17 20:59:19,751][ceph_volume.process][INFO ] stdout inactive [2022-10-17 20:59:19,752][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:19,753][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:19,838][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.block_uuid=ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:19,840][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:19,840][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:19,927][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:19,927][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:20,011][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:20,011][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:20,107][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:20,107][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:20,169][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.block_uuid=ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:20,170][ceph_volume.util.disk][INFO ] opening device /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b to check for BlueStore label [2022-10-17 20:59:20,170][ceph_volume.util.disk][INFO ] opening device /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b to check for BlueStore label [2022-10-17 20:59:20,170][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,173][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-12 [2022-10-17 20:59:20,173][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-12 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DISKSEQ=48 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout MINOR=12 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13268707297 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_NAME=ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_UUID=LVM-ek9Pi62N1n3353csL8gD95jBmIZrcXjFceWlWNVM5JZmi0ZYMr2mRLBf3MOsgff2 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b /dev/disk/by-id/dm-name-ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b /dev/mapper/ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b /dev/disk/by-id/dm-uuid-LVM-ek9Pi62N1n3353csL8gD95jBmIZrcXjFceWlWNVM5JZmi0ZYMr2mRLBf3MOsgff2 [2022-10-17 20:59:20,174][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 20:59:20,174][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,175][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 20:59:20,176][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,253][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b". [2022-10-17 20:59:20,255][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:20,255][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,vg_name=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:20,342][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.block_uuid=ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:20,343][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-0 [2022-10-17 20:59:20,343][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-0 [2022-10-17 20:59:20,345][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-0 unmounted [2022-10-17 20:59:20,345][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,351][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b bs=1M count=10 conv=fsync [2022-10-17 20:59:20,370][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 20:59:20,370][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0174581 s, 601 MB/s [2022-10-17 20:59:20,371][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 20:59:20,371][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 20:59:20,486][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.block_uuid=ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=b30d5f86-ec7e-4915-81f3-bd5cb42a645b,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b/osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b";"ceWlWN-VM5J-Zmi0-ZYMr-2mRL-Bf3M-Osgff2";"6401233518592 [2022-10-17 20:59:20,486][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,488][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 20:59:20,488][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b [2022-10-17 20:59:20,517][ceph_volume.process][INFO ] stderr Removing ceph--b30d5f86--ec7e--4915--81f3--bd5cb42a645b-osd--block--b30d5f86--ec7e--4915--81f3--bd5cb42a645b (253:12) [2022-10-17 20:59:20,565][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" metadata (seqno 3). [2022-10-17 20:59:20,565][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" [2022-10-17 20:59:20,566][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" (seqno 4). [2022-10-17 20:59:20,567][ceph_volume.process][INFO ] stdout Logical volume "osd-block-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" successfully removed [2022-10-17 20:59:20,567][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" from volume group "ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" [2022-10-17 20:59:20,567][ceph_volume.process][INFO ] stdout Volume group "ceph-b30d5f86-ec7e-4915-81f3-bd5cb42a645b" successfully removed [2022-10-17 20:59:20,646][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid b30d5f86-ec7e-4915-81f3-bd5cb42a645b --setuser ceph --setgroup ceph [2022-10-17 21:02:04,624][ceph_volume.main][INFO ] Running command: ceph-volume lvm create --osd-fsid 8b0cd318-f5ca-498e-83fe-15102f631eb4 --bluestore --data ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:04,625][ceph_volume.process][INFO ] Running command: /usr/bin/lsblk -plno KNAME,NAME,TYPE [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop0 /dev/loop0 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop1 /dev/loop1 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop2 /dev/loop2 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop3 /dev/loop3 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop4 /dev/loop4 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop5 /dev/loop5 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/loop6 /dev/loop6 loop [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sda /dev/sda disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdb /dev/sdb disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdc /dev/sdc disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdd /dev/sdd disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sde /dev/sde disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdf /dev/sdf disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdg /dev/sdg disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdh /dev/sdh disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdi /dev/sdi disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdj /dev/sdj disk [2022-10-17 21:02:04,630][ceph_volume.process][INFO ] stdout /dev/sdk /dev/sdk disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/sdk1 /dev/sdk1 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/sdk2 /dev/sdk2 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache0 /dev/bcache0 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache1 /dev/bcache1 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-0 /dev/mapper/vg0-lvhome lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-1 /dev/mapper/vg0-lvvaraudit lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-2 /dev/mapper/vg0-lvvarlog lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-3 /dev/mapper/vg0-lvvartmp lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-4 /dev/mapper/vg0-lvvar lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-5 /dev/mapper/vg0-lvroot lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-6 /dev/mapper/crypt-1fcf7d20-a470-4cba-8b76-fca489b0aee9 crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-7 /dev/mapper/crypt-3d293c18-f62f-4f92-a77d-69b681841c34 crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-8 /dev/mapper/crypt-fe2f7bfd-5034-443e-9ecc-a28b588f14b2 crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-9 /dev/mapper/crypt-5a6d2803-21f9-41d2-a134-decdb9ad2c90 crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-10 /dev/mapper/crypt-90961911-85ba-445d-abb9-548dd4047e1e crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-11 /dev/mapper/crypt-b30d5f86-ec7e-4915-81f3-bd5cb42a645b crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-12 /dev/mapper/crypt-8b0cd318-f5ca-498e-83fe-15102f631eb4 crypt [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/dm-13 /dev/mapper/ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 lvm [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1 /dev/nvme0n1 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p1 /dev/nvme0n1p1 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p2 /dev/nvme0n1p2 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p3 /dev/nvme0n1p3 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p4 /dev/nvme0n1p4 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p5 /dev/nvme0n1p5 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p6 /dev/nvme0n1p6 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p7 /dev/nvme0n1p7 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p8 /dev/nvme0n1p8 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p9 /dev/nvme0n1p9 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/nvme0n1p10 /dev/nvme0n1p10 part [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache2 /dev/bcache2 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache3 /dev/bcache3 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache4 /dev/bcache4 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache5 /dev/bcache5 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache6 /dev/bcache6 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache7 /dev/bcache7 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache8 /dev/bcache8 disk [2022-10-17 21:02:04,631][ceph_volume.process][INFO ] stdout /dev/bcache9 /dev/bcache9 disk [2022-10-17 21:02:04,637][ceph_volume.util.system][ERROR ] Failed to read contents from: /sys/block/sdk/device/sas_address Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/util/system.py", line 124, in get_file_contents contents = open_file.read().strip() OSError: [Errno 19] No such device [2022-10-17 21:02:04,644][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:04,644][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,vg_name=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:04,746][ceph_volume.process][INFO ] stdout ";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:04,746][ceph_volume.util.disk][INFO ] opening device /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 to check for BlueStore label [2022-10-17 21:02:04,746][ceph_volume.util.disk][INFO ] opening device /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 to check for BlueStore label [2022-10-17 21:02:04,747][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-13 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-13 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DISKSEQ=50 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout MINOR=13 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13434768902 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_NAME=ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_UUID=LVM-Idg0ngTz1Eg7GfVMJDGQfcaPCLL4BvnjeVmRa3ypZRavh5qtfIHQwPxOrfjnkRPb [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 21:02:04,750][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 21:02:04,751][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 /dev/disk/by-id/dm-name-ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 /dev/disk/by-id/dm-uuid-LVM-Idg0ngTz1Eg7GfVMJDGQfcaPCLL4BvnjeVmRa3ypZRavh5qtfIHQwPxOrfjnkRPb /dev/mapper/ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 [2022-10-17 21:02:04,751][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 21:02:04,752][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:04,753][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,vg_name=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:04,833][ceph_volume.process][INFO ] stdout ";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:04,834][ceph_volume.api.lvm][WARNING] device is not part of ceph: [2022-10-17 21:02:04,834][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 21:02:04,845][ceph_volume.process][INFO ] stdout AQDMwk1jiQVSMhAAyRilQl5Rs2f5veU10ySlNg== [2022-10-17 21:02:04,845][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:05,060][ceph_volume.process][INFO ] stdout 0 [2022-10-17 21:02:05,061][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:04.899+0000 7f8e49956700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory [2022-10-17 21:02:05,061][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:04.899+0000 7f8e49956700 -1 AuthRegistry(0x7f8e4405f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 21:02:05,063][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:05,063][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,vg_name=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:05,149][ceph_volume.process][INFO ] stdout ";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:05,151][ceph_volume.util.system][WARNING] Executable lvchange not found on the host, will return lvchange as-is [2022-10-17 21:02:05,151][ceph_volume.process][INFO ] Running command: lvchange --addtag ceph.osd_fsid=8b0cd318-f5ca-498e-83fe-15102f631eb4 --addtag ceph.osd_id=0 --addtag ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013 --addtag ceph.cluster_name=ceph --addtag ceph.crush_device_class= --addtag ceph.osdspec_affinity= --addtag ceph.block_device=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 --addtag ceph.block_uuid=eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb --addtag ceph.cephx_lockbox_secret= --addtag ceph.encrypted=0 --addtag ceph.vdo=0 --addtag ceph.type=block /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:05,238][ceph_volume.process][INFO ] stdout Logical volume ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 changed. [2022-10-17 21:02:05,238][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool --gen-print-key [2022-10-17 21:02:05,249][ceph_volume.process][INFO ] stdout AQDNwk1jfHjODhAAxMKF7CG+jAYc2FaaBjSyNg== [2022-10-17 21:02:05,249][ceph_volume.process][INFO ] Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 [2022-10-17 21:02:05,251][ceph_volume.util.system][WARNING] Executable selinuxenabled not in PATH: /var/lib/juju/tools/unit-ceph-osd-0:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin [2022-10-17 21:02:05,251][ceph_volume.process][INFO ] Running command: selinuxenabled [2022-10-17 21:02:05,252][ceph_volume.util.system][INFO ] No SELinux found, skipping call to restorecon [2022-10-17 21:02:05,252][ceph_volume.process][INFO ] Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:05,254][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /dev/dm-13 [2022-10-17 21:02:05,255][ceph_volume.process][INFO ] Running command: /usr/bin/ln -s /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 /var/lib/ceph/osd/ceph-0/block [2022-10-17 21:02:05,257][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap [2022-10-17 21:02:05,318][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.311+0000 7fdaa3624700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T21:02:05.311+0000 7fdaa3624700 -1 AuthRegistry(0x7fda9c05f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 21:02:05,489][ceph_volume.process][INFO ] stderr got monmap epoch 2 [2022-10-17 21:02:05,497][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQDMwk1jiQVSMhAAyRilQl5Rs2f5veU10ySlNg== [2022-10-17 21:02:05,511][ceph_volume.process][INFO ] stdout creating /var/lib/ceph/osd/ceph-0/keyring added entity osd.0 auth(key=AQDMwk1jiQVSMhAAyRilQl5Rs2f5veU10ySlNg==) [2022-10-17 21:02:05,511][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring [2022-10-17 21:02:05,513][ceph_volume.process][INFO ] Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ [2022-10-17 21:02:05,515][ceph_volume.process][INFO ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 8b0cd318-f5ca-498e-83fe-15102f631eb4 --setuser ceph --setgroup ceph [2022-10-17 21:02:05,538][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.531+0000 7fa231e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [2022-10-17 21:02:05,538][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.531+0000 7fa231e8f5c0 -1 bdev(0x560605a3de70 /var/lib/ceph/osd/ceph-0//block) open failed to lock /var/lib/ceph/osd/ceph-0//block: (13) Permission denied [2022-10-17 21:02:05,538][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.531+0000 7fa231e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (13) Permission denied [2022-10-17 21:02:05,538][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.531+0000 7fa231e8f5c0 -1 OSD::mkfs: ObjectStore::mkfs failed with error (13) Permission denied [2022-10-17 21:02:05,538][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.531+0000 7fa231e8f5c0 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0/: (13) Permission denied [2022-10-17 21:02:05,538][ceph_volume.devices.lvm.prepare][ERROR ] lvm prepare was unable to complete Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 8b0cd318-f5ca-498e-83fe-15102f631eb4 --setuser ceph --setgroup ceph [2022-10-17 21:02:05,539][ceph_volume.devices.lvm.prepare][INFO ] will rollback OSD ID creation [2022-10-17 21:02:05,539][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it [2022-10-17 21:02:05,608][ceph_volume.process][INFO ] stderr 2022-10-17T21:02:05.599+0000 7ff3ee734700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory 2022-10-17T21:02:05.599+0000 7ff3ee734700 -1 AuthRegistry(0x7ff3e805f5a0) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin, disabling cephx [2022-10-17 21:02:05,753][ceph_volume.process][INFO ] stderr purged osd.0 [2022-10-17 21:02:05,762][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@0 [2022-10-17 21:02:05,767][ceph_volume.process][INFO ] stdout inactive [2022-10-17 21:02:05,769][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:05,769][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:05,858][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.block_uuid=eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:05,859][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:05,860][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=journal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:05,931][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:05,931][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=db} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:06,015][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:06,015][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=0,ceph.type=wal} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:06,095][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:06,095][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_path=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:06,185][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.block_uuid=eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:06,186][ceph_volume.util.disk][INFO ] opening device /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 to check for BlueStore label [2022-10-17 21:02:06,186][ceph_volume.util.disk][INFO ] opening device /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 to check for BlueStore label [2022-10-17 21:02:06,186][ceph_volume.process][INFO ] Running command: /usr/bin/udevadm info --query=property /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DEVPATH=/devices/virtual/block/dm-13 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DEVNAME=/dev/dm-13 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DEVTYPE=disk [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DISKSEQ=50 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout MAJOR=253 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout MINOR=13 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout SUBSYSTEM=block [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout USEC_INITIALIZED=13434768902 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_UDEV_DISABLE_LIBRARY_FALLBACK_FLAG=1 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_UDEV_PRIMARY_SOURCE_FLAG=1 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_UDEV_RULES=1 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_UDEV_RULES_VSN=2 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_NAME=ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_UUID=LVM-Idg0ngTz1Eg7GfVMJDGQfcaPCLL4BvnjeVmRa3ypZRavh5qtfIHQwPxOrfjnkRPb [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_SUSPENDED=0 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_VG_NAME=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_LV_NAME=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_TABLE_STATE=LIVE [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DM_STATE=ACTIVE [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout DEVLINKS=/dev/mapper/ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 /dev/disk/by-id/dm-uuid-LVM-Idg0ngTz1Eg7GfVMJDGQfcaPCLL4BvnjeVmRa3ypZRavh5qtfIHQwPxOrfjnkRPb /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 /dev/disk/by-id/dm-name-ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 [2022-10-17 21:02:06,190][ceph_volume.process][INFO ] stdout TAGS=:systemd: [2022-10-17 21:02:06,190][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,192][ceph_volume.util.system][WARNING] Executable pvs not found on the host, will return pvs as-is [2022-10-17 21:02:06,192][ceph_volume.process][INFO ] Running command: pvs --noheadings --readonly --units=b --nosuffix --separator=";" -o vg_name,pv_count,lv_count,vg_attr,vg_extent_count,vg_free_count,vg_extent_size /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,269][ceph_volume.process][INFO ] stderr Failed to find device for physical volume "/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4". [2022-10-17 21:02:06,271][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:06,271][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S lv_name=osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,vg_name=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:06,353][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.block_uuid=eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:06,355][ceph_volume.devices.lvm.zap][INFO ] Unmounting /var/lib/ceph/osd/ceph-0 [2022-10-17 21:02:06,355][ceph_volume.process][INFO ] Running command: /usr/bin/umount -v /var/lib/ceph/osd/ceph-0 [2022-10-17 21:02:06,357][ceph_volume.process][INFO ] stderr umount: /var/lib/ceph/osd/ceph-0 unmounted [2022-10-17 21:02:06,357][ceph_volume.process][INFO ] Running command: /usr/sbin/wipefs --all /dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,362][ceph_volume.process][INFO ] Running command: /usr/bin/dd if=/dev/zero of=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4 bs=1M count=10 conv=fsync [2022-10-17 21:02:06,381][ceph_volume.process][INFO ] stderr 10+0 records in 10+0 records out [2022-10-17 21:02:06,381][ceph_volume.process][INFO ] stderr 10485760 bytes (10 MB, 10 MiB) copied, 0.0173354 s, 605 MB/s [2022-10-17 21:02:06,383][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is [2022-10-17 21:02:06,383][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S vg_name=ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size [2022-10-17 21:02:06,490][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.block_uuid=eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb,ceph.cephx_lockbox_secret=,ceph.cluster_fsid=7cbab256-4e46-11ed-addd-13c01276f013,ceph.cluster_name=ceph,ceph.crush_device_class=,ceph.encrypted=0,ceph.osd_fsid=8b0cd318-f5ca-498e-83fe-15102f631eb4,ceph.osd_id=0,ceph.osdspec_affinity=,ceph.type=block,ceph.vdo=0";"/dev/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4/osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4";"ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4";"eVmRa3-ypZR-avh5-qtfI-HQwP-xOrf-jnkRPb";"6401233518592 [2022-10-17 21:02:06,490][ceph_volume.devices.lvm.zap][INFO ] Only 1 LV left in VG, will proceed to destroy volume group ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,492][ceph_volume.util.system][WARNING] Executable vgremove not found on the host, will return vgremove as-is [2022-10-17 21:02:06,492][ceph_volume.process][INFO ] Running command: vgremove -v -f ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4 [2022-10-17 21:02:06,521][ceph_volume.process][INFO ] stderr Removing ceph--8b0cd318--f5ca--498e--83fe--15102f631eb4-osd--block--8b0cd318--f5ca--498e--83fe--15102f631eb4 (253:13) [2022-10-17 21:02:06,569][ceph_volume.process][INFO ] stderr Archiving volume group "ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4" metadata (seqno 3). [2022-10-17 21:02:06,569][ceph_volume.process][INFO ] stderr Releasing logical volume "osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4" [2022-10-17 21:02:06,570][ceph_volume.process][INFO ] stderr Creating volume group backup "/etc/lvm/backup/ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4" (seqno 4). [2022-10-17 21:02:06,570][ceph_volume.process][INFO ] stdout Logical volume "osd-block-8b0cd318-f5ca-498e-83fe-15102f631eb4" successfully removed [2022-10-17 21:02:06,570][ceph_volume.process][INFO ] stderr Removing physical volume "/dev/mapper/crypt-8b0cd318-f5ca-498e-83fe-15102f631eb4" from volume group "ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4" [2022-10-17 21:02:06,570][ceph_volume.process][INFO ] stdout Volume group "ceph-8b0cd318-f5ca-498e-83fe-15102f631eb4" successfully removed [2022-10-17 21:02:06,650][ceph_volume][ERROR ] exception caught by decorator Traceback (most recent call last): File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 59, in newfunc return f(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/main.py", line 46, in main terminal.dispatch(self.mapper, self.argv) File "/usr/lib/python3/dist-packages/ceph_volume/terminal.py", line 194, in dispatch instance.main() File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 77, in main self.create(args) File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/create.py", line 26, in create prepare_step.safe_prepare(args) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 252, in safe_prepare self.prepare() File "/usr/lib/python3/dist-packages/ceph_volume/decorators.py", line 16, in is_root return func(*a, **kw) File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 387, in prepare prepare_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/devices/lvm/prepare.py", line 115, in prepare_bluestore prepare_utils.osd_mkfs_bluestore( File "/usr/lib/python3/dist-packages/ceph_volume/util/prepare.py", line 481, in osd_mkfs_bluestore raise RuntimeError('Command failed with exit code %s: %s' % (returncode, ' '.join(command))) RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 8b0cd318-f5ca-498e-83fe-15102f631eb4 --setuser ceph --setgroup ceph