I think the keying permissions are OK now; however ceph-disk tries to double zap the block device resulting in: 2017-10-20 16:07:45 INFO juju-log mon:1: osdize cmd: ['ceph-disk', 'prepare', '--dmcrypt', '--fs-type', u'xfs', '--zap-disk', '--filestore', u'/dev/vdb'] 2017-10-20 16:07:46 DEBUG mon-relation-changed Creating new GPT entries. 2017-10-20 16:07:46 DEBUG mon-relation-changed Setting name! 2017-10-20 16:07:46 DEBUG mon-relation-changed partNum is 4 2017-10-20 16:07:46 DEBUG mon-relation-changed REALLY setting name! 2017-10-20 16:07:46 DEBUG mon-relation-changed The operation has completed successfully. 2017-10-20 16:07:46 DEBUG mon-relation-changed mke2fs 1.42.13 (17-May-2015) 2017-10-20 16:07:46 DEBUG mon-relation-changed Creating filesystem with 10240 1k blocks and 2560 inodes 2017-10-20 16:07:46 DEBUG mon-relation-changed Filesystem UUID: d3dd2964-ea6c-4812-8218-3264dbfd6406 2017-10-20 16:07:46 DEBUG mon-relation-changed Superblock backups stored on blocks: 2017-10-20 16:07:46 DEBUG mon-relation-changed 8193 2017-10-20 16:07:46 DEBUG mon-relation-changed 2017-10-20 16:07:46 DEBUG mon-relation-changed Allocating group tables: 0/2^H^H^H ^H^H^Hdone 2017-10-20 16:07:46 DEBUG mon-relation-changed Writing inode tables: 0/2^H^H^H ^H^H^Hdone 2017-10-20 16:07:46 DEBUG mon-relation-changed Creating journal (1024 blocks): done 2017-10-20 16:07:46 DEBUG mon-relation-changed Writing superblocks and filesystem accounting information: 0/2^H^H^H ^H^H^Hdone 2017-10-20 16:07:46 DEBUG mon-relation-changed 2017-10-20 16:07:47 DEBUG mon-relation-changed creating /var/lib/ceph/osd-lockbox/94bad3e2-b9a3-45d8-aa9f-7bae30a36167/keyring 2017-10-20 16:07:47 DEBUG mon-relation-changed added entity client.osd-lockbox.94bad3e2-b9a3-45d8-aa9f-7bae30a36167 auth auth(auid = 18446744073709551615 key=AQBSH+pZetCjNxAAeJ2OsQ8y1dIreKEjdwkCXQ== with 0 caps) 2017-10-20 16:07:47 DEBUG mon-relation-changed creating /var/lib/ceph/osd-lockbox/94bad3e2-b9a3-45d8-aa9f-7bae30a36167/osd_keyring 2017-10-20 16:07:47 DEBUG mon-relation-changed added entity osd.0 auth auth(auid = 18446744073709551615 key=AQBSH+pZpYkSMxAAWXV25MOEBx2ghn2KN9JmkQ== with 0 caps) 2017-10-20 16:07:48 DEBUG mon-relation-changed Warning: The kernel is still using the old partition table. 2017-10-20 16:07:48 DEBUG mon-relation-changed The new table will be used at the next reboot or after you 2017-10-20 16:07:48 DEBUG mon-relation-changed run partprobe(8) or kpartx(8) 2017-10-20 16:07:48 DEBUG mon-relation-changed The operation has completed successfully. 2017-10-20 16:07:48 DEBUG mon-relation-changed wipefs: error: /dev/vdb5: probing initialization failed: Device or resource busy 2017-10-20 16:07:48 DEBUG mon-relation-changed ceph-disk: Error: Command '['/sbin/wipefs', '--all', '/dev/vdb5']' returned non-zero exit status 1 2017-10-20 16:07:48 ERROR juju-log mon:1: Unable to initialize device: /dev/vdb 2017-10-20 16:07:48 DEBUG mon-relation-changed Traceback (most recent call last): 2017-10-20 16:07:48 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 541, in 2017-10-20 16:07:48 DEBUG mon-relation-changed hooks.execute(sys.argv) 2017-10-20 16:07:48 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/charmhelpers/core/hookenv.py", line 768, in execute 2017-10-20 16:07:48 DEBUG mon-relation-changed self._hooks[hook_name]() 2017-10-20 16:07:48 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 468, in mon_relation 2017-10-20 16:07:48 DEBUG mon-relation-changed prepare_disks_and_activate() 2017-10-20 16:07:48 DEBUG mon-relation-changed File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/mon-relation-changed", line 375, in prepare_disks_and_activate 2017-10-20 16:07:48 DEBUG mon-relation-changed config('bluestore')) 2017-10-20 16:07:48 DEBUG mon-relation-changed File "lib/ceph/utils.py", line 1367, in osdize 2017-10-20 16:07:48 DEBUG mon-relation-changed bluestore) 2017-10-20 16:07:48 DEBUG mon-relation-changed File "lib/ceph/utils.py", line 1425, in osdize_dev 2017-10-20 16:07:48 DEBUG mon-relation-changed subprocess.check_call(cmd) 2017-10-20 16:07:48 DEBUG mon-relation-changed File "/usr/lib/python2.7/subprocess.py", line 541, in check_call 2017-10-20 16:07:48 DEBUG mon-relation-changed raise CalledProcessError(retcode, cmd) 2017-10-20 16:07:48 DEBUG mon-relation-changed subprocess.CalledProcessError: Command '['ceph-disk', 'prepare', '--dmcrypt', '--fs-type', u'xfs', '--zap-disk', '--filestore', u'/dev/vdb']' returned non-zero exit status 1