Running with --storage flag fails on missing file or directory

[Alex McLeod]

Ran:

juju deploy -n 3 ceph-osd --storage osd-devices=32G,2 --storage osd-journals=8G,1

Machines start up, all 3 ceph-osd units stick on allocating with agent intializing.

Running juju status --storage:

attaching volume 91/137: attaching loop device: attaching loop device to "/var/lib/juju/storage/loop/volume-91-137": losetup: /var/lib/juju/storage/loop/volume-91-137: failed to set up loop device: No such file or directory: exit status 1

SSH into the machine shows /var/lib/juju/storage/loop exists as a directory but no files exist inside

[Liam Young (gnuoy)]

If the units are stuck at 'agent initializing' then the charm has yet to be executed so I don't think this can be a bug in the ceph-mon charm. I'm going to target it at juju. To help them out can you add information about the cloud you are trying to deploy on top of (lxd, aws, opetnstack etc), the version of Juju you are using. If you are doing a lxd deploy could you also confirm you have plenty of diskspace to accommodate the storage being requested.

[Alex McLeod (mcleodalexj)]

Of course. Currently deployed within Ubuntu 20.04.2 LTS using LXC 4.15 and Juju 2.9.4-ubuntu-amd64. Can also confirm that I have well over the expected storage requested available. I appreciate the follow-up and redirect to the Juju team!

[John A Meinel (jameinel)]

So we should be running:
losetup -f --show /var/lib/juju/storage/loop/volume-91-137

And I believe that is where the 'No such file or directory' is coming from.

However, if I try to run something like that on Focal it doesn't seem to do the right thing:
$ sudo losetup -f --show $PWD/blah
losetup: /home/jameinel/dev/go/src/github.com/juju/juju/blah: failed to set up loop device: No such file or directory

if I then create the directory first I get:
$ sudo losetup -f --show $PWD/blah
losetup: /home/jameinel/dev/go/src/github.com/juju/juju/blah: failed to set up loop device: Is a directory

If I do pass it a file, it then lets me mount it:
$ sudo losetup -f --show $PWD/blah
/dev/loop0
losetup: /home/jameinel/dev/go/src/github.com/juju/juju/blah: Warning: file is smaller than 512 bytes; the loop device may be useless or invisible for system tools.

But it seems to be expecting that some other code has actually created the file and given it the right size, etc before we mount it.

What I really don't understand is why the above is using loopback devices for ceph storage. Shouldn't these be coming from the provider?

I suppose if you are testing this on LXD containers or something like that, then that is the only source of storage that we have.

[Launchpad Janitor (janitor)]

[Expired for juju because there has been no activity for 60 days.]