After some more tests, it turns out that this command will fail no matter if I give it a value or not.
It seems the command is executed too early and at that time ceph mon is not initiated yet.
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:05:59 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Ceph is not bootstrapped, skipping upgrade checks.
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:05:59 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Monitor hosts are ['10.5.2.27']
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:00 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Settings for the cluster are: {'fsid': '0be52cf2-ac62-11ee-8f8a-9109e1518e46', 'monitor-secret': 'AQDY+5hlxaxyChAA1kj4CSKU3tq2z1uDNTUgGg=='}
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:00 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Making dir /var/lib/charm/secondary-ceph-mon ceph:ceph 555
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:00 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Making dir /var/run/ceph ceph:ceph 755
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:00 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Making dir /var/lib/ceph/mon/ceph-juju-9d6cae-zaza-34cc90b87300-5 ceph:ceph 755
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:00 WARNING unit.secondary-ceph-mon/0.config-changed logger.go:60 Created symlink /<email address hidden> → /lib/systemd/system/ceph-mon@.service.
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:01 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Waiting for quorum to be reached
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:04 WARNING unit.secondary-ceph-mon/0.config-changed logger.go:60 2024-01-06T07:06:04.580+0000 7f9d064f9700 -1 auth: unable to find a keyring on /var/lib/ceph/mon/ceph-/keyring: (2) No such file or directory
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:04 WARNING unit.secondary-ceph-mon/0.config-changed logger.go:60 2024-01-06T07:06:04.580+0000 7f9d064f9700 -1 AuthRegistry(0x7f9d00060240) no keyring found at /var/lib/ceph/mon/ceph-/keyring, disabling cephx
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:04 INFO unit.secondary-ceph-mon/0.juju-log server.go:316 Making dir /var/lib/ceph/mgr/ceph-juju-9d6cae-zaza-34cc90b87300-5 ceph:ceph 555
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:05 WARNING unit.secondary-ceph-mon/0.config-changed logger.go:60 Created symlink /<email address hidden> → /lib/systemd/system/ceph-mgr@.service.
2024-01-06 07:06:08 [ERROR] unit-secondary-ceph-mon-0.log: 2024-01-06 07:06:07 WARNING unit.secondary-ceph-mon/0.config-changed logger.go:60 Error EINVAL: unrecognized config option 'mgr/prometheus/rbd_stats_pools'
Maybe it needs to wait until mon is bootstrapped then run the command.
After some more tests, it turns out that this command will fail no matter if I give it a value or not.
It seems the command is executed too early and at that time ceph mon is not initiated yet.
2024-01-06 07:06:08 [ERROR] unit-secondary- ceph-mon- 0.log: 2024-01-06 07:05:59 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Ceph is not bootstrapped, skipping upgrade checks. ceph-mon- 0.log: 2024-01-06 07:05:59 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Monitor hosts are ['10.5.2.27'] ceph-mon- 0.log: 2024-01-06 07:06:00 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Settings for the cluster are: {'fsid': '0be52cf2- ac62-11ee- 8f8a-9109e1518e 46', 'monitor-secret': 'AQDY+5hlxaxyCh AA1kj4CSKU3tq2z 1uDNTUgGg= ='} ceph-mon- 0.log: 2024-01-06 07:06:00 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Making dir /var/lib/ charm/secondary -ceph-mon ceph:ceph 555 ceph-mon- 0.log: 2024-01-06 07:06:00 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Making dir /var/run/ceph ceph:ceph 755 ceph-mon- 0.log: 2024-01-06 07:06:00 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Making dir /var/lib/ ceph/mon/ ceph-juju- 9d6cae- zaza-34cc90b873 00-5 ceph:ceph 755 ceph-mon- 0.log: 2024-01-06 07:06:00 WARNING unit.secondary- ceph-mon/ 0.config- changed logger.go:60 Created symlink /<email address hidden> → /lib/systemd/ system/ ceph-mon@ .service. ceph-mon- 0.log: 2024-01-06 07:06:01 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Waiting for quorum to be reached ceph-mon- 0.log: 2024-01-06 07:06:04 WARNING unit.secondary- ceph-mon/ 0.config- changed logger.go:60 2024-01- 06T07:06: 04.580+ 0000 7f9d064f9700 -1 auth: unable to find a keyring on /var/lib/ ceph/mon/ ceph-/keyring: (2) No such file or directory ceph-mon- 0.log: 2024-01-06 07:06:04 WARNING unit.secondary- ceph-mon/ 0.config- changed logger.go:60 2024-01- 06T07:06: 04.580+ 0000 7f9d064f9700 -1 AuthRegistry( 0x7f9d00060240) no keyring found at /var/lib/ ceph/mon/ ceph-/keyring, disabling cephx ceph-mon- 0.log: 2024-01-06 07:06:04 INFO unit.secondary- ceph-mon/ 0.juju- log server.go:316 Making dir /var/lib/ ceph/mgr/ ceph-juju- 9d6cae- zaza-34cc90b873 00-5 ceph:ceph 555 ceph-mon- 0.log: 2024-01-06 07:06:05 WARNING unit.secondary- ceph-mon/ 0.config- changed logger.go:60 Created symlink /<email address hidden> → /lib/systemd/ system/ ceph-mgr@ .service. ceph-mon- 0.log: 2024-01-06 07:06:07 WARNING unit.secondary- ceph-mon/ 0.config- changed logger.go:60 Error EINVAL: unrecognized config option 'mgr/prometheus /rbd_stats_ pools'
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
2024-01-06 07:06:08 [ERROR] unit-secondary-
Maybe it needs to wait until mon is bootstrapped then run the command.