Another sub-problem. It seems when modifying the value via `juju controller-config` the value `agent-logfile-max-size` is stored as a string in the db which is causing problem. Reproducer: 1. Bootstrap a controller and enable HA 2. juju controller-config agent-logfile-max-backups=3 # default is 2 3. Reboot one of the controllers and it won't come back (the controller machine is ok) The machine agent tries start and repeatedly crashes as it can't parse `agent-logfile-max-size`: ``` goroutine 492 [running]: github.com/juju/juju/controller.Config.mustInt(0x5719d60?, {0x58e56c9, 0x19}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/controller/config.go:568 +0xf8 github.com/juju/juju/controller.Config.intOrDefault(...) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/controller/config.go:575 github.com/juju/juju/controller.Config.AgentLogfileMaxBackups(...) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/controller/config.go:867 github.com/juju/juju/apiserver.newServer({{0x623d8c8, 0x9191b20}, {0x0, 0x0}, {0x6233900, 0xc0007fd030}, {0xc0002ac0c0, 0xd}, {0xc0002ac150, 0xd}, ...}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/apiserver/apiserver.go:412 +0xc32 github.com/juju/juju/apiserver.NewServer({{0x623d8c8, 0x9191b20}, {0x0, 0x0}, {0x6233900, 0xc0007fd030}, {0xc0002ac0c0, 0xd}, {0xc0002ac150, 0xd}, ...}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/apiserver/apiserver.go:289 +0xf8 github.com/juju/juju/worker/apiserver.newServerShim({{0x623d8c8, 0x9191b20}, {0x0, 0x0}, {0x6233900, 0xc0007fd030}, {0xc0002ac0c0, 0xd}, {0xc0002ac150, 0xd}, ...}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/worker/apiserver/worker.go:183 +0x58 github.com/juju/juju/worker/apiserver.NewWorker({{0x6273e50, 0xc000ac24e0}, {0x623d8c8, 0x9191b20}, 0xc0007e2600, {0x6250a60, 0xc0000d8800}, 0xc0004a9b00, {0x6215f08, 0xc0007f3e60}, ...}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/worker/apiserver/worker.go:179 +0x4d9 github.com/juju/juju/worker/apiserver.ManifoldConfig.start({{0x587ba94, 0x5}, {0x58a6d86, 0x10}, {0x587bc7e, 0x5}, {0x588ef51, 0xb}, {0x5893781, 0xc}, ...}, ...) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/worker/apiserver/manifold.go:243 +0x843 github.com/juju/juju/cmd/jujud/agent/engine.flagStart.func1({0x6216b38, 0xc000c3c480}) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/cmd/jujud/agent/engine/housing.go:108 +0xb6 github.com/juju/worker/v3/dependency.(*Engine).runWorker.func1() /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/vendor/github.com/juju/worker/v3/dependency/engine.go:518 +0x399 github.com/juju/worker/v3/dependency.(*Engine).runWorker.func2() /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/vendor/github.com/juju/worker/v3/dependency/engine.go:522 +0x53 github.com/juju/worker/v3/dependency.(*Engine).runWorker(0xc00036e480, {0x5889d2e, 0xa}, 0x18da6d0?, 0xc000af2300?, 0xc000c3c480) /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/vendor/github.com/juju/worker/v3/dependency/engine.go:553 +0xeb created by github.com/juju/worker/v3/dependency.(*Engine).requestStart /home/jenkins/workspace/build-juju/build/src/github.com/juju/juju/vendor/github.com/juju/worker/v3/dependency/engine.go:428 +0x6ea 2022-11-19 15:19:47 INFO juju.cmd supercommand.go:56 running jujud [2.9.35 da3416008ea4ce7851a4c967ae191a0044917024 gc go1.19.2] 2022-11-19 15:19:47 DEBUG juju.cmd supercommand.go:57 args: []string{"/var/lib/juju/tools/machine-0/jujud", "machine", "--data-dir", "/var/lib/juju", "--machine-id", "0", "--debug"} ```