invalid sysctl provides no feedback

[Mike Wilson]

If a user inputs invalid sysctl configuration there is no error state shown in juju, but the change doesn't work. It would be nice to have some feedback showing in juju status. This impacts kubernetes-worker and kubernetes-master.

[George Kraft (cynerva)]

Got an example config handy? It would help!

[Mike Wilson (knobby)]

Sorry, I should have included that. What I did was a simply typo with spacing. I was trying to get huge pages enabled on my cluster, so I did:

`juju config kubernetes-worker sysctl="{ net.ipv4.conf.all.forwarding : 1, net.ipv4.neigh.default.gc_thresh1 : 128, net.ipv4.neigh.default.gc_thresh2 : 28672, net.ipv4.neigh.default.gc_thresh3 : 32768, net.ipv6.neigh.default.gc_thresh1 : 128, net.ipv6.neigh.default.gc_thresh2 : 28672, net.ipv6.neigh.default.gc_thresh3 : 32768, fs.inotify.max_user_instances : 8192, fs.inotify.max_user_watches : 1048576, kernel.panic : 10, kernel.panic_on_oops: 1, vm.overcommit_memory : 1, vm.nr_hugepages: 1024 }"

Notice the lack of space after hugepages before the colon. It took me far too long to figure that out. ;)

[George Kraft (cynerva)]

Thanks Mike. That's weird, I would not expect the lack of a space there to be an issue, as the yaml parser should resolve that to the same data with or without the space. And indeed, I can't reproduce the issue with the command you provided. After running it on kubernetes-worker rev 704, the vm.nr_hugepages sysctl was successfully updated.

I -can- reproduce the issue with some other typo, like a missing closing brace:

juju config kubernetes-worker sysctl="{ net.ipv4.conf.all.forwarding : 1, net.ipv4.neigh.default.gc_thresh1 : 128, net.ipv4.neigh.default.gc_thresh2 : 28672, net.ipv4.neigh.default.gc_thresh3 : 32768, net.ipv6.neigh.default.gc_thresh1 : 128, net.ipv6.neigh.default.gc_thresh2 : 28672, net.ipv6.neigh.default.gc_thresh3 : 32768, fs.inotify.max_user_instances : 8192, fs.inotify.max_user_watches : 1048576, kernel.panic : 10, kernel.panic_on_oops: 1, vm.overcommit_memory : 1, vm.nr_hugepages: 1024"

With the above command, the charm gives no indication in its status that the sysctl config is invalid. It logs an error, but that's easy to miss.

The charm code passes the raw config string through[1] to charmhelpers.core.sysctl.create_sysctl, which squashes the YAMLError[2].

[1]: https://github.com/charmed-kubernetes/layer-kubernetes-master-worker-base/blob/809f06c6f6521be59e21859eaebeccd13f4d8c28/reactive/kubernetes_master_worker_base.py#L80
[2]: https://github.com/juju/charm-helpers/blob/21f9f239c4f977afb1720243a6f6de4ae876986c/charmhelpers/core/sysctl.py#L46-L52

[Mike Wilson (knobby)]

Sorry, I was going by memory. It might have been invalid yaml set. I'm glad you're seeing the issue though. I don't think this is a big deal, but it would certainly be nice to fix. I agree with your triage. Thanks! :)