Config item 'version' vanishes under 2.0

[Stuart Bishop]

Setting a config item's value to null in the deployment config removes the item entirely from the service configuration, rather than setting the value to null. This behaviour is different from 1.25 where it works as expected.

The PostgreSQL charm has a config option 'version' in its config.yaml file. This works fine with 1.25. With 1.26alpha3, it disappears. 'config-get' lists only 81 of the 82 options, with 'version' missing.

'version' is used to specify the version of PostgreSQL to install, and is rather important.

[Cheryl Jennings (cherylj)]

I wasn't able to recreate with either a new bootstrap with 1.26 (what's in master), nor with an upgrade from 1.24.7 -> current master.

Stuart, can you attach the logs for:
- machine0
- the postresql unit
- output of juju get postresql?

[Stuart Bishop (stub)]

I managed to reproduce this again. 'juju get postgresql' shows the 'version' configuration setting, but 'juju run --unit=postgresql/1 config-get' has it missing.

The failure occurs using 1.26alpha3 and the lxd provider, bootstrapped with '--upload-tools'. I think I failed to reproduce this earlier when I bootstrapped without --upload-tools.

[Cheryl Jennings (cherylj)]

Stuart - can you set the logging to TRACE, run the 'juju run --unit=postgresql/1 config-get' command again, and send in machine-0.log? You can set it back to WARN once the command completes:

juju set-env logging-config="<root>=TRACE"

This should show the data actually passed back from the state server.

[Stuart Bishop (stub)]

I'm unable to reproduce this today. I have no idea what has changed.

[Cheryl Jennings (cherylj)]

Well, good that it went away, but bad that we don't know why. If you can set the log level to at least DEBUG in case you run into it again, that would help.

[Stuart Bishop (stub)]

Just happened again, in the middle of a test run this time. The deploy 15:29 UTC failed. The previous deploys of the same charm to the same environment (with different configurations and subordinates) worked.

Jan 15 22:29:15 2016-01-15 22:29:15 The following units had errors:
Jan 15 22:29:15 unit: postgresql/15: machine: 23 agent-state: error details: hook failed: "install"

Same symptoms as before. 'juju get postgresql' shows the version config item is there and set to the default value of an empty string. 'juju run --unit=postgresql/15 config-get' shows that the version config item is missing.

[Cheryl Jennings (cherylj)]

I see that in the ConfigSettings returned, the version is null (which is probably why it's omitted): "version":null

Not sure where the default value should be inserted / determined, so I need to look around more.

[Stuart Bishop (stub)]

Here are some less bulky logs from Juju 2.0, where the test run failed with the problem immediately after bootstrap.

[Cheryl Jennings (cherylj)]

@stub - are you still seeing this issue?

[Stuart Bishop (stub)]

I haven't triggered it in a dozen manual deploys.

[Stuart Bishop (stub)]

Just had this happen again under beta7, so I'm reopening this. As before, 'juju run --unit=postgresql/0 "config-get --format=yaml" shows that the version option has gone completely missing:

unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 Traceback (most recent call last):
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 File "/var/lib/juju/agents/unit-postgresql-0/charm/hooks/install", line 19, in <module>
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 main()
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 File "/usr/local/lib/python3.5/dist-packages/charms/reactive/__init__.py", line 72, in main
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 hookenv._run_atstart()
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 File "/usr/local/lib/python3.5/dist-packages/charmhelpers/core/hookenv.py", line 986, in _run_atstart
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 callback(*args, **kwargs)
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 File "/var/lib/juju/agents/unit-postgresql-0/charm/reactive/postgresql/preflight.py", line 51, in block_on_invalid_config
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 config[key] = config[key].lower() # Rewrite to lower case.
unit-postgresql-0: 2016-05-18 07:49:03 INFO unit.postgresql/0.install logger.go:40 KeyError: 'version'

[Stuart Bishop (stub)]

The failure occurs when I set the version to null in the deploy config, rather than the empty string. ie.

pga:
    version: ""
pgb:
    version: null

With the above config and Juju 1.25, both the pga and pgb services can be deployed. With Juju 2.0beta7, the 'version' config item will disappear in the pgb service and it will fail.

The obvious work around is to not do that.

What should happen is config-get --format=yaml should correctly return the value the operator set (null).

[Reed O'Brien (reedobrien)]

On current master I can create a config like

pga:
    version: ""
pgb:
    version: null

I can then deploy like so:

juju deploy postgresql pga --config pg.yaml
juju deploy postgresql pgb --config pg.yaml

And they both go on their merry way. At the end of the day pgb fails to deploy, pga succeeds.

I cannot however make version disappear in the output of

juju config pgb or juju config pgb version

They both return the value of version as "".

I suspect this issue revolves around using null as a value in the yaml config. null is a type, string is a type and null is not string -- null != "". However the version attribute is defined as a string in the postgresql config. In SQL things can be nullable, but that implies absence of a value, not the zero value. Since the type in our config is string I think the default suffices which would be an empty string. At this point (master at 5444b45) it appears that we convert null to "". That said I don't understand why pgb fails to install unless it has something to do with the charm code.

As far as this bug goes, I cannot make version disappear `juju config` output. I'll mark as incomplete if we feel that this is no longer an issue and we can start a new bug for failure to deploy if so desired. Does this seem on track Stuart?

[Stuart Bishop (stub)]

Looks like this has been fixed. Previously, doing what @reedobrien tried would cause the version string to disappear from the output of config-get.

(FWIW, all languages I'm aware of that have a null use it as a value. eg. the default value for uninitialized variables. Normally the default for 'unset' config items should be null, but we use the empty string to work around limitations in the juju command line)