apt's preserve_sources_list should be the same for v2 and v3

[David Britton]

Better formatting: http://paste.ubuntu.com/23564366/

Dec 01 17:29:15 kepler cloud-init[1134]: 2016-12-01 17:29:15,900 - util.py[WARNING]: Running module apt-configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py'>) failed
Dec 01 17:29:15 kepler cloud-init[1134]: [CLOUDINIT] util.py[WARNING]: Running module apt-configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py'>) failed
Dec 01 17:29:15 kepler cloud-init[1134]: [CLOUDINIT] util.py[DEBUG]: Running module apt-configure (<module 'cloudinit.config.cc_apt_configure' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py'>) failed
                                         Traceback (most recent call last):
                                           File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 792, in _run_modules
                                             freq=freq)
                                           File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run
                                             return self._runners.run(name, functor, args, freq, clear_on_fail)
                                           File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run
                                             results = functor(*args)
                                           File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 284, in handle
                                             ocfg = convert_to_v3_apt_format(ocfg)
                                           File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 748, in convert_to_v3_apt_format
                                             cfg = convert_v2_to_v3_apt_format(cfg)
                                           File "/usr/lib/python3/dist-packages/cloudinit/config/cc_apt_configure.py", line 714, in convert_v2_to_v3_apt_format
                                             oldkey))
                                         ValueError: Old and New apt format defined with unequal values True vs False @ apt_preserve_sources_list
D

[Jon Grimm (jgrimm)]

So this is a bug to just make the log more readable when cloud-init receives such a misconfiguration?

[David Britton (dpb)]

logs from machine: bohr

[Scott Moser (smoser)]

Jon,
no, cloud-init is complaining that the apt configuration you gave it is wrong.

Either cloud-init has a bug that it mis-understands v2 and v3 apt formats to be different when they are not, or maas is sending different config in v2 and v3 formats. cloud-init exits failure because of that when configuring apt.

so its not ignorable, apt wont be configured as expected (as it wasn't clear what was expected).

its a bug in cloud-init or in maas.

we need to get the configs.
specifically cloud-init is complaining about the setting:
 apt_preserve_sources_list

[David Britton (dpb)]

https://pastebin.canonical.com/172526/

requested apt configuration from maas. Note, we are not trying to do anything funky here, but this system has been upgraded a few times, maybe something is crufty.

[Scott Moser (smoser)]

So maas sent curtin config like seen in your pastebin. The problem I believe is in lines 15 to 33. maas sends 'debconf_sections' to configure cloud-init in the target. It preseeds 'cloud-init/local-cloud-config' with the value (formatted more nicely):

| apt:
| preserve_sources_list: false
| primary:
| - arches: [default]
| uri: http://archive.ubuntu.com/ubuntu
| proxy:
| http://10.2.0.5:8000/
| security:
| - arches: [default]
| uri: http://archive.ubuntu.com/ubuntu
| apt_preserve_sources_list: true
| apt_proxy: http://10.2.0.5:8000/
| manage_etc_hosts: false
| manual_cache_clean: true

...

The thing that cloud-init is complaining about is that maas declared
 apt_preserve_sources_list: true
and
 apt:
  preserve_sources_list: false

The values for the v2 format (apt_preseve_sources_list) and the v3 format
apt/preserve_sources_list are different, and cloud-init is complaining about that.

[Scott Moser (smoser)]

So there are two fixes to this:
a.) maas send consistent data
b.) cloud-init arbitrarily pick a value and silently use it, never warning anyone about their multiple personalities.

[Jon Grimm (jgrimm)]

Do we have a use case where we'd ever intentionally want inconsistent configs? I can't think of any; if not seems perfectly fine to keep this as a hard error/force consistent data tho indeed we could make some assumptions but that seems pretty fragile.

[Blake Rouse (blake-rouse)]

Yes looks like MAAS should not be sending two different values. I believe we send two different versions for compatibility, but the values for those fields should be the same.

[Adam Collard (adam-collard)]

This bug has not seen any activity in the last 6 months, so it is being automatically closed.

If you are still experiencing this issue, please feel free to re-open.

MAAS Team