Top-level 'network' key results in error using v2 config

Bug #1906187 reported by Christian Dohm
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
Unassigned
cloud-init (Ubuntu)
Fix Released
High
Unassigned
netplan.io (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

I've created a 'network-config' file with Terraform's yamldecode() function that contains (btw. I've tried with the version being a Number w/o quotes and as well as a String as shown here):
---
"network":
  "ethernets":
    "eth0":
      "gateway4": "192.168.1.1"
      "nameservers":
        "addresses":
        - "192.168.1.74"
        - "192.168.1.104"
        "search":
        - "fritz.box"
      "set-name": "eth0"
  "version": "2"
---
After running on Raspberry Pi 4B with 4 GB, created with ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz, the machine's setup fails and /var/log/cloud-init.log reveals that:
---
2020-04-01 17:23:48,649 - util.py[DEBUG]: Reading from /boot/firmware//network-config (quiet=False)
2020-04-01 17:23:48,649 - util.py[DEBUG]: Read 245 bytes from /boot/firmware//network-config
2020-04-01 17:23:48,650 - util.py[DEBUG]: Attempting to load yaml from string of length 240 with allowed root types (<class 'dict'>,)
2020-04-01 17:23:48,652 - util.py[DEBUG]: Attempting to load yaml from string of length 245 with allowed root types (<class 'dict'>,)
2020-04-01 17:23:48,656 - DataSourceNoCloud.py[DEBUG]: Top level network key in network-config but missing 'config' or 'version': {'network': {'ethernets': {'eth0': {'gateway4': '192.168.1.1', 'nameservers': {'addresses': ['192.168.1.74', '192.168.1.104'], 'search': ['fritz.box']}, 'set-name': 'eth0'}}, 'version': '2'}}
---
The corresponding /var/log/clout-init-output.log reveals the trace and that Network won't come up.
---
Cloud-init v. 20.2-45-g5f7825e2-0ubuntu1~20.04.1 running 'init-local' at Wed, 01 Apr 2020 17:23:48 +0000. Up 21.71 seconds.
2020-04-01 17:23:48,796 - util.py[WARNING]: failed stage init-local
failed run of stage init-local
------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in status_wrapper
    ret = functor(name, args)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
    init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 699, in apply_network_config
    net.wait_for_physdevs(netcfg)
  File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 523, in wait_for_physdevs
    physdevs = extract_physdevs(netcfg)
  File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 519, in extract_physdevs
    raise RuntimeError('Unknown network config version: %s' % version)
RuntimeError: Unknown network config version: None
------------------------------------------------------------
Cloud-init v. 20.2-45-g5f7825e2-0ubuntu1~20.04.1 running 'init' at Wed, 01 Apr 2020 17:23:50 +0000. Up 23.69 seconds.
ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++
ci-info: +--------+-------+-----------+-----------+-------+-------------------+
ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
ci-info: +--------+-------+-----------+-----------+-------+-------------------+
ci-info: | eth0 | False | . | . | . | dc:a6:32:b1:78:8e |
ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . |
ci-info: | lo | True | ::1/128 | . | host | . |
ci-info: | wlan0 | False | . | . | . | dc:a6:32:b1:78:8f |
ci-info: +--------+-------+-----------+-----------+-------+-------------------+
ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
ci-info: +-------+-------------+---------+-----------+-------+
ci-info: | Route | Destination | Gateway | Interface | Flags |
ci-info: +-------+-------------+---------+-----------+-------+
ci-info: +-------+-------------+---------+-----------+-------+
2020-04-01 17:23:50,653 - stages.py[WARNING]: Failed to rename devices: Failed to apply network config names: Unknown network config version: None

Related bugs:
 * bug 1798117: juju sends "network" top level key to user.network-config in lxd containers

Revision history for this message
Christian Dohm (chris1294832) wrote :

Should be yamlencode(), it also happens with ubuntu-20.10-preinstalled-server-arm64+raspi.img.xz

affects: ubuntu → netplan.io (Ubuntu)
no longer affects: netplan
Revision history for this message
Ryan Harper (raharper) wrote : Re: [Bug 1906187] [NEW] Version tag is not respected when put last

* Christian Dohm <email address hidden> [2020-11-29 15:50]:
> Public bug reported:
>

Thank you for filing a bug.

> I've created a 'network-config' file with Terraform's yamldecode() function that contains (btw. I've tried with the version being a Number w/o quotes and as well as a String as shown here):
> ---
> "network":
> "ethernets":
> "eth0":
> "gateway4": "192.168.1.1"
> "nameservers":
> "addresses":
> - "192.168.1.74"
> - "192.168.1.104"
> "search":
> - "fritz.box"
> "set-name": "eth0"
> "version": "2"
> ---
> After running on Raspberry Pi 4B with 4 GB, created with ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz, the machine's setup fails and /var/log/cloud-init.log reveals that:
> ---
> 2020-04-01 17:23:48,649 - util.py[DEBUG]: Reading from /boot/firmware//network-config (quiet=False)
> 2020-04-01 17:23:48,649 - util.py[DEBUG]: Read 245 bytes from /boot/firmware//network-config
> 2020-04-01 17:23:48,650 - util.py[DEBUG]: Attempting to load yaml from string of length 240 with allowed root types (<class 'dict'>,)
> 2020-04-01 17:23:48,652 - util.py[DEBUG]: Attempting to load yaml from string of length 245 with allowed root types (<class 'dict'>,)
> 2020-04-01 17:23:48,656 - DataSourceNoCloud.py[DEBUG]: Top level network key in network-config but missing 'config' or 'version': {'network': {'ethernets': {'eth0': {'gateway4': '192.168.1.1', 'nameservers': {'addresses': ['192.168.1.74', '192.168.1.104'], 'search': ['fritz.box']}, 'set-name': 'eth0'}}, 'version': '2'}}

This happens due to the _maybe_remove_top_network method which only
understood network-config v1 format. If the config provided included
the 'network' key, it would remove it. It does some additional
checks (for v1 only) and if they don't match it returns the config
as it was loaded; in this case, with the top-level 'network' key.

Then this network config is passed to cloudinit.net which expects
the DataSource's .network_config property to include a config without a
'network' key; however, in this case, it still has the top-level key
and this confuses the .net's extract_physdevs function.

We should fix _maybe_remove_top_network to handle either v1 or v2;
We may want to accept version field as string and attempt to int() the
value.

As a workaround, if you specify your network config version 2 with the
following changes:

1) remove 'network' key
2) use integer 2 in the 'version' field
3) optionally, your 'set-name' requires use of a 'match' attribute,
   otherwise it's ignored.

  set-name: eth0
  match:
    macaddress: 'AA:BB:CC:DD:EE:FF'

This should let you progress.

Dan Watkins (oddbloke)
Changed in cloud-init (Ubuntu):
status: New → Triaged
importance: Undecided → High
Scott Moser (smoser)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote : Re: Version tag is not respected when put last

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in netplan.io (Ubuntu):
status: New → Confirmed
James Falcon (falcojr)
no longer affects: netplan
Changed in cloud-init:
status: New → Triaged
importance: Undecided → High
James Falcon (falcojr)
Changed in netplan.io (Ubuntu):
status: Confirmed → Invalid
summary: - Version tag is not respected when put last
+ Top-level 'network' key results in error using v2 config
Revision history for this message
James Falcon (falcojr) wrote :
Changed in cloud-init:
status: Triaged → Fix Committed
James Falcon (falcojr)
Changed in cloud-init (Ubuntu):
status: Triaged → Fix Committed
Revision history for this message
Brett Holman (holmanb) wrote : Fixed in cloud-init version 22.2.

This bug is believed to be fixed in cloud-init in version 22.2. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (9.8 KiB)

This bug was fixed in the package cloud-init - 22.2-0ubuntu1~22.10.1

---------------
cloud-init (22.2-0ubuntu1~22.10.1) kinetic; urgency=medium

  * d/control:
    - Build-Depends: add python3-responses and python3-pytest-mock for unittests
    - Suggests: add openssh-server and ssh-import-id
  * d/cloud-init.postinst:
    - remove deprecated emit_upstart from cloud.cfg on upgrade
  * New upstream release.
    - Release 22.2 (LP: #1960939)
    - Fix test due to caplog incompatibility (#1461) [Alberto Contreras]
    - Align rhel custom files with upstream (#1431)
      [Emanuele Giuseppe Esposito]
    - cc_write_files: Improve schema. (#1460) [Alberto Contreras]
    - cli: Redact files with permission errors in commands (#1440)
      [Alberto Contreras] (LP: #1953430)
    - Improve cc_set_passwords. (#1456) [Alberto Contreras]
    - testing: make fake cloud-init wait actually wait (#1459)
    - Scaleway: Fix network configuration for netplan 0.102 and later (#1455)
      [Maxime Corbin]
    - Fix 'ephmeral' typos in disk names(#1452) [Mike Hucka]
    - schema: version schema-cloud-config-v1.json (#1424)
    - cc_modules: set default meta frequency value when no config available
      (#1457)
    - Log generic warning on non-systemd systems. (#1450) [Alberto Contreras]
    - cc_snap.maybe_install_squashfuse no longer needed in Bionic++. (#1448)
      [Alberto Contreras]
    - Drop support of *-sk keys in cc_ssh (#1451) [Alberto Contreras]
    - testing: Fix console_log tests (#1437)
    - tests: cc_set_passoword update for systemd, non-systemd distros (#1449)
    - Fix bug in url_helper/dual_stack() logging (#1426)
    - schema: render schema paths from _CustomSafeLoaderWithMarks (#1391)
    - testing: Make integration tests kinetic friendly (#1441)
    - Handle error if SSH service no present. (#1422)
      [Alberto Contreras] (LP: #1969526)
    - Fix network-manager activator availability and order (#1438)
    - sources/azure: remove reprovisioning marker (#1414) [Chris Patterson]
    - upstart: drop vestigial support for upstart (#1421)
    - testing: Ensure NoCloud detected in test (#1439)
    - Update .github-cla-signers kallioli [Kevin Allioli]
    - Consistently strip top-level network key (#1417) (LP: #1906187)
    - testing: Fix LXD VM metadata test (#1430)
    - testing: Add NoCloud setup for NoCloud test (#1425)
    - Update linters and adapt code for compatibility (#1434) [Paride Legovini]
    - run-container: add support for LXD VMs (#1428) [Paride Legovini]
    - integration-reqs: bump pycloudlib pinned commit (#1427) [Paride Legovini]
    - Fix NoCloud docs (#1423)
    - Docs fixes (#1406)
    - docs: Add docs for module creation (#1415)
    - Remove cheetah from templater (#1416)
    - tests: verify_ordered_items fallback to re.escape if needed (#1420)
    - Misc module cleanup (#1418)
    - docs: Fix doc warnings and enable errors (#1419)
      [Alberto Contreras] (LP: #1876341)
    - Refactor cloudinit.sources.NetworkConfigSource to enum (#1413)
      [Alberto Contreras] (LP: #1874875)
    - Don't fail if IB and Ethernet devices 'collide' (#1411)
    - Use cc_* module meta defintion over hardcoded vars (SC-888) (#1385)
    -...

Changed in cloud-init (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.