In 22.2 cloud-init fails when phone-home module does not have "tries" parameter

[Schmidt Zsolt]

=== Begin SRU Template ===
[Impact]
The cloud-init "phone home" module contains an optional "tries" parameter. In release 22.2, this was accidentally changed to become mandatory. Any previously working cloud-config that didn't contain the "tries" parameter will now cause the module exit with exception. This happened because an overly broad exception handler for converting the 'tries' string to an int was changed to only raise on ValueError. However, if None (or any other non-string) is passed, a TypeError is raised, and this needs to be caught as well.

[Test Case]
1. Launch an Ubuntu instance on any cloud-init supported platform with the following userdata:

#cloud-config
phone_home:
  url: http://192.168.1.1
  post: all

2. By inspecting /var/log/cloud-init.log, ensure the phone home module attempts to make a web request, with the following log:
url_helper.py[DEBUG]: [0/10] open 'http://192.168.1.1' with {'url': 'http://192.168.1.1', 'allow_redirects': True, 'method': 'POST', 'headers': {'User-Agent': 'Cloud-Init/22.2'}} configuration

[Regression Potential]
The parsing exceptions being caught should now be broad enough to handle any configuration we receive, but if not, we would still exit the module with exception.

[Other Info]
Github PR: https://github.com/canonical/cloud-init/pull/1500

=== End SRU Template ===

Initial bug:

Hi!

We have some user-data files where we use the phone-home module of cloud-init.
So far we did not use it's "tries" parameter and everything worked.
However now in version 22.2 there was a change which causes cloud-init to fail.
https://github.com/canonical/cloud-init/compare/22.1...22.2#diff-a4aa83fbb946ba1ea7cf6c8dd5965cd62631dc9cb48d4baa50adddbfef06b82cL108

In our case this change in the exception handling throws a TypeError, instead of the ValueError that is excepted:

File "/usr/lib/python3/dist-packages/cloudinit/config/cc_phone_home.py", line 132, in handle
   tries = int(tries) # type: ignore
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

While we can add the "tries" parameter (and after that everything works just like before),
this exception should be handled properly.

Also according to guidelines:
1. Tell us your cloud provider
None
2. Any appropriate cloud-init configuration you can provide us
phone-home module
3. Perform the following on the system and attach it to this bug:
no logs are necessary

Best regards:
Zsolt

[Julien (xfennec)]

We ran into the exact same issue.

All our VM creations suddenly starts failing yesterday, which was a bit surprising because that's why we were using LTS in the first place ;)

Adding the "tries" parameter to phone_home fixed the issue immediately.

[Chad Smith (chad.smith)]

Upstream PR in flight on this, we expect a quick SRU release to fix this:
https://github.com/canonical/cloud-init/pull/1500

[Chad Smith (chad.smith)]

Upstream commit landed that will address this issue
https://github.com/canonical/cloud-init/commit/a2e6273872c2cafa73530e6c73178c2ebfc2beba

[Chad Smith (chad.smith)]

cloud-init version 22.2-30 has been uploaded to Ubuntu 22.10 Kinetic.

We plan to queue uploads of cloud-init with this specific fix into Ubuntu 18.04. 20.04, 21.10 and 22.04. These fixes should updated cloudimages within a week.

[Steve Langasek (vorlon)]

Hello Schmidt, or anyone else affected,

Accepted cloud-init into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.2-0ubuntu1~22.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Schmidt, or anyone else affected,

Accepted cloud-init into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.2-0ubuntu1~21.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Schmidt, or anyone else affected,

Accepted cloud-init into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.2-0ubuntu1~20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Steve Langasek (vorlon)]

Hello Schmidt, or anyone else affected,

Accepted cloud-init into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/22.2-0ubuntu1~18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[James Falcon (falcojr)]

Created an ad hoc "test" that works with the cloud-init integration testing framework. This test uses the userdata from the bug description and then prints out the relevant section of the logs. The test file and test output are included in the attached tarball.

Note that tracebacks are seen in the logs, but both are expected. The type conversion traceback was the standard behavior before this bug was introduced, but module execution then continues. The web request traceback happens because the address isn't resolvable.

[Schmidt Zsolt (schmidtzsolti91)]

Hi!

Tested by enabling proposed repositories.
Everything works as expected even with the "tries" parameter missing.
Thanks for the quick response.

Br:
Zsolt

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 22.2-0ubuntu1~22.04.2

---------------
cloud-init (22.2-0ubuntu1~22.04.2) jammy; urgency=medium

  * cherry-pick a2e62738: Fix cc_phone_home requiring 'tries' (#1500)
    (LP: #1977952)

 -- James Falcon <email address hidden> Thu, 09 Jun 2022 16:25:29 -0500

[Brian Murray (brian-murray)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 22.2-0ubuntu1~21.10.2

---------------
cloud-init (22.2-0ubuntu1~21.10.2) impish; urgency=medium

  * cherry-pick a2e62738: Fix cc_phone_home requiring 'tries' (#1500)
    (LP: #1977952)

 -- James Falcon <email address hidden> Thu, 09 Jun 2022 16:48:01 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 22.2-0ubuntu1~20.04.2

---------------
cloud-init (22.2-0ubuntu1~20.04.2) focal; urgency=medium

  * cherry-pick a2e62738: Fix cc_phone_home requiring 'tries' (#1500)
    (LP: #1977952)

 -- James Falcon <email address hidden> Thu, 09 Jun 2022 16:53:07 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 22.2-0ubuntu1~18.04.2

---------------
cloud-init (22.2-0ubuntu1~18.04.2) bionic; urgency=medium

  * cherry-pick a2e62738: Fix cc_phone_home requiring 'tries' (#1500)
    (LP: #1977952)

 -- James Falcon <email address hidden> Thu, 09 Jun 2022 16:56:33 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 22.2-64-g1fcd55d6-0ubuntu1~22.10.1

---------------
cloud-init (22.2-64-g1fcd55d6-0ubuntu1~22.10.1) kinetic; urgency=medium

  * d/control: add python3-debconf as Depends and Build-Depends
  * d/gbp.conf d/gbp_format_changelog:
    + git-buildpackage customization for debian/changelog generation
  * New upstream snapshot.
    + tests: mock dns lookup that causes long timeouts (#1555)
    + tox: add unpinned env for do_format and check_format (#1554)
    + cc_ssh_import_id: Substitute deprecated warn (#1553)
      [Alberto Contreras]
    + Remove schema errors from log (#1551) (LP: #1978422) (CVE-2022-2084)
    + Update WebHookHandler to run as background thread (SC-456) (#1491)
      (LP: #1910552)
    + testing: Don't run custom cloud dir test on Bionic (#1542)
    + bash completion: update schema command (#1543) (LP: #1979547)
    + CI: add non-blocking run against the linters tip versions (#1531)
      [Paride Legovini]
    + Change groups within the users schema to support lists and strings
      (#1545) [RedKrieg]
    + make it clear which username should go in the contributing doc (#1546)
    + Pin setuptools for Travis (SC-1136) (#1540)
    + Fix LXD datasource crawl when BOOT enabled (#1537)
    + testing: Fix wrong path in dual stack test (#1538)
    + cloud-config: honor cloud_dir setting (#1523) (LP: #1976564)
      [Alberto Contreras]
    + Add python3-debconf to pkg-deps.json Build-Depends (#1535)
      [Alberto Contreras]
    + redhat spec: udev/rules.d lives under /usr/lib on rhel-based systems
      (#1536)
    + tests/azure: add test coverage for DisableSshPasswordAuthentication
      (#1534) [Chris Patterson]
    + summary: Add david-caro to the cla signers (#1527) [David Caro]
    + Add support for OpenMandriva (https://openmandriva.org/) (#1520)
      [Bernhard Rosenkraenzer]
    + tests/azure: refactor ovf creation (#1533) [Chris Patterson]
    + Improve DataSourceOVF error reporting when script disabled (#1525)
      [rong]
    + tox: integration-tests-jenkins: softfail if only some test failed
      (#1528) [Paride Legovini]
    + CI: drop linters from Travis CI (moved to GH Actions) (#1530)
      [Paride Legovini]
    + sources/azure: remove unused encoding support for customdata (#1526)
      [Chris Patterson]
    + sources/azure: remove unused metadata captured when parsing ovf
      (#1524) [Chris Patterson]
    + sources/azure: remove dscfg parsing from ovf-env.xml (#1522)
      [Chris Patterson]
    + Remove extra space from ec2 dual stack crawl message (#1521)
    + tests/azure: use namespaces in generated ovf-env.xml documents (#1519)
      [Chris Patterson]
    + setup.py: adjust udev/rules default path (#1513)
      [Emanuele Giuseppe Esposito]
    + Add python3-deconf dependency (#1506) [Alberto Contreras]
    + Change match macadress param for network v2 config (#1518)
      [Henrique Caricatti Capozzi]
    + sources/azure: remove unused userdata property from ovf (#1516)
      [Chris Patterson]
    + sources/azure: minor refactoring to network config generation (#1497)
      [Chris Patterson]
    + net: Implement link-local ephemeral ipv6
    + Rename function ...

[Brett Holman (holmanb)]

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[shixuantong (sxt1001)]

What is the relationship between the current bug and CVE-2022-2084? I found out they were related.

https://launchpad.net/bugs/cve/CVE-2022-2084

[James Falcon (falcojr)]

The CVE as added by mistake, likely because of the bot comment including the CVE number in a changelog description. I'm removing the link.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3985