cloud-init crashes with AttributeError: module 'posixpath' has no attribute 'is_file'

[Iain Lane]

=== Begin SRU Template ===
[Impact]
This bug did not affect yakkety or xenial. it was a regression of
in a zesty upload and was fixed.

[Test Case]
just run an instance.

1. launch instance in lxd.
2. enable proposed, update, upgrade cloud-init.
3. lxc reboot $name; sleep 10
4. lxc exec $name -- cat /run/cloud-init/result.json
5. lxc exec $name -- grep Traceback /var/log/cloud-init.log

If there is no stack trace in /var/log/cloud-init.log then this
bug is not present.

[Regression Potential]
The buggy code never made it to a stable release, so a regression
caused by this specific fix is not really possible.
Additionally, look at the commit and you'll convince yourself of
low regression potential.

[Other Info]
Upstream commit:
 https://git.launchpad.net/cloud-init/commit/?id=9040e78feb7c1bc

=== End SRU Template ===

cloud-init 0.7.9-66-g5beecdf8-0ubuntu1. I noticed that diaspora-installer running in autopkgtest was looping. It turns out that /etc/environment (written by a runcmd) hasn't been updated. Looking in console-log I saw the following:

---

[ 7.073261] cloud-init[284]: Cloud-init v. 0.7.9 running 'init-local' at Mon, 20 Mar 2017 11:50:50 +0000. Up 6.93 seconds.
[ 7.078970] cloud-init[284]: 2017-03-20 11:50:50,196 - util.py[WARNING]: failed stage init-local
[ 7.112926] cloud-init[284]: failed run of stage init-local
[ 7.114233] cloud-init[284]: ------------------------------------------------------------
[ 7.116223] cloud-init[284]: Traceback (most recent call last):
[ 7.120348] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 332, in main_init
[ 7.121921] cloud-init[284]: init.fetch(existing=existing)
[ 7.124383] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 349, in fetch
[ 7.128405] cloud-init[284]: return self._get_data_source(existing=existing)
[ 7.132399] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 259, in _get_data_source
[ 7.134033] cloud-init[284]: pkg_list, self.reporter)
[ 7.136243] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 320, in find_source
[ 7.140370] cloud-init[284]: raise DataSourceNotFoundException(msg)
[ 7.141527] cloud-init[284]: cloudinit.sources.DataSourceNotFoundException: Did not find any data source, searched classes: ()
[ 7.145341] cloud-init[284]: During handling of the above exception, another exception occurred:
[ 7.148301] cloud-init[284]: Traceback (most recent call last):
[ 7.152275] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 647, in status_wrapper
[ 7.153846] cloud-init[284]: ret = functor(name, args)
[ 7.156281] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 350, in main_init
[ 7.160275] cloud-init[284]: init.apply_network_config(bring_up=not args.local)
[ 7.161612] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 648, in apply_network_config
[ 7.168504] cloud-init[284]: return self.distro.apply_network_config(netcfg, bring_up=bring_up)
[ 7.172333] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 163, in apply_network_config
[ 7.174047] cloud-init[284]: dev_names = self._write_network_config(netconfig)
[ 7.180476] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/distros/debian.py", line 83, in _write_network_config
[ 7.184319] cloud-init[284]: return self._supported_write_network_config(netconfig)
[ 7.185681] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 78, in _supported_write_network_config
[ 7.192992] cloud-init[284]: name, render_cls = renderers.select(priority=priority)
[ 7.196350] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/net/renderers.py", line 39, in select
[ 7.200655] cloud-init[284]: found = search(priority, target=target, first=True)
[ 7.201941] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/net/renderers.py", line 29, in search
[ 7.208347] cloud-init[284]: if render_mod.available(target):
[ 7.209513] cloud-init[284]: File "/usr/lib/python3/dist-packages/cloudinit/net/eni.py", line 511, in available
[ 7.216033] cloud-init[284]: if not os.path.is_file(eni):
[ 7.217267] cloud-init[284]: AttributeError: module 'posixpath' has no attribute 'is_file'
[ 7.220239] cloud-init[284]: ------------------------------------------------------------
[[0;1;31mFAILED[0m] Failed to start Initial cloud-init job (pre-networking).

---

I don't yet know for sure that this caused my problem, but you probably want to fix it (isfile vs is_file). Introduced in:

   https://git.launchpad.net/cloud-init/commit/?id=5beecdf88b630a397b3722ddb299e9a37ff02737

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-68-gef18b8ac-0ubuntu1

---------------
cloud-init (0.7.9-68-gef18b8ac-0ubuntu1) zesty; urgency=medium

  * New upstream snapshot.
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)

 -- Scott Moser <email address hidden> Mon, 20 Mar 2017 16:21:55 -0400

[Brian Murray (brian-murray)]

Hello Iain, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Iain, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Scott Moser (smoser)]

To verify this... I verified there were no obvious regressions by the
following.
(lxc-proposed-snapshot is
  https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot
It publishes an image to lxd with proposed enabled and cloud-init upgraded.)

$ for r in xenial yakkety; do
   lxc-proposed-snapshot --proposed --publish $r sm0413-$r-proposed; done

$ lxc launch sm0413-xenial-proposed x1674317
$ lxc launch sm0413-xenial-proposed y1674317
$ sleep 30

$ for n in x1674317 y1674317; do
   echo === $n ===
   lxc exec $n -- dpkg-query --show cloud-init
   lxc exec $n cat /run/cloud-init/result.json
   lxc exec $n -- grep WARN /var/log/cloud-init.log || echo no warn
   lxc exec $n -- grep -i traceback /var/log/cloud-init.log || echo no trace
done
=== x1669504 ===

$ for n in x1674317 y1674317; do
> echo === $n ===
> lxc exec $n -- dpkg-query --show cloud-init
> lxc exec $n cat /run/cloud-init/result.json
> lxc exec $n -- grep WARN /var/log/cloud-init.log || echo no warn
> lxc exec $n -- grep -i traceback /var/log/cloud-init.log || echo no trace
> done
=== x1674317 ===
cloud-init 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
{
 "v1": {
  "datasource": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]",
  "errors": []
 }
}
no warn
no trace
=== y1674317 ===
cloud-init 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
{
 "v1": {
  "datasource": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]",
  "errors": []
 }
}
no warn
no trace

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773)
    - ...

[Steve Langasek (vorlon)]

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773...