Fallback networking doesn't handle IOError when reading sys/net/<iface>/carrier

Bug #1625766 reported by J on 2016-09-20
28
This bug affects 4 people
Affects Status Importance Assigned to Milestone
cloud-init
High
Unassigned
cloud-init (Ubuntu)
Medium
Unassigned
Xenial
Undecided
Unassigned
Yakkety
Low
Unassigned

Bug Description

Sometimes reading from /sys/class/net/<iface>/carrier returns an error and is unhangled causing fallback networking to not bring anything up.

[Original Description]

I am running Arch on a KVM vps provider. I installed using this template: Arch Linux 2016.03 64-bit (template). Everything was working fine until I decided to upgrade. I did pacman -Syu and everything upgraded without error until it restarted.

I had to manually install certain python packages. But, I kept getting more errors so I joined IRC.

Here's the log: https://irclogs.ubuntu.com/2016/09/20/%23cloud-init.html

Was told to post it Here to sum up everything

Related branches

J (jwillx) on 2016-09-20
description: updated
Ryan Harper (raharper) on 2016-10-19
description: updated
summary: - Arch issue with start up
+ Fallback networking doesn't handle IOError when reading
+ sys/net/<iface>/carrier
Changed in cloud-init:
importance: Undecided → High
status: New → Triaged
Scott Moser (smoser) on 2016-12-16
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → Medium
Changed in cloud-init:
status: Triaged → Fix Committed
Scott Moser (smoser) on 2016-12-22
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
importance: Undecided → Low
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.9.

Changed in cloud-init:
status: Fix Committed → Fix Released
Brian Murray (brian-murray) wrote :

I'm accepting this cloud-init upload into yakkety-proposed but this particular bug is missing SRU bug information, could it please be added?

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: added: verification-needed

Hello J, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-68-gca3ae67-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser) wrote :

This code path is involved on each boot, but on Ubuntu we never triggered the failure case. Failure case happens when some drivers were in different state. I realize this is not great, but I'm marking this as verification-done based on:

$ release=yakkety
$ lxc launch "ubuntu-daily:$release" "$name"
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-49-g9e904bb-0ubuntu1~16.10.1

$ lxc file pull $name/etc/cloud/build.info -
build_name: server
serial: 20170119

## enable proposed, update
$ m=http://archive.ubuntu.com/ubuntu;
$ echo "deb $m $release-proposed main" | lxc file push - $name/etc/apt/sources.list.d/proposed.list
$ lxc exec $name -- sh -c 'apt update -q && apt install cloud-init' </dev/null
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-68-gca3ae67-0ubuntu1~16.10.1

# clean up to make it look like first boot.
$ lxc exec $name -- sh -c 'cd /var/lib/cloud; for d in *; do [ "$d" = "seed" ] || rm -vRf "$d"; done'
$ lxc exec $name -- sh -c 'rm -f /var/log/cloud*'
$ lxc restart $name

$ lxc exec $name -- grep WARN /var/log/cloud-init.log || echo no warn
no warn

$ lxc lau

tags: added: verification-done
removed: verification-needed

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.8-68-gca3ae67-0ubuntu1~16.10.1

---------------
cloud-init (0.7.8-68-gca3ae67-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cherry-pick: add utility for cherry picking commits from upstream
    into patches in debian/patches.
  * New upstream snapshot.
    - mounts: use mount -a again to accomplish mounts (LP: #1647708)
    - CloudSigma: Fix bug where datasource was not loaded in local search.
      (LP: #1648380)
    - when adding a user, strip whitespace from group list
      [Lars Kellogg-Stedman] (LP: #1354694)
    - fix decoding of utf-8 chars in yaml test
    - Replace usage of sys_netdev_info with read_sys_net (LP: #1625766)
    - fix problems found in python2.6 test.
    - OpenStack: extend physical types to include hyperv, hw_veb, vhost_user.
      (LP: #1642679)
    - tests: fix assumptions that expected no eth0 in system. (LP: #1644043)
    - net/cmdline: Consider ip= or ip6= on command line not only ip=
      (LP: #1639930)
    - Just use file logging by default [Joshua Harlow] (LP: #1643990)
    - Improve formatting for ProcessExecutionError [Wesley Wiedenmeier]
    - flake8: fix trailing white space
    - Doc: various documentation fixes [Sean Bright]
    - cloudinit/config/cc_rh_subscription.py: Remove repos before adding
      [Brent Baude]
    - packages/redhat: fix rpm spec file.
    - main: set TZ in environment if not already set. [Ryan Harper]
    - disk_setup: Use sectors as unit when formatting MBR disks with sfdisk.
      [Daniel Watkins] (LP: #1460715)

 -- Scott Moser <email address hidden> Mon, 19 Dec 2016 15:07:12 -0500

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released

Hello J, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: New → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Brian Murray (brian-murray) wrote :

Hello J, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-0ubuntu1~16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser) wrote :

$ release=xenial
$ lxc launch "ubuntu-daily:$release" "$name"
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.8-49-g9e904bb-0ubuntu1~16.04.4

$ lxc file pull $name/etc/cloud/build.info -
build_name: server
serial: 20170207

## enable proposed, update
$ m=http://archive.ubuntu.com/ubuntu;
$ echo "deb $m $release-proposed main" | lxc file push - $name/etc/apt/sources.list.d/proposed.list
$ lxc exec $name -- sh -c 'apt update -q && apt install cloud-init' </dev/null
$ lxc exec $name -- dpkg-query --show cloud-init
cloud-init 0.7.9-0ubuntu1~16.04.2

# clean up to make it look like first boot.
$ lxc exec $name -- sh -c 'cd /var/lib/cloud; for d in *; do [ "$d" = "seed" ] || rm -vRf "$d"; done'
$ lxc exec $name -- sh -c 'rm -f /var/log/cloud*'
$ lxc restart $name

$ lxc exec $name -- grep WARN /var/log/cloud-init.log || echo no warn
no warn

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.9-0ubuntu1~16.04.2

---------------
cloud-init (0.7.9-0ubuntu1~16.04.2) xenial-proposed; urgency=medium

  * debian/update-grub-legacy-ec2: fix shell syntax error. (LP: #1662221)

cloud-init (0.7.9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/copyright: update License field to include Apache.
  * debian/update-grub-legacy-ec2: fix to include kernels whose config
    has CONFIG_XEN=y (LP: #1379080).
  * debian/patches/azure-use-walinux-agent.patch: continue relying on
    walinux agent in stable release.
  * New upstream release.
    - doc: adjust headers in tests documentation for consistency.
    - pep8: fix issue found in zesty build with pycodestyle.
    - integration test: initial commit of integration test framework
      [Wesley Wiedenmeier]
    - LICENSE: Allow dual licensing GPL-3 or Apache 2.0 [Jon Grimm]
    - Fix config order of precedence, putting kernel command line over system.
      [Wesley Wiedenmeier] (LP: #1582323)
    - pep8: whitespace fix [Scott Moser]
    - Update the list of valid ssh keys. [Michael Felt]
    - network: add ENI unit test for statically rendered routes.
    - set_hostname: avoid erroneously appending domain to fqdn
      [Lars Kellogg-Stedman] (LP: #1647910)
    - doc: change 'nobootwait' to 'nofail' in docs [Anhad Jai Singh]
    - Replace an expired bit.ly link in code comment. [Joshua Harlow]
    - user-groups: fix bug when groups was provided as string and had spaces
      [Scott Moser] (LP: #1354694)
    - when adding a user, strip whitespace from group list
      [Lars Kellogg-Stedman] (LP: #1354694)
    - fix decoding of utf-8 chars in yaml test
    - Replace usage of sys_netdev_info with read_sys_net
      [Joshua Harlow] (LP: #1625766)
    - fix problems found in python2.6 test. [Joshua Harlow]
    - Just use file logging by default [Joshua Harlow] (LP: #1643990)
    - Improve formatting for ProcessExecutionError [Wesley Wiedenmeier]
    - flake8: fix trailing white space
    - Doc: various documentation fixes [Sean Bright]
    - cloudinit/config/cc_rh_subscription.py: Remove repos before adding
      [Brent Baude]
    - packages/redhat: fix rpm spec file.
    - main: set TZ in environment if not already set. [Ryan Harper]

 -- Scott Moser <email address hidden> Mon, 06 Feb 2017 16:18:28 -0500

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers