| 2016-08-19 15:39:30 |
bugproxy |
bug |
|
|
added bug |
| 2016-08-19 15:39:32 |
bugproxy |
tags |
|
architecture-ppc64le bugnameltc-145180 severity-high targetmilestone-inin16041 |
|
| 2016-08-19 15:39:34 |
bugproxy |
attachment added |
|
screen session output https://bugs.launchpad.net/bugs/1615021/+attachment/4724076/+files/screen.out |
|
| 2016-08-19 15:39:36 |
bugproxy |
attachment added |
|
init (modified on initrd) https://bugs.launchpad.net/bugs/1615021/+attachment/4724077/+files/init |
|
| 2016-08-19 15:39:37 |
bugproxy |
attachment added |
|
start-udev (modified on initrd) https://bugs.launchpad.net/bugs/1615021/+attachment/4724078/+files/start-udev |
|
| 2016-08-19 15:39:39 |
bugproxy |
ubuntu: assignee |
|
Taco Screen team (taco-screen-team) |
|
| 2016-08-19 15:39:49 |
bugproxy |
affects |
ubuntu |
systemd (Ubuntu) |
|
| 2016-08-23 12:51:03 |
Breno Leitão |
systemd (Ubuntu): status |
New |
Confirmed |
|
| 2016-08-23 19:21:22 |
Steve Langasek |
affects |
systemd (Ubuntu) |
debian-installer (Ubuntu) |
|
| 2016-08-23 19:25:16 |
Steve Langasek |
bug task added |
|
busybox (Ubuntu) |
|
| 2016-08-23 19:54:20 |
Steve Langasek |
bug task added |
|
systemd (Ubuntu) |
|
| 2016-08-23 20:02:03 |
Steve Langasek |
systemd (Ubuntu): status |
New |
Triaged |
|
| 2016-08-23 20:02:03 |
Steve Langasek |
systemd (Ubuntu): assignee |
|
Martin Pitt (pitti) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
nominated for series |
|
Ubuntu Xenial |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
debian-installer (Ubuntu Xenial) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
busybox (Ubuntu Xenial) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
systemd (Ubuntu Xenial) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
nominated for series |
|
Ubuntu Yakkety |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
debian-installer (Ubuntu Yakkety) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
busybox (Ubuntu Yakkety) |
|
| 2016-08-23 20:02:48 |
Steve Langasek |
bug task added |
|
systemd (Ubuntu Yakkety) |
|
| 2016-08-23 20:05:02 |
Steve Langasek |
busybox (Ubuntu Xenial): status |
New |
Won't Fix |
|
| 2016-08-23 20:05:06 |
Steve Langasek |
busybox (Ubuntu Yakkety): status |
New |
Fix Committed |
|
| 2016-08-23 20:05:33 |
Steve Langasek |
debian-installer (Ubuntu Xenial): status |
New |
Triaged |
|
| 2016-08-23 20:05:36 |
Steve Langasek |
debian-installer (Ubuntu Yakkety): status |
Confirmed |
Triaged |
|
| 2016-08-23 20:05:39 |
Steve Langasek |
systemd (Ubuntu Xenial): status |
New |
Triaged |
|
| 2016-08-23 20:05:50 |
Steve Langasek |
systemd (Ubuntu Xenial): assignee |
|
Martin Pitt (pitti) |
|
| 2016-08-24 04:06:33 |
Martin Pitt |
description |
== Comment: #7 - Guilherme Guaglianoni Piccoli <gpiccoli@br.ibm.com> - 2016-08-19 10:08:07 ==
The normal procedure to perform a Netboot installation of Ubuntu 16.04 is to download the latest vmlinux and initrd.gz files available, and kexec them with no parameters (at least in ppc64el).
We're experiencing a strange issue in which the installer freezes before menus are showed. The system hangs in the point specified below, right after the i40e driver initialization:
[ 11.052832] i40e 0002:01:00.0 enP2p1s0f0: renamed from eth0
[ 11.073976] i40e 0002:01:00.1 enP2p1s0f1: renamed from eth1
[ 11.117799] i40e 0002:01:00.2 enP2p1s0f2: renamed from eth2
[ 11.225745] i40e 0002:01:00.3 enP2p1s0f3: renamed from eth3
***HANG***
The most difficult part in this issue is that it seems to be a timing issue/race condition, and many debug trials end up by avoiding the issue reproduction (heisenbug).
We were successful though in getting logs by booting the kernel with the command-line "BOOT_DEBUG=2" and by changing the initrd in order to enable systemd debug; only the files "init" and "start-udev" were changed in initrd, both attached here.
We've attached here a saved screen session that shows the entire boot process until it gets flooded with lots of messages like:
"starting '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'
'/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/
udev/rules.d/80-net-setup-link.rules': No such file or directory'
seq 3244 queued, 'add' 'pci_bus'
starting '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'
passed 408 byte device to netlink monitor 0x1003cfe8020seq 3236 running'/bin/readlink /etc/udev/rules.d/80-net-setup-l
ink.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules': No such
file or directory'
'/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/
udev/rules.d/80-net-setup-link.rules': No such file or directory'
Process '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules' failed with exit code 2.
PROGRAM '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules' /lib/udev/rules.d/73-usb-net-by-mac.rules:6
passed device to netlink monitor 0x1003d01f730
"
Then it keeps hanged in this stage. We re-tested it by changing the file 73-usb-net-by-mac.rules in initrd, replacing " /etc/udev/rules.d/80-net-setup-link.rules" to "/lib/udev/rules.d/80-net-setup-link.rules", since the former does not exist whereas the latter does. Same issue were observed!
Notice that if we boot the installer with command-line "net.ifnames=0" or "net.ifnames=1", the problem does not reproduces anymore.
We want to ask Canonical's help in investigating this issue.
Thanks,
Guilherme
== Comment: #8 - Guilherme Guaglianoni Piccoli <gpiccoli@br.ibm.com> - 2016-08-19 10:09:51 ==
== Comment: #9 - Guilherme Guaglianoni Piccoli <gpiccoli@br.ibm.com> - 2016-08-19 10:10:31 ==
== Comment: #10 - Guilherme Guaglianoni Piccoli <gpiccoli@br.ibm.com> - 2016-08-19 10:11:49 == |
== Comment: #7 - Guilherme Guaglianoni Piccoli <gpiccoli@br.ibm.com> - 2016-08-19 10:08:07 ==
The normal procedure to perform a Netboot installation of Ubuntu 16.04 is to download the latest vmlinux and initrd.gz files available, and kexec them with no parameters (at least in ppc64el).
We're experiencing a strange issue in which the installer freezes before menus are showed. The system hangs in the point specified below, right after the i40e driver initialization:
[ 11.052832] i40e 0002:01:00.0 enP2p1s0f0: renamed from eth0
[ 11.073976] i40e 0002:01:00.1 enP2p1s0f1: renamed from eth1
[ 11.117799] i40e 0002:01:00.2 enP2p1s0f2: renamed from eth2
[ 11.225745] i40e 0002:01:00.3 enP2p1s0f3: renamed from eth3
***HANG***
The most difficult part in this issue is that it seems to be a timing issue/race condition, and many debug trials end up by avoiding the issue reproduction (heisenbug).
We were successful though in getting logs by booting the kernel with the command-line "BOOT_DEBUG=2" and by changing the initrd in order to enable systemd debug; only the files "init" and "start-udev" were changed in initrd, both attached here.
We've attached here a saved screen session that shows the entire boot process until it gets flooded with lots of messages like:
"starting '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'
'/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/
udev/rules.d/80-net-setup-link.rules': No such file or directory'
seq 3244 queued, 'add' 'pci_bus'
starting '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'
passed 408 byte device to netlink monitor 0x1003cfe8020seq 3236 running'/bin/readlink /etc/udev/rules.d/80-net-setup-l
ink.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules': No such
file or directory'
'/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules'(err) 'failed to execute '/bin/readlink' '/bin/readlink /etc/
udev/rules.d/80-net-setup-link.rules': No such file or directory'
Process '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules' failed with exit code 2.
PROGRAM '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules' /lib/udev/rules.d/73-usb-net-by-mac.rules:6
passed device to netlink monitor 0x1003d01f730
"
Then it keeps hanged in this stage. We re-tested it by changing the file 73-usb-net-by-mac.rules in initrd, replacing " /etc/udev/rules.d/80-net-setup-link.rules" to "/lib/udev/rules.d/80-net-setup-link.rules", since the former does not exist whereas the latter does. Same issue were observed!
Notice that if we boot the installer with command-line "net.ifnames=0" or "net.ifnames=1", the problem does not reproduces anymore.
We want to ask Canonical's help in investigating this issue.
Thanks,
Guilherme
SRU INFORMATION for systemd
===========================
Test case:
* Check what happens for uevents on devices which are not USB network interfaces:
udevadm test /sys/devices/virtual/mem/null
udevadm test /sys/class/net/lo
With the current version these will run
PROGRAM '/bin/readlink /etc/udev/rules.d/80-net-setup-link.rules' /lib/udev/rules.d/73-usb-net-by-mac.rules:6
which is pointless. With the proposed version these should be gone.
* Ensure that the rule still works as intended by connecting an USB network device that has a permanent MAC address (e. g. Android tethering uses a temporary MAC): You should get a MAC-based name like "enx12345678" for it. Now disconnect it again, disable ifnames with
sudo ln -s /dev/null /etc/udev/rules.d/80-net-setup-link.rules
and reconnect the device. You should now get a kernel name like "usb0" for it.
* Regression potential: Errors in the rule could break persistent naming - or its disabling - of USB network interfaces. Running the above test carefully is important to ensure this keeps working. This has little to no actual effect on anything else on the system (aside from a performance impact and spamming logs), so overall the regression potential is low. |
|
| 2016-08-24 04:47:12 |
Martin Pitt |
systemd (Ubuntu Yakkety): status |
Triaged |
Fix Committed |
|
| 2016-08-24 04:47:14 |
Martin Pitt |
systemd (Ubuntu Xenial): status |
Triaged |
In Progress |
|
| 2016-08-24 16:50:57 |
bugproxy |
attachment added |
|
NEW screen output https://bugs.launchpad.net/bugs/1615021/+attachment/4727113/+files/screen.output |
|
| 2016-08-25 15:11:33 |
bugproxy |
attachment added |
|
Boot log for failed run, no debug options https://bugs.launchpad.net/bugs/1615021/+attachment/4727654/+files/bz145180-1.out |
|
| 2016-08-25 22:33:10 |
Launchpad Janitor |
busybox (Ubuntu Yakkety): status |
Fix Committed |
Fix Released |
|
| 2016-08-29 09:35:10 |
Launchpad Janitor |
systemd (Ubuntu Yakkety): status |
Fix Committed |
Fix Released |
|
| 2016-09-07 09:13:01 |
Andy Whitcroft |
systemd (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2016-09-07 09:13:05 |
Andy Whitcroft |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2016-09-07 09:13:07 |
Andy Whitcroft |
bug |
|
|
added subscriber SRU Verification |
| 2016-09-07 09:13:11 |
Andy Whitcroft |
tags |
architecture-ppc64le bugnameltc-145180 severity-high targetmilestone-inin16041 |
architecture-ppc64le bugnameltc-145180 severity-high targetmilestone-inin16041 verification-needed |
|
| 2016-09-10 12:40:01 |
bugproxy |
attachment added |
|
screen session output https://bugs.launchpad.net/bugs/1615021/+attachment/4738165/+files/screen.out |
|
| 2016-09-10 12:40:04 |
bugproxy |
attachment added |
|
init (modified on initrd) https://bugs.launchpad.net/bugs/1615021/+attachment/4738166/+files/init |
|
| 2016-09-10 12:40:07 |
bugproxy |
attachment added |
|
start-udev (modified on initrd) https://bugs.launchpad.net/bugs/1615021/+attachment/4738167/+files/start-udev |
|
| 2016-09-10 12:40:11 |
bugproxy |
attachment added |
|
NEW screen output https://bugs.launchpad.net/bugs/1615021/+attachment/4738168/+files/screen.output |
|
| 2016-09-10 12:40:14 |
bugproxy |
attachment added |
|
Boot log for failed run, no debug options https://bugs.launchpad.net/bugs/1615021/+attachment/4738169/+files/bz145180-1.out |
|
| 2016-09-12 18:54:22 |
Martin Pitt |
tags |
architecture-ppc64le bugnameltc-145180 severity-high targetmilestone-inin16041 verification-needed |
architecture-ppc64le bugnameltc-145180 severity-high targetmilestone-inin16041 verification-done |
|
| 2016-09-14 11:56:40 |
Martin Pitt |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2016-09-14 12:01:19 |
Launchpad Janitor |
systemd (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2016-09-19 19:30:56 |
bugproxy |
bug watch added |
|
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819988 |
|
| 2016-09-19 20:43:05 |
Martin Pitt |
debian-installer (Ubuntu Yakkety): status |
Triaged |
Invalid |
|
| 2016-09-19 20:43:19 |
Martin Pitt |
debian-installer (Ubuntu Xenial): status |
Triaged |
Invalid |
|
