[arm64] compute nodes unstable after upgrading from 4.2 to 4.4 kernel
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Xenial |
Fix Released
|
High
|
Unassigned |
Bug Description
Hi,
In order to investigate bug LP#1531768, we upgraded some arm64 compute nodes (swirlices) to a 4.4 kernel. I think it made the VMs work better, but the hosts became extremely unstable.
After some time, getting a shell on them would be impossible. Connecting on the VSP, you'd get a prompt, and once you typed your username and password, you'd see the motd but the shell would never spawn.
Because of these instability issues, all the arm64 compute nodes are now back on 4.2. However, we managed to capture "perf record" data when a host was failing. I'll attach it to the bug. Perhaps it will give you hints as to what we can do to help you troubleshoot this bug further.
Once we have your instructions, we'll happily reboot one (or a few) nodes to 4.4 to continue troubleshooting.
Thanks !
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jul 12 12:54 seq
crw-rw---- 1 root audio 116, 33 Jul 12 12:54 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.1-0ubuntu3.21
Architecture: arm64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 14.04
Lsusb: Error: command ['lsusb'] failed with exit code 1: unable to initialize libusb: -99
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=screen-
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: console=
ProcVersionSign
RelatedPackageV
linux-
linux-
linux-firmware 1.127.22
RfKill: Error: [Errno 2] No such file or directory
Tags: trusty uec-images
Uname: Linux 4.2.0-41-generic aarch64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
Changed in linux (Ubuntu): | |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
importance: | Undecided → High |
tags: | added: kernel-key |
tags: |
added: kernel-da-key removed: kernel-key |
Changed in linux (Ubuntu): | |
status: | In Progress → Fix Released |
Changed in linux (Ubuntu Xenial): | |
status: | In Progress → Fix Released |
perf data is at : https:/ /private- fileshare. canonical. com/~axino/ swirlix01. perf.data. LP1602577. xz
I'll run apport from this host, note that it's now running 4.2.
Version string of the unstable kernel was : bos01-arm64- 006) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~ 14.04.3) ) #49~14.04.1-Ubuntu SMP Thu Jun 30 22:20:09 UTC 2016 (Ubuntu 4.4.0-30. 49~14.04. 1-generic 4.4.13)
swirlix01 kernel: [ 0.000000] Linux version 4.4.0-30-generic (buildd@