| 2020-02-28 21:31:15 |
Dan Streetman |
description |
crash autopkgtests have began to fail on arm64, as seen here:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/arm64/c/crash/20191223_190616_50c73@/log.gz
The significant difference between this test and the last successful one is that the host kernel has moved from 5.3 to 5.4. As crash needs to understand some internals of the kernel memory, it often needs updates to deal with new kernels.
I've verified that the existing crash still works fine w/ a 5.3 kernel[1], but does not w/ 5.4:
$ sudo crash -st /usr/lib/debug/boot/vmlinux-$(uname -r)
crash: cannot determine VA_BITS_ACTUAL: please use /proc/kcore
However, note hte "please use /proc/kcore" - if I point crash at that instead of whatever it normally uses (/dev/mem?) it is able to start [2]. The same is also true w/ current upstream as of commit 7c2d41e btw - it fails w/o /proc/kcore and works with it.
[1]
ubuntu@d05-3:~$ sudo crash /usr/lib/debug/boot/vmlinux-5.3.0-26-generic
crash 7.2.6
Copyright (C) 2002-2019 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/boot/vmlinux-5.3.0-26-generic
DUMPFILE: /proc/kcore
CPUS: 64
DATE: Wed Jan 8 22:32:40 2020
UPTIME: 4 days, 20:46:37
LOAD AVERAGE: 1.09, 0.76, 0.46
TASKS: 836
NODENAME: d05-3
RELEASE: 5.3.0-26-generic
VERSION: #28-Ubuntu SMP Wed Dec 18 05:39:18 UTC 2019
MACHINE: aarch64 (unknown Mhz)
MEMORY: 127.7 GB
PID: 19415
COMMAND: "crash"
TASK: ffff801464623b00 [THREAD_INFO: ffff801464623b00]
CPU: 13
STATE: TASK_RUNNING (ACTIVE)
crash> quit
[2]
ubuntu@d05-3:~$ sudo crash -st /usr/lib/debug/boot/vmlinux-$(uname -r) /proc/kcore
Thu Jan 9 00:20:20 2020
ubuntu@d05-3:~$ sudo crash /usr/lib/debug/boot/vmlinux-$(uname -r) /proc/kcore
crash 7.2.6
Copyright (C) 2002-2019 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/boot/vmlinux-5.4.0-9-generic
DUMPFILE: /proc/kcore
CPUS: 64
DATE: Thu Jan 9 00:20:57 2020
UPTIME: 03:36:15
LOAD AVERAGE: 0.56, 0.31, 0.26
TASKS: 823
NODENAME: d05-3
RELEASE: 5.4.0-9-generic
VERSION: #12-Ubuntu SMP Mon Dec 16 22:33:31 UTC 2019
MACHINE: aarch64 (unknown Mhz)
MEMORY: 127.7 GB
PID: 44363
COMMAND: "crash"
TASK: ffff00179ae0da00 [THREAD_INFO: ffff00179ae0da00]
CPU: 11
STATE: TASK_RUNNING (ACTIVE)
crash> |
[impact]
autopkgtest failures
[test case]
crash autopkgtests have began to fail on arm64, as seen here:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-focal/focal/arm64/c/crash/20191223_190616_50c73@/log.gz
The significant difference between this test and the last successful one is that the host kernel has moved from 5.3 to 5.4. As crash needs to understand some internals of the kernel memory, it often needs updates to deal with new kernels.
I've verified that the existing crash still works fine w/ a 5.3 kernel[1], but does not w/ 5.4:
$ sudo crash -st /usr/lib/debug/boot/vmlinux-$(uname -r)
crash: cannot determine VA_BITS_ACTUAL: please use /proc/kcore
However, note hte "please use /proc/kcore" - if I point crash at that instead of whatever it normally uses (/dev/mem?) it is able to start [2]. The same is also true w/ current upstream as of commit 7c2d41e btw - it fails w/o /proc/kcore and works with it.
[1]
ubuntu@d05-3:~$ sudo crash /usr/lib/debug/boot/vmlinux-5.3.0-26-generic
crash 7.2.6
Copyright (C) 2002-2019 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/boot/vmlinux-5.3.0-26-generic
DUMPFILE: /proc/kcore
CPUS: 64
DATE: Wed Jan 8 22:32:40 2020
UPTIME: 4 days, 20:46:37
LOAD AVERAGE: 1.09, 0.76, 0.46
TASKS: 836
NODENAME: d05-3
RELEASE: 5.3.0-26-generic
VERSION: #28-Ubuntu SMP Wed Dec 18 05:39:18 UTC 2019
MACHINE: aarch64 (unknown Mhz)
MEMORY: 127.7 GB
PID: 19415
COMMAND: "crash"
TASK: ffff801464623b00 [THREAD_INFO: ffff801464623b00]
CPU: 13
STATE: TASK_RUNNING (ACTIVE)
crash> quit
[2]
ubuntu@d05-3:~$ sudo crash -st /usr/lib/debug/boot/vmlinux-$(uname -r) /proc/kcore
Thu Jan 9 00:20:20 2020
ubuntu@d05-3:~$ sudo crash /usr/lib/debug/boot/vmlinux-$(uname -r) /proc/kcore
crash 7.2.6
Copyright (C) 2002-2019 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/boot/vmlinux-5.4.0-9-generic
DUMPFILE: /proc/kcore
CPUS: 64
DATE: Thu Jan 9 00:20:57 2020
UPTIME: 03:36:15
LOAD AVERAGE: 0.56, 0.31, 0.26
TASKS: 823
NODENAME: d05-3
RELEASE: 5.4.0-9-generic
VERSION: #12-Ubuntu SMP Mon Dec 16 22:33:31 UTC 2019
MACHINE: aarch64 (unknown Mhz)
MEMORY: 127.7 GB
PID: 44363
COMMAND: "crash"
TASK: ffff00179ae0da00 [THREAD_INFO: ffff00179ae0da00]
CPU: 11
STATE: TASK_RUNNING (ACTIVE)
crash>
[regression potential]
autopkgtest change only; any regression would involve false positive/negative tests. |
|
