Ubuntu 18.04 [ briggs ]: "ipcs" command fails with error "invalid structure member offset" in crash prompt.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Fix Released
|
High
|
Canonical Kernel Team | ||
crash (Ubuntu) |
Fix Released
|
High
|
Canonical Kernel Team | ||
Xenial |
Won't Fix
|
Undecided
|
Unassigned | ||
Artful |
Won't Fix
|
Undecided
|
Unassigned | ||
Bionic |
Fix Released
|
High
|
Thadeu Lima de Souza Cascardo | ||
Cosmic |
Fix Released
|
High
|
Canonical Kernel Team |
Bug Description
[Impact]
ipcs command could not be used with the bionic GA kernel on ppc64le.
[Test Case]
The ipcs command has been run after the fix was applied to crash, on live amd64, ppc64el and s390x system. In fact, the new upload introduces a test to verify this bug does not happen again.
[Regression Potential]
We have only introduced the fix for this specific bug, also adding a test for it. It's less likely to be more broken than the current version on the release as an entire new upstream version could be.
-------
== Comment: #0 - PAVITHRA R. PRAKASH <> - 2018-03-29 01:14:47 ==
---Problem Description---
Ubuntu 18.04: "ipcs" command fails with error "invalid structure member offset" in crash prompt.
---Environment--
System Name : ltc-briggs2
Model/Type : P8
Platform : BML
---Uname output---
root@ltc-briggs2:~# uname -a
Linux ltc-briggs2 4.15.0-13-generic #14-Ubuntu SMP Sat Mar 17 13:43:15 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux
---Steps to reproduce--
1. Configure kdump.
2. Trigger crash
3. run crash on captured dump
---Logs----
root@ltc-briggs2:~# dpkg -l|grep makedumpfile
ii makedumpfile 1:1.6.3-1 ppc64el VMcore extraction tool
root@ltc-briggs2:~# dpkg -l|grep crash
ii apport 2.20.9-0ubuntu1 all automatically generate crash reports for debugging
ii crash 7.2.1-1 ppc64el kernel debugging utility, allowing gdb like syntax
ii kdump-tools 1:1.6.3-1 ppc64el scripts and tools for automating kdump (Linux crash dumps)
ii python3-apport 2.20.9-0ubuntu1 all Python 3 library for Apport crash report handling
root@ltc-briggs2:~# dpkg -l|grep kexec
ii kexec-tools 1:2.0.16-1ubuntu1 ppc64el tools to support fast kexec reboots
root@ltc-briggs2:~#
.0-13-generic dump.201803272257 03272257# crash /usr/lib/
crash 7.2.1
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "powerpc64le-
KERNEL: /usr/lib/
DUMPFILE: dump.201803272257 [PARTIAL DUMP]
CPUS: 160
DATE: Tue Mar 27 22:56:58 2018
UPTIME: 00:04:07
LOAD AVERAGE: 1.06, 0.53, 0.20
TASKS: 1734
NODENAME: ltc-briggs2
RELEASE: 4.15.0-13-generic
VERSION: #14-Ubuntu SMP Sat Mar 17 13:43:15 UTC 2018
MACHINE: ppc64le (2926 Mhz)
MEMORY: 512 GB
PANIC: "sysrq: SysRq : Trigger a crash"
PID: 7420
COMMAND: "bash"
TASK: c000003e56c7c600 [THREAD_INFO: c000003e56cb0000]
CPU: 41
STATE: TASK_RUNNING (SYSRQ)
crash> ?
* files mach repeat timer
alias foreach mod runq tree
ascii fuser mount search union
bt gdb net set vm
btop help p sig vtop
dev ipcs ps struct waitq
dis irq pte swap whatis
eval kmem ptob sym wr
exit list ptov sys q
extend log rd task
crash version: 7.2.1 gdb version: 7.6
For help on any command above, enter "help <command>".
For help on input options, enter "help input".
For help on output options, enter "help output".
crash> ipcs
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 70374748930 => 703747482d8 => 703746e8e98 => 703745b1488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash> q
== Comment: #3 - PAVITHRA R. PRAKASH <> - 2018-04-09 00:36:52 ==
(In reply to comment #1)
> Hi Pavithra,
>
> Have you checked in older kernel versions, whether "ipcs" command
> was working ...I think, it is because there were too many changes
> to kernel's IPC code over several kernel versions..possibly crash tool
> don't have the updated changes w.r.t new kernel IPC code
>
> also please check if following "options" are not working:
> >>>
> char *help_ipcs[] = {
> "ipcs",
> "System V IPC facilities",
> "[-smMq] [-n pid|task] [id | addr]",
>
> " This command provides information on the System V IPC facilities. With
> no",
> " arguments, the command will display kernel usage of all three
> factilities.",
> " ",
> " -s show semaphore arrays.",
> " -m show shared memory segments.",
> " -M show shared memory segments with additional details.",
> " -q show message queues.",
> " id show the data associated with this resource ID.",
> " addr show the data associated with this virtual address of a",
> " shmid_kernel, sem_array or msq_queue.",
> "",
> " For kernels supporting namespaces, the -n option may be used to",
> " display the IPC facilities with respect to the namespace of a",
> " specified task:\n",
> " -n pid a process PID.",
> " -n task a hexadecimal task_struct pointer.",
> >>>
>
> Thanks!!
Issue is observed even with old kernel.
KERNEL: /usr/lib/
DUMPFILE: dump.201804090031 [PARTIAL DUMP]
CPUS: 160
DATE: Mon Apr 9 00:30:53 2018
UPTIME: 00:04:24
LOAD AVERAGE: 2.75, 1.56, 0.64
TASKS: 1738
NODENAME: ltc-briggs2
RELEASE: 4.15.0-12-generic
VERSION: #13-Ubuntu SMP Wed Mar 7 21:37:03 UTC 2018
MACHINE: ppc64le (2926 Mhz)
MEMORY: 512 GB
PANIC: "sysrq: SysRq : Trigger a crash"
PID: 6995
COMMAND: "bash"
TASK: c000003f70dcfd00 [THREAD_INFO: c000003f70e78000]
CPU: 64
STATE: TASK_RUNNING (SYSRQ)
crash> ipcs
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 198c7e38930 => 198c7e382d8 => 198c7dd8e98 => 198c7ca1488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash>
-------
crash> ipcs
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 69b16c28930 => 69b16c282d8 => 69b16bc8e98 => 69b16a91488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash> ipcs -s
SEM_ARRAY KEY SEMID UID PERMS NSEMS
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 69b16c2ace0 => 69b16c282d8 => 69b16bc8e98 => 69b16a91488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash> ipcs -m
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 69b16c28930 => 69b16c282d8 => 69b16bc8e98 => 69b16a91488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash> ipcs -M
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
[/usr/bin/crash] error trace: 69b16c28930 => 69b16c282d8 => 69b16bc8e98 => 69b16a91488
ipcs: invalid structure member offset: idr_top
FILE: ipcs.c LINE: 628 FUNCTION: idr_find()
crash> ipcs -q
MSG_QUEUE KEY MSQID UID PERMS USED-BYTES MESSAGES
(none allocated)
tags: | added: architecture-ppc64le bugnameltc-166199 severity-high targetmilestone-inin1804 |
Changed in ubuntu: | |
assignee: | nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) |
affects: | ubuntu → crash (Ubuntu) |
Changed in ubuntu-power-systems: | |
importance: | Undecided → High |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
tags: | added: triage-g |
Changed in ubuntu-power-systems: | |
status: | New → Triaged |
Changed in crash (Ubuntu): | |
assignee: | Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Kernel Team (canonical-kernel-team) |
importance: | Undecided → High |
description: | updated |
Changed in crash (Ubuntu Bionic): | |
status: | New → In Progress |
assignee: | nobody → Thadeu Lima de Souza Cascardo (cascardo) |
importance: | Undecided → High |
Changed in crash (Ubuntu Artful): | |
status: | New → Won't Fix |
Changed in crash (Ubuntu Xenial): | |
status: | New → Won't Fix |
Changed in ubuntu-power-systems: | |
status: | Triaged → In Progress |
description: | updated |
description: | updated |
description: | updated |
Changed in ubuntu-power-systems: | |
status: | In Progress → Fix Committed |
Changed in ubuntu-power-systems: | |
status: | Fix Committed → Fix Released |
------- Comment From <email address hidden> 2018-04-20 11:30 EDT-------
The ipr structure was removed in the following commit in the kernel:
commit 0a835c4f090af2c 76fc2932c539c3b 32fd21fbbb
Author: Matthew Wilcox <email address hidden>
Date: Tue Dec 20 10:27:56 2016 -0500
Reimplement IDR and IDA using the radix tree
The IDR is very similar to the radix tree. It has some functionality that
the radix tree did not have (alloc next free, cyclic allocation, a
callback-based for_each, destroy tree), which is readily implementable on
top of the radix tree. A few small changes were needed in order to use a
tag to represent nodes with free space below them. More extensive
changes were needed to support storing NULL as a valid entry in an IDR.
Plain radix trees still interpret NULL as a not-present entry.
The IDA is reimplemented as a client of the newly enhanced radix tree. As
in the current implementation, it uses a bitmap at the last level of the
tree.
It seems that crash didn't adjust to this change and this is causing this following error.