kernel crash dump create error on Huawei server for 18.04.3 hwe-kernel

[Zhanglei Mao]

coping data get io error

[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Started Availability of block devices.
[ 185.379378] kdump-tools[1695]: Starting kdump-tools: * running makedumpfile -c -d 31 /proc/vmcore /var/crash/201912310519/dump-incomplete
Copying data :[ 421.979156] print_req_error: I/O error, dev sdb, sector 82830936 flags 0
 [ 54.8 %] | [ 422.081318] bcache: bch_count_io_errors() sdb1: IO error on reading from cache, recovering.

[Zhanglei Mao (zhanglei-mao)]

ubuntu@infra1:~$ sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 93.1G 0 loop /var/lib/lxd/storage-pools/default
sda 8:0 0 1.1T 0 disk
├─sda1 8:1 0 512M 0 part /boot/efi
└─sda2 8:2 0 1.1T 0 part
  ├─infra1--vg-root 253:0 0 1.1T 0 lvm /
  └─infra1--vg-swap_1 253:1 0 976M 0 lvm [SWAP]
sdb 8:16 0 557.9G 0 disk
sdc 8:32 0 557.9G 0 disk
ubuntu@infra1:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 126G 0 126G 0% /dev
tmpfs 26G 2.4M 26G 1% /run
/dev/mapper/infra1--vg-root 1.1T 313G 729G 31% /
tmpfs 126G 54M 126G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 126G 0 126G 0% /sys/fs/cgroup
/dev/sda1 511M 6.1M 505M 2% /boot/efi
tmpfs 100K 0 100K 0% /var/lib/lxd/shmounts
tmpfs 100K 0 100K 0% /var/lib/lxd/devlxd
/dev/loop0 94G 6.2G 86G 7% /var/lib/lxd/storage-pools/default
tmpfs 26G 0 26G 0% /run/user/1000
ubuntu@infra1:~$ cd /var/crash/
ubuntu@infra1:/var/crash$ ls -lhs
total 8.7M
8.7M -rw-r----- 1 bind bind 8.7M Dec 24 05:39 _usr_sbin_named.115.crash
ubuntu@infra1:/var/crash$

root@node1:~# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x4000000
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.0.0-37-generic
kdump initrd:
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-5.0.0-37-generic
current state: ready to kdump

kexec command:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/vmlinuz-5.0.0-37-generic root=UUID=d9281738-de5a-4564-852e-f07c29ea4817 ro default_hugepagesz=1GB transparent_hugepage=never solcpus=4-13,32-41,18-27,46-55 console=tty0 console=ttyS0 systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
root@node1:~# dmesg |grep -i crash
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-5.0.0-37-generic root=UUID=d9281738-de5a-4564-852e-f07c29ea4817 ro default_hugepagesz=1GB hugepagesz=1GB hugepages=192 transparent_hugepage=never solcpus=4-13,32-41,18-27,46-55 console=tty0 console=ttyS0 crashkernel=2G-:512M
[ 0.027777] Reserving 512MB of memory at 64MB for crashkernel (System RAM: 261726MB)
[ 1.921643] Kernel command line: BOOT_IMAGE=/vmlinuz-5.0.0-37-generic root=UUID=d9281738-de5a-4564-852e-f07c29ea4817 ro default_hugepagesz=1GB hugepagesz=1GB hugepages=192 transparent_hugepage=never solcpus=4-13,32-41,18-27,46-55 console=tty0 console=ttyS0 crashkernel=2G-:512M
[ 73.044719] pstore: Using crash dump compression: deflate
[ 79.287259] megaraid_sas 0000:1c:00.0: firmware crash dump : no
root@node1:~#

[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Started Availability of block devices.
[ ...

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1862720

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tom Reynolds (tomreyn)]

"I/O error, dev sdb, sector 82830936 flags 0" would seem to suggest this is a hardware error (physically defective storage media sdb), have you ruled this out, yet?

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]