| 2017-04-11 18:39:57 |
bugproxy |
bug |
|
|
added bug |
| 2017-04-11 18:40:00 |
bugproxy |
tags |
|
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin1704 |
|
| 2017-04-11 18:40:02 |
bugproxy |
attachment added |
|
console log https://bugs.launchpad.net/bugs/1681909/+attachment/4860328/+files/kdump_over_ssh_firestone_NV |
|
| 2017-04-11 18:40:05 |
bugproxy |
attachment added |
|
console log https://bugs.launchpad.net/bugs/1681909/+attachment/4860329/+files/kdump_over_ssh_firestone_NV |
|
| 2017-04-11 18:40:22 |
bugproxy |
attachment added |
|
sosreport https://bugs.launchpad.net/bugs/1681909/+attachment/4860330/+files/sosreport-ltc-firep3-20170308001351.tar.xz |
|
| 2017-04-11 18:40:25 |
bugproxy |
attachment added |
|
Console log of successful dump capture after adding a time delay of 'sleep 30' https://bugs.launchpad.net/bugs/1681909/+attachment/4860331/+files/kdump-with-sleep30-ssh.log |
|
| 2017-04-11 18:40:26 |
bugproxy |
ubuntu: assignee |
|
Taco Screen team (taco-screen-team) |
|
| 2017-04-11 18:40:31 |
bugproxy |
affects |
ubuntu |
makedumpfile (Ubuntu) |
|
| 2017-04-26 14:15:59 |
Manoj Iyer |
bug task added |
|
ubuntu-power-systems |
|
| 2017-05-08 19:40:20 |
Manoj Iyer |
makedumpfile (Ubuntu): assignee |
Taco Screen team (taco-screen-team) |
Nish Aravamudan (nacc) |
|
| 2017-05-08 19:40:25 |
Manoj Iyer |
makedumpfile (Ubuntu): importance |
Undecided |
High |
|
| 2017-06-01 15:18:05 |
Manoj Iyer |
tags |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin1704 |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin1704 ubuntu-17.04 |
|
| 2017-07-10 10:49:42 |
bugproxy |
tags |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin1704 ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin16043 ubuntu-17.04 |
|
| 2017-07-19 16:04:52 |
Manoj Iyer |
ubuntu-power-systems: importance |
Undecided |
High |
|
| 2017-07-31 14:38:02 |
Manoj Iyer |
tags |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin16043 ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin16043 triage-a ubuntu-17.04 |
|
| 2017-08-14 17:29:53 |
David Britton |
makedumpfile (Ubuntu): assignee |
Nish Aravamudan (nacc) |
|
|
| 2017-08-14 17:45:19 |
Manoj Iyer |
makedumpfile (Ubuntu): assignee |
|
Canonical Kernel Team (canonical-kernel-team) |
|
| 2017-08-14 19:04:43 |
Joseph Salisbury |
tags |
architecture-ppc64le bugnameltc-152306 severity-high targetmilestone-inin16043 triage-a ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 kernel-da-key severity-high targetmilestone-inin16043 triage-a ubuntu-17.04 |
|
| 2017-08-21 13:49:59 |
Andrew Cloke |
ubuntu-power-systems: assignee |
|
Canonical Kernel Team (canonical-kernel-team) |
|
| 2017-09-11 13:51:41 |
Manoj Iyer |
tags |
architecture-ppc64le bugnameltc-152306 kernel-da-key severity-high targetmilestone-inin16043 triage-a ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 kernel-da-key severity-high targetmilestone-inin16043 triage-r ubuntu-17.04 |
|
| 2017-11-28 17:10:55 |
Andrew Cloke |
tags |
architecture-ppc64le bugnameltc-152306 kernel-da-key severity-high targetmilestone-inin16043 triage-r ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-r ubuntu-17.04 |
|
| 2017-12-04 14:36:12 |
Andrew Cloke |
ubuntu-power-systems: status |
New |
Triaged |
|
| 2018-02-26 14:51:24 |
Andrew Cloke |
ubuntu-power-systems: status |
Triaged |
Incomplete |
|
| 2018-02-26 14:51:43 |
Andrew Cloke |
tags |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-r ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-g ubuntu-17.04 |
|
| 2018-03-05 15:41:34 |
Andrew Cloke |
summary |
Ubuntu 17.04: dump is not captured in remote host when kdump over ssh is configured on firestone. |
dump is not captured in remote host when kdump over ssh is configured on firestone. |
|
| 2018-03-05 15:41:37 |
Frank Heimes |
tags |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-g ubuntu-17.04 |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-g |
|
| 2018-03-05 15:55:06 |
bugproxy |
attachment added |
|
Console log of successful dump capture after adding a time delay of 'sleep 30' https://bugs.launchpad.net/bugs/1681909/+attachment/5070039/+files/kdump-with-sleep30-ssh.log |
|
| 2018-03-06 14:40:44 |
bugproxy |
attachment added |
|
Console log - Kdump failure attempt with Ubuntu 16.04 https://bugs.launchpad.net/bugs/1681909/+attachment/5070786/+files/console_kdump_1604.log |
|
| 2018-03-06 16:31:43 |
bugproxy |
attachment added |
|
Dmesg - Kdump failed attempt with Ubuntu 16.04 https://bugs.launchpad.net/bugs/1681909/+attachment/5070860/+files/dmesg_kdump_ubuntu1604.txt |
|
| 2018-03-19 14:08:50 |
Manoj Iyer |
summary |
dump is not captured in remote host when kdump over ssh is configured on firestone. |
[18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
|
| 2018-03-19 14:09:08 |
Manoj Iyer |
summary |
[18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
[Feat req18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
|
| 2018-03-19 14:28:22 |
Manoj Iyer |
ubuntu-power-systems: importance |
High |
Low |
|
| 2018-03-19 14:28:24 |
Manoj Iyer |
makedumpfile (Ubuntu): importance |
High |
Low |
|
| 2018-04-05 16:11:42 |
Frank Heimes |
summary |
[Feat req18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
[Feat 18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
|
| 2018-04-05 16:29:23 |
bugproxy |
tags |
architecture-ppc64le bugnameltc-152306 kernel-da-key ppc64el-kdump severity-high targetmilestone-inin16043 triage-g |
architecture-ppc64le targetmilestone-inin16043 |
|
| 2018-05-14 17:22:50 |
Frank Heimes |
tags |
architecture-ppc64le targetmilestone-inin16043 |
architecture-ppc64le kernel-da-key ppc64el-kdump targetmilestone-inin16043 triage-g |
|
| 2018-07-02 13:39:41 |
Frank Heimes |
summary |
[Feat 18.10]dump is not captured in remote host when kdump over ssh is configured on firestone. |
[FEAT 18.10] dump is not captured in remote host when kdump over ssh is configured on firestone. |
|
| 2019-05-29 16:46:13 |
Manoj Iyer |
makedumpfile (Ubuntu): assignee |
Canonical Kernel Team (canonical-kernel-team) |
|
|
| 2019-05-29 16:46:17 |
Manoj Iyer |
ubuntu-power-systems: assignee |
Canonical Kernel Team (canonical-kernel-team) |
|
|
| 2019-06-10 13:52:25 |
Manoj Iyer |
makedumpfile (Ubuntu): status |
New |
Incomplete |
|
| 2019-06-12 19:14:48 |
Guilherme G. Piccoli |
summary |
[FEAT 18.10] dump is not captured in remote host when kdump over ssh is configured on firestone. |
kdump is not captured in remote host when kdump over ssh is configured |
|
| 2019-06-12 19:14:55 |
Guilherme G. Piccoli |
bug task deleted |
makedumpfile (Ubuntu) |
|
|
| 2019-06-12 19:16:07 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu) |
|
| 2019-06-12 19:16:16 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu): status |
New |
Confirmed |
|
| 2019-06-12 19:16:18 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu): importance |
Undecided |
Medium |
|
| 2019-06-12 19:16:21 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu): assignee |
|
Guilherme G. Piccoli (gpiccoli) |
|
| 2019-06-12 19:16:47 |
Guilherme G. Piccoli |
bug |
|
|
added subscriber Murilo Fossa Vicentini |
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu Bionic) |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
nominated for series |
|
Ubuntu Cosmic |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu Cosmic) |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
nominated for series |
|
Ubuntu Xenial |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu Xenial) |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
nominated for series |
|
Ubuntu Eoan |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu Eoan) |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
nominated for series |
|
Ubuntu Disco |
|
| 2019-06-12 19:17:27 |
Guilherme G. Piccoli |
bug task added |
|
makedumpfile (Ubuntu Disco) |
|
| 2019-06-12 19:17:37 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): status |
New |
Confirmed |
|
| 2019-06-12 19:17:39 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Bionic): status |
New |
Confirmed |
|
| 2019-06-12 19:17:41 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Cosmic): status |
New |
Confirmed |
|
| 2019-06-12 19:17:43 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Disco): status |
New |
Confirmed |
|
| 2019-06-12 19:17:45 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): importance |
Undecided |
Medium |
|
| 2019-06-12 19:17:46 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Bionic): importance |
Undecided |
Medium |
|
| 2019-06-12 19:17:48 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Cosmic): importance |
Undecided |
Medium |
|
| 2019-06-12 19:17:50 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Disco): importance |
Undecided |
Medium |
|
| 2019-06-12 19:17:51 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Disco): assignee |
|
Guilherme G. Piccoli (gpiccoli) |
|
| 2019-06-12 19:17:52 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Cosmic): assignee |
|
Guilherme G. Piccoli (gpiccoli) |
|
| 2019-06-12 19:17:54 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Bionic): assignee |
|
Guilherme G. Piccoli (gpiccoli) |
|
| 2019-06-12 19:17:56 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): assignee |
|
Guilherme G. Piccoli (gpiccoli) |
|
| 2019-06-12 19:18:40 |
Guilherme G. Piccoli |
tags |
architecture-ppc64le kernel-da-key ppc64el-kdump targetmilestone-inin16043 triage-g |
architecture-ppc64le kernel-da-key ppc64el-kdump sts triage-g |
|
| 2019-06-24 14:18:24 |
Andrew Cloke |
ubuntu-power-systems: status |
Incomplete |
Confirmed |
|
| 2019-07-04 20:02:11 |
Guilherme G. Piccoli |
attachment added |
|
0001-NOT-UPSTREAM-virtio-net-Add-droppkt-param-to-force-d.patch https://bugs.launchpad.net/ubuntu-power-systems/+bug/1681909/+attachment/5275113/+files/0001-NOT-UPSTREAM-virtio-net-Add-droppkt-param-to-force-d.patch |
|
| 2019-07-04 20:33:51 |
Guilherme G. Piccoli |
description |
== Comment: #0 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-07 05:00:29 ==
---Problem Description---
Ubuntu 17.04: dump is not captured in remote host when kdump over ssh is configured on firestone.
---Steps to Reproduce---
1. Configure kdump.
2. Check whether kdump is operational using ?# kdump-config show?.
3. Install ?kernel-debuginfo? and ?kernel-debuginfo-common? rpms.
4. Setup password less ssh connection, generate rsa key.
# ssh-keygen -t rsa
5. verify id_rsa and id_rsa.pub are created under /root/.ssh/
6. Edit /etc/default/kdump-tools and add below entries.
SSH="ubuntu@9.114.15.239"
SSH_KEY=/root/.ssh/id_rsa
7. Propagate RSA key.
# kdump-config propagate
8. Restart kdump service.
# kdump-config load
9. Trigger Crash using below commands.
# echo "1" > /proc/sys/kernel/sysrq
# echo "c" > /proc/sysrq-trigger
10. Verify dump is available in remote server in configured path.
Machine details
===========
$ ipmitool -I lanplus -H 9.47.70.3 -U ADMIN -P admin sol activate
$ ssh ubuntu@9.47.70.29
PW: shriya101
Attaching logs
== Comment: #1 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-07 05:01:42 ==
== Comment: #5 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-07 23:19:46 ==
Hi,
Attaching the logs.
Network info:
root@ltc-firep3:~# hwinfo --network
36: None 00.0: 10700 Loopback
[Created at net.126]
Unique ID: ZsBS.GQNx7L4uPNA
SysFS ID: /class/net/lo
Hardware Class: network interface
Model: "Loopback network interface"
Device File: lo
Link detected: yes
Config Status: cfg=new, avail=yes, need=no, active=unknown
37: None 00.0: 10701 Ethernet
[Created at net.126]
Unique ID: 2lHw.ndpeucax6V1
Parent ID: mIXc.aXC4wIvegH8
SysFS ID: /class/net/enP33p3s0f2
SysFS Device Link: /devices/pci0021:00/0021:00:00.0/0021:01:00.0/0021:02:01.0/0021:03:00.2
Hardware Class: network interface
Model: "Ethernet network interface"
Driver: "tg3"
Driver Modules: "tg3"
Device File: enP33p3s0f2
HW Address: 98:be:94:03:18:4a
Permanent HW Address: 98:be:94:03:18:4a
Link detected: no
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #15 (Ethernet controller)
38: None 00.0: 10701 Ethernet
[Created at net.126]
Unique ID: 7Onn.ndpeucax6V1
Parent ID: sx0U.aXC4wIvegH8
SysFS ID: /class/net/enP33p3s0f0
SysFS Device Link: /devices/pci0021:00/0021:00:00.0/0021:01:00.0/0021:02:01.0/0021:03:00.0
Hardware Class: network interface
Model: "Ethernet network interface"
Driver: "tg3"
Driver Modules: "tg3"
Device File: enP33p3s0f0
HW Address: 98:be:94:03:18:48
Permanent HW Address: 98:be:94:03:18:48
Link detected: yes
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #16 (Ethernet controller)
39: None 00.0: 10701 Ethernet
[Created at net.126]
Unique ID: VwX_.ndpeucax6V1
Parent ID: DUng.aXC4wIvegH8
SysFS ID: /class/net/enP33p3s0f3
SysFS Device Link: /devices/pci0021:00/0021:00:00.0/0021:01:00.0/0021:02:01.0/0021:03:00.3
Hardware Class: network interface
Model: "Ethernet network interface"
Driver: "tg3"
Driver Modules: "tg3"
Device File: enP33p3s0f3
HW Address: 98:be:94:03:18:4b
Permanent HW Address: 98:be:94:03:18:4b
Link detected: no
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #25 (Ethernet controller)
40: None 00.0: 10701 Ethernet
[Created at net.126]
Unique ID: bZ1s.ndpeucax6V1
Parent ID: J7HY.aXC4wIvegH8
SysFS ID: /class/net/enP33p3s0f1
SysFS Device Link: /devices/pci0021:00/0021:00:00.0/0021:01:00.0/0021:02:01.0/0021:03:00.1
Hardware Class: network interface
Model: "Ethernet network interface"
Driver: "tg3"
Driver Modules: "tg3"
Device File: enP33p3s0f1
HW Address: 98:be:94:03:18:49
Permanent HW Address: 98:be:94:03:18:49
Link detected: no
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #4 (Ethernet controller)
root@ltc-firep3:~#
Thanks,
Pavithra
== Comment: #6 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-07 23:20:47 ==
== Comment: #7 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-07 23:21:27 ==
== Comment: #8 - Urvashi Jawere <urjawere@in.ibm.com> - 2017-03-08 02:48:15 ==
I am able to see some errors in syslog ;
auxiliary
Mar 7 04:57:44 ltc-firep3 systemd-resolved[3486]: DNSSEC validation failed for question 114.15.239:/home/ubuntu/test IN SOA: failed-auxiliary
Mar 7 04:57:44 ltc-firep3 systemd-resolved[3486]: DNSSEC validation failed for question 9.114.15.239:/home/ubuntu/test IN DS: failed-auxiliary
Mar 7 04:57:44 ltc-firep3 systemd-resolved[3486]: DNSSEC validation failed for question 9.114.15.239:/home/ubuntu/test IN SOA: failed-auxiliary
Mar 7 04:57:44 ltc-firep3 systemd-resolved[3486]: DNSSEC validation failed for question 9.114.15.239:/home/ubuntu/test IN A: failed-auxiliary
Mar 7 04:57:44 ltc-firep3 systemd-resolved[3486]: Server 9.12.16.2 does not support DNSSEC, downgrading to non-DNSSEC mode.
Mar 7 04:57:44 ltc-firep3 kdump-config: /root/.ssh/id_rsa failed to be sent to ubuntu@9.114.15.239:/home/ubuntu/test
Mar 7 04:58:04 ltc-firep3 systemd[1]: Reloading.
Mar 7 04:59:15 ltc-firep3 systemd[1]: Reloading.
Mar 7 04:59:16 ltc-firep3 kdump-config: propagated ssh key /root/.ssh/id_rsa to server ubuntu@9.114.15.239
.
.
.
Mar 7 05:06:55 ltc-firep3 systemd[1]: Started Accounts Service.
Mar 7 05:06:56 ltc-firep3 kdump-tools[3498]: Starting kdump-tools: Modified cmdline:root=UUID=1e76cfd5-988c-46f4-bdc4-39fe1ed01152 ro quiet splash irqpoll nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0 elfcorehdr=155136K
Mar 7 05:06:57 ltc-firep3 kdump-tools[3498]: * loaded kdump kernel
Mar 7 05:06:57 ltc-firep3 kdump-tools: /sbin/kexec -p --command-line="root=UUID=1e76cfd5-988c-46f4-bdc4-39fe1ed01152 ro quiet splash irqpoll nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
Mar 7 05:06:57 ltc-firep3 kdump-tools: loaded kdump kernel
Mar 7 05:06:57 ltc-firep3 systemd[1]: Started Kernel crash dump capture service.
Mar 7 05:06:57 ltc-firep3 apport[3584]: ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/linux-image-4.10.0-9-generic-201703060521.crash'
Mar 7 05:06:57 ltc-firep3 apport[3584]: ...done.
== Comment: #18 - Hari Krishna Bathini <hbathini@in.ibm.com> - 2017-03-28 06:55:20 ==
Looks like tg3 module was not needed after all. Interesting thing though is
even after enP34p1s0f0 is up (ifup) and network.online target is reached,
network was not really active. It took about 30 seconds, after reaching
network.online target, for the network to be active, even on a normal boot.
Adding this wait time in kdump script, before saving dump, ensured that
vmcore is captured successful. Attaching the log for the same..
Not sure why enP34p1s0f0 is taking that long to configure/initialize. Even so,
this delay should be part of ifup/network-online.target if it is inevitable,
so that network is pingable after network-online.target
Thanks
Hari
== Comment: #19 - Hari Krishna Bathini <hbathini@in.ibm.com> - 2017-03-28 07:01:52 ==
The workaround snippet adding delay in kdump script:
--- kdump-config.orig 2017-03-28 03:35:17.753542107 -0500
+++ kdump-config 2017-03-28 06:59:22.887576623 -0500
@@ -761,6 +761,7 @@
KDUMP_DMESGFILE="$KDUMP_STAMPDIR/dmesg.$KDUMP_STAMP"
ERROR=0
+ sleep 30
ssh -i $KDUMP_SSH_KEY $KDUMP_REMOTE_HOST mkdir -p $KDUMP_STAMPDIR
ERROR=$?
# If remote connections fails, no need to continue
---
Thanks
Hari
== Comment: #20 - PAVITHRA R. PRAKASH <pavrampu@in.ibm.com> - 2017-03-30 01:33:56 ==
(In reply to comment #19)
> The workaround snippet adding delay in kdump script:
>
>
> --- kdump-config.orig 2017-03-28 03:35:17.753542107 -0500
> +++ kdump-config 2017-03-28 06:59:22.887576623 -0500
> @@ -761,6 +761,7 @@
> KDUMP_DMESGFILE="$KDUMP_STAMPDIR/dmesg.$KDUMP_STAMP"
> ERROR=0
>
> + sleep 30
> ssh -i $KDUMP_SSH_KEY $KDUMP_REMOTE_HOST mkdir -p $KDUMP_STAMPDIR
> ERROR=$?
> # If remote connections fails, no need to continue
>
> ---
>
> Thanks
> Hari
With above workaround dump captured successfully in remote host.
Thanks,
Pavithra
== Comment: #22 - Hari Krishna Bathini <hbathini@in.ibm.com> - 2017-04-10 22:14:27 ==
(In reply to comment #18)
> Created attachment 117088 [details]
> Console log of successful dump capture after adding a time delay of 'sleep
> 30'
>
> Looks like tg3 module was not needed after all. Interesting thing though is
> even after enP34p1s0f0 is up (ifup) and network.online target is reached,
> network was not really active. It took about 30 seconds, after reaching
> network.online target, for the network to be active, even on a normal boot.
> Adding this wait time in kdump script, before saving dump, ensured that
> vmcore is captured successful. Attaching the log for the same..
>
> Not sure why enP34p1s0f0 is taking that long to configure/initialize. Even
> so,
> this delay should be part of ifup/network-online.target if it is inevitable,
> so that network is pingable after network-online.target
Hi Canonical,
Since this falls outside the realm of kdump, should we add a NET_WAIT_TIME field
in /etc/default/kdump-tools file that defaults to 0 but can be changed when the
user sees timing troubles?
Thanks
Hari |
[Impact]
* Kdump over network (like NFS mount or SSH dump) relies on network-online target from systemd. Even so, there are some NICs that report "Link Up" state but aren't ready to transmit packets. This is a generally bad behavior that is credited probably to NIC firmware delays, usually not fixable from drivers. Some adapters known to act like this are bnx2x, tg3 and ixgbe.
* Kdump is a mechanism that may be a last resort to debug complex/hard to reproduce issues, so it's interesting to increase its reliability / resilience. We then propose here a solution/quirk to this issue on network dump by adding a retry/delay mechanism; if it's a network dump, kdump will retry some times and sleep between the attempts in order to exclude the case of NICs that aren't ready yet but will soon be able to transmit packets.
* Although first reported by IBM in PowerPC arch, the scope for this issue is the NIC, and it was later reported in x86 arch too.
[Test case]
Usually it's difficult to naturally reproduce this issue in a deterministic way, but we have an artificial test case on comment #24 of this LP.
Also, we have a report from this bug in which the user managed to reproduce the problem consistently - it's fixed after testing our solution.
[Regression potential]
There's not a clear regression potential here since it's just a retry/delay mechanism. Some potential problems may come from bad coding in the script.
The delay between attempts is only 3 sec per iteration, so it shouldn't block the kdump progress for a high amount of time at once. |
|
| 2019-07-04 20:34:05 |
Guilherme G. Piccoli |
bug |
|
|
added subscriber Thadeu Lima de Souza Cascardo |
| 2019-07-04 20:37:25 |
Guilherme G. Piccoli |
attachment added |
|
lp1681909_eoan.debdiff https://bugs.launchpad.net/ubuntu-power-systems/+bug/1681909/+attachment/5275117/+files/lp1681909_eoan.debdiff |
|
| 2019-07-04 20:37:37 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): status |
Confirmed |
In Progress |
|
| 2019-07-04 20:37:40 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Bionic): status |
Confirmed |
In Progress |
|
| 2019-07-04 20:37:43 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Cosmic): status |
Confirmed |
In Progress |
|
| 2019-07-04 20:37:48 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Disco): status |
Confirmed |
In Progress |
|
| 2019-07-04 20:37:50 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Eoan): status |
Confirmed |
In Progress |
|
| 2019-07-04 20:38:09 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Disco): assignee |
Guilherme G. Piccoli (gpiccoli) |
Thadeu Lima de Souza Cascardo (cascardo) |
|
| 2019-07-04 20:38:17 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Cosmic): assignee |
Guilherme G. Piccoli (gpiccoli) |
Thadeu Lima de Souza Cascardo (cascardo) |
|
| 2019-07-04 20:38:26 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Bionic): assignee |
Guilherme G. Piccoli (gpiccoli) |
Thadeu Lima de Souza Cascardo (cascardo) |
|
| 2019-07-04 20:38:33 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): assignee |
Guilherme G. Piccoli (gpiccoli) |
Thadeu Lima de Souza Cascardo (cascardo) |
|
| 2019-07-04 20:48:53 |
Frank Heimes |
ubuntu-power-systems: status |
Confirmed |
In Progress |
|
| 2019-07-05 00:20:50 |
Ubuntu Foundations Team Bug Bot |
tags |
architecture-ppc64le kernel-da-key ppc64el-kdump sts triage-g |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g |
|
| 2019-07-05 00:20:56 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2019-07-08 02:59:58 |
Adolfo Jayme Barrientos |
bug |
|
|
added subscriber Adolfo Jayme |
| 2019-07-20 19:19:46 |
Thadeu Lima de Souza Cascardo |
attachment added |
|
SRU for disco https://bugs.launchpad.net/ubuntu-power-systems/+bug/1681909/+attachment/5278173/+files/makedumpfile_1.6.5-1ubuntu1.1_disco.debdiff |
|
| 2019-07-20 19:20:06 |
Thadeu Lima de Souza Cascardo |
attachment added |
|
SRU for bionic https://bugs.launchpad.net/ubuntu-power-systems/+bug/1681909/+attachment/5278174/+files/makedumpfile_1.6.5-1ubuntu1~18.04.2_bionic.debdiff |
|
| 2019-07-22 13:21:54 |
Guilherme G. Piccoli |
bug |
|
|
added subscriber STS Sponsors |
| 2019-07-22 16:32:21 |
Eric Desrochers |
makedumpfile (Ubuntu Cosmic): status |
In Progress |
Won't Fix |
|
| 2019-07-23 13:47:05 |
Eric Desrochers |
description |
[Impact]
* Kdump over network (like NFS mount or SSH dump) relies on network-online target from systemd. Even so, there are some NICs that report "Link Up" state but aren't ready to transmit packets. This is a generally bad behavior that is credited probably to NIC firmware delays, usually not fixable from drivers. Some adapters known to act like this are bnx2x, tg3 and ixgbe.
* Kdump is a mechanism that may be a last resort to debug complex/hard to reproduce issues, so it's interesting to increase its reliability / resilience. We then propose here a solution/quirk to this issue on network dump by adding a retry/delay mechanism; if it's a network dump, kdump will retry some times and sleep between the attempts in order to exclude the case of NICs that aren't ready yet but will soon be able to transmit packets.
* Although first reported by IBM in PowerPC arch, the scope for this issue is the NIC, and it was later reported in x86 arch too.
[Test case]
Usually it's difficult to naturally reproduce this issue in a deterministic way, but we have an artificial test case on comment #24 of this LP.
Also, we have a report from this bug in which the user managed to reproduce the problem consistently - it's fixed after testing our solution.
[Regression potential]
There's not a clear regression potential here since it's just a retry/delay mechanism. Some potential problems may come from bad coding in the script.
The delay between attempts is only 3 sec per iteration, so it shouldn't block the kdump progress for a high amount of time at once. |
[Impact]
* Kdump over network (like NFS mount or SSH dump) relies on network-online target from systemd. Even so, there are some NICs that report "Link Up" state but aren't ready to transmit packets. This is a generally bad behavior that is credited probably to NIC firmware delays, usually not fixable from drivers. Some adapters known to act like this are bnx2x, tg3 and ixgbe.
* Kdump is a mechanism that may be a last resort to debug complex/hard to reproduce issues, so it's interesting to increase its reliability / resilience. We then propose here a solution/quirk to this issue on network dump by adding a retry/delay mechanism; if it's a network dump, kdump will retry some times and sleep between the attempts in order to exclude the case of NICs that aren't ready yet but will soon be able to transmit packets.
* Although first reported by IBM in PowerPC arch, the scope for this issue is the NIC, and it was later reported in x86 arch too.
[Test case]
Usually it's difficult to naturally reproduce this issue in a deterministic way, but we have an artificial test case on comment #24 of this LP.
Also, we have a report from this bug in which the user managed to reproduce the problem consistently - it's fixed after testing our solution.
[Regression potential]
There's not a clear regression potential here since it's just a retry/delay mechanism. Some potential problems may come from bad coding in the script.
The delay between attempts is only 3 sec per iteration, so it shouldn't block the kdump progress for a high amount of time at once.
[Other information]
Salsa Debian commit:
https://salsa.debian.org/debian/makedumpfile/commit/d63ba95337988be1eac8c8c76d90825ff5c6d17f |
|
| 2019-07-23 14:00:37 |
Eric Desrochers |
makedumpfile (Ubuntu Eoan): status |
In Progress |
Fix Committed |
|
| 2019-07-23 14:15:55 |
Guilherme G. Piccoli |
makedumpfile (Ubuntu Xenial): status |
In Progress |
Won't Fix |
|
| 2019-07-23 19:49:25 |
Eric Desrochers |
bug |
|
|
added subscriber Eric Desrochers |
| 2019-08-16 15:02:44 |
Eric Desrochers |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g |
|
| 2019-08-19 19:58:44 |
Guilherme G. Piccoli |
branch linked |
|
lp:~gpiccoli/britney/hints-ubuntu |
|
| 2019-08-20 16:24:14 |
Launchpad Janitor |
makedumpfile (Ubuntu Eoan): status |
Fix Committed |
Fix Released |
|
| 2019-08-27 14:33:34 |
Eric Desrochers |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2019-08-28 13:42:53 |
Andy Whitcroft |
makedumpfile (Ubuntu Disco): status |
In Progress |
Fix Committed |
|
| 2019-08-28 13:42:56 |
Andy Whitcroft |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2019-08-28 13:42:58 |
Andy Whitcroft |
bug |
|
|
added subscriber SRU Verification |
| 2019-08-28 13:43:05 |
Andy Whitcroft |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g verification-needed verification-needed-disco |
|
| 2019-08-28 13:50:10 |
Andy Whitcroft |
makedumpfile (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2019-08-28 13:50:17 |
Andy Whitcroft |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g verification-needed verification-needed-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g verification-needed verification-needed-bionic verification-needed-disco |
|
| 2019-08-28 14:07:21 |
Andrew Cloke |
ubuntu-power-systems: status |
In Progress |
Fix Committed |
|
| 2019-08-29 12:01:28 |
Eric Desrochers |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts sts-sponsor-slashd triage-g verification-needed verification-needed-bionic verification-needed-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-needed verification-needed-bionic verification-needed-disco |
|
| 2019-08-29 12:01:48 |
Eric Desrochers |
removed subscriber STS Sponsors |
|
|
|
| 2019-08-30 18:54:08 |
Guilherme G. Piccoli |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-needed verification-needed-bionic verification-needed-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-done verification-done-bionic verification-done-disco |
|
| 2019-09-30 14:02:44 |
Andrew Cloke |
ubuntu-power-systems: status |
Fix Committed |
In Progress |
|
| 2019-09-30 14:02:47 |
Andrew Cloke |
makedumpfile (Ubuntu Bionic): status |
Fix Committed |
In Progress |
|
| 2019-09-30 14:02:50 |
Andrew Cloke |
makedumpfile (Ubuntu Disco): status |
Fix Committed |
In Progress |
|
| 2019-10-23 13:56:10 |
Andy Whitcroft |
makedumpfile (Ubuntu Disco): status |
In Progress |
Fix Committed |
|
| 2019-10-23 13:56:21 |
Andy Whitcroft |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-done verification-done-bionic verification-done-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-done-bionic verification-needed verification-needed-disco |
|
| 2019-10-23 13:59:32 |
Andy Whitcroft |
makedumpfile (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2019-10-23 13:59:40 |
Andy Whitcroft |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-done-bionic verification-needed verification-needed-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-needed verification-needed-bionic verification-needed-disco |
|
| 2019-10-31 20:13:44 |
Guilherme G. Piccoli |
tags |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-needed verification-needed-bionic verification-needed-disco |
architecture-ppc64le kernel-da-key patch ppc64el-kdump sts triage-g verification-done verification-done-bionic verification-done-disco |
|
| 2019-11-07 15:29:43 |
Guilherme G. Piccoli |
bug |
|
|
added subscriber Guilherme G. Piccoli |
| 2019-11-11 15:56:26 |
Launchpad Janitor |
makedumpfile (Ubuntu Disco): status |
Fix Committed |
Fix Released |
|
| 2019-11-11 15:56:38 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2019-11-11 15:57:55 |
Launchpad Janitor |
makedumpfile (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2019-11-11 17:41:53 |
Guilherme G. Piccoli |
ubuntu-power-systems: status |
In Progress |
Fix Released |
|
| 2022-08-16 06:27:31 |
Kalpana Shetty |
bug |
|
|
added subscriber Kalpana Shetty |
