Activity log for bug #1795659

Date Who What changed Old value New value Message
2018-10-02 13:58:19 Stefan Stranz bug added bug
2018-10-02 13:58:19 Stefan Stranz attachment added dmesg from /var/crash https://bugs.launchpad.net/bugs/1795659/+attachment/5195853/+files/dmesg.201810020941
2018-10-02 15:14:08 Joseph Salisbury affects linux-hwe (Ubuntu) linux (Ubuntu)
2018-10-02 15:14:08 Joseph Salisbury linux (Ubuntu): importance Undecided High
2018-10-02 15:14:08 Joseph Salisbury linux (Ubuntu): status New Triaged
2018-10-02 15:14:26 Joseph Salisbury tags kernel-da-key
2018-10-02 15:14:42 Joseph Salisbury linux (Ubuntu): status Triaged Incomplete
2018-12-02 04:17:54 Launchpad Janitor linux (Ubuntu): status Incomplete Expired
2019-04-10 13:52:26 Guilherme G. Piccoli bug added subscriber Guilherme G. Piccoli
2019-04-10 14:06:22 Guilherme G. Piccoli bug watch added https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830771
2019-04-10 14:06:31 Guilherme G. Piccoli linux (Ubuntu): assignee Guilherme G. Piccoli (gpiccoli)
2019-04-10 14:06:35 Guilherme G. Piccoli linux (Ubuntu): status Expired Confirmed
2019-04-10 14:06:49 Guilherme G. Piccoli nominated for series Ubuntu Bionic
2019-04-10 14:06:49 Guilherme G. Piccoli bug task added linux (Ubuntu Bionic)
2019-04-10 14:07:00 Guilherme G. Piccoli linux (Ubuntu Bionic): status New Confirmed
2019-04-10 14:07:02 Guilherme G. Piccoli linux (Ubuntu Bionic): importance Undecided High
2019-04-10 14:07:04 Guilherme G. Piccoli linux (Ubuntu Bionic): assignee Guilherme G. Piccoli (gpiccoli)
2019-04-11 14:02:45 Guilherme G. Piccoli tags kernel-da-key kernel-da-key sts
2019-05-23 14:17:34 Guilherme G. Piccoli nominated for series Ubuntu Disco
2019-05-23 14:17:34 Guilherme G. Piccoli bug task added linux (Ubuntu Disco)
2019-05-23 14:17:34 Guilherme G. Piccoli nominated for series Ubuntu Cosmic
2019-05-23 14:17:34 Guilherme G. Piccoli bug task added linux (Ubuntu Cosmic)
2019-05-23 14:17:46 Guilherme G. Piccoli linux (Ubuntu Cosmic): status New Confirmed
2019-05-23 14:17:48 Guilherme G. Piccoli linux (Ubuntu Disco): status New Confirmed
2019-05-23 14:17:50 Guilherme G. Piccoli linux (Ubuntu Cosmic): importance Undecided High
2019-05-23 14:17:52 Guilherme G. Piccoli linux (Ubuntu Disco): importance Undecided High
2019-05-23 14:17:54 Guilherme G. Piccoli linux (Ubuntu Cosmic): assignee Guilherme G. Piccoli (gpiccoli)
2019-05-23 14:17:56 Guilherme G. Piccoli linux (Ubuntu Disco): assignee Guilherme G. Piccoli (gpiccoli)
2019-07-16 21:30:36 Guilherme G. Piccoli linux (Ubuntu Disco): status Confirmed Fix Released
2019-07-16 21:30:42 Guilherme G. Piccoli linux (Ubuntu Cosmic): status Confirmed Won't Fix
2019-07-16 21:30:46 Guilherme G. Piccoli linux (Ubuntu Bionic): status Confirmed In Progress
2019-07-17 18:45:40 Guilherme G. Piccoli summary kernel panic using CIFS share smb2_push_mandatory_locks kernel panic using CIFS share in smb2_push_mandatory_locks()
2019-07-17 18:49:36 Guilherme G. Piccoli description Description: Ubuntu 16.04.5 LTS Release: 16.04 Kernel: 4.15.0-36-generic #39~16.04.1-Ubuntu SMP Tue Sep 25 08:59:23 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Under load, getting a system crash when accessing files on an SMB3 share. dmesg from crash attached. I can upload the crash dump if needed. Share is mounted with the following options: "ro,_netdev,username=*****,password=*****,domain=*****,vers=3.02,sec=ntlmsspi,nounix,noserverino,nobrl,cache=none" Dmesg points to the cifs module [ 2192.662345] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 [ 2192.662407] IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] [Impact] * We got reports of a kernel crash in cifs module with the following signature: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] PGD 0 P4D 0 RIP: 0010:smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] Call Trace: cifs_oplock_break+0x12f/0x3d0 [cifs] process_one_work+0x14d/0x410 worker_thread+0x4b/0x460 kthread+0x105/0x140 [...] Low-level analysis (decodecode script output and the objdump of the function) revealed that we are crashing in a NULL ptr dereference when trying to access "cfile->tlink"; below, a snippet of the objdump at function smb2_push_mandatory_locks(): [...] mov 0x10(%r14),%r15 # %r15 = cifsFileInfo *cfile mov 0x18(%r14),%rbx # %rbx = cifsLockInfo *li = (fdlocks->locks) lea 0x18(%r14),%r12 mov 0x90(%r15),%rax # %rax = struct tcon_link *tlink (cfile->tlink) cmp %r12,%rbx mov 0x38(%rax),%rax # <--- TRAP [trying to get cifs_tcon *tl_tcon] [...] * After discussing the issue with CIFS maintainers (Steve French and Pavel Shilovsky) they suggested commit b98749cac4a6 ("CIFS: keep FileInfo handle live during oplock break") [http://git.kernel.org/linus/b98749cac4a6] as a fix for multiple reports of this kind of crash. * The fix was sent to stable kernels and is present in Ubuntu kernels 5.0 and newer. We are requesting the SRU for this patch here in order to fix the crashes, after reports of successful testing with the patch (see below section) and since the patch is restricted to the cifs module scope and accepted on linux stable. * Alternatively the issue is known to be avoided when oplocks are disabled using "cifs.enable_oplocks=N" module parameter. [Test case] * Unfortunately we cannot reproduce the issue. The patch proposed here was validated by us with xfstests (instructions followed from https://wiki.samba.org/index.php/Xfstesting-cifs) and fio. Also, we have a user report of test validation using LISA (https://github.com/LIS/LISAv2). * Using xfstest with the exclusions proposed in the link above we managed to get the same results as a non-patched kernel, i.e., the same tests failed in both kernels, we didn't get worse results with the patch. Fio also didn't show noticeable performance regression with the patch. [Regression potential] * The patch was validated by the cifs filesystem maintainers (in fact they suggested its inclusion in Ubuntu) and by the aforementioned tests; also, the scope is restricted to cifs only so the likelihood of regressions is considered low. *Due to the nature of the code modification (add a new reference of a file handler and manipulate it in different places), I consider that if we have a regression it'll manifest as deadlock/blocked tasks, not something more serious like crashes or data corruption.
2019-07-17 18:50:12 Guilherme G. Piccoli description [Impact] * We got reports of a kernel crash in cifs module with the following signature: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] PGD 0 P4D 0 RIP: 0010:smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] Call Trace: cifs_oplock_break+0x12f/0x3d0 [cifs] process_one_work+0x14d/0x410 worker_thread+0x4b/0x460 kthread+0x105/0x140 [...] Low-level analysis (decodecode script output and the objdump of the function) revealed that we are crashing in a NULL ptr dereference when trying to access "cfile->tlink"; below, a snippet of the objdump at function smb2_push_mandatory_locks(): [...] mov 0x10(%r14),%r15 # %r15 = cifsFileInfo *cfile mov 0x18(%r14),%rbx # %rbx = cifsLockInfo *li = (fdlocks->locks) lea 0x18(%r14),%r12 mov 0x90(%r15),%rax # %rax = struct tcon_link *tlink (cfile->tlink) cmp %r12,%rbx mov 0x38(%rax),%rax # <--- TRAP [trying to get cifs_tcon *tl_tcon] [...] * After discussing the issue with CIFS maintainers (Steve French and Pavel Shilovsky) they suggested commit b98749cac4a6 ("CIFS: keep FileInfo handle live during oplock break") [http://git.kernel.org/linus/b98749cac4a6] as a fix for multiple reports of this kind of crash. * The fix was sent to stable kernels and is present in Ubuntu kernels 5.0 and newer. We are requesting the SRU for this patch here in order to fix the crashes, after reports of successful testing with the patch (see below section) and since the patch is restricted to the cifs module scope and accepted on linux stable. * Alternatively the issue is known to be avoided when oplocks are disabled using "cifs.enable_oplocks=N" module parameter. [Test case] * Unfortunately we cannot reproduce the issue. The patch proposed here was validated by us with xfstests (instructions followed from https://wiki.samba.org/index.php/Xfstesting-cifs) and fio. Also, we have a user report of test validation using LISA (https://github.com/LIS/LISAv2). * Using xfstest with the exclusions proposed in the link above we managed to get the same results as a non-patched kernel, i.e., the same tests failed in both kernels, we didn't get worse results with the patch. Fio also didn't show noticeable performance regression with the patch. [Regression potential] * The patch was validated by the cifs filesystem maintainers (in fact they suggested its inclusion in Ubuntu) and by the aforementioned tests; also, the scope is restricted to cifs only so the likelihood of regressions is considered low. *Due to the nature of the code modification (add a new reference of a file handler and manipulate it in different places), I consider that if we have a regression it'll manifest as deadlock/blocked tasks, not something more serious like crashes or data corruption. [Impact] * We got reports of a kernel crash in cifs module with the following signature: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] PGD 0 P4D 0 RIP: 0010:smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] Call Trace:  cifs_oplock_break+0x12f/0x3d0 [cifs]  process_one_work+0x14d/0x410  worker_thread+0x4b/0x460  kthread+0x105/0x140 [...] * Low-level analysis (decodecode script output and the objdump of the function) revealed that we are crashing in a NULL ptr dereference when trying to access "cfile->tlink"; below, a snippet of the objdump at function smb2_push_mandatory_locks(): [...] mov 0x10(%r14),%r15 # %r15 = cifsFileInfo *cfile mov 0x18(%r14),%rbx # %rbx = cifsLockInfo *li = (fdlocks->locks) lea 0x18(%r14),%r12 mov 0x90(%r15),%rax # %rax = struct tcon_link *tlink (cfile->tlink) cmp %r12,%rbx mov 0x38(%rax),%rax # <--- TRAP [trying to get cifs_tcon *tl_tcon] [...] * After discussing the issue with CIFS maintainers (Steve French and Pavel Shilovsky) they suggested commit b98749cac4a6 ("CIFS: keep FileInfo handle live during oplock break") [http://git.kernel.org/linus/b98749cac4a6] as a fix for multiple reports of this kind of crash. * The fix was sent to stable kernels and is present in Ubuntu kernels 5.0 and newer. We are requesting the SRU for this patch here in order to fix the crashes, after reports of successful testing with the patch (see below section) and since the patch is restricted to the cifs module scope and accepted on linux stable. * Alternatively the issue is known to be avoided when oplocks are disabled using "cifs.enable_oplocks=N" module parameter. [Test case] * Unfortunately we cannot reproduce the issue. The patch proposed here was validated by us with xfstests (instructions followed from https://wiki.samba.org/index.php/Xfstesting-cifs) and fio. Also, we have a user report of test validation using LISA (https://github.com/LIS/LISAv2). * Using xfstest with the exclusions proposed in the link above we managed to get the same results as a non-patched kernel, i.e., the same tests failed in both kernels, we didn't get worse results with the patch. Fio also didn't show noticeable performance regression with the patch. [Regression potential] * The patch was validated by the cifs filesystem maintainers (in fact they suggested its inclusion in Ubuntu) and by the aforementioned tests; also, the scope is restricted to cifs only so the likelihood of regressions is considered low. * Due to the nature of the code modification (add a new reference of a file handler and manipulate it in different places), I consider that if we have a regression it'll manifest as deadlock/blocked tasks, not something more serious like crashes or data corruption.
2019-07-19 02:49:58 Khaled El Mously linux (Ubuntu Bionic): status In Progress Fix Committed
2019-07-23 15:56:26 Marcelo Cerri bug task added linux-azure (Ubuntu)
2019-07-23 15:56:49 Marcelo Cerri nominated for series Ubuntu Xenial
2019-07-23 15:56:49 Marcelo Cerri bug task added linux (Ubuntu Xenial)
2019-07-23 15:56:49 Marcelo Cerri bug task added linux-azure (Ubuntu Xenial)
2019-07-23 15:56:57 Marcelo Cerri linux (Ubuntu Xenial): status New Invalid
2019-07-23 15:57:17 Marcelo Cerri linux-azure (Ubuntu Bionic): status New Invalid
2019-07-23 15:57:30 Marcelo Cerri linux-azure (Ubuntu Cosmic): status New Invalid
2019-07-23 15:57:58 Marcelo Cerri linux-azure (Ubuntu Disco): status New Invalid
2019-07-23 15:58:14 Marcelo Cerri linux-azure (Ubuntu): importance Undecided Critical
2019-07-23 15:58:14 Marcelo Cerri linux-azure (Ubuntu): status New Confirmed
2019-07-23 15:58:35 Marcelo Cerri linux-azure (Ubuntu Xenial): importance Undecided Critical
2019-07-23 19:27:28 Marcelo Cerri linux-azure (Ubuntu Xenial): status New Fix Committed
2019-07-24 20:24:17 Brad Figg tags kernel-da-key sts cscc kernel-da-key sts
2019-07-25 18:32:55 Ubuntu Kernel Bot tags cscc kernel-da-key sts cscc kernel-da-key sts verification-needed-bionic
2019-07-29 06:58:31 Launchpad Janitor linux-azure (Ubuntu Xenial): status Fix Committed Fix Released
2019-07-31 13:51:26 Guilherme G. Piccoli linux-azure (Ubuntu): status Confirmed Fix Released
2019-07-31 13:52:13 Guilherme G. Piccoli linux (Ubuntu Xenial): assignee Guilherme G. Piccoli (gpiccoli)
2019-07-31 14:27:50 Guilherme G. Piccoli tags cscc kernel-da-key sts verification-needed-bionic cscc kernel-da-key sts verification-done-bionic
2019-08-07 08:34:05 Ubuntu Kernel Bot tags cscc kernel-da-key sts verification-done-bionic cscc kernel-da-key sts verification-done-bionic verification-needed-xenial
2019-08-13 11:27:47 Launchpad Janitor linux (Ubuntu Bionic): status Fix Committed Fix Released
2019-08-13 11:27:47 Launchpad Janitor cve linked 2000-1134
2019-08-13 11:27:47 Launchpad Janitor cve linked 2007-3852
2019-08-13 11:27:47 Launchpad Janitor cve linked 2008-0525
2019-08-13 11:27:47 Launchpad Janitor cve linked 2009-0416
2019-08-13 11:27:47 Launchpad Janitor cve linked 2011-4834
2019-08-13 11:27:47 Launchpad Janitor cve linked 2015-1838
2019-08-13 11:27:47 Launchpad Janitor cve linked 2015-7442
2019-08-13 11:27:47 Launchpad Janitor cve linked 2016-7489
2019-08-13 11:27:47 Launchpad Janitor cve linked 2018-5383
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-10126
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-1125
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-12614
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-12818
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-12819
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-12984
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-13233
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-13272
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-2101
2019-08-13 11:27:47 Launchpad Janitor cve linked 2019-3846
2019-11-04 16:04:02 Guillaume Penin bug added subscriber Guillaume Penin
2019-11-06 16:40:25 Jay Vosburgh bug added subscriber Jay Vosburgh
2020-01-30 14:23:34 Guilherme G. Piccoli description [Impact] * We got reports of a kernel crash in cifs module with the following signature: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] PGD 0 P4D 0 RIP: 0010:smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] Call Trace:  cifs_oplock_break+0x12f/0x3d0 [cifs]  process_one_work+0x14d/0x410  worker_thread+0x4b/0x460  kthread+0x105/0x140 [...] * Low-level analysis (decodecode script output and the objdump of the function) revealed that we are crashing in a NULL ptr dereference when trying to access "cfile->tlink"; below, a snippet of the objdump at function smb2_push_mandatory_locks(): [...] mov 0x10(%r14),%r15 # %r15 = cifsFileInfo *cfile mov 0x18(%r14),%rbx # %rbx = cifsLockInfo *li = (fdlocks->locks) lea 0x18(%r14),%r12 mov 0x90(%r15),%rax # %rax = struct tcon_link *tlink (cfile->tlink) cmp %r12,%rbx mov 0x38(%rax),%rax # <--- TRAP [trying to get cifs_tcon *tl_tcon] [...] * After discussing the issue with CIFS maintainers (Steve French and Pavel Shilovsky) they suggested commit b98749cac4a6 ("CIFS: keep FileInfo handle live during oplock break") [http://git.kernel.org/linus/b98749cac4a6] as a fix for multiple reports of this kind of crash. * The fix was sent to stable kernels and is present in Ubuntu kernels 5.0 and newer. We are requesting the SRU for this patch here in order to fix the crashes, after reports of successful testing with the patch (see below section) and since the patch is restricted to the cifs module scope and accepted on linux stable. * Alternatively the issue is known to be avoided when oplocks are disabled using "cifs.enable_oplocks=N" module parameter. [Test case] * Unfortunately we cannot reproduce the issue. The patch proposed here was validated by us with xfstests (instructions followed from https://wiki.samba.org/index.php/Xfstesting-cifs) and fio. Also, we have a user report of test validation using LISA (https://github.com/LIS/LISAv2). * Using xfstest with the exclusions proposed in the link above we managed to get the same results as a non-patched kernel, i.e., the same tests failed in both kernels, we didn't get worse results with the patch. Fio also didn't show noticeable performance regression with the patch. [Regression potential] * The patch was validated by the cifs filesystem maintainers (in fact they suggested its inclusion in Ubuntu) and by the aforementioned tests; also, the scope is restricted to cifs only so the likelihood of regressions is considered low. * Due to the nature of the code modification (add a new reference of a file handler and manipulate it in different places), I consider that if we have a regression it'll manifest as deadlock/blocked tasks, not something more serious like crashes or data corruption. NOTICE: The new patch merge is being worked on https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1856949 - if you face this issue, please report there! [Impact] * We got reports of a kernel crash in cifs module with the following signature: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] PGD 0 P4D 0 RIP: 0010:smb2_push_mandatory_locks+0x10e/0x3b0 [cifs] Call Trace:  cifs_oplock_break+0x12f/0x3d0 [cifs]  process_one_work+0x14d/0x410  worker_thread+0x4b/0x460  kthread+0x105/0x140 [...] * Low-level analysis (decodecode script output and the objdump of the function) revealed that we are crashing in a NULL ptr dereference when trying to access "cfile->tlink"; below, a snippet of the objdump at function smb2_push_mandatory_locks(): [...] mov 0x10(%r14),%r15 # %r15 = cifsFileInfo *cfile mov 0x18(%r14),%rbx # %rbx = cifsLockInfo *li = (fdlocks->locks) lea 0x18(%r14),%r12 mov 0x90(%r15),%rax # %rax = struct tcon_link *tlink (cfile->tlink) cmp %r12,%rbx mov 0x38(%rax),%rax # <--- TRAP [trying to get cifs_tcon *tl_tcon] [...] * After discussing the issue with CIFS maintainers (Steve French and Pavel Shilovsky) they suggested commit b98749cac4a6 ("CIFS: keep FileInfo handle live during oplock break") [http://git.kernel.org/linus/b98749cac4a6] as a fix for multiple reports of this kind of crash. * The fix was sent to stable kernels and is present in Ubuntu kernels 5.0 and newer. We are requesting the SRU for this patch here in order to fix the crashes, after reports of successful testing with the patch (see below section) and since the patch is restricted to the cifs module scope and accepted on linux stable. * Alternatively the issue is known to be avoided when oplocks are disabled using "cifs.enable_oplocks=N" module parameter. [Test case] * Unfortunately we cannot reproduce the issue. The patch proposed here was validated by us with xfstests (instructions followed from https://wiki.samba.org/index.php/Xfstesting-cifs) and fio. Also, we have a user report of test validation using LISA (https://github.com/LIS/LISAv2). * Using xfstest with the exclusions proposed in the link above we managed to get the same results as a non-patched kernel, i.e., the same tests failed in both kernels, we didn't get worse results with the patch. Fio also didn't show noticeable performance regression with the patch. [Regression potential] * The patch was validated by the cifs filesystem maintainers (in fact they suggested its inclusion in Ubuntu) and by the aforementioned tests; also, the scope is restricted to cifs only so the likelihood of regressions is considered low. * Due to the nature of the code modification (add a new reference of a file handler and manipulate it in different places), I consider that if we have a regression it'll manifest as deadlock/blocked tasks, not something more serious like crashes or data corruption.
2020-07-14 14:53:28 Guilherme G. Piccoli linux (Ubuntu): status Confirmed Fix Released