Bug #1454903 “Btrfs Fails to Mount Drive - btrfs-transaction blo...” : Bugs : linux package : Ubuntu

Revision history for this message

jdfalk (johnathan-falk) wrote on 2015-05-14:

#1

AlsaInfo.txt Edit (43.3 KiB, text/plain; charset="utf-8")
CRDA.txt Edit (238 bytes, text/plain; charset="utf-8")
CurrentDmesg.txt Edit (117.9 KiB, text/plain; charset="utf-8")
Dependencies.txt Edit (2.3 KiB, text/plain; charset="utf-8")
IwConfig.txt Edit (175 bytes, text/plain; charset="utf-8")
JournalErrors.txt Edit (39.2 KiB, text/plain; charset="utf-8")
Lspci.txt Edit (39.3 KiB, text/plain; charset="utf-8")
Lsusb.txt Edit (664 bytes, text/plain; charset="utf-8")
ProcCpuinfo.txt Edit (3.8 KiB, text/plain; charset="utf-8")
ProcInterrupts.txt Edit (2.6 KiB, text/plain; charset="utf-8")
ProcModules.txt Edit (5.8 KiB, text/plain; charset="utf-8")
UdevDb.txt Edit (163.4 KiB, text/plain; charset="utf-8")
WifiSyslog.txt Edit (149.7 KiB, text/plain; charset="utf-8")

Revision history for this message

Brad Figg (brad-figg) wrote on 2015-05-14: Status changed to Confirmed

#2

This change was made by a bot.

Changed in linux (Ubuntu):
status:	New → Confirmed

Revision history for this message

Joseph Salisbury (jsalisbury) wrote on 2015-05-14:

#3

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.1 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.1-rc3-vivid/

Changed in linux (Ubuntu):
importance:	Undecided → Medium
status:	Confirmed → Incomplete

Alberto Salvia Novella (es20490446e) on 2015-05-14

Changed in linux (Ubuntu):
importance:	Medium → High

Revision history for this message

jdfalk (johnathan-falk) wrote on 2015-05-15: Re: [Bug 1454903] Re: Btrfs Fails to Mount Drive - btrfs-transaction blocked for more than 120 seconds.

#4

Download full text (4.7 KiB)

It slowly got worse until my whole system wouldn't start. Finally I ran
btrfs check --repair /dev/sdx1 from a usb boot drive and it said: "cache
and super generation don't match, space cache will be invalidated"

After that it mounted right away. However I will proceed with the newer
kernel as we don't know what caused the corruption in the first place and
why it didn't resolve this automagically.
On Thu, May 14, 2015 at 11:01 AM Alberto Salvia Novella <
<email address hidden>> wrote:

> ** Changed in: linux (Ubuntu)
> Importance: Medium => High
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1454903
>
> Title:
> Btrfs Fails to Mount Drive - btrfs-transaction blocked for more than
> 120 seconds.
>
> Status in linux package in Ubuntu:
> Incomplete
>
> Bug description:
> Call Trace:
> kernel: ffff880408609800 ffff8803e9cba200 ffff8803d06f7398
> ffff8803bc89fe00
> kernel: 0000000000014200 ffff88040ce9f5c0 ffff880407f9d850
> ffff8803bc89fdd8
> kernel: ffff8803bc89fdc8 ffff880407f9d850 0000000000014200
> ffff8803bc89ffd8
> kernel: btrfs-transacti D ffff8803bc89fdc8 0 4187 2 0x00000000
> kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
> kernel: Not tainted 3.19.0-16-generic #16-Ubuntu
> kernel: INFO: task btrfs-transacti:4187 blocked for more than 120
> seconds.
> kernel: [<ffffffff817c934d>] system_call_fastpath+0x16/0x1b
> kernel: [<ffffffff812175db>] SyS_mount+0x8b/0xd0
> kernel: [<ffffffff812169a4>] do_mount+0x204/0xb20
> kernel: [<ffffffff81213b4b>] vfs_kern_mount+0x6b/0x120
> kernel: [<ffffffff8119b805>] ? __alloc_percpu+0x15/0x20
> kernel: [<ffffffff811f7428>] mount_fs+0x38/0x1c0
> kernel: [<ffffffffc0333b00>] btrfs_mount+0x850/0x920 [btrfs]
> kernel: [<ffffffffc035db23>] open_ctree+0x1813/0x2090 [btrfs]
> kernel: [<ffffffffc039f9c0>] ? replay_one_extent+0x6b0/0x6b0 [btrfs]
> kernel: [<ffffffffc03a2214>] btrfs_recover_log_trees+0x1d4/0x470 [btrfs]
> kernel: [<ffffffffc039abbb>] walk_log_tree+0xbb/0x1a0 [btrfs]
> kernel: [<ffffffffc039a679>] walk_down_log_tree+0x1d9/0x420 [btrfs]
> kernel: [<ffffffffc0382f34>] ? alloc_extent_buffer+0x34/0x350 [btrfs]
> kernel: [<ffffffffc039b04e>] insert_orphan_item+0x5e/0x90 [btrfs]
>
> ProblemType: Bug
> DistroRelease: Ubuntu 15.04
> Package: linux-image-3.19.0-16-generic 3.19.0-16.16
> ProcVersionSignature: Ubuntu 3.19.0-16.16-generic 3.19.3
> Uname: Linux 3.19.0-16-generic x86_64
> ApportVersion: 2.17.2-0ubuntu1
> Architecture: amd64
> AudioDevicesInUse:
> USER PID ACCESS COMMAND
> /dev/snd/controlC0: jdfalk 3444 F.... pulseaudio
> Date: Wed May 13 19:03:10 2015
> HibernationDevice: RESUME=/dev/mapper/sda5_crypt
> InstallationDate: Installed on 2015-05-10 (4 days ago)
> InstallationMedia: Ubuntu-Server 15.04 "Vivid Vervet" - Release amd64
> (20150422)
> MachineType: Gigabyte Technology Co., Ltd. To be filled by O.E.M.
> ProcEnviron:
> TERM=xterm-256color
> PATH=(custom, no user)
> LANG=en_US.UTF-8
> SHELL=/bin/bash
> ProcFB: 0 inteldrmfb
> ProcKerne...

It slowly got worse until my whole system wouldn't start. Finally I ran
btrfs check --repair /dev/sdx1 from a usb boot drive and it said: "cache
and super generation don't match, space cache will be invalidated"

After that it mounted right away. However I will proceed with the newer
kernel as we don't know what caused the corruption in the first place and
why it didn't resolve this automagically.
On Thu, May 14, 2015 at 11:01 AM Alberto Salvia Novella <
es20490446e@gmail.com> wrote:

> ** Changed in: linux (Ubuntu)
>    Importance: Medium => High
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1454903
>
> Title:
>   Btrfs Fails to Mount Drive - btrfs-transaction blocked for more than
>   120 seconds.
>
> Status in linux package in Ubuntu:
>   Incomplete
>
> Bug description:
>   Call Trace:
>   kernel:  ffff880408609800 ffff8803e9cba200 ffff8803d06f7398
> ffff8803bc89fe00
>   kernel:  0000000000014200 ffff88040ce9f5c0 ffff880407f9d850
> ffff8803bc89fdd8
>   kernel:  ffff8803bc89fdc8 ffff880407f9d850 0000000000014200
> ffff8803bc89ffd8
>   kernel: btrfs-transacti D ffff8803bc89fdc8     0  4187      2 0x00000000
>   kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
> message.
>   kernel:       Not tainted 3.19.0-16-generic #16-Ubuntu
>   kernel: INFO: task btrfs-transacti:4187 blocked for more than 120
> seconds.
>   kernel:  [<ffffffff817c934d>] system_call_fastpath+0x16/0x1b
>   kernel:  [<ffffffff812175db>] SyS_mount+0x8b/0xd0
>   kernel:  [<ffffffff812169a4>] do_mount+0x204/0xb20
>   kernel:  [<ffffffff81213b4b>] vfs_kern_mount+0x6b/0x120
>   kernel:  [<ffffffff8119b805>] ? __alloc_percpu+0x15/0x20
>   kernel:  [<ffffffff811f7428>] mount_fs+0x38/0x1c0
>   kernel:  [<ffffffffc0333b00>] btrfs_mount+0x850/0x920 [btrfs]
>   kernel:  [<ffffffffc035db23>] open_ctree+0x1813/0x2090 [btrfs]
>   kernel:  [<ffffffffc039f9c0>] ? replay_one_extent+0x6b0/0x6b0 [btrfs]
>   kernel:  [<ffffffffc03a2214>] btrfs_recover_log_trees+0x1d4/0x470 [btrfs]
>   kernel:  [<ffffffffc039abbb>] walk_log_tree+0xbb/0x1a0 [btrfs]
>   kernel:  [<ffffffffc039a679>] walk_down_log_tree+0x1d9/0x420 [btrfs]
>   kernel:  [<ffffffffc0382f34>] ? alloc_extent_buffer+0x34/0x350 [btrfs]
>   kernel:  [<ffffffffc039b04e>] insert_orphan_item+0x5e/0x90 [btrfs]
>
>   ProblemType: Bug
>   DistroRelease: Ubuntu 15.04
>   Package: linux-image-3.19.0-16-generic 3.19.0-16.16
>   ProcVersionSignature: Ubuntu 3.19.0-16.16-generic 3.19.3
>   Uname: Linux 3.19.0-16-generic x86_64
>   ApportVersion: 2.17.2-0ubuntu1
>   Architecture: amd64
>   AudioDevicesInUse:
>    USER        PID ACCESS COMMAND
>    /dev/snd/controlC0:  jdfalk     3444 F.... pulseaudio
>   Date: Wed May 13 19:03:10 2015
>   HibernationDevice: RESUME=/dev/mapper/sda5_crypt
>   InstallationDate: Installed on 2015-05-10 (4 days ago)
>   InstallationMedia: Ubuntu-Server 15.04 "Vivid Vervet" - Release amd64
> (20150422)
>   MachineType: Gigabyte Technology Co., Ltd. To be filled by O.E.M.
>   ProcEnviron:
>    TERM=xterm-256color
>    PATH=(custom, no user)
>    LANG=en_US.UTF-8
>    SHELL=/bin/bash
>   ProcFB: 0 inteldrmfb
>   ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.19.0-16-generic.efi.signed
> root=UUID=51a981ad-aeb0-4bd7-91c6-2d493155c812 ro rootflags=subvol=@
> crashkernel=384M-:128M
>   PulseList: Error: command ['pacmd', 'list'] failed with exit code 1: No
> PulseAudio daemon running, or not running as session daemon.
>   RelatedPackageVersions:
>    linux-restricted-modules-3.19.0-16-generic N/A
>    linux-backports-modules-3.19.0-16-generic  N/A
>    linux-firmware                             1.143
>   RfKill:
>
>   SourcePackage: linux
>   UdevLog: Error: [Errno 2] No such file or directory: '/var/log/udev'
>   UpgradeStatus: No upgrade log present (probably fresh install)
>   dmi.bios.date: 01/06/2014
>   dmi.bios.vendor: American Megatrends Inc.
>   dmi.bios.version: F18i
>   dmi.board.asset.tag: To be filled by O.E.M.
>   dmi.board.name: Z77X-D3H
>   dmi.board.vendor: Gigabyte Technology Co., Ltd.
>   dmi.board.version: x.x
>   dmi.chassis.asset.tag: To Be Filled By O.E.M.
>   dmi.chassis.type: 3
>   dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
>   dmi.chassis.version: To Be Filled By O.E.M.
>   dmi.modalias:
> dmi:bvnAmericanMegatrendsInc.:bvrF18i:bd01/06/2014:svnGigabyteTechnologyCo.,Ltd.:pnTobefilledbyO.E.M.:pvrTobefilledbyO.E.M.:rvnGigabyteTechnologyCo.,Ltd.:rnZ77X-D3H:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvrToBeFilledByO.E.M.:
>   dmi.product.name: To be filled by O.E.M.
>   dmi.product.version: To be filled by O.E.M.
>   dmi.sys.vendor: Gigabyte Technology Co., Ltd.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1454903/+subscriptions
>

Revision history for this message

jdfalk (johnathan-falk) wrote on 2015-05-15: JournalErrors.txt

#5

JournalErrors.txt Edit (10.6 KiB, text/plain)

apport information

tags:	added: apport-collected
description:	updated

Revision history for this message

jdfalk (johnathan-falk) wrote on 2015-05-15:

#6

Updated to the requested mainline kernel.
uname --all
Linux 4.1.0-040100rc3-generic #201505102036 SMP Mon May 11 00:37:27 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I don't want to tag this as fixed by the upstream kernel because we don't know what actually caused the issue in the first place.

Changed in linux (Ubuntu):
status:	Incomplete → Confirmed

jdfalk (johnathan-falk) on 2015-08-09

tags:

added: kernel-fixed-upstream

Revision history for this message

Thomas Mayer (thomas303) wrote on 2017-05-07:

#7

I've just seen this with 16.04, together with latest HES 4.8 kernel:

uname -a
Linux server 4.8.0-51-generic #54~16.04.1-Ubuntu SMP Wed Apr 26 16:00:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

That happened during a btrfs-scrub:

May 7 02:11:52 server kernel: [17884.471597] [<ffffffffc04cbc73>] wait_current_trans.isra.21+0xd3/0x120 [btrfs]
May 7 02:11:52 server kernel: [17884.471617] [<ffffffffc04ce43f>] start_transaction+0x27f/0x4c0 [btrfs]
May 7 02:11:52 server kernel: [17884.471635] [<ffffffffc04ce6d7>] btrfs_join_transaction+0x17/0x20 [btrfs]
May 7 02:11:52 server kernel: [17884.471654] [<ffffffffc04d4b45>] btrfs_dirty_inode+0x35/0xd0 [btrfs]
May 7 02:11:52 server kernel: [17884.471672] [<ffffffffc04da57d>] btrfs_setattr+0x9d/0x330 [btrfs]
May 7 02:11:52 server kernel: [17884.471697] INFO: task btrfs:3559 blocked for more than 120 seconds.
May 7 02:11:52 server kernel: [17884.471851] btrfs D ffff937fa2257b68 0 3559 3518 0x00000000
May 7 02:11:52 server kernel: [17884.471887] [<ffffffffc04cb862>] wait_for_commit.isra.16+0x52/0xa0 [btrfs]
May 7 02:11:52 server kernel: [17884.471909] [<ffffffffc04cc205>] btrfs_wait_for_commit+0x145/0x170 [btrfs]
May 7 02:11:52 server kernel: [17884.471935] [<ffffffffc04bb227>] btrfs_inc_block_group_ro+0x67/0x1b0 [btrfs]
May 7 02:11:52 server kernel: [17884.471961] [<ffffffffc0532d77>] scrub_enumerate_chunks+0x217/0x670 [btrfs]
May 7 02:11:52 server kernel: [17884.471994] [<ffffffffc05344ea>] btrfs_scrub_dev+0x1fa/0x540 [btrfs]
May 7 02:11:52 server kernel: [17884.472019] [<ffffffffc050c25c>] btrfs_ioctl+0x1a7c/0x2010 [btrfs]

Revision history for this message

AlexanderYT (arcam) wrote on 2017-06-16:

#8

Download full text (20.9 KiB)

I can confirm

# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

# uptime g6-1
02:13:43 up 17 days, 6:40, 1 user, load average: 404.83, 402.10, 397.12

[Thu Jun 15 19:30:38 2017] device eth1 left promiscuous mode
[Thu Jun 15 19:31:12 2017] device eth1 entered promiscuous mode
[Thu Jun 15 19:34:04 2017] device eth1 left promiscuous mode
[Thu Jun 15 19:35:59 2017] device eth1 entered promiscuous mode
[Thu Jun 15 19:38:44 2017] device eth1 left promiscuous mode
[Thu Jun 15 21:32:25 2017] device eth2 entered promiscuous mode
[Thu Jun 15 22:04:15 2017] device eth2 left promiscuous mode
[Thu Jun 15 22:04:46 2017] device eth2 entered promiscuous mode
[Thu Jun 15 22:04:52 2017] device eth2 left promiscuous mode
[Fri Jun 16 16:23:36 2017] device eth1 entered promiscuous mode
[Fri Jun 16 20:02:57 2017] device eth1 left promiscuous mode
[Fri Jun 16 22:39:54 2017] INFO: task btrfs-transacti:1604 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017] Tainted: G I 4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] btrfs-transacti D ffff8817e65e7b38 0 1604 2 0x00000000
[Fri Jun 16 22:39:54 2017] ffff8817e65e7b38 ffffffffc03dbf65 ffff880be8c3f2c0 ffff8817e51472c0
[Fri Jun 16 22:39:54 2017] ffff8817e65e8000 00000000ffffff00 ffff8817e51472c0 ffff8817e51472c0
[Fri Jun 16 22:39:54 2017] ffff88137fc714c8 ffff8817e65e7b50 ffffffff81838545 ffff88137fc71530
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017] [<ffffffffc03dbf65>] ? btrfs_submit_bio_hook+0xe5/0x1b0 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017] [<ffffffffc04158e6>] btrfs_tree_read_lock+0xe6/0x140 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017] [<ffffffffc03aeba4>] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc03b3cab>] btrfs_search_slot+0x74b/0xa00 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffff810f611c>] ? ktime_get+0x3c/0xb0
[Fri Jun 16 22:39:54 2017] [<ffffffffc03cf27e>] btrfs_lookup_inode+0x3e/0xc0 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc042fe99>] __btrfs_update_delayed_inode+0x69/0x210 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc03ae90a>] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc04305b1>] __btrfs_run_delayed_items+0x571/0x620 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc03aeb06>] ? btrfs_free_path+0x26/0x30 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc0430cf3>] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc03d9ecb>] btrfs_commit_transaction+0x28b/0xa90 [btrfs]
[Fri Jun 16 22:39:54 2017] [<ffffffffc03d52e9>] transaction...

I can confirm

# lsb_release -a                                                                                                                                       
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial

# uptime                                                                                                                                               g6-1
 02:13:43 up 17 days,  6:40,  1 user,  load average: 404.83, 402.10, 397.12

[Thu Jun 15 19:30:38 2017] device eth1 left promiscuous mode
[Thu Jun 15 19:31:12 2017] device eth1 entered promiscuous mode
[Thu Jun 15 19:34:04 2017] device eth1 left promiscuous mode
[Thu Jun 15 19:35:59 2017] device eth1 entered promiscuous mode
[Thu Jun 15 19:38:44 2017] device eth1 left promiscuous mode
[Thu Jun 15 21:32:25 2017] device eth2 entered promiscuous mode
[Thu Jun 15 22:04:15 2017] device eth2 left promiscuous mode
[Thu Jun 15 22:04:46 2017] device eth2 entered promiscuous mode
[Thu Jun 15 22:04:52 2017] device eth2 left promiscuous mode
[Fri Jun 16 16:23:36 2017] device eth1 entered promiscuous mode
[Fri Jun 16 20:02:57 2017] device eth1 left promiscuous mode
[Fri Jun 16 22:39:54 2017] INFO: task btrfs-transacti:1604 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] btrfs-transacti D ffff8817e65e7b38     0  1604      2 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff8817e65e7b38 ffffffffc03dbf65 ffff880be8c3f2c0 ffff8817e51472c0
[Fri Jun 16 22:39:54 2017]  ffff8817e65e8000 00000000ffffff00 ffff8817e51472c0 ffff8817e51472c0
[Fri Jun 16 22:39:54 2017]  ffff88137fc714c8 ffff8817e65e7b50 ffffffff81838545 ffff88137fc71530
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03dbf65>] ? btrfs_submit_bio_hook+0xe5/0x1b0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc04158e6>] btrfs_tree_read_lock+0xe6/0x140 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03aeba4>] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03b3cab>] btrfs_search_slot+0x74b/0xa00 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810f611c>] ? ktime_get+0x3c/0xb0
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03cf27e>] btrfs_lookup_inode+0x3e/0xc0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc042fe99>] __btrfs_update_delayed_inode+0x69/0x210 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ae90a>] ? btrfs_alloc_path+0x1a/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc04305b1>] __btrfs_run_delayed_items+0x571/0x620 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03aeb06>] ? btrfs_free_path+0x26/0x30 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc0430cf3>] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03d9ecb>] btrfs_commit_transaction+0x28b/0xa90 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03d52e9>] transaction_kthread+0x229/0x240 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03d50c0>] ? btrfs_cleanup_transaction+0x570/0x570 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810a0be8>] kthread+0xd8/0xf0
[Fri Jun 16 22:39:54 2017]  [<ffffffff810a0b10>] ? kthread_create_on_node+0x1e0/0x1e0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183ca0f>] ret_from_fork+0x3f/0x70
[Fri Jun 16 22:39:54 2017]  [<ffffffff810a0b10>] ? kthread_create_on_node+0x1e0/0x1e0
[Fri Jun 16 22:39:54 2017] INFO: task cmahostd:3707 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] cmahostd        D ffff8817e813fd48     0  3707      1 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff8817e813fd48 ffff8817e47b8000 ffff8817e9702640 ffff8817e47b8000
[Fri Jun 16 22:39:54 2017]  ffff8817e8140000 ffff8816d038b068 ffff8816d038b080 ffff8813f8582000
[Fri Jun 16 22:39:54 2017]  ffff8816d038b000 ffff8817e813fd60 ffffffff81838545 ffff8817e47b8000
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183b3f0>] rwsem_down_read_failed+0xe0/0x140
[Fri Jun 16 22:39:54 2017]  [<ffffffff81406bc4>] call_rwsem_down_read_failed+0x14/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183a990>] ? down_read+0x20/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8127fe0e>] proc_pid_cmdline_read+0xae/0x530
[Fri Jun 16 22:39:54 2017]  [<ffffffff81347570>] ? security_file_permission+0xa0/0xc0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e8d8>] __vfs_read+0x18/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120eea6>] vfs_read+0x86/0x130
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120fbf5>] SyS_read+0x55/0xc0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task rs:main Q:Reg:16960 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] rs:main Q:Reg   D ffff88004ad6fba8     0 16960  16795 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff88004ad6fba8 ffff880bd5c417b0 ffff8817e9701980 ffff880b4cc4bfc0
[Fri Jun 16 22:39:54 2017]  ffff88004ad70000 ffff880bd5c417cc ffff880bd5c417b0 0000000162a12000
[Fri Jun 16 22:39:54 2017]  ffff88137fe31bf0 ffff88004ad6fbc0 ffffffff81838545 0000000162a12fff
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f90b2>] lock_extent_bits+0x1a2/0x1f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ec060>] lock_and_cleanup_extent_if_need+0xe0/0x1f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ed1b6>] __btrfs_buffered_write+0x1d6/0x5f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f0ab1>] btrfs_file_write_iter+0x171/0x560 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff817ac0f5>] ? inet_sendmsg+0x65/0xa0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e60b>] new_sync_write+0x9b/0xe0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e676>] __vfs_write+0x26/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120eff9>] vfs_write+0xa9/0x1a0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120fcb5>] SyS_write+0x55/0xc0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task mongod:17018 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] mongod          D ffff8815c7e0b7c0     0 17018  16795 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff8815c7e0b7c0 ffff8809da646120 ffff8817e97072c0 ffff8817d90cb300
[Fri Jun 16 22:39:54 2017]  ffff8815c7e0c000 00000000ffffff00 ffff8817d90cb300 ffff8817d90cb300
[Fri Jun 16 22:39:54 2017]  ffff88137fc714c8 ffff8815c7e0b7d8 ffffffff81838545 ffff88137fc71530
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc04158e6>] btrfs_tree_read_lock+0xe6/0x140 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03aeba4>] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03b3cab>] btrfs_search_slot+0x74b/0xa00 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff811ed497>] ? kmem_cache_alloc+0x187/0x1f0
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03cd369>] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e101e>] run_delalloc_nocow+0x15e/0xb40 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810b7e9d>] ? dequeue_task_fair+0x51d/0x8b0
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f99e8>] ? test_range_bit+0x48/0x130 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e1d6e>] run_delalloc_range+0x36e/0x3d0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f9dd5>] writepage_delalloc.isra.35+0x105/0x170 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03fbaea>] __extent_writepage+0xca/0x330 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03fc031>] extent_write_cache_pages.isra.31.constprop.51+0x2e1/0x3d0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810bdb73>] ? load_balance+0x1d3/0x980
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03fd2ec>] extent_writepages+0x5c/0x90 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03de760>] ? btrfs_real_readdir+0x5e0/0x5e0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03dc408>] btrfs_writepages+0x28/0x30 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff8119d5de>] do_writepages+0x1e/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff81190356>] __filemap_fdatawrite_range+0xc6/0x100
[Fri Jun 16 22:39:54 2017]  [<ffffffff81190463>] filemap_fdatawrite_range+0x13/0x20
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f0530>] btrfs_fdatawrite_range+0x20/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f0579>] start_ordered_ops+0x19/0x30 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f05fe>] btrfs_sync_file+0x6e/0x3b0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810f08dc>] ? hrtimer_nanosleep+0xdc/0x210
[Fri Jun 16 22:39:54 2017]  [<ffffffff8124336b>] vfs_fsync_range+0x4b/0xb0
[Fri Jun 16 22:39:54 2017]  [<ffffffffc06217b6>] ? kvm_on_user_return+0x66/0xa0 [kvm]
[Fri Jun 16 22:39:54 2017]  [<ffffffff811cc58e>] SyS_msync+0x17e/0x1f0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task mongod:17099 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] mongod          D ffff880a30693758     0 17099  16795 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff880a30693758 ffff880a30693770 ffff8817e9706600 ffff880b63830cc0
[Fri Jun 16 22:39:54 2017]  ffff880a30694000 00000000ffffff00 ffff880b63830cc0 ffff880b63830cc0
[Fri Jun 16 22:39:54 2017]  ffff88137fc714c8 ffff880a30693770 ffffffff81838545 ffff88137fc71530
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc04158e6>] btrfs_tree_read_lock+0xe6/0x140 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03aeba4>] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03b3cab>] btrfs_search_slot+0x74b/0xa00 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03cd369>] btrfs_lookup_file_extent+0x49/0x60 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03de8cc>] btrfs_get_extent+0x16c/0xae0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ea221>] btrfs_get_blocks_direct+0x231/0x780 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff8124db11>] do_blockdev_direct_IO+0xd01/0x3410
[Fri Jun 16 22:39:54 2017]  [<ffffffff8102d8c7>] ? __switch_to+0x437/0x5c0
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e3850>] ? finish_ordered_fn+0x20/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e9ff0>] ? btrfs_page_exists_in_range+0x100/0x100 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e3850>] ? finish_ordered_fn+0x20/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff81250263>] __blockdev_direct_IO+0x43/0x50
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e3850>] ? finish_ordered_fn+0x20/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03dd6d0>] btrfs_direct_IO+0x170/0x320 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e3850>] ? finish_ordered_fn+0x20/0x20 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810b7300>] ? dequeue_entity+0x400/0xa80
[Fri Jun 16 22:39:54 2017]  [<ffffffff81190c09>] generic_file_direct_write+0xb9/0x180
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f0be2>] btrfs_file_write_iter+0x2a2/0x560 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e60b>] new_sync_write+0x9b/0xe0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e676>] __vfs_write+0x26/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120eff9>] vfs_write+0xa9/0x1a0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120fcb5>] SyS_write+0x55/0xc0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task java:17639 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] java            D ffff880034eabba8     0 17639  17174 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff880034eabba8 ffff88143fd736f0 ffff8817e97072c0 ffff880b62035940
[Fri Jun 16 22:39:54 2017]  ffff880034eac000 ffff88143fd7370c ffff88143fd736f0 00000000012e4000
[Fri Jun 16 22:39:54 2017]  ffff8813efc23a60 ffff880034eabbc0 ffffffff81838545 00000000012fffff
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f90b2>] lock_extent_bits+0x1a2/0x1f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ec060>] lock_and_cleanup_extent_if_need+0xe0/0x1f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ed1b6>] __btrfs_buffered_write+0x1d6/0x5f0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff811010a4>] ? futex_wait+0x254/0x280
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03f0ab1>] btrfs_file_write_iter+0x171/0x560 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff811009b0>] ? futex_wake+0x90/0x170
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e60b>] new_sync_write+0x9b/0xe0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e676>] __vfs_write+0x26/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120eff9>] vfs_write+0xa9/0x1a0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120fcb5>] SyS_write+0x55/0xc0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task java:17748 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] java            D ffff88174e853be0     0 17748  17174 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff88174e853be0 000000006795d709 ffff880be8c3f2c0 ffff881671a41980
[Fri Jun 16 22:39:54 2017]  ffff88174e854000 00000000ffffff00 ffff881671a41980 ffff881671a41980
[Fri Jun 16 22:39:54 2017]  ffff88137fc714c8 ffff88174e853bf8 ffffffff81838545 ffff88137fc71530
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffffc04158e6>] btrfs_tree_read_lock+0xe6/0x140 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4210>] ? wake_atomic_t_function+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03aeba4>] btrfs_read_lock_root_node+0x34/0x50 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03b3cab>] btrfs_search_slot+0x74b/0xa00 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc0452d98>] ? btrfs_crc32c+0x68/0x90 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03cc637>] btrfs_check_dir_item_collision+0x77/0x120 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03e8800>] btrfs_rename2+0x130/0x7b0 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff81219600>] ? generic_permission+0x110/0x190
[Fri Jun 16 22:39:54 2017]  [<ffffffff8121c819>] vfs_rename+0x5d9/0x930
[Fri Jun 16 22:39:54 2017]  [<ffffffff81347200>] ? security_kernel_fw_from_file+0x50/0x70
[Fri Jun 16 22:39:54 2017]  [<ffffffff8122169f>] SyS_rename+0x39f/0x3c0
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task java:17755 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] java            D ffff881531a9bc68     0 17755  17174 0x00000000
[Fri Jun 16 22:39:54 2017]  ffff881531a9bc68 ffff881796444fd0 ffff880be8c3e600 ffff8817e80ee600
[Fri Jun 16 22:39:54 2017]  ffff881531a9c000 ffff88179644507c ffff8817e80ee600 00000000ffffffff
[Fri Jun 16 22:39:54 2017]  ffff881796445080 ffff881531a9bc80 ffffffff81838545 ffff881796445078
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffff818387ee>] schedule_preempt_disabled+0xe/0x10
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183a429>] __mutex_lock_slowpath+0xb9/0x130
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183a4bf>] mutex_lock+0x1f/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8121d9fc>] path_openat+0x4bc/0x1330
[Fri Jun 16 22:39:54 2017]  [<ffffffff8121fa61>] do_filp_open+0x91/0x100
[Fri Jun 16 22:39:54 2017]  [<ffffffff8122d366>] ? __alloc_fd+0x46/0x190
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120df08>] do_sys_open+0x138/0x2a0
[Fri Jun 16 22:39:54 2017]  [<ffffffff81103bf1>] ? SyS_futex+0x81/0x180
[Fri Jun 16 22:39:54 2017]  [<ffffffff8120e08e>] SyS_open+0x1e/0x20
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c672>] entry_SYSCALL_64_fastpath+0x16/0x71
[Fri Jun 16 22:39:54 2017] INFO: task java:17338 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] java            D ffff880bbdfc7af8     0 17338  16795 0x00000100
[Fri Jun 16 22:39:54 2017]  ffff880bbdfc7af8 ffff880bbdfc7b28 ffff8817e61d3300 ffff880be51aa640
[Fri Jun 16 22:39:54 2017]  ffff880bbdfc8000 ffff880befb16dc0 7fffffffffffffff ffffffff81838d40
[Fri Jun 16 22:39:54 2017]  ffff880bbdfc7c58 ffff880bbdfc7b10 ffffffff81838545 0000000000000000
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838d40>] ? bit_wait+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183b695>] schedule_timeout+0x1b5/0x270
[Fri Jun 16 22:39:54 2017]  [<ffffffff81837e96>] ? __schedule+0x3b6/0xa30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183c497>] ? _raw_spin_lock_irqsave+0x37/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c3d75>] ? finish_wait+0x55/0x70
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838d40>] ? bit_wait+0x60/0x60
[Fri Jun 16 22:39:54 2017]  [<ffffffff81837a74>] io_schedule_timeout+0xa4/0x110
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838d5b>] bit_wait_io+0x1b/0x70
[Fri Jun 16 22:39:54 2017]  [<ffffffff818388ed>] __wait_on_bit+0x5d/0x90
[Fri Jun 16 22:39:54 2017]  [<ffffffff8118e3cb>] wait_on_page_bit+0xcb/0xf0
[Fri Jun 16 22:39:54 2017]  [<ffffffff810c4250>] ? autoremove_wake_function+0x40/0x40
[Fri Jun 16 22:39:54 2017]  [<ffffffffc03ea903>] btrfs_page_mkwrite+0x193/0x380 [btrfs]
[Fri Jun 16 22:39:54 2017]  [<ffffffff811bdee6>] do_page_mkwrite+0x56/0xb0
[Fri Jun 16 22:39:54 2017]  [<ffffffff811c0298>] do_wp_page+0x158/0x6c0
[Fri Jun 16 22:39:54 2017]  [<ffffffff811009b0>] ? futex_wake+0x90/0x170
[Fri Jun 16 22:39:54 2017]  [<ffffffff810ef770>] ? __hrtimer_init+0x90/0x90
[Fri Jun 16 22:39:54 2017]  [<ffffffff811c1a90>] handle_mm_fault+0xce0/0x1820
[Fri Jun 16 22:39:54 2017]  [<ffffffff81837e96>] ? __schedule+0x3b6/0xa30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8106b527>] __do_page_fault+0x197/0x400
[Fri Jun 16 22:39:54 2017]  [<ffffffff8106b7b2>] do_page_fault+0x22/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183e7f8>] page_fault+0x28/0x30
[Fri Jun 16 22:39:54 2017] INFO: task java:17563 blocked for more than 120 seconds.
[Fri Jun 16 22:39:54 2017]       Tainted: G          I     4.4.0-67-generic #88-Ubuntu
[Fri Jun 16 22:39:54 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[Fri Jun 16 22:39:54 2017] java            D ffff880b5ae0fde8     0 17563  16795 0x00000100
[Fri Jun 16 22:39:54 2017]  ffff880b5ae0fde8 ffff880befa0f240 ffff880be93bf2c0 ffff880be7a32640
[Fri Jun 16 22:39:54 2017]  ffff880b5ae10000 ffff8816d038b068 ffff8816d038b080 ffff880b5ae0ff58
[Fri Jun 16 22:39:54 2017]  ffff880be7a32640 ffff880b5ae0fe00 ffffffff81838545 ffff880be7a32640
[Fri Jun 16 22:39:54 2017] Call Trace:
[Fri Jun 16 22:39:54 2017]  [<ffffffff81838545>] schedule+0x35/0x80
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183b3f0>] rwsem_down_read_failed+0xe0/0x140
[Fri Jun 16 22:39:54 2017]  [<ffffffff81406bc4>] call_rwsem_down_read_failed+0x14/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183a990>] ? down_read+0x20/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8106b705>] __do_page_fault+0x375/0x400
[Fri Jun 16 22:39:54 2017]  [<ffffffff8118b04b>] ? fire_user_return_notifiers+0x3b/0x50
[Fri Jun 16 22:39:54 2017]  [<ffffffff8106b7b2>] do_page_fault+0x22/0x30
[Fri Jun 16 22:39:54 2017]  [<ffffffff8183e7f8>] page_fault+0x28/0x30

Revision history for this message

AlexanderYT (arcam) wrote on 2017-06-16:

#9

Linux g6-1 4.4.0-67-generic #88-Ubuntu SMP Wed Mar 8 16:34:45 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Revision history for this message

Konrad Wojas (kwojas) wrote on 2017-06-18:

#10

I upgraded from kernel 4.4 to 4.8 on 16.04 LTS in the hope the issue would go away. My hang is in btrfs-cleaner, not sure if this is the same bug or not.

This related thread is interesting but does not seem to resolve it: https://patchwork.kernel.org/patch/8284721/

Linux nuc 4.8.0-54-generic #57~16.04.1-Ubuntu SMP Wed May 24 16:22:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Jun 19 01:51:14 nuc kernel: INFO: task btrfs-cleaner:408 blocked for more than 120 seconds.
Jun 19 01:51:14 nuc kernel: Not tainted 4.8.0-54-generic #57~16.04.1-Ubuntu
Jun 19 01:51:14 nuc kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 19 01:51:14 nuc kernel: btrfs-cleaner D ffff9478128d3d00 0 408 2 0x00000000
Jun 19 01:51:14 nuc kernel: ffff9478128d3d00 ffff9478128d3cc8 ffff94781c2a6ac0 ffff9478137a0000
Jun 19 01:51:14 nuc kernel: ffffffffa4cc7492 ffff9478128d4000 ffff947813e841f0 ffff947813e84000
Jun 19 01:51:14 nuc kernel: ffff947813e841f0 0000000000000001 ffff9478128d3d18 ffffffffa5496045
Jun 19 01:51:14 nuc kernel: Call Trace:
Jun 19 01:51:14 nuc kernel: [<ffffffffa4cc7492>] ? autoremove_wake_function+0x12/0x40
Jun 19 01:51:14 nuc kernel: [<ffffffffa5496045>] schedule+0x35/0x80
Jun 19 01:51:14 nuc kernel: [<ffffffffc079fc73>] wait_current_trans.isra.21+0xd3/0x120 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffa4cc7480>] ? wake_atomic_t_function+0x60/0x60
Jun 19 01:51:14 nuc kernel: [<ffffffffc07a248f>] start_transaction+0x2cf/0x4c0 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffc07a2698>] btrfs_start_transaction+0x18/0x20 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffc0788f72>] btrfs_drop_snapshot+0x4b2/0x800 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffa5495b87>] ? __schedule+0x307/0x790
Jun 19 01:51:14 nuc kernel: [<ffffffffc07a1624>] btrfs_clean_one_deleted_snapshot+0xb4/0x100 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffc0797dbd>] cleaner_kthread+0x15d/0x1d0 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffc0797c60>] ? btrfs_destroy_pinned_extent+0xf0/0xf0 [btrfs]
Jun 19 01:51:14 nuc kernel: [<ffffffffa4ca3fb8>] kthread+0xd8/0xf0
Jun 19 01:51:14 nuc kernel: [<ffffffffa549aa9f>] ret_from_fork+0x1f/0x40
Jun 19 01:51:14 nuc kernel: [<ffffffffa4ca3ee0>] ? kthread_create_on_node+0x1e0/0x1e0

I upgraded from kernel 4.4 to 4.8 on 16.04 LTS in the hope the issue would go away. My hang is in btrfs-cleaner, not sure if this is the same bug or not.

This related thread is interesting but does not seem to resolve it: https://patchwork.kernel.org/patch/8284721/

Linux nuc 4.8.0-54-generic #57~16.04.1-Ubuntu SMP Wed May 24 16:22:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Jun 19 01:51:14 nuc kernel: INFO: task btrfs-cleaner:408 blocked for more than 120 seconds.
Jun 19 01:51:14 nuc kernel:       Not tainted 4.8.0-54-generic #57~16.04.1-Ubuntu
Jun 19 01:51:14 nuc kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 19 01:51:14 nuc kernel: btrfs-cleaner   D ffff9478128d3d00     0   408      2 0x00000000
Jun 19 01:51:14 nuc kernel:  ffff9478128d3d00 ffff9478128d3cc8 ffff94781c2a6ac0 ffff9478137a0000
Jun 19 01:51:14 nuc kernel:  ffffffffa4cc7492 ffff9478128d4000 ffff947813e841f0 ffff947813e84000
Jun 19 01:51:14 nuc kernel:  ffff947813e841f0 0000000000000001 ffff9478128d3d18 ffffffffa5496045
Jun 19 01:51:14 nuc kernel: Call Trace:
Jun 19 01:51:14 nuc kernel:  [<ffffffffa4cc7492>] ? autoremove_wake_function+0x12/0x40
Jun 19 01:51:14 nuc kernel:  [<ffffffffa5496045>] schedule+0x35/0x80
Jun 19 01:51:14 nuc kernel:  [<ffffffffc079fc73>] wait_current_trans.isra.21+0xd3/0x120 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffa4cc7480>] ? wake_atomic_t_function+0x60/0x60
Jun 19 01:51:14 nuc kernel:  [<ffffffffc07a248f>] start_transaction+0x2cf/0x4c0 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffc07a2698>] btrfs_start_transaction+0x18/0x20 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffc0788f72>] btrfs_drop_snapshot+0x4b2/0x800 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffa5495b87>] ? __schedule+0x307/0x790
Jun 19 01:51:14 nuc kernel:  [<ffffffffc07a1624>] btrfs_clean_one_deleted_snapshot+0xb4/0x100 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffc0797dbd>] cleaner_kthread+0x15d/0x1d0 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffc0797c60>] ? btrfs_destroy_pinned_extent+0xf0/0xf0 [btrfs]
Jun 19 01:51:14 nuc kernel:  [<ffffffffa4ca3fb8>] kthread+0xd8/0xf0
Jun 19 01:51:14 nuc kernel:  [<ffffffffa549aa9f>] ret_from_fork+0x1f/0x40
Jun 19 01:51:14 nuc kernel:  [<ffffffffa4ca3ee0>] ? kthread_create_on_node+0x1e0/0x1e0

Revision history for this message

Kai-Heng Feng (kaihengfeng) wrote on 2017-07-04:

#11

The best way is to ask maintainer on linux-btrfs.

Alexej (nebu0email.tg) on 2017-07-19

Changed in linux (Ubuntu):
status:	Confirmed → Fix Released

Revision history for this message

Daniel (hackie) wrote on 2017-08-14:

#12

@Alexej Do you have more information about the fix? Which version? Upstream?

Ubuntu
linux package

Btrfs Fails to Mount Drive - btrfs-transaction blocked for more than 120 seconds.

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntulinux package

Btrfs Fails to Mount Drive - btrfs-transaction blocked for more than 120 seconds.

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntu
linux package