On one of my servers I've reverted to the 3.19.0-58-generic kernel. I still can't mount these broken filesystems. root@ceph-store4:~# mount /var/lib/ceph/osd/ceph-2 Killed root@ceph-store4:~# dmesg -T ... [Fri Apr 29 11:09:27 2016] XFS (bcache4): Mounting V5 Filesystem [Fri Apr 29 11:09:27 2016] XFS (bcache4): Starting recovery (logdev: internal) [Fri Apr 29 11:09:29 2016] XFS (bcache4): _xfs_buf_find: Block out of range: block 0x8e8e05f48, EOFS 0x1d1c0bea0 [Fri Apr 29 11:09:29 2016] ------------[ cut here ]------------ [Fri Apr 29 11:09:29 2016] WARNING: CPU: 2 PID: 5427 at /build/linux-lts-vivid-GISjUd/linux-lts-vivid-3.19.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x289/0x2d0 [xfs]() [Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c [Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Not tainted 3.19.0-58-generic #64~14.04.1-Ubuntu [Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014 [Fri Apr 29 11:09:29 2016] 0000000000000000 ffff88080aaeb928 ffffffff817b6c3d 0000000000000000 [Fri Apr 29 11:09:29 2016] ffffffffc0931e58 ffff88080aaeb968 ffffffff81075f3a ffff88080aaeb988 [Fri Apr 29 11:09:29 2016] ffff881008135980 0000000000000008 0000000000000000 ffff88080aaeba78 [Fri Apr 29 11:09:29 2016] Call Trace: [Fri Apr 29 11:09:29 2016] [] dump_stack+0x63/0x81 [Fri Apr 29 11:09:29 2016] [] warn_slowpath_common+0x8a/0xc0 [Fri Apr 29 11:09:29 2016] [] warn_slowpath_null+0x1a/0x20 [Fri Apr 29 11:09:29 2016] [] _xfs_buf_find+0x289/0x2d0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_buf_get_map+0x2a/0x280 [xfs] [Fri Apr 29 11:09:29 2016] [] ? xfs_free_ag_extent+0x453/0x750 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_trans_get_buf_map+0x111/0x1a0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_btree_get_bufs+0x50/0x60 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_alloc_fix_freelist+0x1a7/0x4c0 [xfs] [Fri Apr 29 11:09:29 2016] [] ? kmem_zone_alloc+0x77/0x100 [xfs] [Fri Apr 29 11:09:29 2016] [] ? radix_tree_lookup+0xd/0x10 [Fri Apr 29 11:09:29 2016] [] ? xfs_perag_get+0x2a/0xd0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_free_extent+0x8a/0x110 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efi+0x170/0x1b0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_finish+0x21/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_log_mount_finish+0x34/0x50 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_mountfs+0x509/0x760 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_fill_super+0x2c6/0x340 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_bdev+0x1b0/0x1f0 [Fri Apr 29 11:09:29 2016] [] ? xfs_parseargs+0xbe0/0xbe0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_mount+0x15/0x20 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_fs+0x39/0x1b0 [Fri Apr 29 11:09:29 2016] [] vfs_kern_mount+0x6b/0x120 [Fri Apr 29 11:09:29 2016] [] do_mount+0x224/0xce0 [Fri Apr 29 11:09:29 2016] [] ? __get_free_pages+0xe/0x50 [Fri Apr 29 11:09:29 2016] [] ? copy_mount_options+0x3a/0x160 [Fri Apr 29 11:09:29 2016] [] SyS_mount+0x8b/0xe0 [Fri Apr 29 11:09:29 2016] [] system_call_fastpath+0x16/0x1b [Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625ce ]--- [Fri Apr 29 11:09:29 2016] XFS (bcache4): _xfs_buf_find: Block out of range: block 0x8e8e05f48, EOFS 0x1d1c0bea0 [Fri Apr 29 11:09:29 2016] ------------[ cut here ]------------ [Fri Apr 29 11:09:29 2016] WARNING: CPU: 2 PID: 5427 at /build/linux-lts-vivid-GISjUd/linux-lts-vivid-3.19.0/fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x289/0x2d0 [xfs]() [Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c [Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Tainted: G W 3.19.0-58-generic #64~14.04.1-Ubuntu [Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014 [Fri Apr 29 11:09:29 2016] 0000000000000000 ffff88080aaeb928 ffffffff817b6c3d 0000000000000000 [Fri Apr 29 11:09:29 2016] ffffffffc0931e58 ffff88080aaeb968 ffffffff81075f3a ffff88080aaeb988 [Fri Apr 29 11:09:29 2016] ffff881008135980 0000000000000008 ffff8808075be700 ffff88080aaeba78 [Fri Apr 29 11:09:29 2016] Call Trace: [Fri Apr 29 11:09:29 2016] [] dump_stack+0x63/0x81 [Fri Apr 29 11:09:29 2016] [] warn_slowpath_common+0x8a/0xc0 [Fri Apr 29 11:09:29 2016] [] warn_slowpath_null+0x1a/0x20 [Fri Apr 29 11:09:29 2016] [] _xfs_buf_find+0x289/0x2d0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_buf_get_map+0x1f3/0x280 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_trans_get_buf_map+0x111/0x1a0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_btree_get_bufs+0x50/0x60 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_alloc_fix_freelist+0x1a7/0x4c0 [xfs] [Fri Apr 29 11:09:29 2016] [] ? kmem_zone_alloc+0x77/0x100 [xfs] [Fri Apr 29 11:09:29 2016] [] ? radix_tree_lookup+0xd/0x10 [Fri Apr 29 11:09:29 2016] [] ? xfs_perag_get+0x2a/0xd0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_free_extent+0x8a/0x110 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efi+0x170/0x1b0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_finish+0x21/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_log_mount_finish+0x34/0x50 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_mountfs+0x509/0x760 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_fill_super+0x2c6/0x340 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_bdev+0x1b0/0x1f0 [Fri Apr 29 11:09:29 2016] [] ? xfs_parseargs+0xbe0/0xbe0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_mount+0x15/0x20 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_fs+0x39/0x1b0 [Fri Apr 29 11:09:29 2016] [] vfs_kern_mount+0x6b/0x120 [Fri Apr 29 11:09:29 2016] [] do_mount+0x224/0xce0 [Fri Apr 29 11:09:29 2016] [] ? __get_free_pages+0xe/0x50 [Fri Apr 29 11:09:29 2016] [] ? copy_mount_options+0x3a/0x160 [Fri Apr 29 11:09:29 2016] [] SyS_mount+0x8b/0xe0 [Fri Apr 29 11:09:29 2016] [] system_call_fastpath+0x16/0x1b [Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625cf ]--- [Fri Apr 29 11:09:29 2016] BUG: unable to handle kernel NULL pointer dereference at 00000000000000f8 [Fri Apr 29 11:09:29 2016] IP: [] xfs_trans_binval+0x16/0x120 [xfs] [Fri Apr 29 11:09:29 2016] PGD 8083d5067 PUD 80ad6a067 PMD 0 [Fri Apr 29 11:09:29 2016] Oops: 0000 [#1] SMP [Fri Apr 29 11:09:29 2016] Modules linked in: bridge openvswitch ipmi_devintf xfs dcdbas intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd joydev sb_edac dm_multipath edac_core scsi_dh mei_me lpc_ich mei ipmi_si 8250_fintek ipmi_msghandler wmi acpi_pad mac_hid acpi_power_meter shpchp 8021q garp mrp stp llc lp parport btrfs bcache hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 bnx2x raid0 mpt2sas ptp multipath raid_class pps_core mdio scsi_transport_sas nvme linear libcrc32c [Fri Apr 29 11:09:29 2016] CPU: 2 PID: 5427 Comm: mount Tainted: G W 3.19.0-58-generic #64~14.04.1-Ubuntu [Fri Apr 29 11:09:29 2016] Hardware name: Dell Inc. PowerEdge R720xd/0HJK12, BIOS 2.4.3 07/09/2014 [Fri Apr 29 11:09:29 2016] task: ffff880809f23110 ti: ffff88080aae8000 task.ti: ffff88080aae8000 [Fri Apr 29 11:09:29 2016] RIP: 0010:[] [] xfs_trans_binval+0x16/0x120 [xfs] [Fri Apr 29 11:09:29 2016] RSP: 0018:ffff88080aaeba68 EFLAGS: 00010292 [Fri Apr 29 11:09:29 2016] RAX: 0000000000000000 RBX: ffff8807f6c96000 RCX: 0000000000000866 [Fri Apr 29 11:09:29 2016] RDX: 0000000000000865 RSI: 0000000000000000 RDI: ffff8807f6c96000 [Fri Apr 29 11:09:29 2016] RBP: ffff88080aaeba88 R08: 000000000001efe0 R09: ffff88080fa3efe0 [Fri Apr 29 11:09:29 2016] R10: ffffffffc08e934c R11: ffffea00201d6f00 R12: ffff88080aaebb70 [Fri Apr 29 11:09:29 2016] R13: ffff8807f6c96000 R14: 0000000000000000 R15: ffff881007e6cfc0 [Fri Apr 29 11:09:29 2016] FS: 00007f378fedd880(0000) GS:ffff88080fa20000(0000) knlGS:0000000000000000 [Fri Apr 29 11:09:29 2016] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [Fri Apr 29 11:09:29 2016] CR2: 00000000000000f8 CR3: 0000000808e0d000 CR4: 00000000000407e0 [Fri Apr 29 11:09:29 2016] Stack: [Fri Apr 29 11:09:29 2016] ffff8807f6c96000 ffff88080aaebb70 ffff88080bb2c800 ffff881007f59400 [Fri Apr 29 11:09:29 2016] ffff88080aaebb58 ffffffffc08ac5e2 ffffffff00000000 0000000800000002 [Fri Apr 29 11:09:29 2016] ffffffff00000002 ffff8808075d7a80 0000000000000000 0000000000000000 [Fri Apr 29 11:09:29 2016] Call Trace: [Fri Apr 29 11:09:29 2016] [] xfs_alloc_fix_freelist+0x1b2/0x4c0 [xfs] [Fri Apr 29 11:09:29 2016] [] ? kmem_zone_alloc+0x77/0x100 [xfs] [Fri Apr 29 11:09:29 2016] [] ? radix_tree_lookup+0xd/0x10 [Fri Apr 29 11:09:29 2016] [] ? xfs_perag_get+0x2a/0xd0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_free_extent+0x8a/0x110 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efi+0x170/0x1b0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_process_efis.isra.27+0x64/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xlog_recover_finish+0x21/0xb0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_log_mount_finish+0x34/0x50 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_mountfs+0x509/0x760 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_fill_super+0x2c6/0x340 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_bdev+0x1b0/0x1f0 [Fri Apr 29 11:09:29 2016] [] ? xfs_parseargs+0xbe0/0xbe0 [xfs] [Fri Apr 29 11:09:29 2016] [] xfs_fs_mount+0x15/0x20 [xfs] [Fri Apr 29 11:09:29 2016] [] mount_fs+0x39/0x1b0 [Fri Apr 29 11:09:29 2016] [] vfs_kern_mount+0x6b/0x120 [Fri Apr 29 11:09:29 2016] [] do_mount+0x224/0xce0 [Fri Apr 29 11:09:29 2016] [] ? __get_free_pages+0xe/0x50 [Fri Apr 29 11:09:29 2016] [] ? copy_mount_options+0x3a/0x160 [Fri Apr 29 11:09:29 2016] [] SyS_mount+0x8b/0xe0 [Fri Apr 29 11:09:29 2016] [] system_call_fastpath+0x16/0x1b [Fri Apr 29 11:09:29 2016] Code: 75 eb e9 53 ff ff ff 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 56 49 89 f6 41 55 49 89 fd 41 54 53 <4c> 8b a6 f8 00 00 00 66 66 66 66 90 41 f6 44 24 78 04 74 5b 5b [Fri Apr 29 11:09:29 2016] RIP [] xfs_trans_binval+0x16/0x120 [xfs] [Fri Apr 29 11:09:29 2016] RSP [Fri Apr 29 11:09:29 2016] CR2: 00000000000000f8 [Fri Apr 29 11:09:29 2016] ---[ end trace fe0de8e0241625d0 ]---