soft-crash: task flush blocked for more than 120 seconds

[Paul Bohm]

since installing maverick on an amazon m1.large install i'm crashing every 1-2 days. install is the alestic maverick EBS backed image: ami-ca1f4f8f

also had the same happen once to a mounted volume (but not the root fs). i have a very basic setup with purely userspace apps - nothing that should be out of the ordinary.

thinking about reinstalling as 32bit and if that doesn't help downgrading since i didn't have problems before maverick (was not on 64 bit before though).

Oct 24 18:03:22 solaris kernel: [394565.670057] INFO: task flush-202:97:1047 blocked for more than 120 seconds.
Oct 24 18:03:22 solaris kernel: [394565.670074] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:03:22 solaris kernel: [394565.670081] flush-202:97 D ffff880003dce980 0 1047 2 0x00000000
Oct 24 18:03:22 solaris kernel: [394565.670087] ffff8801d6dbf620 0000000000000246 ffff880100000e58 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670092] ffff8801d6dbffd8 0000000000015980 ffff8801d6dbffd8 ffff8801d6db2dc0
Oct 24 18:03:22 solaris kernel: [394565.670095] 0000000000015980 0000000000015980 ffff8801d6dbffd8 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670100] Call Trace:
Oct 24 18:03:22 solaris kernel: [394565.670114] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:03:22 solaris kernel: [394565.670120] [<ffffffff812a2f3c>] get_request_wait+0xcc/0x1a0
Oct 24 18:03:22 solaris kernel: [394565.670125] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:03:22 solaris kernel: [394565.670130] [<ffffffff8129ccd7>] ? elv_merge+0x1d7/0x210
Oct 24 18:03:22 solaris kernel: [394565.670133] [<ffffffff812a30a3>] __make_request+0x93/0x4b0
Oct 24 18:03:22 solaris kernel: [394565.670137] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:03:22 solaris kernel: [394565.670140] [<ffffffff812a1c83>] generic_make_request+0x1b3/0x540
Oct 24 18:03:22 solaris kernel: [394565.670146] [<ffffffff81102cc5>] ? mempool_alloc_slab+0x15/0x20
Oct 24 18:03:22 solaris kernel: [394565.670151] [<ffffffff811ec861>] ? ext4_mark_iloc_dirty+0x71/0x90
Oct 24 18:03:22 solaris kernel: [394565.670154] [<ffffffff812a2092>] submit_bio+0x82/0x110
Oct 24 18:03:22 solaris kernel: [394565.670158] [<ffffffff8117c269>] submit_bh+0xf9/0x140
Oct 24 18:03:22 solaris kernel: [394565.670162] [<ffffffff8117e240>] __block_write_full_page+0x1d0/0x3b0
Oct 24 18:03:22 solaris kernel: [394565.670164] [<ffffffff8117dab0>] ? end_buffer_async_write+0x0/0x190
Oct 24 18:03:22 solaris kernel: [394565.670168] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:03:22 solaris kernel: [394565.670171] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:03:22 solaris kernel: [394565.670173] [<ffffffff8117ed50>] block_write_full_page_endio+0xe0/0x120
Oct 24 18:03:22 solaris kernel: [394565.670176] [<ffffffff811e9ed0>] ? ext4_bh_delay_or_unwritten+0x0/0x30
Oct 24 18:03:22 solaris kernel: [394565.670179] [<ffffffff8117eda5>] block_write_full_page+0x15/0x20
Oct 24 18:03:22 solaris kernel: [394565.670182] [<ffffffff811ed3c3>] ext4_writepage+0x1d3/0x380
Oct 24 18:03:22 solaris kernel: [394565.670185] [<ffffffff811eb1dc>] mpage_da_submit_io+0x14c/0x1c0
Oct 24 18:03:22 solaris kernel: [394565.670188] [<ffffffff811ef7e3>] ? mpage_da_map_blocks+0x1c3/0x370
Oct 24 18:03:22 solaris kernel: [394565.670191] [<ffffffff811f0395>] ext4_da_writepages+0x4f5/0x640
Oct 24 18:03:22 solaris kernel: [394565.670195] [<ffffffff8110a791>] do_writepages+0x21/0x40
Oct 24 18:03:22 solaris kernel: [394565.670200] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0
Oct 24 18:03:22 solaris kernel: [394565.670203] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280
Oct 24 18:03:22 solaris kernel: [394565.670206] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0
Oct 24 18:03:22 solaris kernel: [394565.670209] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0
Oct 24 18:03:22 solaris kernel: [394565.670212] [<ffffffff8117680c>] wb_do_writeback+0x17c/0x190
Oct 24 18:03:22 solaris kernel: [394565.670217] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10
Oct 24 18:03:22 solaris kernel: [394565.670220] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160
Oct 24 18:03:22 solaris kernel: [394565.670223] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0
Oct 24 18:03:22 solaris kernel: [394565.670228] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100
Oct 24 18:03:22 solaris kernel: [394565.670231] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100
Oct 24 18:03:22 solaris kernel: [394565.670233] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:03:22 solaris kernel: [394565.670238] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:03:22 solaris kernel: [394565.670242] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:03:22 solaris kernel: [394565.670244] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:03:22 solaris kernel: [394565.670282] INFO: task jbd2/sdg1-8:1824 blocked for more than 120 seconds.
Oct 24 18:03:22 solaris kernel: [394565.670289] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:03:22 solaris kernel: [394565.670295] jbd2/sdg1-8 D ffff880003dce980 0 1824 2 0x00000000
Oct 24 18:03:22 solaris kernel: [394565.670299] ffff8801d77f3a50 0000000000000246 0000000000000000 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670303] ffff8801d77f3fd8 0000000000015980 ffff8801d77f3fd8 ffff8801d70e2dc0
Oct 24 18:03:22 solaris kernel: [394565.670306] 0000000000015980 0000000000015980 ffff8801d77f3fd8 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670310] Call Trace:
Oct 24 18:03:22 solaris kernel: [394565.670313] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:03:22 solaris kernel: [394565.670316] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:03:22 solaris kernel: [394565.670319] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:03:22 solaris kernel: [394565.670321] [<ffffffff815a27ef>] __wait_on_bit+0x5f/0x90
Oct 24 18:03:22 solaris kernel: [394565.670324] [<ffffffff81100cb3>] wait_on_page_bit+0x73/0x80
Oct 24 18:03:22 solaris kernel: [394565.670327] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:03:22 solaris kernel: [394565.670330] [<ffffffff8110b5d5>] ? pagevec_lookup_tag+0x25/0x40
Oct 24 18:03:22 solaris kernel: [394565.670333] [<ffffffff8110a503>] write_cache_pages+0x193/0x3d0
Oct 24 18:03:22 solaris kernel: [394565.670336] [<ffffffff81109400>] ? __writepage+0x0/0x40
Oct 24 18:03:22 solaris kernel: [394565.670340] [<ffffffff810d1244>] ? __delayacct_blkio_end+0x34/0x60
Oct 24 18:03:22 solaris kernel: [394565.670343] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:03:22 solaris kernel: [394565.670346] [<ffffffff81006b1b>] ? __raw_callee_save_xen_irq_enable+0x11/0x26
Oct 24 18:03:22 solaris kernel: [394565.670349] [<ffffffff8110a764>] generic_writepages+0x24/0x30
Oct 24 18:03:22 solaris kernel: [394565.670355] [<ffffffff81227a37>] journal_submit_data_buffers+0xb7/0x170
Oct 24 18:03:22 solaris kernel: [394565.670359] [<ffffffff812280dc>] jbd2_journal_commit_transaction+0x30c/0x1350
Oct 24 18:03:22 solaris kernel: [394565.670363] [<ffffffff815a40ce>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Oct 24 18:03:22 solaris kernel: [394565.670366] [<ffffffff81070933>] ? try_to_del_timer_sync+0x83/0xe0
Oct 24 18:03:22 solaris kernel: [394565.670370] [<ffffffff8122d87d>] kjournald2+0xbd/0x220
Oct 24 18:03:22 solaris kernel: [394565.670373] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:03:22 solaris kernel: [394565.670376] [<ffffffff8122d7c0>] ? kjournald2+0x0/0x220
Oct 24 18:03:22 solaris kernel: [394565.670378] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:03:22 solaris kernel: [394565.670381] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:03:22 solaris kernel: [394565.670384] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:03:22 solaris kernel: [394565.670387] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:03:22 solaris kernel: [394565.670399] INFO: task python:27611 blocked for more than 120 seconds.
Oct 24 18:03:22 solaris kernel: [394565.670405] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:03:22 solaris kernel: [394565.670411] python D ffff880003dec980 0 27611 1 0x00000000
Oct 24 18:03:22 solaris kernel: [394565.670415] ffff880190a01998 0000000000000282 ffff880003ded518 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670419] ffff880190a01fd8 0000000000015980 ffff880190a01fd8 ffff8801d4fd5b80
Oct 24 18:03:22 solaris kernel: [394565.670422] 0000000000015980 0000000000015980 ffff880190a01fd8 0000000000015980
Oct 24 18:03:22 solaris kernel: [394565.670427] Call Trace:
Oct 24 18:03:22 solaris kernel: [394565.670430] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:03:22 solaris kernel: [394565.670434] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:03:22 solaris kernel: [394565.670436] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:03:22 solaris kernel: [394565.670439] [<ffffffff815a269a>] __wait_on_bit_lock+0x5a/0xc0
Oct 24 18:03:22 solaris kernel: [394565.670442] [<ffffffff81100a97>] __lock_page+0x67/0x70
Oct 24 18:03:22 solaris kernel: [394565.670444] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:03:22 solaris kernel: [394565.670447] [<ffffffff811008ce>] ? find_get_page+0x1e/0xa0
Oct 24 18:03:22 solaris kernel: [394565.670450] [<ffffffff811020b2>] find_lock_page+0x52/0x80
Oct 24 18:03:22 solaris kernel: [394565.670452] [<ffffffff8110212a>] grab_cache_page_write_begin+0x4a/0xc0
Oct 24 18:03:22 solaris kernel: [394565.670456] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:03:22 solaris kernel: [394565.670459] [<ffffffff811f1964>] ext4_da_write_begin+0x144/0x290
Oct 24 18:03:22 solaris kernel: [394565.670464] [<ffffffff810ffe62>] generic_perform_write+0xc2/0x1d0
Oct 24 18:03:22 solaris kernel: [394565.670468] [<ffffffff81202348>] ? __ext4_journal_stop+0x68/0x90
Oct 24 18:03:22 solaris kernel: [394565.670471] [<ffffffff811760a2>] ? __mark_inode_dirty+0x42/0x1d0
Oct 24 18:03:22 solaris kernel: [394565.670474] [<ffffffff810fffd4>] generic_file_buffered_write+0x64/0xa0
Oct 24 18:03:22 solaris kernel: [394565.670477] [<ffffffff811028e0>] __generic_file_aio_write+0x240/0x470
Oct 24 18:03:22 solaris kernel: [394565.670482] [<ffffffff8149fff3>] ? sock_sendmsg+0xf3/0x120
Oct 24 18:03:22 solaris kernel: [394565.670485] [<ffffffff81102b75>] generic_file_aio_write+0x65/0xd0
Oct 24 18:03:22 solaris kernel: [394565.670488] [<ffffffff811e77a9>] ext4_file_write+0x39/0xb0
Oct 24 18:03:22 solaris kernel: [394565.670491] [<ffffffff81152bea>] do_sync_write+0xda/0x120
Oct 24 18:03:22 solaris kernel: [394565.670494] [<ffffffff810043c6>] ? xen_mc_flush+0x96/0x1c0
Oct 24 18:03:22 solaris kernel: [394565.670499] [<ffffffff81052213>] ? finish_task_switch+0x53/0xd0
Oct 24 18:03:22 solaris kernel: [394565.670503] [<ffffffff8128f218>] ? apparmor_file_permission+0x18/0x20
Oct 24 18:03:22 solaris kernel: [394565.670507] [<ffffffff8125e7b6>] ? security_file_permission+0x16/0x20
Oct 24 18:03:22 solaris kernel: [394565.670510] [<ffffffff81152ec8>] vfs_write+0xb8/0x1a0
Oct 24 18:03:22 solaris kernel: [394565.670513] [<ffffffff81153711>] sys_write+0x51/0x80
Oct 24 18:03:22 solaris kernel: [394565.670516] [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b
Oct 24 18:05:22 solaris kernel: [394685.660124] INFO: task flush-202:97:1047 blocked for more than 120 seconds.
Oct 24 18:05:22 solaris kernel: [394685.660144] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:05:22 solaris kernel: [394685.660151] flush-202:97 D ffff880003dce980 0 1047 2 0x00000000
Oct 24 18:05:22 solaris kernel: [394685.660156] ffff8801d6dbf620 0000000000000246 ffff880100000e58 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660160] ffff8801d6dbffd8 0000000000015980 ffff8801d6dbffd8 ffff8801d6db2dc0
Oct 24 18:05:22 solaris kernel: [394685.660164] 0000000000015980 0000000000015980 ffff8801d6dbffd8 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660168] Call Trace:
Oct 24 18:05:22 solaris kernel: [394685.660179] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:05:22 solaris kernel: [394685.660184] [<ffffffff812a2f3c>] get_request_wait+0xcc/0x1a0
Oct 24 18:05:22 solaris kernel: [394685.660189] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:05:22 solaris kernel: [394685.660193] [<ffffffff8129ccd7>] ? elv_merge+0x1d7/0x210
Oct 24 18:05:22 solaris kernel: [394685.660197] [<ffffffff812a30a3>] __make_request+0x93/0x4b0
Oct 24 18:05:22 solaris kernel: [394685.660201] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:05:22 solaris kernel: [394685.660204] [<ffffffff812a1c83>] generic_make_request+0x1b3/0x540
Oct 24 18:05:22 solaris kernel: [394685.660208] [<ffffffff81102cc5>] ? mempool_alloc_slab+0x15/0x20
Oct 24 18:05:22 solaris kernel: [394685.660213] [<ffffffff811ec861>] ? ext4_mark_iloc_dirty+0x71/0x90
Oct 24 18:05:22 solaris kernel: [394685.660216] [<ffffffff812a2092>] submit_bio+0x82/0x110
Oct 24 18:05:22 solaris kernel: [394685.660219] [<ffffffff8117c269>] submit_bh+0xf9/0x140
Oct 24 18:05:22 solaris kernel: [394685.660222] [<ffffffff8117e240>] __block_write_full_page+0x1d0/0x3b0
Oct 24 18:05:22 solaris kernel: [394685.660225] [<ffffffff8117dab0>] ? end_buffer_async_write+0x0/0x190
Oct 24 18:05:22 solaris kernel: [394685.660229] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:05:22 solaris kernel: [394685.660232] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:05:22 solaris kernel: [394685.660234] [<ffffffff8117ed50>] block_write_full_page_endio+0xe0/0x120
Oct 24 18:05:22 solaris kernel: [394685.660237] [<ffffffff811e9ed0>] ? ext4_bh_delay_or_unwritten+0x0/0x30
Oct 24 18:05:22 solaris kernel: [394685.660240] [<ffffffff8117eda5>] block_write_full_page+0x15/0x20
Oct 24 18:05:22 solaris kernel: [394685.660243] [<ffffffff811ed3c3>] ext4_writepage+0x1d3/0x380
Oct 24 18:05:22 solaris kernel: [394685.660246] [<ffffffff811eb1dc>] mpage_da_submit_io+0x14c/0x1c0
Oct 24 18:05:22 solaris kernel: [394685.660249] [<ffffffff811ef7e3>] ? mpage_da_map_blocks+0x1c3/0x370
Oct 24 18:05:22 solaris kernel: [394685.660252] [<ffffffff811f0395>] ext4_da_writepages+0x4f5/0x640
Oct 24 18:05:22 solaris kernel: [394685.660256] [<ffffffff8110a791>] do_writepages+0x21/0x40
Oct 24 18:05:22 solaris kernel: [394685.660260] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0
Oct 24 18:05:22 solaris kernel: [394685.660263] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280
Oct 24 18:05:22 solaris kernel: [394685.660267] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0
Oct 24 18:05:22 solaris kernel: [394685.660269] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0
Oct 24 18:05:22 solaris kernel: [394685.660272] [<ffffffff8117680c>] wb_do_writeback+0x17c/0x190
Oct 24 18:05:22 solaris kernel: [394685.660277] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10
Oct 24 18:05:22 solaris kernel: [394685.660280] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160
Oct 24 18:05:22 solaris kernel: [394685.660283] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0
Oct 24 18:05:22 solaris kernel: [394685.660288] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100
Oct 24 18:05:22 solaris kernel: [394685.660291] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100
Oct 24 18:05:22 solaris kernel: [394685.660293] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:05:22 solaris kernel: [394685.660297] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:05:22 solaris kernel: [394685.660300] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:05:22 solaris kernel: [394685.660303] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:05:22 solaris kernel: [394685.660331] INFO: task jbd2/sdg1-8:1824 blocked for more than 120 seconds.
Oct 24 18:05:22 solaris kernel: [394685.660338] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:05:22 solaris kernel: [394685.660344] jbd2/sdg1-8 D ffff880003dce980 0 1824 2 0x00000000
Oct 24 18:05:22 solaris kernel: [394685.660347] ffff8801d77f3a50 0000000000000246 0000000000000000 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660351] ffff8801d77f3fd8 0000000000015980 ffff8801d77f3fd8 ffff8801d70e2dc0
Oct 24 18:05:22 solaris kernel: [394685.660354] 0000000000015980 0000000000015980 ffff8801d77f3fd8 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660358] Call Trace:
Oct 24 18:05:22 solaris kernel: [394685.660361] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:05:22 solaris kernel: [394685.660364] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:05:22 solaris kernel: [394685.660366] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:05:22 solaris kernel: [394685.660369] [<ffffffff815a27ef>] __wait_on_bit+0x5f/0x90
Oct 24 18:05:22 solaris kernel: [394685.660371] [<ffffffff81100cb3>] wait_on_page_bit+0x73/0x80
Oct 24 18:05:22 solaris kernel: [394685.660374] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:05:22 solaris kernel: [394685.660377] [<ffffffff8110b5d5>] ? pagevec_lookup_tag+0x25/0x40
Oct 24 18:05:22 solaris kernel: [394685.660380] [<ffffffff8110a503>] write_cache_pages+0x193/0x3d0
Oct 24 18:05:22 solaris kernel: [394685.660383] [<ffffffff81109400>] ? __writepage+0x0/0x40
Oct 24 18:05:22 solaris kernel: [394685.660387] [<ffffffff810d1244>] ? __delayacct_blkio_end+0x34/0x60
Oct 24 18:05:22 solaris kernel: [394685.660389] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:05:22 solaris kernel: [394685.660392] [<ffffffff81006b1b>] ? __raw_callee_save_xen_irq_enable+0x11/0x26
Oct 24 18:05:22 solaris kernel: [394685.660396] [<ffffffff8110a764>] generic_writepages+0x24/0x30
Oct 24 18:05:22 solaris kernel: [394685.660401] [<ffffffff81227a37>] journal_submit_data_buffers+0xb7/0x170
Oct 24 18:05:22 solaris kernel: [394685.660404] [<ffffffff812280dc>] jbd2_journal_commit_transaction+0x30c/0x1350
Oct 24 18:05:22 solaris kernel: [394685.660407] [<ffffffff815a40ce>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Oct 24 18:05:22 solaris kernel: [394685.660411] [<ffffffff81070933>] ? try_to_del_timer_sync+0x83/0xe0
Oct 24 18:05:22 solaris kernel: [394685.660415] [<ffffffff8122d87d>] kjournald2+0xbd/0x220
Oct 24 18:05:22 solaris kernel: [394685.660418] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:05:22 solaris kernel: [394685.660420] [<ffffffff8122d7c0>] ? kjournald2+0x0/0x220
Oct 24 18:05:22 solaris kernel: [394685.660423] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:05:22 solaris kernel: [394685.660425] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:05:22 solaris kernel: [394685.660429] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:05:22 solaris kernel: [394685.660431] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:05:22 solaris kernel: [394685.660440] INFO: task python:27611 blocked for more than 120 seconds.
Oct 24 18:05:22 solaris kernel: [394685.660447] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:05:22 solaris kernel: [394685.660453] python D ffff880003dec980 0 27611 1 0x00000000
Oct 24 18:05:22 solaris kernel: [394685.660457] ffff880190a01998 0000000000000282 ffff880003ded518 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660460] ffff880190a01fd8 0000000000015980 ffff880190a01fd8 ffff8801d4fd5b80
Oct 24 18:05:22 solaris kernel: [394685.660464] 0000000000015980 0000000000015980 ffff880190a01fd8 0000000000015980
Oct 24 18:05:22 solaris kernel: [394685.660467] Call Trace:
Oct 24 18:05:22 solaris kernel: [394685.660470] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:05:22 solaris kernel: [394685.660473] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:05:22 solaris kernel: [394685.660475] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:05:22 solaris kernel: [394685.660478] [<ffffffff815a269a>] __wait_on_bit_lock+0x5a/0xc0
Oct 24 18:05:22 solaris kernel: [394685.660481] [<ffffffff81100a97>] __lock_page+0x67/0x70
Oct 24 18:05:22 solaris kernel: [394685.660483] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:05:22 solaris kernel: [394685.660486] [<ffffffff811008ce>] ? find_get_page+0x1e/0xa0
Oct 24 18:05:22 solaris kernel: [394685.660488] [<ffffffff811020b2>] find_lock_page+0x52/0x80
Oct 24 18:05:22 solaris kernel: [394685.660491] [<ffffffff8110212a>] grab_cache_page_write_begin+0x4a/0xc0
Oct 24 18:05:22 solaris kernel: [394685.660494] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:05:22 solaris kernel: [394685.660497] [<ffffffff811f1964>] ext4_da_write_begin+0x144/0x290
Oct 24 18:05:22 solaris kernel: [394685.660501] [<ffffffff810ffe62>] generic_perform_write+0xc2/0x1d0
Oct 24 18:05:22 solaris kernel: [394685.660505] [<ffffffff81202348>] ? __ext4_journal_stop+0x68/0x90
Oct 24 18:05:22 solaris kernel: [394685.660508] [<ffffffff811760a2>] ? __mark_inode_dirty+0x42/0x1d0
Oct 24 18:05:22 solaris kernel: [394685.660511] [<ffffffff810fffd4>] generic_file_buffered_write+0x64/0xa0
Oct 24 18:05:22 solaris kernel: [394685.660514] [<ffffffff811028e0>] __generic_file_aio_write+0x240/0x470
Oct 24 18:05:22 solaris kernel: [394685.660519] [<ffffffff8149fff3>] ? sock_sendmsg+0xf3/0x120
Oct 24 18:05:22 solaris kernel: [394685.660522] [<ffffffff81102b75>] generic_file_aio_write+0x65/0xd0
Oct 24 18:05:22 solaris kernel: [394685.660525] [<ffffffff811e77a9>] ext4_file_write+0x39/0xb0
Oct 24 18:05:22 solaris kernel: [394685.660528] [<ffffffff81152bea>] do_sync_write+0xda/0x120
Oct 24 18:05:22 solaris kernel: [394685.660531] [<ffffffff810043c6>] ? xen_mc_flush+0x96/0x1c0
Oct 24 18:05:22 solaris kernel: [394685.660536] [<ffffffff81052213>] ? finish_task_switch+0x53/0xd0
Oct 24 18:05:22 solaris kernel: [394685.660539] [<ffffffff8128f218>] ? apparmor_file_permission+0x18/0x20
Oct 24 18:05:22 solaris kernel: [394685.660546] [<ffffffff8125e7b6>] ? security_file_permission+0x16/0x20
Oct 24 18:05:22 solaris kernel: [394685.660549] [<ffffffff81152ec8>] vfs_write+0xb8/0x1a0
Oct 24 18:05:22 solaris kernel: [394685.660552] [<ffffffff81153711>] sys_write+0x51/0x80
Oct 24 18:05:22 solaris kernel: [394685.660555] [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b
Oct 24 18:07:22 solaris kernel: [394805.670116] INFO: task flush-202:97:1047 blocked for more than 120 seconds.
Oct 24 18:07:22 solaris kernel: [394805.670138] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:07:22 solaris kernel: [394805.670146] flush-202:97 D ffff880003dce980 0 1047 2 0x00000000
Oct 24 18:07:22 solaris kernel: [394805.670150] ffff8801d6dbf620 0000000000000246 ffff880100000e58 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670155] ffff8801d6dbffd8 0000000000015980 ffff8801d6dbffd8 ffff8801d6db2dc0
Oct 24 18:07:22 solaris kernel: [394805.670158] 0000000000015980 0000000000015980 ffff8801d6dbffd8 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670163] Call Trace:
Oct 24 18:07:22 solaris kernel: [394805.670177] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:07:22 solaris kernel: [394805.670183] [<ffffffff812a2f3c>] get_request_wait+0xcc/0x1a0
Oct 24 18:07:22 solaris kernel: [394805.670188] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:07:22 solaris kernel: [394805.670192] [<ffffffff8129ccd7>] ? elv_merge+0x1d7/0x210
Oct 24 18:07:22 solaris kernel: [394805.670195] [<ffffffff812a30a3>] __make_request+0x93/0x4b0
Oct 24 18:07:22 solaris kernel: [394805.670200] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:07:22 solaris kernel: [394805.670204] [<ffffffff812a1c83>] generic_make_request+0x1b3/0x540
Oct 24 18:07:22 solaris kernel: [394805.670209] [<ffffffff81102cc5>] ? mempool_alloc_slab+0x15/0x20
Oct 24 18:07:22 solaris kernel: [394805.670216] [<ffffffff811ec861>] ? ext4_mark_iloc_dirty+0x71/0x90
Oct 24 18:07:22 solaris kernel: [394805.670219] [<ffffffff812a2092>] submit_bio+0x82/0x110
Oct 24 18:07:22 solaris kernel: [394805.670224] [<ffffffff8117c269>] submit_bh+0xf9/0x140
Oct 24 18:07:22 solaris kernel: [394805.670227] [<ffffffff8117e240>] __block_write_full_page+0x1d0/0x3b0
Oct 24 18:07:22 solaris kernel: [394805.670230] [<ffffffff8117dab0>] ? end_buffer_async_write+0x0/0x190
Oct 24 18:07:22 solaris kernel: [394805.670234] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:07:22 solaris kernel: [394805.670237] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:07:22 solaris kernel: [394805.670240] [<ffffffff8117ed50>] block_write_full_page_endio+0xe0/0x120
Oct 24 18:07:22 solaris kernel: [394805.670243] [<ffffffff811e9ed0>] ? ext4_bh_delay_or_unwritten+0x0/0x30
Oct 24 18:07:22 solaris kernel: [394805.670246] [<ffffffff8117eda5>] block_write_full_page+0x15/0x20
Oct 24 18:07:22 solaris kernel: [394805.670249] [<ffffffff811ed3c3>] ext4_writepage+0x1d3/0x380
Oct 24 18:07:22 solaris kernel: [394805.670254] [<ffffffff811eb1dc>] mpage_da_submit_io+0x14c/0x1c0
Oct 24 18:07:22 solaris kernel: [394805.670257] [<ffffffff811ef7e3>] ? mpage_da_map_blocks+0x1c3/0x370
Oct 24 18:07:22 solaris kernel: [394805.670260] [<ffffffff811f0395>] ext4_da_writepages+0x4f5/0x640
Oct 24 18:07:22 solaris kernel: [394805.670265] [<ffffffff8110a791>] do_writepages+0x21/0x40
Oct 24 18:07:22 solaris kernel: [394805.670270] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0
Oct 24 18:07:22 solaris kernel: [394805.670273] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280
Oct 24 18:07:22 solaris kernel: [394805.670277] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0
Oct 24 18:07:22 solaris kernel: [394805.670280] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0
Oct 24 18:07:22 solaris kernel: [394805.670283] [<ffffffff8117680c>] wb_do_writeback+0x17c/0x190
Oct 24 18:07:22 solaris kernel: [394805.670288] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10
Oct 24 18:07:22 solaris kernel: [394805.670291] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160
Oct 24 18:07:22 solaris kernel: [394805.670294] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0
Oct 24 18:07:22 solaris kernel: [394805.670300] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100
Oct 24 18:07:22 solaris kernel: [394805.670302] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100
Oct 24 18:07:22 solaris kernel: [394805.670305] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:07:22 solaris kernel: [394805.670309] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:07:22 solaris kernel: [394805.670313] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:07:22 solaris kernel: [394805.670316] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:07:22 solaris kernel: [394805.670351] INFO: task jbd2/sdg1-8:1824 blocked for more than 120 seconds.
Oct 24 18:07:22 solaris kernel: [394805.670358] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:07:22 solaris kernel: [394805.670364] jbd2/sdg1-8 D ffff880003dce980 0 1824 2 0x00000000
Oct 24 18:07:22 solaris kernel: [394805.670368] ffff8801d77f3a50 0000000000000246 0000000000000000 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670371] ffff8801d77f3fd8 0000000000015980 ffff8801d77f3fd8 ffff8801d70e2dc0
Oct 24 18:07:22 solaris kernel: [394805.670375] 0000000000015980 0000000000015980 ffff8801d77f3fd8 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670379] Call Trace:
Oct 24 18:07:22 solaris kernel: [394805.670382] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:07:22 solaris kernel: [394805.670385] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:07:22 solaris kernel: [394805.670387] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:07:22 solaris kernel: [394805.670390] [<ffffffff815a27ef>] __wait_on_bit+0x5f/0x90
Oct 24 18:07:22 solaris kernel: [394805.670393] [<ffffffff81100cb3>] wait_on_page_bit+0x73/0x80
Oct 24 18:07:22 solaris kernel: [394805.670396] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:07:22 solaris kernel: [394805.670399] [<ffffffff8110b5d5>] ? pagevec_lookup_tag+0x25/0x40
Oct 24 18:07:22 solaris kernel: [394805.670402] [<ffffffff8110a503>] write_cache_pages+0x193/0x3d0
Oct 24 18:07:22 solaris kernel: [394805.670405] [<ffffffff81109400>] ? __writepage+0x0/0x40
Oct 24 18:07:22 solaris kernel: [394805.670409] [<ffffffff810d1244>] ? __delayacct_blkio_end+0x34/0x60
Oct 24 18:07:22 solaris kernel: [394805.670412] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:07:22 solaris kernel: [394805.670415] [<ffffffff81006b1b>] ? __raw_callee_save_xen_irq_enable+0x11/0x26
Oct 24 18:07:22 solaris kernel: [394805.670418] [<ffffffff8110a764>] generic_writepages+0x24/0x30
Oct 24 18:07:22 solaris kernel: [394805.670424] [<ffffffff81227a37>] journal_submit_data_buffers+0xb7/0x170
Oct 24 18:07:22 solaris kernel: [394805.670427] [<ffffffff812280dc>] jbd2_journal_commit_transaction+0x30c/0x1350
Oct 24 18:07:22 solaris kernel: [394805.670431] [<ffffffff815a40ce>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Oct 24 18:07:22 solaris kernel: [394805.670434] [<ffffffff81070933>] ? try_to_del_timer_sync+0x83/0xe0
Oct 24 18:07:22 solaris kernel: [394805.670438] [<ffffffff8122d87d>] kjournald2+0xbd/0x220
Oct 24 18:07:22 solaris kernel: [394805.670441] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:07:22 solaris kernel: [394805.670443] [<ffffffff8122d7c0>] ? kjournald2+0x0/0x220
Oct 24 18:07:22 solaris kernel: [394805.670446] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:07:22 solaris kernel: [394805.670449] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:07:22 solaris kernel: [394805.670452] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:07:22 solaris kernel: [394805.670455] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 24 18:07:22 solaris kernel: [394805.670466] INFO: task python:27611 blocked for more than 120 seconds.
Oct 24 18:07:22 solaris kernel: [394805.670473] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:07:22 solaris kernel: [394805.670479] python D ffff880003dec980 0 27611 1 0x00000000
Oct 24 18:07:22 solaris kernel: [394805.670483] ffff880190a01998 0000000000000282 ffff880003ded518 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670487] ffff880190a01fd8 0000000000015980 ffff880190a01fd8 ffff8801d4fd5b80
Oct 24 18:07:22 solaris kernel: [394805.670491] 0000000000015980 0000000000015980 ffff880190a01fd8 0000000000015980
Oct 24 18:07:22 solaris kernel: [394805.670494] Call Trace:
Oct 24 18:07:22 solaris kernel: [394805.670497] [<ffffffff81100ac0>] ? sync_page+0x0/0x50
Oct 24 18:07:22 solaris kernel: [394805.670500] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:07:22 solaris kernel: [394805.670502] [<ffffffff81100afd>] sync_page+0x3d/0x50
Oct 24 18:07:22 solaris kernel: [394805.670505] [<ffffffff815a269a>] __wait_on_bit_lock+0x5a/0xc0
Oct 24 18:07:22 solaris kernel: [394805.670508] [<ffffffff81100a97>] __lock_page+0x67/0x70
Oct 24 18:07:22 solaris kernel: [394805.670511] [<ffffffff8107f0c0>] ? wake_bit_function+0x0/0x40
Oct 24 18:07:22 solaris kernel: [394805.670513] [<ffffffff811008ce>] ? find_get_page+0x1e/0xa0
Oct 24 18:07:22 solaris kernel: [394805.670516] [<ffffffff811020b2>] find_lock_page+0x52/0x80
Oct 24 18:07:22 solaris kernel: [394805.670518] [<ffffffff8110212a>] grab_cache_page_write_begin+0x4a/0xc0
Oct 24 18:07:22 solaris kernel: [394805.670522] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:07:22 solaris kernel: [394805.670525] [<ffffffff811f1964>] ext4_da_write_begin+0x144/0x290
Oct 24 18:07:22 solaris kernel: [394805.670530] [<ffffffff810ffe62>] generic_perform_write+0xc2/0x1d0
Oct 24 18:07:22 solaris kernel: [394805.670534] [<ffffffff81202348>] ? __ext4_journal_stop+0x68/0x90
Oct 24 18:07:22 solaris kernel: [394805.670537] [<ffffffff811760a2>] ? __mark_inode_dirty+0x42/0x1d0
Oct 24 18:07:22 solaris kernel: [394805.670540] [<ffffffff810fffd4>] generic_file_buffered_write+0x64/0xa0
Oct 24 18:07:22 solaris kernel: [394805.670543] [<ffffffff811028e0>] __generic_file_aio_write+0x240/0x470
Oct 24 18:07:22 solaris kernel: [394805.670548] [<ffffffff8149fff3>] ? sock_sendmsg+0xf3/0x120
Oct 24 18:07:22 solaris kernel: [394805.670551] [<ffffffff81102b75>] generic_file_aio_write+0x65/0xd0
Oct 24 18:07:22 solaris kernel: [394805.670554] [<ffffffff811e77a9>] ext4_file_write+0x39/0xb0
Oct 24 18:07:22 solaris kernel: [394805.670558] [<ffffffff81152bea>] do_sync_write+0xda/0x120
Oct 24 18:07:22 solaris kernel: [394805.670564] [<ffffffff810043c6>] ? xen_mc_flush+0x96/0x1c0
Oct 24 18:07:22 solaris kernel: [394805.670570] [<ffffffff81052213>] ? finish_task_switch+0x53/0xd0
Oct 24 18:07:22 solaris kernel: [394805.670573] [<ffffffff8128f218>] ? apparmor_file_permission+0x18/0x20
Oct 24 18:07:22 solaris kernel: [394805.670578] [<ffffffff8125e7b6>] ? security_file_permission+0x16/0x20
Oct 24 18:07:22 solaris kernel: [394805.670581] [<ffffffff81152ec8>] vfs_write+0xb8/0x1a0
Oct 24 18:07:22 solaris kernel: [394805.670584] [<ffffffff81153711>] sys_write+0x51/0x80
Oct 24 18:07:22 solaris kernel: [394805.670587] [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b
Oct 24 18:09:22 solaris kernel: [394925.660085] INFO: task flush-202:97:1047 blocked for more than 120 seconds.
Oct 24 18:09:22 solaris kernel: [394925.660099] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 24 18:09:22 solaris kernel: [394925.660106] flush-202:97 D ffff880003dce980 0 1047 2 0x00000000
Oct 24 18:09:22 solaris kernel: [394925.660111] ffff8801d6dbf620 0000000000000246 ffff880100000e58 0000000000015980
Oct 24 18:09:22 solaris kernel: [394925.660116] ffff8801d6dbffd8 0000000000015980 ffff8801d6dbffd8 ffff8801d6db2dc0
Oct 24 18:09:22 solaris kernel: [394925.660119] 0000000000015980 0000000000015980 ffff8801d6dbffd8 0000000000015980
Oct 24 18:09:22 solaris kernel: [394925.660124] Call Trace:
Oct 24 18:09:22 solaris kernel: [394925.660135] [<ffffffff815a2173>] io_schedule+0x73/0xc0
Oct 24 18:09:22 solaris kernel: [394925.660140] [<ffffffff812a2f3c>] get_request_wait+0xcc/0x1a0
Oct 24 18:09:22 solaris kernel: [394925.660147] [<ffffffff8107f080>] ? autoremove_wake_function+0x0/0x40
Oct 24 18:09:22 solaris kernel: [394925.660154] [<ffffffff8129ccd7>] ? elv_merge+0x1d7/0x210
Oct 24 18:09:22 solaris kernel: [394925.660157] [<ffffffff812a30a3>] __make_request+0x93/0x4b0
Oct 24 18:09:22 solaris kernel: [394925.660162] [<ffffffff81006adf>] ? __raw_callee_save_xen_restore_fl+0x11/0x1e
Oct 24 18:09:22 solaris kernel: [394925.660165] [<ffffffff812a1c83>] generic_make_request+0x1b3/0x540
Oct 24 18:09:22 solaris kernel: [394925.660169] [<ffffffff81102cc5>] ? mempool_alloc_slab+0x15/0x20
Oct 24 18:09:22 solaris kernel: [394925.660174] [<ffffffff811ec861>] ? ext4_mark_iloc_dirty+0x71/0x90
Oct 24 18:09:22 solaris kernel: [394925.660177] [<ffffffff812a2092>] submit_bio+0x82/0x110
Oct 24 18:09:22 solaris kernel: [394925.660181] [<ffffffff8117c269>] submit_bh+0xf9/0x140
Oct 24 18:09:22 solaris kernel: [394925.660184] [<ffffffff8117e240>] __block_write_full_page+0x1d0/0x3b0
Oct 24 18:09:22 solaris kernel: [394925.660188] [<ffffffff8117dab0>] ? end_buffer_async_write+0x0/0x190
Oct 24 18:09:22 solaris kernel: [394925.660191] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:09:22 solaris kernel: [394925.660194] [<ffffffff811ee800>] ? noalloc_get_block_write+0x0/0x30
Oct 24 18:09:22 solaris kernel: [394925.660197] [<ffffffff8117ed50>] block_write_full_page_endio+0xe0/0x120
Oct 24 18:09:22 solaris kernel: [394925.660200] [<ffffffff811e9ed0>] ? ext4_bh_delay_or_unwritten+0x0/0x30
Oct 24 18:09:22 solaris kernel: [394925.660203] [<ffffffff8117eda5>] block_write_full_page+0x15/0x20
Oct 24 18:09:22 solaris kernel: [394925.660206] [<ffffffff811ed3c3>] ext4_writepage+0x1d3/0x380
Oct 24 18:09:22 solaris kernel: [394925.660208] [<ffffffff811eb1dc>] mpage_da_submit_io+0x14c/0x1c0
Oct 24 18:09:22 solaris kernel: [394925.660211] [<ffffffff811ef7e3>] ? mpage_da_map_blocks+0x1c3/0x370
Oct 24 18:09:22 solaris kernel: [394925.660214] [<ffffffff811f0395>] ext4_da_writepages+0x4f5/0x640
Oct 24 18:09:22 solaris kernel: [394925.660219] [<ffffffff8110a791>] do_writepages+0x21/0x40
Oct 24 18:09:22 solaris kernel: [394925.660223] [<ffffffff81175356>] writeback_single_inode+0xe6/0x3f0
Oct 24 18:09:22 solaris kernel: [394925.660226] [<ffffffff81175ab5>] writeback_sb_inodes+0x195/0x280
Oct 24 18:09:22 solaris kernel: [394925.660229] [<ffffffff811762d0>] writeback_inodes_wb+0xa0/0x1b0
Oct 24 18:09:22 solaris kernel: [394925.660232] [<ffffffff8117662b>] wb_writeback+0x24b/0x2b0
Oct 24 18:09:22 solaris kernel: [394925.660235] [<ffffffff8117680c>] wb_do_writeback+0x17c/0x190
Oct 24 18:09:22 solaris kernel: [394925.660240] [<ffffffff8106ffd0>] ? process_timeout+0x0/0x10
Oct 24 18:09:22 solaris kernel: [394925.660243] [<ffffffff81176873>] bdi_writeback_task+0x53/0x160
Oct 24 18:09:22 solaris kernel: [394925.660246] [<ffffffff8107ef47>] ? bit_waitqueue+0x17/0xd0
Oct 24 18:09:22 solaris kernel: [394925.660250] [<ffffffff81119cb6>] bdi_start_fn+0x86/0x100
Oct 24 18:09:22 solaris kernel: [394925.660252] [<ffffffff81119c30>] ? bdi_start_fn+0x0/0x100
Oct 24 18:09:22 solaris kernel: [394925.660255] [<ffffffff8107eb26>] kthread+0x96/0xa0
Oct 24 18:09:22 solaris kernel: [394925.660258] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
Oct 24 18:09:22 solaris kernel: [394925.660262] [<ffffffff815a465d>] ? retint_restore_args+0x5/0x6
Oct 24 18:09:22 solaris kernel: [394925.660265] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
Oct 25 04:01:07 solaris kernel: [430429.909446] vbd vbd-2145: 16 Device in use; refusing to close

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: linux-image-2.6.35-22-virtual 2.6.35-22.35
Regression: Yes
Reproducible: Yes
ProcVersionSignature: Ubuntu 2.6.35-22.35-virtual 2.6.35.4
Uname: Linux 2.6.35-22-virtual x86_64
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access /dev/snd/: No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
Date: Thu Oct 28 09:00:21 2010
Ec2AMI: ami-ca1f4f8f
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: us-west-1b
Ec2InstanceType: m1.large
Ec2Kernel: aki-9ba0f1de
Ec2Ramdisk: unavailable
Lspci:

Lsusb: Error: command ['lsusb'] failed with exit code 1:
ProcCmdLine: root=LABEL=uec-rootfs ro console=hvc0
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcModules: acpiphp 18752 0 - Live 0xffffffffa0000000
SourcePackage: linux