I cannot find how it might be related, but we are seeing massive filesystem corruptions in virtual guests on kvm in Lucid. Host was running several kernels, from stock Lucid up to 3.0.0-14-server. Guests were booted with several different kernels as well. We also changed storag backen form qcow, to raw and eventually to lvm base with no avail. Usuall message just after going to RO: [2012-03-01 04:39:06] EXT4-fs (vda): error count: 10 [2012-03-01 04:39:06] EXT4-fs (vda): initial error at 1323754623: htree_dirblock_to_tree:586: inode 371080: block 1229922 [2012-03-01 04:39:06] EXT4-fs (vda): last error at 1329327878: ext4_remount:3754: inode 170313: block 543763 [2012-03-02 04:40:50] EXT4-fs (vda): error count: 10 [2012-03-02 04:40:50] EXT4-fs (vda): initial error at 1323754623: htree_dirblock_to_tree:586: inode 371080: block 1229922 [2012-03-02 04:40:50] EXT4-fs (vda): last error at 1329327878: ext4_remount:3754: inode 170313: block 543763 [2012-03-03 04:42:38] EXT4-fs (vda): error count: 10 [2012-03-03 04:42:38] EXT4-fs (vda): initial error at 1323754623: htree_dirblock_to_tree:586: inode 371080: block 1229922 [2012-03-03 04:42:38] EXT4-fs (vda): last error at 1329327878: ext4_remount:3754: inode 170313: block 543763 [2012-03-04 04:44:25] EXT4-fs (vda): error count: 10 [2012-03-04 04:44:25] EXT4-fs (vda): initial error at 1323754623: htree_dirblock_to_tree:586: inode 371080: block 1229922 [2012-03-04 04:44:25] EXT4-fs (vda): last error at 1329327878: ext4_remount:3754: inode 170313: block 543763 [2012-03-04 20:34:20] EXT4-fs error (device vda): htree_dirblock_to_tree:587: inode #171186: block 546842: comm chown: bad entry in directory: rec_len is smaller than minimal - offset=0(0), inode=4210740, rec_len=0, name_len=0 [2012-03-04 20:34:20] Aborting journal on device vda-8. [2012-03-04 20:34:20] EXT4-fs (vda): Remounting filesystem read-only [2012-03-05 04:46:13] EXT4-fs (vda): error count: 11 [2012-03-05 04:46:13] EXT4-fs (vda): initial error at 1323754623: htree_dirblock_to_tree:586: inode 371080: block 1229922 [2012-03-05 04:46:13] EXT4-fs (vda): last error at 1330893259: htree_dirblock_to_tree:587: inode 171186: block 546842 Or [20768.343508] EXT3-fs error (device vda): htree_dirblock_to_tree: bad entry in directory #837494: rec_len is smaller than minimal - offset=0, inode=4210740, rec_len=0, name_len=0 [20768.348149] Aborting journal on device vda. [20768.352064] EXT3-fs (vda): error: remounting filesystem read-only [20768.396397] __journal_remove_journal_head: freeing b_committed_data [20768.396405] __journal_remove_journal_head: freeing b_committed_data [20768.396407] __journal_remove_journal_head: freeing b_committed_data [20769.700102] ------------[ cut here ]------------ [20769.700125] WARNING: at /build/buildd/linux-lts-backport-oneiric-3.0.0/fs/ext3/inode.c:1571 ext3_ordered_writepage+0x223/0x250() [20769.700127] Hardware name: Bochs [20769.700128] Modules linked in: nfs lockd fscache auth_rpcgss nfs_acl sunrpc psmouse serio_raw virtio_balloon i2c_piix4 raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov floppy raid6_pq async_tx raid1 raid0 multipath linear [20769.700146] Pid: 2496, comm: flush-253:0 Not tainted 3.0.0-14-server #23~lucid1-Ubuntu [20769.700147] Call Trace: [20769.700155] [] warn_slowpath_common+0x7f/0xc0 [20769.700158] [] warn_slowpath_null+0x1a/0x20 [20769.700160] [] ext3_ordered_writepage+0x223/0x250 [20769.700166] [] __writepage+0x17/0x40 [20769.700169] [] write_cache_pages+0x241/0x4d0 [20769.700171] [] ? set_page_dirty+0x70/0x70 [20769.700173] [] generic_writepages+0x51/0x80 [20769.700176] [] do_writepages+0x35/0x40 [20769.700180] [] writeback_single_inode+0x10e/0x280 [20769.700183] [] writeback_sb_inodes+0xe3/0x1b0 [20769.700185] [] writeback_inodes_wb+0xa4/0x170 [20769.700187] [] wb_writeback+0x2f3/0x430 [20769.700191] [] ? _raw_spin_lock_irqsave+0x2f/0x40 [20769.700194] [] wb_do_writeback+0x21f/0x270 [20769.700196] [] bdi_writeback_thread+0xaa/0x270 [20769.700199] [] ? wb_do_writeback+0x270/0x270 [20769.700203] [] kthread+0x96/0xa0 [20769.700206] [] kernel_thread_helper+0x4/0x10 [20769.700208] [] ? kthread_worker_fn+0x190/0x190 [20769.700211] [] ? gs_change+0x13/0x13 [20769.700212] ---[ end trace 787438b409e1b580 ]--- [2012-02-23 10:01:54] plackup[1101]: segfault at 404034 ip 0000000000404034 sp 00007fff61aad508 error 14 in perl[601000+1000] [2012-02-23 10:03:57] plackup[1092]: segfault at 404034 ip 0000000000404034 sp 00007fff61aad458 error 14 in perl[601000+1000] [2012-02-23 10:09:53] EXT4-fs error (device vda): htree_dirblock_to_tree:587: inode #135110: block 533749: comm grep: bad entry in directory: rec_len is smaller than minimal - offset=0(0), inode=4210740, rec_len=0, name_len=0 [2012-02-23 10:09:53] Aborting journal on device vda-8. [2012-02-23 10:09:53] EXT4-fs (vda): Remounting filesystem read-only [2012-02-23 10:11:41] EXT4-fs error (device vda): htree_dirblock_to_tree:587: inode #135110: block 533749: comm grep: bad entry in directory: rec_len is smaller than minimal - offset=0(0), inode=4210740, rec_len=0, name_len=0 [2012-02-23 10:12:15] EXT4-fs error (device vda): htree_dirblock_to_tree:587: inode #135110: block 533749: comm grep: bad entry in directory: rec_len is smaller than minimal - offset=0(0), inode=4210740, rec_len=0, name_len=0 [2012-02-23 10:13:15] EXT4-fs error (device vda): htree_dirblock_to_tree:587: inode #135110: block 533749: comm grep: bad entry in directory: rec_len is smaller than minimal - offset=0(0), inode=4210740, rec_len=0, name_len=0 Interesting is inode 4210740 referred by all crashes (although I don't know why)