Comment 0 for bug 861656

Revision history for this message
Serge Hallyn (serge-hallyn) wrote : many ext4 errors

My host has xfs rootfs. It runs openstack with lxc, which creates the container root filesystems on qemu-nbd block devices with ext4fs by default.

For probably 1 in 8 containers, when the host is working hard, I end up with ext4 errors which in the end prevent the host from cleanly rebooting. http://people.canonical.com/~serge/20110927_001.jpg is one example. Here is another one:

[ 178.736380] JBD2: I/O error detected when updating journal superblock for nbd13-8.
[ 178.736567] journal commit I/O error
[ 182.374588] br100: port 4(veth2) entering forwarding state
[ 182.376240] device veth2 left promiscuous mode
[ 182.376245] br100: port 4(veth2) entering disabled state
[ 182.983726] nbd14: NBD_DISCONNECT
[ 182.983979] nbd14: Receive control failed (result -32)
[ 182.984075] nbd14: queue cleared
[ 182.984525] nbd14: Attempted send on closed socket
[ 182.984530] end_request: I/O error, dev nbd14, sector 328280
[ 182.984533] Buffer I/O error on device nbd14, logical block 41035
[ 182.984538] EXT4-fs warning (device nbd14): ext4_end_bio:242: I/O error writing to inode 32327 (offset 0 size 4096 starting block 41036)
[ 182.984597] nbd14: Attempted send on closed socket
[ 182.984600] end_request: I/O error, dev nbd14, sector 328256
[ 182.984602] Buffer I/O error on device nbd14, logical block 41032
[ 182.984605] EXT4-fs warning (device nbd14): ext4_end_bio:242: I/O error writing to inode 32342 (offset 0 size 4096 starting block 41033)
[ 182.984770] JBD2: Detected IO errors while flushing file data on nbd14-8
[ 182.984776] nbd14: Attempted send on closed socket
[ 182.984779] end_request: I/O error, dev nbd14, sector 1049856
[ 182.984794] Aborting journal on device nbd14-8.
[ 182.984802] nbd14: Attempted send on closed socket
[ 182.984804] end_request: I/O error, dev nbd14, sector 1048576
[ 182.984807] Buffer I/O error on device nbd14, logical block 131072
[ 182.984809] lost page write due to I/O error on nbd14
[ 182.984812] JBD2: I/O error detected when updating journal superblock for nbd14-8.
[ 182.984825] journal commit I/O error
[ 184.789984] br100: port 3(veth1) entering forwarding state
[ 184.791634] device veth1 left promiscuous mode
[ 184.791639] br100: port 3(veth1) entering disabled state
[ 185.236242] nbd12: NBD_DISCONNECT
[ 185.236362] nbd12: Receive control failed (result -32)
[ 185.236418] nbd12: queue cleared
[ 185.236734] nbd12: Attempted send on closed socket
[ 185.236737] end_request: I/O error, dev nbd12, sector 328280
[ 185.236740] Buffer I/O error on device nbd12, logical block 41035
[ 185.236745] EXT4-fs warning (device nbd12): ext4_end_bio:242: I/O error writing to inode 32327 (offset 0 size 4096 starting block 41036)
[ 185.236804] nbd12: Attempted send on closed socket
[ 185.236806] end_request: I/O error, dev nbd12, sector 328256
[ 185.236809] Buffer I/O error on device nbd12, logical block 41032
[ 185.236812] EXT4-fs warning (device nbd12): ext4_end_bio:242: I/O error writing to inode 32342 (offset 0 size 4096 starting block 41033)
[ 185.236998] JBD2: Detected IO errors while flushing file data on nbd12-8
[ 185.237005] nbd12: Attempted send on closed socket
[ 185.237007] end_request: I/O error, dev nbd12, sector 1049856
[ 185.237025] Aborting journal on device nbd12-8.
[ 185.237034] nbd12: Attempted send on closed socket
[ 185.237036] end_request: I/O error, dev nbd12, sector 1048576
[ 185.237039] Buffer I/O error on device nbd12, logical block 131072
[ 185.237041] lost page write due to I/O error on nbd12
[ 185.237044] JBD2: I/O error detected when updating journal superblock for nbd12-8.
[ 185.237059] journal commit I/O error
[ 185.342363] nbd14: Attempted send on closed socket
[ 185.342369] end_request: I/O error, dev nbd14, sector 0
[ 185.342374] Buffer I/O error on device nbd14, logical block 0
[ 185.342375] lost page write due to I/O error on nbd14
[ 185.342388] EXT4-fs error (device nbd14): ext4_put_super:771: Couldn't clean up the journal
[ 185.342392] EXT4-fs (nbd14): Remounting filesystem read-only
[ 185.342394] EXT4-fs (nbd14): previous I/O error to superblock detected
[ 185.342399] nbd14: Attempted send on closed socket
[ 185.342401] end_request: I/O error, dev nbd14, sector 0
[ 185.342404] Buffer I/O error on device nbd14, logical block 0
[ 185.342405] lost page write due to I/O error on nbd14
[ 185.352231] nbd12: Attempted send on closed socket
[ 185.352236] end_request: I/O error, dev nbd12, sector 0
[ 185.352239] Buffer I/O error on device nbd12, logical block 0
[ 185.352241] lost page write due to I/O error on nbd12
[ 185.352252] EXT4-fs error (device nbd12): ext4_put_super:771: Couldn't clean up the journal
[ 185.352256] EXT4-fs (nbd12): Remounting filesystem read-only
[ 185.352258] EXT4-fs (nbd12): previous I/O error to superblock detected

[ 178.733767] nbd13: NBD_DISCONNECT
[ 178.733865] nbd13: Receive control failed (result -32)
[ 178.733911] nbd13: shutting down socket
[ 178.733914] nbd13: queue cleared
[ 178.734250] nbd13: Attempted send on closed socket
[ 178.734254] end_request: I/O error, dev nbd13, sector 328280
[ 178.734257] Buffer I/O error on device nbd13, logical block 41035
[ 178.734262] EXT4-fs warning (device nbd13): ext4_end_bio:242: I/O error writing to inode 32327 (offset 0 size 4096 starting block 41036)
[ 178.734334] nbd13: Attempted send on closed socket
[ 178.734336] end_request: I/O error, dev nbd13, sector 328256
[ 178.734338] Buffer I/O error on device nbd13, logical block 41032
[ 178.734342] EXT4-fs warning (device nbd13): ext4_end_bio:242: I/O error writing to inode 32342 (offset 0 size 4096 starting block 41033)
[ 178.736325] JBD2: Detected IO errors while flushing file data on nbd13-8
[ 178.736339] nbd13: Attempted send on closed socket
[ 178.736343] end_request: I/O error, dev nbd13, sector 1049856
[ 178.736361] Aborting journal on device nbd13-8.
[ 178.736370] nbd13: Attempted send on closed socket
[ 178.736372] end_request: I/O error, dev nbd13, sector 1048576
[ 178.736375] Buffer I/O error on device nbd13, logical block 131072
[ 178.736377] lost page write due to I/O error on nbd13