Comment 0 for bug 1668129

Revision history for this message
Pete Cheslock (pete-cheslock) wrote :

On the AWS i3 instance class - when putting the new NVME storage disks under high IO load - seeing data corruption and errors in dmesg

[ 662.884390] blk_update_request: I/O error, dev nvme0n1, sector 120063912
[ 662.887824] Buffer I/O error on dev nvme0n1, logical block 14971093, lost async page write
[ 662.891254] Buffer I/O error on dev nvme0n1, logical block 14971094, lost async page write
[ 662.895591] Buffer I/O error on dev nvme0n1, logical block 14971095, lost async page write
[ 662.899873] Buffer I/O error on dev nvme0n1, logical block 14971096, lost async page write
[ 662.904179] Buffer I/O error on dev nvme0n1, logical block 14971097, lost async page write
[ 662.908458] Buffer I/O error on dev nvme0n1, logical block 14971098, lost async page write
[ 662.912287] Buffer I/O error on dev nvme0n1, logical block 14971099, lost async page write
[ 662.916047] Buffer I/O error on dev nvme0n1, logical block 14971100, lost async page write
[ 662.920285] Buffer I/O error on dev nvme0n1, logical block 14971101, lost async page write
[ 662.924565] Buffer I/O error on dev nvme0n1, logical block 14971102, lost async page write
[ 663.645530] blk_update_request: I/O error, dev nvme0n1, sector 120756912
<snip>
[ 1012.752265] blk_update_request: I/O error, dev nvme0n1, sector 3744
[ 1012.755396] buffer_io_error: 194552 callbacks suppressed
[ 1012.755398] Buffer I/O error on dev nvme0n1, logical block 20, lost async page write
[ 1012.759248] Buffer I/O error on dev nvme0n1, logical block 21, lost async page write
[ 1012.763368] Buffer I/O error on dev nvme0n1, logical block 22, lost async page write
[ 1012.767271] Buffer I/O error on dev nvme0n1, logical block 23, lost async page write
[ 1012.771314] Buffer I/O error on dev nvme0n1, logical block 24, lost async page write

Able to replicate this with a bonnie++ stress test.

bonnie++ -d /mnt/test/ -r 1000

Linux i-0d76e144d85f487cf 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux