Ubuntu16.04:KVM - xfs_log_force: error -5 returned after stress test running after a day #345
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
The Ubuntu-power-systems project |
Invalid
|
High
|
Canonical Kernel Team | ||
linux (Ubuntu) |
Invalid
|
High
|
Canonical Kernel Team | ||
Xenial |
Invalid
|
High
|
Canonical Kernel Team |
Bug Description
== Comment: #0 - Chanh H. Nguyen <email address hidden> - 2016-09-08 15:01:30 ==
We have installed 16.04.1 on our guest and running IO stress test against via virtual qlogic disk. After running around 8 hrs, this error popping up "xfs_imap_to_bp: xfs_trans_
Jul 19 19:03:41 tinyg2 kernel: [65047.974604] XFS (vdb9): metadata I/O error: block 0x2f140 ("xfs_trans_
Jul 19 19:03:41 tinyg2 kernel: [65047.975885] XFS (vdb9): xfs_imap_to_bp: xfs_trans_
Jul 19 19:03:41 tinyg2 kernel: [65047.976069] XFS (vdb9): xfs_do_
Jul 19 19:03:41 tinyg2 kernel: [65047.976457] XFS (vdb9): metadata I/O error: block 0x703d54 ("xlog_iodone") error 5 numblks 64
Jul 19 19:03:41 tinyg2 kernel: [65047.976662] XFS (vdb9): xfs_do_
Jul 19 19:03:41 tinyg2 kernel: [65047.976995] XFS (vdb9): Log I/O Error Detected. Shutting down filesystem
Jul 19 19:03:41 tinyg2 kernel: [65047.977114] XFS (vdb9): Please umount the filesystem and rectify the problem(s)
follow with all of these error that quickly fill up dmesg log....
Jul 19 19:04:37 tinyg2 kernel: [65103.515386] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:05:07 tinyg2 kernel: [65133.595369] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:05:37 tinyg2 kernel: [65163.679358] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:06:07 tinyg2 kernel: [65193.755383] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:06:37 tinyg2 kernel: [65223.835396] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:06:51 tinyg2 kernel: [65237.091455] EXT4-fs (loop1): mounting ext2 file system using the ext4 subsystem
Jul 19 19:06:51 tinyg2 kernel: [65237.093219] EXT4-fs (loop1): mounted filesystem without journal. Opts: (null)
Jul 19 19:06:51 tinyg2 kernel: [65237.623893] Process accounting resumed
Jul 19 19:07:07 tinyg2 kernel: [65253.919361] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:07:37 tinyg2 kernel: [65283.995393] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:08:08 tinyg2 kernel: [65314.075378] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:08:38 tinyg2 kernel: [65344.155397] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:09:08 tinyg2 kernel: [65374.235367] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:09:38 tinyg2 kernel: [65404.319354] XFS (vdb9): xfs_log_force: error -5 returned.
Jul 19 19:10:08 tinyg2 kernel: [65434.395368] XFS (vdb9): xfs_log_force: error -5 returned.
== Comment: #1 - Chanh H. Nguyen <email address hidden> - 2016-09-08 15:02:27 ==
We recently re-created this bug on our new system "cressg2" which is using a different SAN on kernel -36.
My Ubuntu 16.04.1 guest, cressg2, are running the IO and other stress test, and it is hitting "xfs_log_force: error -5 returned" problem after it runs the tests for 15 hours. It is using our lab storage (SVC SAN storage configuation with access to DS8000 Volume) and switch, compare to tinyg3, so it is using different storage and fabric switch.
root@cressg2:~# uname -a
Linux cressg2 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:00:57 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux
[Wed Sep 7 10:26:48 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:18 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:32 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:33 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:33 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:48 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:48 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:49 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:27:56 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:28:18 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:28:48 2016] XFS (vdb8): xfs_log_force: error -5 returned.
[Wed Sep 7 10:29:18 2016] XFS (vdb8): xfs_log_force: error -5 returned.
summary: |
- ISST-LTE:Briggs:Ubuntu16.04:KVM - xfs_log_force: error -5 returned after - stress test running after a day #345 + Ubuntu16.04:KVM - xfs_log_force: error -5 returned after stress test + running after a day #345 |
tags: | added: triage-g |
Changed in ubuntu-power-systems: | |
importance: | Undecided → High |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
Changed in ubuntu-power-systems: | |
status: | Triaged → Incomplete |
Changed in linux (Ubuntu): | |
assignee: | Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Canonical Kernel Team (canonical-kernel-team) |
Changed in linux (Ubuntu Xenial): | |
assignee: | nobody → Canonical Kernel Team (canonical-kernel-team) |
tags: |
added: targetmilestone-inin--- removed: targetmilestone-inin16045 |
tags: | added: cscc |
Default Comment by Bridge