ata1.00: exception Emask 0x50 SAct 0x1 SErr 0x480800 action 0x6 frozen

[Sasa Paporovic]

I am on Ubuntu 11.10 64bit and testing Ubuntu12.04 in kvm-qemu0.14.1+noroms-0ubuntu6.3 on it.

On running Ubuntu12.04 in kvm-qemu my Ubuntu 11.10 host kernel throws ata exeptions of different kinds.

One high repeated:

Mar 26 13:44:07 ubuntu kernel: [ 3941.779080] ata1.00: exception Emask 0x50 SAct 0x1 SErr 0x480800 action 0x6 frozen
Mar 26 13:44:07 ubuntu kernel: [ 3941.779092] ata1.00: irq_stat 0x08000000, interface fatal error
Mar 26 13:44:07 ubuntu kernel: [ 3941.779101] ata1: SError: { HostInt 10B8B Handshk }
Mar 26 13:44:07 ubuntu kernel: [ 3941.779113] ata1.00: failed command: WRITE FPDMA QUEUED
Mar 26 13:44:07 ubuntu kernel: [ 3941.779128] ata1.00: cmd 61/58:00:1c:1e:1f/00:00:03:00:00/40 tag 0 ncq 45056 out
Mar 26 13:44:07 ubuntu kernel: [ 3941.779131] res 40/00:04:1c:1e:1f/00:00:03:00:00/40 Emask 0x50 (ATA bus error)
Mar 26 13:44:07 ubuntu kernel: [ 3941.779138] ata1.00: status: { DRDY }
Mar 26 13:44:07 ubuntu kernel: [ 3941.779148] ata1: hard resetting link
Mar 26 13:44:07 ubuntu kernel: [ 3942.268089] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Mar 26 13:44:07 ubuntu kernel: [ 3942.276491] ata1.00: configured for UDMA/133
Mar 26 13:44:07 ubuntu kernel: [ 3942.292077] ata1: EH complete

Interestingly the ata entries stopped when I switch of the Ubuntu12.04 guest.

They may be caused by guest Ubuntu12.04 kernel3.2.x I/O errors which I could observe there in.

I will attachadditional the 12.04 guest logs when and if I get it manage to access them.

Anyway, the host kernel(3.0.x series) should not be affected of guest kernel misbehaviour. This me cause in short time a corrupted filesystem.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-generic 3.0.0.17.20
ProcVersionSignature: Ubuntu 3.0.0-17.30-generic 3.0.22
Uname: Linux 3.0.0-17-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
ApportVersion: 1.23-0ubuntu4
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: user 2318 F.... pulseaudio
CRDA: Error: [Errno 2] Datei oder Verzeichnis nicht gefunden
Card0.Amixer.info:
 Card hw:0 'SB'/'HDA ATI SB at 0xfe7f4000 irq 16'
   Mixer name : 'Realtek ALC889'
   Components : 'HDA:10ec0889,14627596,00100004'
   Controls : 37
   Simple ctrls : 22
Date: Mon Mar 26 14:08:43 2012
HibernationDevice: RESUME=UUID=377d403d-6b03-486e-b35e-4df4e6ca1e74
InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64+mac (20111011)
MachineType: MICRO-STAR INTERNATIONAL CO.,LTD MS-7596
ProcEnviron:
 PATH=(custom, no User Name)
 LANG=de_DE.UTF-8
 SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.0.0-17-generic root=UUID=ab49260a-c922-44e8-9686-9e7848aad0ab ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.0.0-17-generic N/A
 linux-backports-modules-3.0.0-17-generic N/A
 linux-firmware 1.60.1
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/03/2010
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: V1.8
dmi.board.asset.tag: To Be Filled By O.E.M.
dmi.board.name: 760GM -E51 (MS-7596)
dmi.board.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
dmi.board.version: 1.0
dmi.chassis.asset.tag: To Be Filled By O.E.M.
dmi.chassis.type: 3
dmi.chassis.vendor: MICRO-STAR INTERNATIONAL CO.,LTD
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrV1.8:bd09/03/2010:svnMICRO-STARINTERNATIONALCO.,LTD:pnMS-7596:pvr1.0:rvnMICRO-STARINTERNATIONALCO.,LTD:rn760GM-E51(MS-7596):rvr1.0:cvnMICRO-STARINTERNATIONALCO.,LTD:ct3:cvr1.0:
dmi.product.name: MS-7596
dmi.product.version: 1.0
dmi.sys.vendor: MICRO-STAR INTERNATIONAL CO.,LTD

[Sasa Paporovic (melchiaros)]

Can not gather the Ubuntu12.04 guest logs. The I/O errors have prevent the logging/writing to the virtual harddrive.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.3 kernel[1] (Not a kernel in the daily directory). Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag(Only that one tag, please leave the other tags). This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text.

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[1] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.3-precise/

[Richard Eames (naddiseo)]

@melchiaros regarding bug #958188, I'm not sure if mine is a duplicate or not. Mine is specifically related to being an SSD, because I also have a HDD installed which doesn't fail. Let me know if there's anything I can do to help diagnose mine to see if it is at all related to this (#965213).

[Sasa Paporovic (melchiaros)]

I have set this to kernel-unable-to test-upstream.

If somene of the subscribers is able and willing to test, feel free.

[Sasa Paporovic (melchiaros)]

Additional:

I have made several installations following each other on the same sytem with Ubuntu11.10.

With linux-generic 3.0.0.16.x Ubuntu serie is this problem not observable to me.

With linux-generic 3.0.0.17.x Ubuntu serie it occures regular!

[Sasa Paporovic (melchiaros)]

Regular short after appearing in the kern.log my system is crashing.

[Julien (julien-t43)]

I got this problem yesterday and this morning. It disappears without explanation now.

Hardware is a samsung N230 with a just changed disk to WD Momentus XT (hybrid SSD) 750GB. It worked fine 5 days before getting this problem during one day and a half.
System is Lubuntu 11.10 freshly installed with the new disk with a multiboot Windows. The latter seems to have not been affected by disk problem. explore2extfs seems to be fine from windows, but lubuntu liveusb key seems to also have the problem in its log (but of course, it booted ok). I couldn't boot the local system, it was switched to busybox shell (initrd so) before doing anything (either in normal boot, recovery mode, or previous kernel 3.0.0-17, 3.0.0-12).
libata.force=noncq and libata.noacpi=1 didn't change a thing.
fsck found a lot of error in the interval and "fixed" them (with afterwork sadly ...) but didn't change the problem also.

I don't know if it's samsung related as seen in Bug 285892, Bios related, kernel, or others, but this bug seems to last a long time.

Other links I found
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/285892?comments=all
http://forum.ubuntu-fr.org/viewtopic.php?id=235884
http://forums.contribs.org/index.php?topic=46165.0
https://wiki.archlinux.org/index.php/Samsung_N140
https://wiki.archlinux.org/index.php/Samsung_N130
http://www.samsung.com/fr/support/model/NP-N230-JA01FR-downloads

dmesg
>>>
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.0.0-12-generic (buildd@vernadsky) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #20-Ubuntu SMP Fri Oct 7 14:50:42 UTC 2011 (Ubuntu 3.0.0-12.20-generic 3.0.4)
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] NSC Geode by NSC
[ 0.000000] Cyrix CyrixInstead
[ 0.000000] Centaur CentaurHauls
[ 0.000000] Transmeta GenuineTMx86
[ 0.000000] Transmeta TransmetaCPU
[ 0.000000] UMC UMC UMC UMC
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 000000000009dc00 (usable)
[ 0.000000] BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000ce000 - 00000000000d0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000dc000 - 00000000000e0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
[ 0.000000] BIOS-e820: 0000000000100000 - 000000007f5b0000 (usable)
[ 0.000000] BIOS-e820: 000000007f5b0000 - 000000007f5c0000 (ACPI data)
[ 0.000000] BIOS-e820: 000000007f5c0000 - 000000007f5c3000 (ACPI NVS)
[ 0.000000] BIOS-e820: 000000007f5c3000 - 0000000080000000 (reserved)
[ 0.000000] BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
[ 0.000000] BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
[ 0.000000] BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[ 0.000000] Notice: NX (Execute Disable) protection cannot be enabled in hardware: non-PAE kernel!
[ 0.000000] NX (Execute Disable) protection: approximated ...

[Sasa Paporovic (melchiaros)]

I have found what it is.

In respect to:

http://lime-technology.com/wiki/index.php?title=The_Analysis_of_Drive_Issues

with the text passage(citing):

"Drive Interface Issues

These are problems with the cables and connections to the drive, both power and data, or the quality of the power supplied. If your errors match one of these, then almost certainly, your drive is completely fine. There have been many drives returned or thrown out, after numerous errors similar to the following issues, that were entirely the fault of the cables or power or connectors used, NOT the drive itself.
"

-> You may read on on the linked website for which kernel entries are indicating this.

----------------------------------------------------

I went in the next computer store and came back with a new SATA data and a new SATA energy cable(arround 20€ -> propably cheaper when you have time to look for).

I changed both against my old ones.

I have had runned the system since many days to be sure under haesy hardrive load(virtulisation with qemu-kvm with many guests in parallel).

-> No ata1 errors anymore with the same harddrive!

A deeper look at the cables shows me that the plastic connector of the SATA energy supply to the harddrive was deformed a bit.
It was nearly invisible but seems to be enough to interrupt the harddrive and causes this errors.

----------------------------------------------------

I would like to bring this issue up as a blueprint for Ubuntu/apport, because it was a time stealing one which is not easy to identify.

The system should inform the user/superuser when there are entries of this type in the kernel log that this affected with the possibility of defect cable or cable connectors to the harddrive.

I let this open for now, so that user affected by this can find it.

[David Barker (dbark)]

I am seeing similar symptoms.
With a Sony "Y Series" laptop (containing an AMD/ATI SBx00 SATA interface), whenever AC power is removed I see similar messages in dmesg.

There are no issues at all under AC power, only when operating on battery. The issue is present whether booting under battery power or booting under AC and then switching to battery.

I suspect it may be a pm-utils or kernel regression (based on similar bugs reported circa 2010).

The issue was not present with Ubuntu or XUbuntu 11.10 however is present in both Ubuntu and XUbuntu 12.04.
I have tried using 12.04 beta 2 and daily (as of 16/04/2012).
I have also tried using the most recent mainline kernel and still experienced the issue.

I will attach detailed logs once I have access to the machine again.

[hsiyao (hsiyao)]

Hi,

I came via https://bugs.launchpad.net/ubuntu/+source/linux/+bug/285892?comments=all which said, it would be a duplicate of this bug. I experience the bug after having changed from a legacy hdd to a ssd (ocz agility 3), running Ubuntu 11.10, all cables are fine, but everytime there are excessive write or read actions on this drive I experience a system freeze with the following appearing in the syslog. This is perfectly reproducable by just copying a big file.

kernel: [ 1958.016076] ata3: lost interrupt (Status 0x50)
kernel: [ 1958.016102] ata3.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
kernel: [ 1958.016111] ata3.01: failed command: READ DMA
kernel: [ 1958.016131] ata3.01: cmd c8/00:18:40:0b:50/00:00:00:00:00/fb tag 0 dma 12288 in
kernel: [ 1958.016144] res 40/00:00:00:4f:c2/00:00:00:00:00/10 Emask 0x4 (timeout)
kernel: [ 1958.016151] ata3.01: status: { DRDY }
kernel: [ 1958.016168] ata3: soft resetting link
kernel: [ 1958.220628] ata3.01: configured for UDMA/33
kernel: [ 1958.220636] ata3.01: device reported invalid CHS sector 0
kernel: [ 1958.220647] ata3: EH complete
kernel: [ 1989.088064] ata3: lost interrupt (Status 0x50)
kernel: [ 1989.088090] ata3.01: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
kernel: [ 1989.088099] ata3.01: failed command: WRITE DMA EXT
kernel: [ 1989.088122] ata3.01: cmd 35/00:20:b0:b2:20/00:01:07:00:00/f0 tag 0 dma 147456 out
kernel: [ 1989.088124] res 40/00:00:00:4f:c2/00:00:00:00:00/10 Emask 0x4 (timeout)
kernel: [ 1989.088129] ata3.01: status: { DRDY }
kernel: [ 1989.088142] ata3: soft resetting link
kernel: [ 1989.292604] ata3.01: configured for UDMA/33
kernel: [ 1989.292609] ata3.01: device reported invalid CHS sector 0
kernel: [ 1989.292615] ata3: EH complete

The support from ocz told me, that I might have a chance resolving this by either applying a firmware update or the "SATA3 to SATA2" fix. The latter seemed very interesting to me, could it be the main reason for the whole issue?
I haven't tried yet since the ssd sits in an all in one pc that can't boot from usb, has a broken dvd drive, and is not really built for being repaired often, so I'd like to make sure it really solves the issue. And since I'm reading a lot about people having no problems with another OS I keep on waiting for a bit more ;)
Anyone got a opinion about this sata3 to sata2 fix or whether it's OS or hardware connected?

[bastafidli (ubuntu-bastafidli)]

I have had this same or very similar issue on my fresh new Kubuntu 12.04 install with Gigabyte ga-990fxa-ud3 v1.1 motherboard with the following disks

[ 2.069621] ata4.00: ATA-8: ST31500341AS, SD1B, max UDMA/133
[ 2.078992] ata2.00: ATA-8: WDC WD5000AAKS-00YGA0, 12.01C02, max UDMA/133
[ 2.084532] ata5.00: ATA-9: SAMSUNG SSD 830 Series, CXM03B1Q, max UDMA/133
[ 2.128540] ata1.00: ATA-7: ST3500830AS, 3.AFE, max UDMA/133
[ 2.410522] ata3.00: ATA-8: WDC WD20EADS-00S2B0, 01.00A01, max UDMA/133

Here is example of errors I was receiving

Nov 17 21:17:03 desktop kernel: [287733.791309] ata3.00: exception Emask 0x0 SAct 0xff SErr 0x0 action 0x0
Nov 17 21:17:03 desktop kernel: [287733.791320] ata3.00: irq_stat 0x40000008
Nov 17 21:17:03 desktop kernel: [287733.791329] ata3.00: failed command: READ FPDMA QUEUED
Nov 17 21:17:03 desktop kernel: [287733.791345] ata3.00: cmd 60/08:38:3f:11:00/00:00:9f:00:00/40 tag 7 ncq 4096 in
Nov 17 21:17:03 desktop kernel: [287733.791349] res 41/40:00:3f:11:00/00:00:9f:00:00/40 Emask 0x409 (media error) <F>
Nov 17 21:17:03 desktop kernel: [287733.791357] ata3.00: status: { DRDY ERR }
Nov 17 21:17:03 desktop kernel: [287733.791362] ata3.00: error: { UNC }
Nov 17 21:17:03 desktop kernel: [287733.797639] ata3.00: configured for UDMA/133
Nov 17 21:17:03 desktop kernel: [287733.797670] ata3: EH complete
Nov 17 21:17:05 desktop kernel: [287736.447090] ata3.00: exception Emask 0x0 SAct 0xff SErr 0x0 action 0x0
Nov 17 21:17:05 desktop kernel: [287736.447100] ata3.00: irq_stat 0x40000008
Nov 17 21:17:05 desktop kernel: [287736.447109] ata3.00: failed command: READ FPDMA QUEUED
Nov 17 21:17:05 desktop kernel: [287736.447124] ata3.00: cmd 60/08:00:3f:11:00/00:00:9f:00:00/40 tag 0 ncq 4096 in
Nov 17 21:17:05 desktop kernel: [287736.447127] res 41/40:00:3f:11:00/00:00:9f:00:00/40 Emask 0x409 (media error) <F>
Nov 17 21:17:05 desktop kernel: [287736.447136] ata3.00: status: { DRDY ERR }
Nov 17 21:17:05 desktop kernel: [287736.447141] ata3.00: error: { UNC }
Nov 17 21:17:05 desktop kernel: [287736.453208] ata3.00: configured for UDMA/133
Nov 17 21:17:05 desktop kernel: [287736.453237] sd 2:0:0:0: [sdc] Unhandled sense code
Nov 17 21:17:05 desktop kernel: [287736.453243] sd 2:0:0:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 17 21:17:05 desktop kernel: [287736.453251] sd 2:0:0:0: [sdc] Sense Key : Medium Error [current] [descriptor]
Nov 17 21:17:05 desktop kernel: [287736.453261] Descriptor sense data with sense descriptors (in hex):
Nov 17 21:17:05 desktop kernel: [287736.453266] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
Nov 17 21:17:05 desktop kernel: [287736.453286] 9f 00 11 3f
Nov 17 21:17:05 desktop kernel: [287736.453295] sd 2:0:0:0: [sdc] Add. Sense: Unrecovered read error - auto reallocate failed
Nov 17 21:17:05 desktop kernel: [287736.453306] sd 2:0:0:0: [sdc] CDB: Read(10): 28 00 9f 00 11 3f 00 00 08 00
Nov 17 21:17:05 desktop kernel: [287736.453325] end_request: I/O error, dev sdc, sector 2667581759
Nov 17 21:17:05 desktop kernel: [287736.453369] ata3: EH complete
Nov 17 21:17:05 desktop kernel: [287736.453946] EXT4-fs error (device dm-3): __ext4_get...

[Doug Smythies (dsmythies)]

This comment is directed at David Barker and post #13. Why? Because the timing and versions that work / not work fit my issue.
David: I wonder if you might post your logs. And does the issue still occur for you?

In my case, I have isolated the issue to revision numbers 2760 and 2761 of udev (launchpad reference), which was late in the 12.04 pre-release cycle. I have been searching for someone else with the same issue.

References:
https://bugs.launchpad.net/ubuntu/+source/udev/+bug/986654
http://www.smythies.com/~doug/network/hd_race/index.html
http://ubuntuforums.org/showthread.php?t=1958838

[Doug Smythies (dsmythies)]

Sorry, For this line:

"This comment is directed at David Barker and post #13"

I meant to write:

"This comment is directed at David Barker and post #12"

[Udo Giacomozzi (udo-launchpad)]

Hi, I thought I contribute some additional info to this problem.

We have about ~50 nearly identical all-in-one PCs out there that are experiencing relatively often the exact problem described here.

The industrial-grade hardware is based on Intel Atom N270 (1.6 GHz) - see full specs here: http://www.cartft.com/catalog/il/1325

We use different kinds of non-mechanical hard disks:
- either a Kingston 8GB SSD SATA drive (most are using these, then we tried using other drives to avoid the problem we're talking about here)
- or a SATA-to-CompactFlash adapter with a 4GB SanDisk Extreme CompactFlash card (we use those cards - without adapter - in other configurations and never had any problems with them)
- or a Transcend TS8GHSD310 8GB SSD SATA drive

They are all running the same Linux image based on Debian 6, Kernel 2.6.38.6 and run 24/7. The system has been specifically modified to avoid any unnecessary Flash writes. All partitions are r/w and occasional writes are possible, but overall there are certainly far less than 1 MByte writes per day.

Basically we're only using half of the 8 GB drive space so these drives have a lot of room for wear leveling.

In the past months we had a number of failures that were mostly related to the frozen SATA link described in this bug. The drives would not even recover with a software reboot - the PCs need to be powered down and up again.

We're using high quality cables and also tried to use different cables. Note that these machines are firmly mounted, have no moving parts whatsoever and are located in lonely places - i.e. there is no mechanical reason why the SATA cable should suddenly have problems.

We only started recently to log in detail which hardware failed but I can defenitely say that it happens relatively often with the Kingston drives and it happened at least once with the CF adapter. I have no bad records for the Transcend drives so far but only five machines are using them so far. I wouldn't be surprised if even those fail.

My personal opinion is that there is some sort of incompatibility between the Linux drivers and the SATA controller or SATA drives that reveals itself only in certain "rare" situations. But other that that I'm clueless.

These machines most of the time do nothing but wait, so CPU time and disk accesses are very low.

BTW, the same Linux image is used in a number of other machines without SATA drives (they have CF card slots) and never had any comparable problems. Also I was never able to reproduce the problem in our office, which makes testing very hard.

I hope this helps to narrow down the problem. Let me know if I can be of any further help.

[penalvch (penalvch)]

Sasa Paporovic, this bug report is being closed due to your last comment https://bugs.launchpad.net/ubuntu/+source/linux/+bug/965213/comments/11 regarding this being due to hardware failure. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.