[Lenovo ThinkPad W510] Sudden Read-Only Filesystems

[Axel Pospischil]

++++++++++++++++++++++++++++++++++++++++++++++++++++
[UPDATE 05th March 2014]

The lenovo service was here and changed the mainboard.
No problems any more with a Crucial M500 480 GB and therefore
I think witch any other SSD on the W510.

So the problem turned out a real hardware problem.
The lenovo diagnostic cd however did not turn out any problem.
So to make a long story short: If you encounter any problems like in
this thread, don't heasitate to contact the lenovo service and insist for
a repair. In my case, I had to contact the service twice because I am using
hardware that is not part of the supported lenovo device list.

++++++++++++++++++++++++++++++++++++++++++++++++++++

After a while of running the system, the SSD goes into a readonly file system mode.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: linux-image-3.11.0-15-generic 3.11.0-15.23
ProcVersionSignature: Ubuntu 3.11.0-15.23-generic 3.11.10
Uname: Linux 3.11.0-15-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu2.2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: apos 3270 F.... pulseaudio
 /dev/snd/controlC0: apos 3270 F.... pulseaudio
Date: Mon Jan 6 01:08:09 2014
HibernationDevice: RESUME=UUID=00c4f34a-e81e-49fb-a41b-548542b09bdf
InstallationDate: Installed on 2014-01-02 (3 days ago)
InstallationMedia: Ubuntu 13.10 "Saucy Salamander" - Release amd64 (20131016.1)
MachineType: LENOVO 4391E46
MarkForUpload: True
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.11.0-15-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash libata.force=1:3.0G,2:1,5G libata.force=noncq
RelatedPackageVersions:
 linux-restricted-modules-3.11.0-15-generic N/A
 linux-backports-modules-3.11.0-15-generic N/A
 linux-firmware 1.116
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 10/03/2012
dmi.bios.vendor: LENOVO
dmi.bios.version: 6NET84WW (1.45 )
dmi.board.name: 4391E46
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr6NET84WW(1.45):bd10/03/2012:svnLENOVO:pn4391E46:pvrThinkPadW510:rvnLENOVO:rn4391E46:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 4391E46
dmi.product.version: ThinkPad W510
dmi.sys.vendor: LENOVO
---
ApportVersion: 2.13.1-0ubuntu1
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: apos 3204 F.... pulseaudio
 /dev/snd/controlC0: apos 3204 F.... pulseaudio
CurrentDesktop: Unity
DistroRelease: Ubuntu 14.04
HibernationDevice: RESUME=UUID=7290992b-11df-4d5c-a9bc-579dafe5eb61
InstallationDate: Installed on 2014-01-08 (4 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Alpha amd64 (20140105)
MachineType: LENOVO 4391E46
NonfreeKernelModules: nvidia
Package: linux (not installed)
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-1-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
ProcVersionSignature: Ubuntu 3.13.0-1.16-generic 3.13.0-rc7
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-1-generic N/A
 linux-backports-modules-3.13.0-1-generic N/A
 linux-firmware 1.121
Tags: trusty
Uname: Linux 3.13.0-1-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip libvirtd lpadmin mythtv plugdev sambashare sudo vboxusers
dmi.bios.date: 10/03/2012
dmi.bios.vendor: LENOVO
dmi.bios.version: 6NET84WW (1.45 )
dmi.board.name: 4391E46
dmi.board.vendor: LENOVO
dmi.board.version: Not Available
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvr6NET84WW(1.45):bd10/03/2012:svnLENOVO:pn4391E46:pvrThinkPadW510:rvnLENOVO:rn4391E46:rvrNotAvailable:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 4391E46
dmi.product.version: ThinkPad W510
dmi.sys.vendor: LENOVO
---
ApportVersion: 2.12.7-0ubuntu6
Architecture: amd64
CurrentDesktop: Unity
DistroRelease: Ubuntu 14.04
InstallationDate: Installed on 2014-01-08 (6 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Alpha amd64 (20140105)
MarkForUpload: True
NonfreeKernelModules: nvidia
Package: linux (not installed)
Tags: trusty
Uname: Linux 3.12.7-031207-generic x86_64
UnreportableReason: The running kernel is not an Ubuntu kernel
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip libvirtd lpadmin mythtv plugdev sambashare sudo vboxusers

[penalvch (penalvch)]

Axel Pospischil, thank you for providing a fresh report. Could you please confirm this issue exists with the latest development release of Ubuntu? ISO images are available from http://cdimage.ubuntu.com/daily-live/current/ . If the issue remains, please just make a comment to this.

If reproducible, could you also please test the latest upstream kernel available (not the daily folder) following https://wiki.ubuntu.com/KernelMainlineBuilds ? It will allow additional upstream developers to examine the issue. Once you've tested the upstream kernel, please comment on which kernel version specifically you tested. If this bug is fixed in the mainline kernel, please add the following tags:
kernel-fixed-upstream
kernel-fixed-upstream-VERSION-NUMBER

where VERSION-NUMBER is the version number of the kernel you tested. For example:
kernel-fixed-upstream-v3.13-rc7

This can be done by clicking on the yellow circle with a black pencil icon next to the word Tags located at the bottom of the bug description. As well, please remove the tag:
needs-upstream-testing

If the mainline kernel does not fix this bug, please add the following tags:
kernel-bug-exists-upstream
kernel-bug-exists-upstream-VERSION-NUMBER

As well, please remove the tag:
needs-upstream-testing

Once testing of the upstream kernel is complete, please mark this bug's Status as Confirmed. Please let us know your results. Thank you for your understanding.

[Axel Pospischil (apos)]

Hi,.

I am now testing with a fresh install ob Ubuntu 14.04 LTS.
Yesterday evening I installed a new system.

(Since I had problems getting nvidia and mainline kernel to run, I decided to got this way).

I wrote a little script, that sends me an mail with dmesg output, if the system goes into frozen state.

I will investigate, if something happens.

Greets.

[Axel Pospischil (apos)]

kernel 3.12.0-1-generic (AMD 64)

Twice today I recognized a freeze.

BUT:

* the system becomes stable again after a while
* ata1 will be reset
* system becomes unstable (eth0 network, wlan still works)

You can see some message, that come from a service, that every minute tries to touch and remove a file, and applies the "sync" command.

See the end of the syslog.txt.

Jan 9 19:52:23 madagaskar kernel: [ 8913.794328] ata1.00: status: { DRDY }
Jan 9 19:52:23 madagaskar kernel: [ 8913.794332] ata1.00: failed command: WRITE FPDMA QUEUED
Jan 9 19:52:23 madagaskar kernel: [ 8913.794341] ata1.00: cmd 61/40:f0:28:67:19/00:00:00:00:00/40 tag 30 ncq 32768 out
Jan 9 19:52:23 madagaskar kernel: [ 8913.794341] res 40/00:48:e8:50:19/00:00:00:00:00/40 Emask 0x44 (timeout)
Jan 9 19:52:23 madagaskar kernel: [ 8913.794346] ata1.00: status: { DRDY }
Jan 9 19:52:23 madagaskar kernel: [ 8913.794353] ata1: hard resetting link
Jan 9 19:52:23 madagaskar kernel: [ 8914.113935] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Jan 9 19:52:23 madagaskar kernel: [ 8914.117008] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
Jan 9 19:52:23 madagaskar kernel: [ 8914.117017] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jan 9 19:52:23 madagaskar kernel: [ 8914.117023] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
Jan 9 19:52:23 madagaskar kernel: [ 8914.117292] ata1.00: failed to get NCQ Send/Recv Log Emask 0x1
Jan 9 19:52:23 madagaskar kernel: [ 8914.118656] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
Jan 9 19:52:23 madagaskar kernel: [ 8914.118663] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
Jan 9 19:52:23 madagaskar kernel: [ 8914.118668] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
Jan 9 19:52:23 madagaskar kernel: [ 8914.118886] ata1.00: failed to get NCQ Send/Recv Log Emask 0x1
Jan 9 19:52:23 madagaskar kernel: [ 8914.118892] ata1.00: configured for UDMA/33
Jan 9 19:52:23 madagaskar kernel: [ 8914.134078] ata1: EH complete

[Axel Pospischil (apos)]

Given kernel was wrong:

linux-image-3.13.0-1-generic 3.13.0-1.16

Collection apport informations at the moment ...

[Axel Pospischil (apos)]

System freezes with above kernel ...

But when I try to investigate, I cannot use the machine for debugging because the system is in a read-only state.

I need some help to investigate the problem:

How can I trigger the kernel event "ata1.00: status: { DRDY }" to run a script (e.g. sende me an email with part of dmesg)?

[Axel Pospischil (apos)]

Today I made a backup of the drive to another 500GB disk since the problem occured during the last days after a certain time of beeing up with the pc.

Today:
1. I updated the Samsung EVO's firmware the the one from october 13th 2013.
2. I dist-upgraded to kernel 3.13.0-1.16 on Ubuntu 12.04

~$ dpkg -l linux-image*
ii linux-image-3.13.0-1-generic 3.13.0-1.16 amd64
ii linux-image-extra-3.13.0-1-generic 3.13.0-1.16 amd64

- I did NOT apply any additional kernel parameters:
~$ dmesg | grep BOOT
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.13.0-1-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash
[ 0.000000] ACPI: BOOT 00000000c3ffefd8 000028 (v01 LENOVO TP-6N 00001450 LTP 00000001)
[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.13.0-1-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash

After some minutes dmesg reports "hard resetting link" of the ata port:

~$ dmesg | egrep "ata1|scsci|BOOT"
[ 2062.684026] ata1: hard resetting link
[ 2063.003975] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 2063.005927] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x100)
[ 2063.005935] ata1.00: revalidation failed (errno=-5)
[ 2063.005944] ata1: limiting SATA link speed to 1.5 Gbps
[ 2068.003208] ata1: hard resetting link
[ 2068.323217] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[ 2068.326509] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[ 2068.326519] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
[ 2068.326525] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
[ 2068.326722] ata1.00: supports DRM functions and may not be fully accessible
[ 2068.326812] ata1.00: failed to get NCQ Send/Recv Log Emask 0x1
[ 2068.328137] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[ 2068.328145] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out
[ 2068.328151] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out
[ 2068.328355] ata1.00: supports DRM functions and may not be fully accessible
[ 2068.328465] ata1.00: failed to get NCQ Send/Recv Log Emask 0x1
[ 2068.328475] ata1.00: configured for UDMA/133
[ 2068.343087] ata1: EH complete

Machine is still working until now. But I expect a freeze even with the new kernel.
It the problem persists, I'll test the other drive an will report again.

[Axel Pospischil (apos)]

apport information

[penalvch (penalvch)]

Axel Pospischil, the Trusty kernel is not the upstream mainline kernel noted in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1266305/comments/2 .

[Axel Pospischil (apos)]

Thanks Christopher for your answer.

I could not test the upstream kernel due to massive problems with the nvidia card.
And I first wanted to the test the standard distribution - as advised.
Sorry for the wrong tagging this way ..

Nevertheless: tomorrow I try to return the drive, because it turns out, that several other people using Thinkpads have the same problem with exact this drive, the the Samsung 840 EVO.
Other versions like the Samsung EVO 840 PRO (!) series, the Crucial M4 or Plextor Model M5P are reported to work even in older machines.

- http://www.sevenforums.com/bsod-help-support/309288-bsod-after-few-minutes-5-55-minutes-samsung-ssd-840-evo.html
- [GERMAN] http://thinkpad-forum.de/threads/168841-Freezes-nach-Einbau-einer-neuen-SSD?p=1698795#post1698795

Due to the fact, that it is not likely, that lenovo will fix this issue in the BIOS or that the kernel can handle this, I am returning the drive and use a compatible model.

I am not a kernel guy, but I don't think, that a real hardware problem can be fixed?!
If I am wrong please correct me and I will give it a try again.

Meanwhile I replaced today the 500 GB Samsung 840 EVO with a 500GB Seagate Momentus XT Hybird, which layed arround.
It is working like a charm without any errors ...

I will investigate further.

Greets
Axel

[penalvch (penalvch)]

Axel Pospischil, one thing you may want to check into is if the firmware for the drive is the latest as per http://www.samsung.com/global/business/semiconductor/samsungssd/downloads.html . You may check what firmware you have currently via:
sudo hdparm -I /dev/sda

[Axel Pospischil (apos)]

Thanks Christopher for your advice.

Yes the firmware of the Samsung EVO 840 was updated to the latest one (last firmware Dec.' 13) with the Samsung Magician software. I checked the update twice after a reboot.

I have to add: the new replacement drive is an exact 1:1 copy of the SSD, and shows no problems at all.

So I really think - and with above links - this is a hardware related problem between the Samsung EVO 840 and the hardware of the Thinkpad W510's Intel Corporation 5 Series/3400 Series Chipset 6 port SATA AHCI Controller [8086:3b2f] (rev 06) or other components. Side effect is, that network components are also not working as they should, after the hard reset issued by the kernel..

Due to the fact, that this is (and will be) a much common drive for a lot of people, I think this is an important bug.

I will investigate further ...

[Axel Pospischil (apos)]

A few hours ago I got a new Samsung EVO 840 Pro. I made a clone of the actual system.

I booted up, stress tested the system (4GB file video conversion while running a virtual machine, ...).
It runs for 3 hours now. Not any single problem so far.

MY PERSONAL CONCLUSION:
DON'T USE THE SAMSUNG EVO 840 INSIDE THE THINKPAD W510.
SAMSUNG EVO 840 PRO (!) WORKS FINE.

Probably other older Thinkpad models are affected as well.

Please understand, that I could not afford a 350 € harddisc and a1500 € laptop just for testing purposes any more.
This is a productive system.

We should change the status to "won't fix".

[penalvch (penalvch)]

Axel Pospischil, thank you for your comment. Assuming the issue isn't due to poorly written firmware (despite being latest), hardware error, etc., then if you intend to continue providing troubleshooting information, we may continue pursuing a resolution. However, if you intend to send back the Samsung EVO 840, and use the Samsung EVO 840 Pro, this would be marked Invalid, not Won't Fix, as noted in https://wiki.ubuntu.com/Bugs/Status . Won't Fix would be more appropriate, for example, if the issue was in fact due to bad firmware, and to write around that in the kernel was deemed outside of scope for the kernel.

[Axel Pospischil (apos)]

Hi Christopher, thank you for the explanations.

I returned the EVO 840 and got the EVO 840 Pro instead from my local reseller.
So I cannot test any more into the subject.

I learned a lot about filing bugs in ubuntu, so this was very interesting.
Thank you for your advise so far.

I found another link to the problem
* http://www.howtoeverything.net/linux/hardware/ubuntu-freeze-issue-after-ssd-upgrade

Due to the fact, that the drive was returned, please mark the bug invalid.

[Axel Pospischil (apos)]

Due to the fact, that the drive was returned but was OK (tested in another system), I marked the bug "Opinion" so that it is closed.

[penalvch (penalvch)]

Axel Pospischil, this bug report is being closed due to your last comment https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1266305/comments/15 regarding you no longer have the hardware. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status. Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.

[Axel Pospischil (apos)]

Hi, unfortunately my enthusiasm was a little bit too big.

Yesterday I got a freeze again, but this time a little time after wakeup from standby.

- I tried then the kernel 3.12.7-031207-generic (which is the last trusty kernel for the 3.12 line).
- I uninstalled tlp ro minimize side effects.
- I booted with libata.force=1:3.0G,2:1,5G libata.force=noncq libata.noacpi=1

I restarted the system at Tue, 14 Jan 2014 17:45:34 +0100.
Today at Wed, 15 Jan 2014 09:06:18 +010 I got a freeze of the ata2 port (which is the dvd).
I tried to eject the drive and did an undock with the attached script. This provoked again hard setting the ata2 link.
As side effekt the ethernet interface stopped working. Also the ata2 (DVD) is not accessible any more.

So far.

I am attaching the dmesg output from yesterday evening, which was ok and the dmesg from today with the errors.

[Axel Pospischil (apos)]

Please give me a few days of doing a clear testing plan about this.

[Axel Pospischil (apos)]

apport information

[Axel Pospischil (apos)]

Unfortunately the lttng-modules for the kernel 3.13 are not installable.
Any advise for another tool to trap kernel messages?

https://bugs.launchpad.net/ubuntu/+source/lttng-modules/+bug/1267652

[penalvch (penalvch)]

Axel Pospischil, given that you have changed SSD models, could you please file a new report via a terminal:
sudo apt-get update && sudo apt-get -y dist-upgrade && ubuntu-bug linux

For more on this, please read the official Ubuntu documentation:
Ubuntu Bug Control and Ubuntu Bug Squad: https://wiki.ubuntu.com/Bugs/BestPractices#X.2BAC8-Reporting.Focus_on_One_Issue
Ubuntu Kernel Team: https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Filing_Kernel_Bug_reports
Ubuntu Community: https://help.ubuntu.com/community/ReportingBugs#Bug_reporting_etiquette

When opening up the new report, please feel free to subscribe me to it.

Thank you for your understanding.

Helpful bug reporting tips:
https://wiki.ubuntu.com/ReportingBugs

[Axel Pospischil (apos)]

#######################################################

>> I filed a new bug due to new "new hardware" and new environment: bug #1269777

#######################################################

Please be aware that this is a duplicate bug of the above one and bug #1265309

[Axel Pospischil (apos)]

++++++++++++++++++++++++++++++++++++++++++++++++++++
[UPDATE 05th March 2014]

The lenovo service was here and changed the mainboard.
No problems any more with a Crucial M500 480 GB and therefore
I think witch any other SSD on the W510.

So the problem turned out a real hardware problem.
The lenovo diagnostic cd however did not turn out any problem.
So to make a long story short: If you encounter any problems like in
this thread, don't heasitate to contact the lenovo service and insist for
a repair. In my case, I had to contact the service twice because I am using
hardware that is not part of the supported lenovo device list.

++++++++++++++++++++++++++++++++++++++++++++++++++++