No TRIM via USB

[Néher Márton]

Hi,

As USB attached SSDs are becoming quite big and affordable, along comes the idea of installing systems on these.
But with current USB drivers, some functionality seems to be missing from USB subsystem.

The system does not recognize it is an SSD, but it is corrected manually:
# cat /etc/udev/rules.d/10-forcessd.rules
SUBSYSTEM=="block", ATTRS{vendor}=="SanDisk", ATTRS{model}=="Extreme", KERNEL=="sd?", ATTR{queue/rotational}="0"

# cat /sys/block/sdb/queue/rotational
0

Checked hdparm, it is saying I have TRIM on the device:
# hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
 Model Number: SanDisk pSSD
(...)
Commands/features:
 Enabled Supported:
(...)
    * Data Set Management TRIM supported (limit 8 blocks)
    * Deterministic read ZEROs after TRIM
(...)

Here is appropriate dmesg:
[ 3.815604] usb 2-8: Manufacturer: SunplusIT INC.
[ 4.122002] usb 3-3: new SuperSpeed USB device number 2 using xhci_hcd
[ 4.138402] usb 3-3: New USB device found, idVendor=0781, idProduct=5580
[ 4.138404] usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 4.138405] usb 3-3: Product: Extreme
[ 4.138406] usb 3-3: Manufacturer: SanDisk
[ 4.138407] usb 3-3: SerialNumber: AA011109131654094942
[ 4.141393] usb-storage 3-3:1.0: USB Mass Storage device detected
[ 4.141431] scsi0 : usb-storage 3-3:1.0
[ 4.141703] usbcore: registered new interface driver usb-storage

And:
[ 5.331311] sd 0:0:0:0: [sdb] 122544516 512-byte logical blocks: (62.7 GB/58.4 GiB)
[ 5.331608] sd 0:0:0:0: [sdb] Write Protect is off
[ 5.331611] sd 0:0:0:0: [sdb] Mode Sense: 33 00 00 08
[ 5.331855] sd 0:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 5.339849] sdb: sdb1 sdb2 sdb3 < sdb5 >
[ 5.340783] sd 0:0:0:0: [sdb] Attached SCSI disk

From here I'll be using /boot formatted to ext4 to eliminate luks,lvm and btrfs from the equasion:

# mount |grep sdb2
/dev/sdb2 on /boot type ext4 (rw,noexec,discard)

# fstrim -v /boot/
fstrim: /boot/: FITRIM ioctl failed: Operation not supported

# strace fstrim -v /boot/
(...)
open("/boot/", O_RDONLY) = 3
ioctl(3, FITRIM, 0x7fffdded85c0) = -1 EOPNOTSUPP (Operation not supported)
(...)

During debugging, I've tried compiling a kernel with UAS module to check, it booted, system was g changed with the trim - as it wouldn't been used at all.

Any ideas why TRIM is not working on USB?
Any timeframes for possible fix?
Any workarounds maybe?
---
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: dome 2460 F.... pulseaudio
 /dev/snd/pcmC1D0p: dome 2460 F...m pulseaudio
 /dev/snd/controlC0: dome 2460 F.... pulseaudio
DistroRelease: Ubuntu 14.04
HibernationDevice: RESUME=UUID=bafebd1d-6988-438f-afd4-b41bb8608616
InstallationDate: Installed on 2014-06-12 (56 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
MachineType: LENOVO 20AR001AUK
Package: linux (not installed)
ProcEnviron:
 LANGUAGE=en_US
 TERM=xterm
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-24-generic root=/dev/mapper/system-root ro rootflags=subvol=@ cryptopts=target=crypter,source=/dev/disk/by-uuid/dc149cdf-62f9-4e29-a048-dd0403f51d56,lvm=system,discard quiet splash crashkernel=384M-:128M vt.handoff=7
ProcVersionSignature: Ubuntu 3.13.0-24.47-generic 3.13.9
PulseList:
 Error: command ['pacmd', 'list'] failed with exit code 1: Home directory not accessible: Permission denied
 No PulseAudio daemon running, or not running as session daemon.
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-24-generic N/A
 linux-backports-modules-3.13.0-24-generic N/A
 linux-firmware 1.127.5
Tags: trusty
Uname: Linux 3.13.0-24-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

_MarkForUpload: True
dmi.bios.date: 03/28/2014
dmi.bios.vendor: LENOVO
dmi.bios.version: GJET75WW (2.25 )
dmi.board.asset.tag: Not Available
dmi.board.name: 20AR001AUK
dmi.board.vendor: LENOVO
dmi.board.version: 0B98401 PRO
dmi.chassis.asset.tag: No Asset Information
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Not Available
dmi.modalias: dmi:bvnLENOVO:bvrGJET75WW(2.25):bd03/28/2014:svnLENOVO:pn20AR001AUK:pvrThinkPadT440s:rvnLENOVO:rn20AR001AUK:rvr0B98401PRO:cvnLENOVO:ct10:cvrNotAvailable:
dmi.product.name: 20AR001AUK
dmi.product.version: ThinkPad T440s
dmi.sys.vendor: LENOVO

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1336541/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1336541

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Néher Márton (neher-marton)]

apport information

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.16 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

If you are unable to test the mainline kernel, for example it will not boot, please add the tag: 'kernel-unable-to-test-upstream'.
Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.16-utopic/

[ted (ted-linux-kernel-bugs)]

On a fully updated F20 machine hdparm shows that TRIM is supported on this Corsair
USB stick but both "fstrim" and "mount -o discard" commands fail
I have replicated the problem using an Addonics eSATAp to USB3 adaptor and a modern SSD.
ioctl/FITRIM problem

Details
ja@paxos ~ 1$ uname -a
Linux paxos 3.15.10-200.fc20.x86_64 #1 SMP Thu Aug 14 15:39:24 UTC 2014
x86_64 x86_64 x86_64 GNU/Linux

[root@paxos:~]$ hdparm -I /dev/sdb
ATA device, with non-removable media
        Model Number: Voyager GTX
        Serial Number: FF1807470C0800110923
        Firmware Revision: S9FM01.7
        Transport: Serial, ATA8-AST, SATA 1.0a, SATA II
Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
...
           * Data Set Management TRIM supported (limit 8 blocks)
...
------------------------------------------------------------------------
root@paxos:~]$ mount |grep sdb
/dev/sdb1 on /media/gtx type ext4 (rw,nosuid,nodev,noexec,noatime,seclabel,discard,data=ordered)
//-----------------------------------------------------------------
dmesg -ew
[Aug23 08:45] EXT4-fs (sdb1): mounting with "discard" option, but the device does not support discard
[ +0.000002] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: discard
[ +0.000004] SELinux: initialized (dev sdb1, type ext4), uses xattr
//-----------------------------------------------------------------
[root@paxos:~]$ cat /sys/block/sdb/queue/rotational
0
//-----------------------------------------------------------------
[root@paxos:~]$ fstrim -v /media/gtx
fstrim: /media/gtx: discard operation not supported.
//-----------------------------------------------------------------
[root@paxos:~]$ strace fstrim /media/gtx
...
stat("/media/gtx", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
open("/media/gtx", O_RDONLY) = 3
ioctl(3, FITRIM, 0x7fffee2fb700) = -1 EOPNOTSUPP (Operation not supported)
...

The latest references to the problem that I can find are given here
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1336541
http://askubuntu.com/questions/262154/trim-and-ssd-with-usb-3-0-enclosure-does-not-work-uasp-not-supported
but I can find no reference to a likely, timely solution

[Ｖ字龍(Vdragon) (vdragon)]

Hi, provide my study here :-)

As far as I can see to support TRIM via USB bridge the USB bridge chip MUST support SCSI / ATA Translation(SAT)'s ATA PASS THROUGH command in order to pass the TRIM command to the disk.

Reading drive's S.M.A.R.T. data is similar to TRIM which they both require working ATA PASS THROUGH command, you may checkout following webpages for more info:

* [USB – smartmontools](https://www.smartmontools.org/wiki/USB)
* [Supported_USB-Devices – smartmontools](https://www.smartmontools.org/wiki/Supported_USB-Devices)

and YES, external HDD closure's controller may not provide this functionality, make sure to checkout the chip's datasheet before purchasing any new one(while some may support this functionality via controller firmware upgrade most of them don't)

However even you can read S.M.A.R.T. data using smartmontools utility, TRIM is still (currently) not possible for unknown reason, might be the usb-storage kernel module issue I guess.

[Dan Parslow (djp-ubuntu)]

TRIM is accessible via "hdparm --trim-sector-ranges" on a USB device supporting ATA PASS THROUGH:

Device is an SSD in a USB 3.0 adapter identified in lshw as follows:

Bus 001 Device 005: ID 174c:55aa ASMedia Technology Inc. ASMedia 2105 SATA bridge

Adapter is hosting a 180GB Intel 530 M.2 NGFF SSD

fstrim fails as described with "FITRIM ioctl failed: Operation not supported" even though the hdparm invocation works fine. I've tested full ext4 filesystem trims using wiper.sh, the utility by Mark Lord that was bundled for a while with hdparm until fstrim came out. This utility works by wrapping the hdparm invocation.

So it definitely seems to be the case that the USB driver stack is declining to implement TRIM even in cases where it is technically able to do so. Given the extremely high performance of USB3+, it seems very likely that USB-connected SSDs will become increasingly usual.

[Dan Parslow (djp-ubuntu)]

Correction to the above, that's the lsusb identification of the adapter. lshw identifies it as

     *-scsi
          physical id: 1
          bus info: usb@1:2.4
          logical name: scsi0
        *-disk
             description: SCSI Disk
             product: 2115
             vendor: ASMT
             physical id: 0.0.0
             bus info: scsi@0:0.0.0
             logical name: /dev/sda
             version: 0
             serial: 00000000000000000000
             size: 167GiB (180GB)
             capabilities: gpt-1.00 partitioned partitioned:gpt
             configuration: ansiversion=6 guid=0c8c6c54-9b77-45eb-ac02-39017ab4e289 sectorsize=512

[Dan Parslow (djp-ubuntu)]

It seems that if the device is handled by the usb-storage driver, TRIM is as of this writing unsupported and likely to remain so.

usb/storage/scsiglue.c sets skip_vpd_pages to true,
source/drivers/scsi/sd.c will not query for the necessary block limits to support discard if skip_vpd_pages is true.

The bug poster should note per comment #1 in their dmesg excerpt that their device is using the usb-storage driver

However, my Intel 530 SSD in the ASMT 2115 enclosure is acquired by the uas driver rather than usb-storage. TRIM is not supported there, possibly because of a failing in the bridge controller's translation of the SCSI unmap method to ATA TRIM.

hdparm (and presumably blkdiscard, which I haven't tried) succeed because they use ATA commands directly without going through a SCSI translation layer either in usb-storage (disabled) or in the enclosure firmware (possibly broken?).

[Tom Yan (tom-ty89)]

The ASMedia bridges (as of the USB 3.1 Gen 2 ASM1351) simply does NOT have UNMAP->TRIM implemented at all. It does support the "ATA PASSTHROUGH" SCSI command, and it does not block TRIM on that, so hdparm works.

The only one I've seen that supports UNMAP->TRIM is JMicron JMS567.

However, the implementation seems to be only partially done. Apparently it does not support / has issue with queued UNMAP commands though (assuming the uas driver in the kernel is doing everything right). Therefore, blkdiscard/fstrim will work fine with the usb-storage driver (`u` quirk), but will have issue with the uas driver on it.

Also, for some reason (maybe the aforementioned one), it has the LBPME bit set to 0. So you need to change the provisioning_mode sysfs file with `echo -n unmap` too (though that can be necessary anyway when the usb-storage driver (u quirk) is in use, since it prefers Read Capacity (10), which does not check the LBPME bit at all).

Unlike the ASMedia chips, it seems to block TRIM in its ATA PASSTHROUGH btw.

[Elias Abacioglu (raboo)]

similar or same bug reported at upstream
https://bugzilla.kernel.org/show_bug.cgi?id=83181

[ralph (ralph-linux-kernel-bugs)]

The referenced launchpad bug suggests hdparm(8)'s --trim-sector-ranges works because it uses ATA commands directly. fstrim(8) fails, even with ATA-passthrough SCSI command that lets smartctl(8) work, due to lack of usb-storage driver support. Interesting comments start at #18 (no anchor available!) at https://bugs.launchpad.net/ubuntu/+source/linux-lts-xenial/+bug/1336541

Could someone knowledgeable in the this area take a look and the comments and give a summary of the problem. Perhaps it can then be moved off the `io_other' component.