df and fsck on empty exfat sdcard shows space used

[Joe Harrington]

Description: Ubuntu 16.04.4 LTS
Release: 16.04

exfat-utils:
  Installed: 1.2.3-1
  Candidate: 1.2.3-1
  Version table:
 *** 1.2.3-1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status

Expected: df of exfat sdcard filesystem shows used amount commensurate with space associated with files and directories, accounting for finite block/cluster sizes.

Actual: df shows much more space used than possible to account for in files and directories.

I have an xdhc card (Samsung 64GB Evo UHC-1) with an exFAT filesystem. There are very few files or directories. du says 1.1MB is used, but df says nearly 1GB is used. The report is corroborated elsewhere (see below).

df -h says:

    /dev/mmcblk0p1 60G 925M 59G 2% /media/jh/disk

fsck says:

# fsck /dev/mmcblk0p1
fsck from util-linux 2.27.1
exfatfsck 1.2.3
Checking file system on /dev/mmcblk0p1.
File system version 1.0
Sector size 512 bytes
Cluster size 128 KB
Volume size 60 GB
Used space 924 MB
Available space 59 GB
Totally 4 directories and 6 files.
File system checking finished. No errors found.

Use of card:

It goes in my GoPro Hero 4 Black camera. I pull the card, pop it into my Ubuntu box, and mount it to transfer videos and photos. Then I delete them (I may cp or rsync and then rm, or more commonly mv). After the transfer, there are 6 files in 4 directories (plus the root). The files total 327 bytes. The directories are 128k each. Even using full 128k clusters for the files doesn't approach 925MB. No dotfiles are on the card. What's taking up nearly a gig?

I believe the 925M figure has been rising over time, even though the content I can see after removing photos and videos has not changed. Mostly, I record a small number of large (4GB) videos (GoPro hardware enforces this), but recently I did have several hundred photos in a time-lapse video recorded as single files.

Here's what's on the card after deleting the photos and videos:

    /media/jh/disk $ find . | xargs ls -ld
    drwx------ 1 jh jh 131072 Dec 31 1969 .
    drwx------ 1 jh jh 131072 Mar 6 10:11 ./DCIM
    drwx------ 1 jh jh 131072 Mar 6 10:19 ./DCIM/100GOPRO
    -rwx------ 1 jh jh 8 Mar 1 14:47 ./DCIM/leinfo.sav
    -rwx------ 1 jh jh 139 Jan 1 2015 ./Get_started_with_GoPro.url
    drwx------ 1 jh jh 131072 Jul 19 2017 ./LOST.DIR
    -rwx------ 1 jh jh 0 Jan 7 2015 ./mdb.db
    -rwx------ 1 jh jh 0 Jan 7 2015 ./mdb.log
    drwx------ 1 jh jh 131072 Dec 7 11:50 ./MISC
    -rwx------ 1 jh jh 19 Dec 27 2014 ./MISC/card
    -rwx------ 1 jh jh 161 Feb 5 2017 ./MISC/version.txt

Here's the mount entry:

    /dev/mmcblk0p1 on /media/jh/disk type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)

A forum question indicates a similar problem:

https://askubuntu.com/questions/1010088/how-can-an-exfat-filesystem-be-53-used-with-zero-files-and-directories/1012467#1012467

That person reformatted the card (rebuilt the filesytem) and the used space dropped to near zero. A commenter suggested doing fsck, but this changed nothing. The commenter suggested that the exfat volume bitmap might be involved. However, I've filled my card many times and it's not marking the whole card as full. Perhaps just one block per deleted file is not reset or some other weirdness where a subset of blocks is not reset on delete. This would explain the slowly growing "used" space. Note that in the forum report, the files were deleted on a Mac, whereas for me everything has been in Linux.

Thanks,

--jh--

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: exfat-utils 1.2.3-1
ProcVersionSignature: Ubuntu 4.13.0-36.40~16.04.1-generic 4.13.13
Uname: Linux 4.13.0-36-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
CurrentDesktop: XFCE
Date: Wed Mar 7 12:10:53 2018
InstallationDate: Installed on 2017-06-17 (262 days ago)
InstallationMedia: Ubuntu 16.04.2 LTS "Xenial Xerus" - Release amd64 (20170215.2)
SourcePackage: exfat-utils
UpgradeStatus: No upgrade log present (probably fresh install)