date modified is wrong for files on an exfat formatted drive

[Tim Crowther]

When using exfat formatted drives (e.g. my camera card) with focal fossa any access causes the date modified to be set, even when it would not normally be set, and it is set a month into the future.

Installing exfat-fuse and exfat-utils results in the correct behaviour.

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubuntu-release-upgrader-core 1:20.04.18
ProcVersionSignature: Ubuntu 5.4.0-21.25-generic 5.4.27
Uname: Linux 5.4.0-21-generic x86_64
ApportVersion: 2.20.11-0ubuntu26
Architecture: amd64
CasperMD5CheckResult: skip
CrashDB: ubuntu
CurrentDesktop: ubuntu:GNOME
Date: Mon Apr 13 17:27:30 2020
InstallationDate: Installed on 2020-04-12 (1 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Beta amd64 (20200409)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_GB:en
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_GB.UTF-8
 SHELL=/bin/bash
SourcePackage: ubuntu-release-upgrader
Symptom: dist-upgrade
UpgradeStatus: No upgrade log present (probably fresh install)

[gueba (gueba)]

I can confirm this bug on a life-system xubuntu 20.04.

In addition to the increase of one month in date modified, sometimes the date accessed shows "1979-12-31 00:00:00"

[Player (anonymousplayer)]

I can confirm this bug on a fresh install of ubuntu 20.04.

My steps to reproduce the bug:
Connected an external drive formatted as exFAT.
Pick a file and note down its modified date.
Copy the file to another disk.
Unmount and remount the exFAT drive.
Check the modified date of the file, it's incremented by 30 days.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Sebastian Gürtler (seguar)]

For it hasn't been corrected in the regular updates, I also confirm that it's still present on ubuntu 20.04, while using an external 4 TB hard drive. I'd like to emphasize the importance of this bug, for it may lead to complications and erroneous deletions if I used the drive for synchronizing folders between different systems.
(older files have a date in the future so newer versions are deleted and "updated" !)

[Sebastian Gürtler (seguar)]

... I have to add: exfat-fuse and exfat-utils are in the "newest version" 1.3.0-1 and the bug still exists!

[Sebastian Gürtler (seguar)]

The error occurs even without writing to the hard disk. I didn't find the exact procedure to reproduce the error, it seems to happen during mounting or unmounting, maybe only from nautilus, but even in this case I don't have the changes of the date any time. The Date doesn't change by 30 days but 1 month and there are only some files affected, but repeatedly. Even that not consistently.
On my hard drive there is only one directory and the subdirectories affected:
unmount + save remove from nautilus/mount automatically via entry in fstab, listings from terminal, from a single session on Nov 18 2020(!), here only the files "Fuga.blend1, Materials.blend1, Mixereinstellungen.vmsms" affeced - I have no idea, what the common feature may be.

sebastian@osamu:/media/4tbp/Eigene Videoprojekte/Fuga BWV 863$ ls -l
insgesamt 106752
-rwxr-xr-x 1 root root 1016 Dez 9 2020 Ablauf.txt
-rwxr-xr-x 1 root root 2990398 Dez 5 2020 'BWV 863 (I, 18 gis-moll) Selbst.mp3'
-rwxr-xr-x 1 root root 1325052 Dez 12 2020 Fuga.blend
-rwxr-xr-x 1 root root 1325148 Jul 9 2021 Fuga.blend1
-rwxr-xr-x 1 root root 73990 Dez 9 2020 'Fuga BWV 863.aup'
drwxr-xr-x 4 root root 131072 Dez 5 2020 'Fuga BWV 863_data'
-rwxr-xr-x 1 root root 231661 Dez 18 2020 Fuga_take2.aup
drwxr-xr-x 4 root root 131072 Dez 18 2020 Fuga_take2_data
-rwxr-xr-x 1 root root 4376783 Dez 9 2020 'Leuchtender Boden Cycles.png'
-rwxr-xr-x 1 root root 1185996 Dez 9 2020 Materials.blend
-rwxr-xr-x 1 root root 1185996 Jul 9 2021 Materials.blend1
-rwxr-xr-x 1 root root 2966841 Dez 9 2020 MIDI.mp3
-rwxr-xr-x 1 root root 151 Jul 9 2021 Mixereinstellungen.vmsms
-rwxr-xr-x 1 root root 249378 Dez 5 2020 Synchro.kdenlive
drwxr-xr-x 2 root root 131072 Dez 18 2020 take2_video
-rwxr-xr-x 1 root root 84884 Dez 6 2020 Test1.kdenlive
-rwxr-xr-x 1 root root 91579391 Jan 6 2021 Test1.webm
drwxr-xr-x 2 root root 131072 Dez 5 2020 'Video Test Alphaclips'
drwxr-xr-x 2 root root 131072 Dez 5 2020 'Video Test-Rohdaten'

no file used, just removed and mounted again:

-rwxr-xr-x 1 root root 1016 Dez 9 2020 Ablauf.txt
-rwxr-xr-x 1 root root 2990398 Dez 5 2020 'BWV 863 (I, 18 gis-moll) Selbst.mp3'
-rwxr-xr-x 1 root root 1325052 Dez 12 2020 Fuga.blend
-rwxr-xr-x 1 root root 1325148 Aug 9 2021 Fuga.blend1
-rwxr-xr-x 1 root root 73990 Dez 9 2020 'Fuga BWV 863.aup'
drwxr-xr-x 4 root root 131072 Dez 5 2020 'Fuga BWV 863_data'
-rwxr-xr-x 1 root root 231661 Jan 18 2021 Fuga_take2.aup
drwxr-xr-x 4 root root 131072 Dez 18 2020 Fuga_take2_data
-rwxr-xr-x 1 root root 4376783 Dez 9 2020 'Leuchtender Boden Cycles.png'
-rwxr-xr-x 1 root root 1185996 Dez 9 2020 Materials.blend
-rwxr-xr-x 1 root root 1185996 Aug 9 2021 Materials.blend1
-rwxr-xr-x 1 root root 2966841 Dez 9 2020 MIDI.mp3
-rwxr-xr-x 1 root root 151 Aug 9 2021 Mixereinstellungen.vmsms
-rwxr-xr-x 1 root root 249378 Dez 5 2020 Synchro.kdenlive
drwxr-xr-x 2 root root 131072 Dez 18 2020 take2_video
-rwxr-xr-x 1 root root 84884 Dez 6 2020 Test1.kdenlive
-rwxr-xr-x 1 root root 91579391 Jan 6 2021 Test1.webm
drwxr-xr-x 2 root root 131072 Dez 5...

[Sebastian Gürtler (seguar)]

I made some experiments to define the behaviour, the problem is, that 3 computers with installed ubuntu 20.04.1 LTS behave(d) differently (installing fuse-exfat 1.3.0 stopped the described behaviour in two of the computers, seems, that nautilus is able to bypass fuse-exfat in some conditions).

I made the tests in an alternative terminal session without active GUI and manually mounting without the fstab-entries.

I repeatedly mounted the device, created files, read them with cat, modified them with touch. I did not apply status changes. Then I unmounted, mounted read-only, checked changes and changeability, seeing, that mount with -r option really protected the file system from being changed.

touch xy creates a file, the time applied to the system was correct in atime, ctime, mtime but highly fractioned below milliseconds, which is not really in accordance to the exfat system where the time resolution is 2 seconds and in an additional field 10 ms.

if unmounted and mounted read only again the newly created files were advanced 1 month and the seconds were rounded to the next lower even number (conforming to the time resolution of 2 seconds in the mtime and ctime, the atime was set to Dec, 31, 1979, 1:00 (+001).
cat and touch (which results in the expected error message) made no changes to the files, also if unmounted and remounted rw.

The same results came with remounting read/writeable.

After using cat the mtime and ctime remains unchanged (1 month advanced), atime is set to the correct timestamp.
After using touch all three timestamps are updated to the actual time and date (without fraction, just even seconds!)

if unmounted and mounted again files that were only read, showed a further advance of the date by one month in mtime and ctime and again Dec, 31,1979 in atime.
The touched files advanced by 1 month in mtime and ctime and also showed Dec, 31, 1979 in atime.
Files that were only listed showed no change (remained the one month advanced as happend by the unmounting after their creation).

It seems that the procedure that writes some buffers back to the hard drive during unmounting has bugs in converting to the exfat file entries. The month in the c time-structure is 0-11, in exfat 1-12, this may be an issue. The field for last access seems to be left empty. And the fractions below the 2 seconds seem to be omitted in the procedures that work with the filesystem even before flushing the buffers, but they seem to be used in the procedure, that creates a file newly.

After installing fuse-exfat this behaviour stopped in one computer, in the other not (where I will try uninstall and reinstall...). A bit less important, but now with cat even the atime stays unchanged completely (other than with ext4); ctime is changed together with mtime.

[Oleg (lega4)]

Same issue here with the files copied from my Sony a6000 camera. Any suggestions how to fix it?

>After installing fuse-exfat
It gives me "E: Unable to locate package fuse-exfat"

[Sebastian Gürtler (seguar)]

The fix is quite offtopic, but for this special case it should be a solution: the date of the pictures should be also preserved in the EXIF-Data. You can read the exif-data of the files and set the timestamp from this data.
I used another tool under windows for this so far, but exiftool should do the job:

Excerpt from the help-message:

exiftool "-FileModifyDate<DateTimeOriginal" dir
         Use the original date from the meta information to set the same
         file's filesystem modification date for all images in a directory.
         (Note that "-TagsFromFile @" is assumed if no other -TagsFromFile
         is specified when redirecting information as in this example.)

Hope that works,

Sebastian

[Sebastian Gürtler (seguar)]

I think I found the bug in the linux 5.4, but I have no idea whether or how to submit a patch...
The handling of exfat changes completely in the following (but not LTS) versions.

In the file drivers/staging/exfat/exfat_super.c, in lines 60-

static void exfat_time_fat2unix(struct timespec64 *ts, struct date_time_t *tp)
{
 ts->tv_sec = mktime64(tp->Year + 1980, tp->Month + 1, tp->Day, // shouldn't it be tp->Month, tp->Day .... ?????!!!
         tp->Hour, tp->Minute, tp->Second);

 ts->tv_nsec = tp->MilliSecond * NSEC_PER_MSEC;
}

static void exfat_time_unix2fat(struct timespec64 *ts, struct date_time_t *tp)
{
 time64_t second = ts->tv_sec;
 struct tm tm;

 time64_to_tm(second, 0, &tm);
[...]
 tp->MilliSecond = ts->tv_nsec / NSEC_PER_MSEC;
 tp->Second = tm.tm_sec;
 tp->Minute = tm.tm_min;
 tp->Hour = tm.tm_hour;
 tp->Day = tm.tm_mday;
 tp->Month = tm.tm_mon + 1;
 tp->Year = tm.tm_year + 1900 - 1980;
}

The meaning of the field Month in struct date_time_t has no documentation in the header exfat.h, but the function unix2fat produces a month from 1-12 in date_time_t by adding 1 to the result of time64_to_tm.

[Pierre Wilch (bugfollower)]

I had the same problem.
- Since this bug only appeared by me by consulting the proprieties of the file with Nautilus, and did not occur with linux mint 20 and Nemo, I reported it as bug 1890958 in Nautilus.
- Since updates beginning of the year 2021 I can no longer reproduce it. It seems to be solved.
- Ubuntu team, can you confirm it here (or not?), so people who spent time to report this to help you could sleep on their two ears and use ubuntu 20.04 again, with no fear of such dangerous bugs?

[Seth Arnold (seth-arnold)]

I added the linux source package to this bug because I've heard this commit addresses the issue:

https://github.com/gregkh/linux/commit/099340d3e758cca06a82bf5dcff8b9a8acbdcb0a

Thanks

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1872504

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Tim Crowther (applejack)]

As far as I can tell at the moment this bug appears to have been fixed.