ntfs-3g Unsupported "Archive" reparse point

[David]

######## Description ########

I believe I've encountered a new type of reparse point that's unsupported by the current ntfs-3g library. I've tested this with a basic Windows 10 installation, and the reparse point seems to be on a Microsoft Edge executable file.
The current plugins (available at http://jp-andre.pagesperso-orange.fr/advanced-ntfs-3g.html#download) currently support reparse points for (all of which I have installed on my system):

1) System Compression
2) Deduplicated files
3) OneDrive

I attempted to invoke the "file" operation on a given Windows 10 file, entitled "MicrosoftEdge.exe".

######## Expected Outcome ########

I expected to be able to read the file either in the form of a "symlink" or a normal file.

######## Actual Outcome ########

I encountered the dreaded "unsupported reparse point":

david@david-VirtualBox:~$ file /mnt/windows_10/Users/user/AppData/Local/Microsoft/WindowsApps/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/MicrosoftEdge.exe
Users/user/AppData/Local/Microsoft/WindowsApps/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/MicrosoftEdge.exe: broken symbolic link to unsupported reparse point

Which I believe is a currently unsupported "Archive" reparse point, which I've determined by running "ntfsinfo" on the file. It identifies the File Attribute as an "Archive" reparse point:

david@david-VirtualBox:~$ sudo ntfsinfo -F /Users/user/AppData/Local/Microsoft/WindowsApps/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/MicrosoftEdge.exe /dev/sdb2
Dumping Inode 89554 (0x15dd2)
Upd. Seq. Array Off.: 48 (0x30)
Upd. Seq. Array Count: 3 (0x3)
Upd. Seq. Number: 4 (0x4)
LogFile Seq. Number: 0xf3122ff
MFT Record Seq. Numb.: 1 (0x1)
Number of Hard Links: 2 (0x2)
Attribute Offset: 56 (0x38)
MFT Record Flags: IN_USE
Bytes Used: 736 (0x2e0) bytes
Bytes Allocated: 1024 (0x400) bytes
Next Attribute Instance: 5 (0x5)
MFT Padding: 00 00
Dumping attribute $STANDARD_INFORMATION (0x10) from mft record 89554 (0x15dd2)
 Resident: Yes
 Attribute flags: 0x0000
 Attribute instance: 0 (0x0)
 Data size: 72 (0x48)
 Resident flags: 0x00
 File Creation Time: Wed Jan 16 18:01:51 2019 UTC
 File Altered Time: Wed Jan 16 18:01:51 2019 UTC
 MFT Changed Time: Wed Jan 16 18:01:51 2019 UTC
 Last Accessed Time: Wed Jan 16 18:01:51 2019 UTC
 File attributes: ARCHIVE REPARSE_POINT (0x00000420)
 Maximum versions: 0
 Version number: 0
 Class ID: 0
 User ID: 0 (0x0)
 Security ID: 1652 (0x674)
 Quota charged: 0 (0x0)
 Update Sequence Number: 5612560 (0x55a410)
Dumping attribute $FILE_NAME (0x30) from mft record 89554 (0x15dd2)
 Resident: Yes
 Attribute flags: 0x0000
 Attribute instance: 3 (0x3)
 Data size: 90 (0x5a)
 Resident flags: 0x01
 Parent directory: 89553 (0x15dd1)
 File Creation Time: Wed Jan 16 18:01:51 2019 UTC
 File Altered Time: Wed Jan 16 18:01:51 2019 UTC
 MFT Changed Time: Wed Jan 16 18:01:51 2019 UTC
 Last Accessed Time: Wed Jan 16 18:01:51 2019 UTC
 Allocated Size: 0 (0x0)
 Data Size: 0 (0x0)
 Filename Length: 12 (0xc)
 File attributes: ARCHIVE (0x00000020)
 Namespace: DOS
 Filename: 'MICROS~1.EXE'
Dumping attribute $FILE_NAME (0x30) from mft record 89554 (0x15dd2)
 Resident: Yes
 Attribute flags: 0x0000
 Attribute instance: 2 (0x2)
 Data size: 100 (0x64)
 Resident flags: 0x01
 Parent directory: 89553 (0x15dd1)
 File Creation Time: Wed Jan 16 18:01:51 2019 UTC
 File Altered Time: Wed Jan 16 18:01:51 2019 UTC
 MFT Changed Time: Wed Jan 16 18:01:51 2019 UTC
 Last Accessed Time: Wed Jan 16 18:01:51 2019 UTC
 Allocated Size: 0 (0x0)
 Data Size: 0 (0x0)
 Filename Length: 17 (0x11)
 File attributes: ARCHIVE (0x00000020)
 Namespace: Win32
 Filename: 'MicrosoftEdge.exe'
Dumping attribute $DATA (0x80) from mft record 89554 (0x15dd2)
 Resident: Yes
 Attribute flags: 0x0000
 Attribute instance: 1 (0x1)
 Data size: 0 (0x0)
 Resident flags: 0x00
Dumping attribute $REPARSE_POINT (0xc0) from mft record 89554 (0x15dd2)
 Resident: Yes
 Attribute flags: 0x0000
 Attribute instance: 4 (0x4)
 Data size: 280 (0x118)
 Resident flags: 0x00
 Reparse tag: 0x8000001b
 Data length: 272 (0x110)
 Data: 0x030000004d006900630072006f0073006f00660074002e004d00690063007200...
End of inode reached

######## System/Environment Details ########

david@david-VirtualBox:~$ lsb_release -rd
Description: Ubuntu 18.04.1 LTS
Release: 18.04

david@david-VirtualBox:~$ apt-cache policy ntfs-3g
ntfs-3g:
  Installed: 1:2017.3.23-2
  Candidate: 1:2017.3.23-2
  Version table:
 *** 1:2017.3.23-2 500
        500 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
        100 /var/lib/dpkg/status

The "getfattr" is:

david@david-VirtualBox:~$ getfattr -h -n system.ntfs_reparse_data -e hex /mnt/windows_10/Users/user/AppData/Local/Microsoft/WindowsApps/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/MicrosoftEdge.exe
getfattr: Removing leading '/' from absolute path names
# file: mnt/windows_10/Users/user/AppData/Local/Microsoft/WindowsApps/Microsoft.MicrosoftEdge_8wekyb3d8bbwe/MicrosoftEdge.exe
system.ntfs_reparse_data=0x1b00008010010000030000004d006900630072006f0073006f00660074002e004d006900630072006f0073006f006600740045006400670065005f003800770065006b0079006200330064003800620062007700650000004d006900630072006f0073006f00660074002e004d006900630072006f0073006f006600740045006400670065005f003800770065006b0079006200330064003800620062007700650021004d006900630072006f0073006f00660074004500640067006500000043003a005c00570069006e0064006f00770073005c00530079007300740065006d00330032005c00530079007300740065006d005500570050004c00610075006e0063006800650072002e00650078006500000031000000

And my ntfs-3g configuration is as follows (per reporting standards at the bottom of http://jp-andre.pagesperso-orange.fr/junctions.html#other):

david@david-VirtualBox:~$ $(which ntfs-3g) -help 2>&1 | grep ration
ntfs-3g 2017.3.23 integrated FUSE 28 - Third Generation NTFS Driver
  Configuration type 7, XATTRS are on, POSIX ACLS are on

david@david-VirtualBox:~$ file $(which ntfs-3g)
/bin/ntfs-3g: setuid ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=587e8bf3601c8587afd520f8cbf9c95bd73a5c25, stripped

david@david-VirtualBox:~$ md5sum $(which ntfs-3g)
a917787eae50117dec8646d6187af00a /bin/ntfs-3g

david@david-VirtualBox:~$ ls -ld $(strings $(which ntfs-3g) | grep ntfs-plugin | sed -e 's/ntfs-plugin.*//')
drwxr-xr-x 2 root root 4096 Jan 21 16:11 /usr/lib/x86_64-linux-gnu/ntfs-3g/

david@david-VirtualBox:~$ md5sum $(strings $(which ntfs-3g) | grep ntfs-plugin | sed -e 's/%08lx/*/')
1d5331d5d6bf8a79f60e0d1f87b77094 /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so
63a1ccbb2d16581761ea60339a5c6d26 /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000017.so
9db6f8288ed3856578e1ec3372310a35 /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-9000301a.so

[Jean-Pierre (jean-pierre-andre)]

Thank you for your detailed report.

> I encountered the dreaded "unsupported reparse point":

And this is caused by Microsoft designing new organizations of files to implement new features. These features are not documented and may be meaningless to a Linux user.

> File attributes: ARCHIVE REPARSE_POINT (0x00000420)

The archive flag is a traditional one, it means the file has been modified since it was last backup'ed.

> Reparse tag: 0x8000001b

The only thing I know about this tag is that it means "IO_REPARSE_TAG_APPEXECLINK". Your report shows this file has no data at all, so this must be some cloud file which has to be downloaded to be run.

The reparse data itself only shows the name of the file (Microsoft.MicrosoftEdge_8wekyb3d8bbweMicrosoft.MicrosoftEdge_8wekyb3d8bbwe!MicrosoftEdge) and the local launcher (C:\Windows\System32\SystemUWPLauncher.exe), so the cloud file must be at a location defined by your Microsoft account.

Accessing cloud storage is far beyond the ntfs-3g scope. Moreover running a Windows browser executable on Linux is likely to pose some challenges to usual emulators. The only thing I can reasonably do is to design a crude plugin which will return EREMOTE ("The object is remote") instead of an unsupported symlink.

[David (dshunfen)]

Many thanks. Appreciate the quick response.

I totally agree that it would be unreasonable to actually invoke some executable code to accomplish reading a file off of the filesystem.
In my mind, the main purpose of ntfs-3g (and any FS driver) is to at a minimum provide a view into the content on the mounted filesystem.

That being the case, I suppose what I would expect to be able to do here in the case of the "executable link" is:
1) Have it manifest itself as a normal file, with the content being the reparse point byte content
2) Have it manifest itself as a broken symlink to the URL target
I can see how both of those don't do justice to the actual reparse point itself, however (at least in my case) I think either of those would ideally be a better "default".

Practically, what would the EREMOTE plugin you proposed do? Would that just return a more specific error rather than "unsupported reparse point"? If my two options above don't make any sense to implement, I like the idea of having a more specific error for general users encountering this type of thing, especially if we could include the target in the message.

[Jean-Pierre (jean-pierre-andre)]

> 2) Have it manifest itself as a broken symlink to the URL target

This sounds like a good approximation. The target of the symlink could be formatted so that it shows what it is about. In your case this could be
C:\Windows\System32\SystemUWPLauncher.exe(Microsoft.MicrosoftEdge_8wekyb3d8bbwe, Microsoft.MicrosoftEdge_8wekyb3d8bbwe!MicrosoftEdge)
and this should be understandable as meaning to start the launcher with two arguments. I prefer not to translate the "\"s to make clear the link cannot be used on Linux. This is not an URL, and I have no idea of what the URL used by Windows 10 looks like (and it probably contains an authentication token).

At first glance, this is possible with your current ntfs-3g version (a few days needed to implement).

> Practically, what would the EREMOTE plugin you proposed do?

If you did "cat the-reparse-file", you would get "Error : The object is remote", with consequences depending on the context (typically shell or application aborted).

[David (dshunfen)]

Thank you. I think if it would be possible to have both of those things, it would be very helpful.

That is, showing the broken symlink to the URL location, and when one tries to do something like "cat"ing the file, it would indicate that it's a reparse point pointing to a remote file.

If those things both make sense to do and don't "break the rules" of how we represent filesystems, I think having them would shed a little more light on to the file rather than the generic exception/error "unsupported reparse point".

[Jean-Pierre (jean-pierre-andre)]

> I think if it would be possible to have both of those things, it would be very helpful

Not possible : either the file is a symlink or it is of unknown type.

I have made a proposal for you to try, available on :
http://jp-andre.pagesperso-orange.fr/execlink.zip
(however you will have to adapt the readme, which is related to another plugin)

IMHO this does not add much value to the standard "Unsupported reparse point".