The test environment does not support the XDG Special User Directories

[Naël]

[XDG Special User Directories]

The XDG Special User Directories are a set of (eight) localized user directories for storing documents, downloads, music, pictures, videos, and so on. Their paths are defined for each user in the file $XDG_CONFIG_HOME/user-dirs.dirs, where XDG_CONFIG_HOME defaults to $HOME/.config.

Applications can read this file to find the directories in a locale-independent manner. Déjà-Dup uses the function GLib.Environment.get_user_special_dir to do so, in deja-dup/libdeja/DirHandling.vala. For instance, on a French locale and with default XDG Special User Directories, GLib.Environment.get_user_special_dir(UserDirectory.MUSIC) returns the path to the directory "Musique" in the user's home directory.

The file $XDG_CONFIG_HOME/user-dirs.dirs and the names of the XDG Special User Directories are updated by xdg-user-dirs-update according to the user's locale at the start of each user session. If the file or the directories don't exist, xdg-user-dirs-update creates them, based on the user's locale and the system default values in /etc/xdg/user-dirs.defaults.

[Non-support by the test shell]

The test shell changes XDG_CONFIG_HOME to /tmp/dd-<temp>/config, but does not retrieve user-dirs.dirs in the process, or points to it, or recreates it with xdg-user-dirs-update.

As a result the new $XDG_CONFIG_HOME/user-dirs.dirs points to a non-existent file, and applications started in the test environment cannot find the user's special directories. The function GLib.Environment.get_user_special_dir returns "null" every time it is called, save for the desktop directory, because it falls back to "$HOME/Desktop" for compatibility with existing practise.

[Impact]

This is why the exclude list displayed in deja-dup-preferences (started from the test shell) does not feature the $DOWNLOAD directory that features in the default exclude-list key of the schema org.gnome.DejaDup: ['$DOWNLOAD', '$TRASH'].

This is why including $DOCUMENTS, $DOWNLOAD, $MUSIC, $PICTURES, $VIDEOS, $TEMPLATES and/or $PUBLIC_SHARE in the include-list and exclude-list keys of org.gnome.DejaDup does not lead to them being passed to duplicity as --include and --exclude options. That puzzled me to no end when I was testing a hack to solve bug 1549776, see comments 8, 11, 12.

From what I understand, $DOWNLOAD also features as an exclude in a number of test scripts (in deja-dup/libdeja/tests/scripts) which are run by the test shell during make check. It would be reduced to "null" there too.

[Fix]

Simply adding one of the following lines to the test shell's script, in the section "Create fake home environment", ensures that the user's current XDG Special User Directories are found, by creating a file user-dirs.dirs in the new XDG_CONFIG_HOME:

  ln -s "${HOME}/.config/user-dirs.dirs" "${XDG_CONFIG_HOME}/user-dirs.dirs"
  cp "${HOME}/.config/user-dirs.dirs" "${XDG_CONFIG_HOME}/user-dirs.dirs"
  xdg-user-dirs-update

Alternatively, new XDG Special User Directories can be created in the test environment /tmp/dd-<temp>, instead of using the user's current ones, which are not adapted to tests since they contain the user's data rather than unimportant test data. In theory, the simplest way to do so is to point HOME to the test environment as I suggest in bug 1660174, and then issue xdg-user-dirs-update:

  HOME="${ROOTDIR}"
  xdg-user-dirs-update

But this doesn't work because xdg-user-dirs-update gets the user's home directory from HOME only if it cannot get it from /etc/passwd. So no XDG Special User Directories are created in the new HOME, since they already exist in the home directory mentioned in /etc/passwd. A file user-dirs.dirs is created in the new XDG_CONFIG_HOME though, since there isn't any. But since the paths defined in this file are relative to $HOME, and that no XDG Special User Directories are created in the new HOME, the paths point to non-existent directories.

So we have to do everything by hand, for instance with:

  OLD_HOME="${HOME}"
  HOME="${ROOTDIR}"

  if [ -f "${OLD_HOME}/.config/user-dirs.dirs" \
    -a -f "${OLD_HOME}/.config/user-dirs.locale" ]
  then

    cp "${OLD_HOME}/.config/user-dirs.dirs" \
       "${XDG_CONFIG_HOME}/user-dirs.dirs"
    cp "${OLD_HOME}/.config/user-dirs.locale" \
       "${XDG_CONFIG_HOME}/user-dirs.locale"

    . "${XDG_CONFIG_HOME}/user-dirs.dirs"
    for d in DESKTOP DOCUMENTS DOWNLOAD MUSIC PICTURES \
             PUBLICSHARE TEMPLATES VIDEOS
    do
      eval mkdir -p "\${XDG_${d}_DIR}"
    done

  fi

[Patch]

Attached is a patch for deja-dup/libdeja/DirHandling.vala that just adds printfs to the function parse_keywords in order to print the values of the XDG directories in the log file ("DEJA_DUP_DEBUG=1 deja-dup --backup > dd.log"). All added lines are prefixed with LUKE: for easy greping (name of the user I tested with).

[References]

https://wiki.archlinux.org/index.php/XDG_user_directories
https://www.freedesktop.org/wiki/Software/xdg-user-dirs
https://valadoc.org/glib-2.0/GLib.Environment.get_user_special_dir.html
xdg-user-dirs-update(1)
xdg-user-dir(1)
user-dirs.dirs(5)
user-dirs.default(5)

[Naël (nathanael-naeri)]

And here is a patch that implements the fix I suggest, plus those I suggest for bugs 1660174 and 1660224 as an added bonus (on top of the current revision 1570):

  1660174: set HOME to the test environment /tmp/dd-<temp>
  1660224: include a trash can in the test environment
  1660342: setup XDG Special User Directories in the test environment

The whole thing makes the fake home environment more similar to the actual one IMHO.

[Naël (nathanael-naeri)]

The patch I included in comment 2 is wrong, my bad. I probably got mixed up when I tested it yesterday.

Running xdg-user-dirs-update after setting HOME to the test environment /tmp/dd-<temp> does not create XDG Special User Directories in this directory as I thought it did, because xdg-user-dirs-update gets the user's home directory from HOME only if it cannot get it from /etc/passwd (see function get_home_dir in its source code, file xdg-user-dirs-update.c).

It does create a file user-dirs.dirs in the new XDG_CONFIG_HOME though, since there isn't any. But since the paths defined in this file are relative to $HOME, and that no XDG Special User Directories were created in the new HOME, the paths point to non-existent directories.

So we have to do everything by hand instead of using xdg-user-dirs-update. I attach here a new, hopefully correct version of the patch I suggested in comment 2 to fix bugs 1660174 1660224 1660342. I have also changed the bug's description, section [Fix].

[Vej (vej)]

This had been released as a part of Déjà Dup 34.4.