backup-manager is not removing the correct files

[David Ayers]

I have set:

# Number of days we have to keep an archive (Time To Live)
export BM_ARCHIVE_TTL="2"
export BM_LOGGER_LEVEL="info"

yet have a weekly anachron job which creates my archives. I'm not sure whether the "2" is actually meant to mean "two days" or "two" backups. But in any case the processing seems wrong:

Anacron is invoking the weekly script on 2013-12-3 and is removing only the md5 file 20131126 yet not the corresponding archive files. After the processing below I still have:

-rw------- 1 root root 71 Dez 3 08:43 schiefer-20131203.md5
-rw-rw---- 1 root root 2391242 Nov 26 10:06 schiefer-etc.20131126.master.tar.bz2
-rw------- 1 root root 2390977 Dez 3 08:43 schiefer-etc.20131203.master.tar.bz2
-rw-rw---- 1 root root 7583581916 Nov 26 11:37 schiefer-home-ayers.20131126.master.tar.bz2
-rw-rw---- 1 root root 4779 Nov 26 10:06 schiefer-usr-local.20131126.master.tar.bz2
-rw-rw---- 1 root root 1173007204 Nov 26 10:18 schiefer-var-lib-home.20131126.master.tar.bz2

3 files from last weeks backup. Because only the md5file was removed yet not the corresponding archives leads to the message: No md5file found: /var/archives/schiefer-20131126.md5

(There are other disturbing issues but I'll open separate bugs for them.)

Dec 3 08:43:18 schiefer backup-manager[4301]: info * Getting lock for backup-manager 4301 with /etc/backup-manager.conf
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Cleaning /var/archives
Dec 3 08:43:18 schiefer backup-manager-purge[5163]: info * Reading archives from STDIN - BackupManager::Dialog::print_info (/usr/bin/backup-manager-purge l. 302)
Dec 3 08:43:18 schiefer backup-manager-purge[5163]: info * Outdating archives made before 20131201 - BackupManager::Dialog::print_info (/usr/bin/backup-manager-purge l. 245)
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Removing archive "/var/archives/schiefer-20131126.md5".
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Removing archive "/var/archives/schiefer-etc.20131122.master.tar.bz2".
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Removing archive "/var/archives/schiefer-home-ayers.20131122.master.tar.bz2".
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Removing archive "/var/archives/schiefer-usr-local.20131122.master.tar.bz2".
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Removing archive "/var/archives/schiefer-var-lib-home.20131122.master.tar.bz2".
Dec 3 08:43:18 schiefer backup-manager[4301]: info * Using method "tarball".
Dec 3 08:43:21 schiefer backup-manager[4301]: error * No md5file found: /var/archives/schiefer-20131126.md5
Dec 3 08:43:21 schiefer backup-manager[4301]: info * Running post-command: /usr/local/sbin/backup-manager-post

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: backup-manager 0.7.9-3
ProcVersionSignature: Ubuntu 3.2.0-56.86-generic 3.2.51
Uname: Linux 3.2.0-56-generic x86_64
ApportVersion: 2.0.1-0ubuntu17.6
Architecture: amd64
Date: Tue Dec 3 12:56:34 2013
InstallationMedia: Ubuntu 12.04.1 LTS "Precise Pangolin" - Release amd64 (20120823.1)
MarkForUpload: True
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=de_AT:de
 TERM=xterm
 PATH=(custom, no user)
 LANG=de_AT.UTF-8
 SHELL=/bin/bash
SourcePackage: backup-manager
UpgradeStatus: No upgrade log present (probably fresh install)

[David Ayers (ayers)]

After adding some Debug-Logs to the scripts, it seems this my be a result of a previous abort of a backup run which failed to produce an MD5 file. Will only a maximum of one file per prefix be removed?

[David Ayers (ayers)]

Unfortunately that suspicion was moot.

It seems the final decision is made in backup-manager-purge and one difference seems to be that the md5 file doesn't have a "master":

        # we can outdate a master only if a younger master exists
        if ($data->{master}) {
            foreach my $master_date (
                keys %{$g_rh_archives->{$data->{name}}{pathByDateMasters}}) {
                if ($master_date > $data->{date}) {
#> This is where none of the other files get pushed since master_date == $date->{date}
                    push @{$ra_outdated}, $archive;
                    last;
                }
            }
        }

        # here the archive is deprecated, its date is < to $purge_date
        else {
            # if BM_ARCHIVE_STRICTPURGE is true, we can only purge
            # an archive prefixed with BM_ARCHIVE_PREFIX
            next if (($ENV{BM_ARCHIVE_STRICTPURGE} eq "true") and
                     ($data->{prefix} ne $ENV{BM_ARCHIVE_PREFIX}));

            # now, we're sure we can outdate the archive
            push @{$ra_outdated}, $archive;
#> This is where the md5 file gets pushed
        }

So currently I'm still wondering on whether some analog date check should be added when no master exists, or whether the date check against the master could possibly be bogus.

But I'm also starting to believe that the issue really only manifests since the weekly backup (which requires these master checks) is paired with a TTL of merely 2 days. The intent is to insure that the backup is done automatically at least once a week, but only 2 sets of archives should be retained due to space constraints. But I guess that cannot be be expressed correctly.