Comment 0 for bug 1630516

Good afternoon.
I have started seeing something very similar to Debian Dug 734688 "Logs are not rotated for a month" but in the latest Ubuntu LTS (16.04). I seem to have
$ logrotate --version
logrotate 3.8.7
bundled in it. A few weeks ago I started getting root emails such as this:
> Subject: Cron <root@warden> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
>
> /etc/cron.daily/logrotate:
> error: error creating output file /var/log/munin/munin-node.log.1: File exists
> run-parts: /etc/cron.daily/logrotate exited with return code 1
When I inspected the area of concern I was able to see that there was an existing .1 file.
manager@warden:/var/log/munin$ ll
total 580
drwxr-xr-x 2 munin adm 4096 Sep 27 06:31 ./
drwxr-xr-x 13 root syslog 4096 Oct 5 06:26 ../
-rw-r--r-- 1 root root 3440 Sep 26 13:39 munin-node-configure.log
-rw-r--r-- 1 root root 490251 Oct 5 10:25 munin-node.log
-rw-r--r-- 1 root root 56598 Sep 21 02:01 munin-node.log.1
-rw-r--r-- 1 root root 24576 Aug 31 02:01 munin-node.log.2
-rw-r--r-- 1 root root 1906 Sep 19 06:25 munin-node.log.8.gz
The contents of the munin-node.log file seem to run from the 19th September until today. Unlike other parts of this bug the .1 and .2 files do not seem to be already compressed.

I deleted all but the munin-node.log file to see if it would resolve the problem and was going to leave it at that. Then I noticed that I have had another Ubuntu machine which has been sending similar emails for the past week:
> Subject: Cron <root@trac> test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
>
> /etc/cron.daily/logrotate:
> error: error creating output file /var/log/syslog.1.gz: File exists
> run-parts: /etc/cron.daily/logrotate exited with return code 1
Different file, different machine but a very similar error message.

Checking on the syslog file I can see that it better fits with other reports on this bug as my duplicated .1 files has a corresponding .1.gz file.
manager@trac:/var/log$ ll syslog*
-rw-r----- 1 syslog adm 918492 Oct 5 10:30 syslog
-rw-r----- 1 syslog adm 125819 Sep 30 06:25 syslog.1
-rw-r----- 1 syslog adm 20638 Oct 2 02:01 syslog.1.gz
-rw-r----- 1 syslog adm 41989 Sep 30 02:00 syslog.2.gz
-rw-r----- 1 syslog adm 18654 Sep 28 02:01 syslog.3.gz
-rw-r----- 1 syslog adm 31720 Sep 26 06:40 syslog.4.gz
-rw-r----- 1 syslog adm 33151 Sep 25 02:01 syslog.5.gz
-rw-r----- 1 syslog adm 17290 Sep 23 02:01 syslog.6.gz
-rw-r----- 1 syslog adm 39275 Sep 21 06:35 syslog.7.gz

There has been some speculation that a full or nearly full /var partition would cause this issue. I can confirm that /var is part of / on my systems and that presently both of them have several gigabytes of space. I run Munin an Icinga to monitor system state. Neither of these show / being completely full in the past month. They have both had /boot fill significantly. Trac had a highest use value of / being 99.28% full in the past year but warden has only had a peak of 33% in the past year.

A quick search of the internet suggests a couple of other people reporting similar issues:
https://github.com/gitlabhq/gitlabhq/issues/6894
http://raspberrypi.stackexchange.com/questions/22545/why-are-system-logs-not-rotating

I do not believe that I have altered by logrotate configuration files but here is a copy of the ones I know about:
$ cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

$ cat /etc/logrotate.d/rsyslog
/var/log/syslog
{
            rotate 7
            daily
            missingok
            notifempty
            delaycompress
            compress
            postrotate
                        reload rsyslog >/dev/null 2>&1 || true
            endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
            rotate 4
            weekly
            missingok
            notifempty
            compress
            delaycompress
            sharedscripts
            postrotate
                        reload rsyslog >/dev/null 2>&1 || true
            endscript
}

manager@trac:/var/log$ cat /etc/logrotate.d/munin-node
/var/log/munin/munin-node.log {
            daily
            missingok
            rotate 7
            compress
            copytruncate
            notifempty
            create 640 root root
}

I am confident that is all the files which are relative to the configuration:
manager@trac:/var/log$ rgrep syslog /etc/logrotate.*
/etc/logrotate.conf:# use the syslog group by default, since this is the owning group
/etc/logrotate.conf:# of /var/log/syslog.
/etc/logrotate.conf:su root syslog
/etc/logrotate.d/ufw: invoke-rc.d rsyslog reload >/dev/null 2>&1 || true
/etc/logrotate.d/rsyslog:/var/log/syslog
/etc/logrotate.d/rsyslog: reload rsyslog >/dev/null 2>&1 || true
/etc/logrotate.d/rsyslog: reload rsyslog >/dev/null 2>&1 || true
manager@trac:/var/log$ rgrep munin /etc/logrotate.*
/etc/logrotate.d/munin-node:/var/log/munin/munin-node.log

If there is anything else I can do to help diagnose the issue please let me know.

Usual boiler plate bits:
"trac" machine:
$ lsb_release -rd
Description: Ubuntu 14.04.5 LTS
Release: 14.04

$ apt-cache policy logrotate
logrotate:
  Installed: 3.8.7-1ubuntu1
  Candidate: 3.8.7-1ubuntu1
  Version table:
 *** 3.8.7-1ubuntu1 0
        500 http://gb.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
        100 /var/lib/dpkg/status

"warden" machine:
$ lsb_release -rd
Description: Ubuntu 16.04.1 LTS
Release: 16.04

$ apt-cache policy logrotate
logrotate:
  Installed: 3.8.7-2ubuntu2
  Candidate: 3.8.7-2ubuntu2
  Version table:
 *** 3.8.7-2ubuntu2 500
        500 http://gb.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
N: Ignoring file '50unattended-upgrades.ucf-dist' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: logrotate 3.8.7-1ubuntu1
ProcVersionSignature: Ubuntu 4.2.0-42.49~14.04.1-generic 4.2.8-ckt12
Uname: Linux 4.2.0-42-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.21
Architecture: amd64
Date: Wed Oct 5 11:14:53 2016
InstallationDate: Installed on 2015-12-14 (295 days ago)
InstallationMedia: Ubuntu-Server 14.04.3 LTS "Trusty Tahr" - Beta amd64 (20150805)
SourcePackage: logrotate
UpgradeStatus: No upgrade log present (probably fresh install)