We have "delaycompress" option enabled in /etc/logrotate.conf globally.
This option orders logrotate to copytruncate the log and leave it renamed as <log file name>.1
Next time logrotate is started by cron - it compresses these <log file name>.1 first, removes the uncompressed source log and only after it starts new rotation cycle on the available logs. At least it is meant to be so
At the same time we have "sharedscripts" option enabled in /etc/logrotate.d/fuel.nodaily configuration file.
This option creates the big list of files to be rotated first and runs logrotate with the entire list instead of calling it against the each log file.
Having both these options at the same time leads to the following unwanted effects:
1. Looks like "sharedscripts" prevails over "delaycompress". So, when logrotate is started, it creates the list of files to rotate first instead of compressing the files, delayed from the previous rotation. As a results the delayed files are included to the rotation list. After that, logrotate attempts to start rotation ans stops completely on the first error like following:
#################################################
"error: error creating output file /var/log/docker-logs/mcollective.log.1: File exists"
#################################################
No rotation happens at all after such errors.
2. If error happened with any single log file rotation and "sharedscripts" is enabled - the remained logs are skipped from rotation as well.
Solution for the both cases is to not use "sharedscripts" option. Delaycompress may be necessary for some kinds of log sources, sharedscripts may be safely removed.
Note: this issue is better visible on the environments with a lot of log files - number of files increases the chance of logrotate failure.
++++++++++++++++++++++++++++++++++++++++++++++
Additional info on MOS version:
VERSION:
feature_groups:
- mirantis
production: "docker"
release: "7.0"
openstack_version: "2015.1.0-7.0"
api: "1.0"
build_number: "187"
build_id: "2015-08-18_03-05-20"
nailgun_sha: "4710801a2f4a6d61d652f8f1e64215d9dde37d2e"
python-fuelclient_sha: "4c74a60aa60c06c136d9197c7d09fa4f8c8e2863"
fuel-agent_sha: "57145b1d8804389304cd04322ba0fb3dc9d30327"
fuel-nailgun-agent_sha: "e01693992d7a0304d926b922b43f3b747c35964c"
astute_sha: "e24ca066bf6160bc1e419aaa5d486cad1aaa937d"
fuel-library_sha: "0062e69db17f8a63f85996039bdefa87aea498e1"
fuel-ostf_sha: "17786b86b78e5b66d2b1c15500186648df10c63d"
fuelmain_sha: "c9dad194e82a60bf33060eae635fff867116a9ce"
Fix proposed to branch: master
Review: https:/