neutron/ovs-agent.log isn't rotated on HA controllers

Bug #1298957 reported by Gleb
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Confirmed
Medium
Bogdan Dobrelya

Bug Description

{"ostf_sha": "83ada35fec2664089e07fdc0d34861ae2a4d948a", "fuelmain_sha": "17eed776b30886851ae0042fa7a30184f5cd8eb6", "astute_sha": "8b2059a37be9bd82df49f684822727b4df4c511b", "release": "4.0", "nailgun_sha": "ac02e18990cd652db6577ce42bdea9838076c63c", "fuellib_sha": "098f381ff8a528a39d3b6f17ea70955baeb159e8"}

Here is the normal logrotate config

/var/log/neutron/ovs-agent.log {
    daily
    missingok
    compress
    delaycompress
    notifempty
}

After logrotate log file change the name from ovs-agent.log to ovs-agent.log.1 and ovs agent continues to write th this file.
On compute nodes this logrotate config works well. Probably the reason is corosync/pacemaker.

I tried using postrotate section
    postrotate
      /bin/kill `cat /var/run/resource-agents/p_neutron-plugin-openvswitch-agent.pid 2> /dev/null` 2> /dev/null || true
    endscript

but it cause migration l3-agent dhcp-agent beacuse in 4.0 they depend on ovs-agent and metadata agent.

Changed in fuel:
importance: Undecided → Medium
assignee: nobody → Bogdan Dobrelya (bogdando)
Changed in fuel:
milestone: none → 5.0
Changed in fuel:
status: New → Confirmed
Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

Services should be at least SIGHUP-ed in order to ensure file descriptors updated once logs has been rotated.

This issue is also related to https://bugs.launchpad.net/oslo/+bug/1276694 - in Fuel<=4.x (with Havana MOS) OS services do not support SIGHUP yet.

According to the mentioned related #1276694 issue, after OS I-3 (Fuel 5.0 and cannot be backported) we should ensure all OS services (except murano*, glance* which do not support SIGHUP yet) must be SIGHUPPed in postrotate sections of 10-fuel.conf & 20-fuel.conf (please note: in Fuel we use these ones to ensure log rotation policy for deployed environments).

Revision history for this message
Evgeny Kozhemyakin (ekozhemyakin) wrote :

pls, note: it's possible to use "copytruncate" logrotate's directive for the services which don't support SIGHUP.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

We use this option, indeed. See https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/openstack/templates/20-fuel.conf.erb#L30
(and the same for 10-fuel.conf.erb)
1) As you can see, all neutron logs are ensured to be rotated
2) But services cannot be reloaded (no support for SIGHUP yet IceHouse), hence it was decided do not restart them to refresh fd for logfiles and wait for an appropriate SIGHUP support for Openstack services - https://bugs.launchpad.net/oslo/+bug/1276694
3) Thus, in Havana and earlier releases, the issues with log rotation could happen due to the issue mentioned above.
4) The only w/a I can sugest is to ensure forcibly restarting for affected OS services - just put corresponding instructions into the 10-fuel.conf and 20-fuel.conf.

That's why this issue was marked as a duplicate of https://bugs.launchpad.net/fuel/+bug/1276512

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.