[SRU] virtlogd not rotating logs properly

[David Negreira]

[Impact]

 When one enables max_age_days on /etc/libvirt/virtlogd.conf, for example by setting it to max_age_days = 1, the log files are not rotated correctly and we see this error on virtlogd:

Feb 28 08:58:57 dixie virtlogd[7205]: internal error: Failed to parse rotated index from ''

[Test Case]

1) Launch a couple of dummy VMs to ensure that there are some logs in /var/log/libvirt
2) Edit /etc/libvirt/virtlogd.conf and ensure that the max_age_days setting is uncommented as per the following:
   max_age_days = 1
3) sudo systemctl restart virtlogd
4) Wait 24 hours. (There is no way to accelerate the timer unless one edits the source code)
5) Verify that logs are cleaned up and there are no errors in syslog or virtlogd journal like the following:
   Feb 28 08:58:57 dixie virtlogd[7205]: internal error: Failed to parse rotated index from ''

[Where problems could occur]

There could be some issues with virtlogd, but the code path only executes when max_age_days is different than 0.
 In the event of a regression, one could disable the setting by commenting max_age_days.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "lp2072647-libvirt.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

[Heitor Alves de Siqueira (halves)]

Thanks for the patch, David!

I think this needs to target Oracular as well, right? I'm adding that to the bug, and will unsubscribe ~ubuntu-sponsors as devel needs to be fixed first.

Once you have uploaded a debdiff or MR for Oracular, feel free to re-subscribe the ~ubuntu-sponsors team.

[David Negreira (dnegreira)]

debdiff for noble

[David Negreira (dnegreira)]

debdiff for oracular

[David Negreira (dnegreira)]

Hi Heitor
Thanks for your comment/review, I have now added a debdiff for oracular as well as modified the the origin to point to the commit where the fix is done upstream, so this is now ready to be reviewed again, hence I re-subscribed ubuntu-sponsors.

[Mitchell Dzurick (mitchdz)]

Hi, the libvirt merge for Oracular is under way and should come out soon. I say let's just wait for the merge, and then we can process the noble SRU.

[Dan Bungert (dbungert)]

Marking fix released for Oracular, as this is present in upstream and released for Oracular in the version 10.5.0-1ubuntu1 upload.

[Dan Bungert (dbungert)]

Looking at this bug, it's almost ready for upload, but I'd suggest a changelog tweak. I like to focus on the symptom of the issue to make it easier for users who want to understand the changelog - something about "internal error: Failed to parse rotated index" maybe, as the relationship to max_age_days may not be immediately obvious. What do you think?

[David Negreira (dnegreira)]

Hi Mitchel,

Thanks for the update. I'm happy to hear the fix is now in Oracular.

Dan, I have now uploaded a debdiff with a different changelog. Let me know if the format is okay.

[Lukas Märdian (slyon)]

The new changelog from debdiff in comment #10 provides more context around "Failed to parse rotated index", as suggested by Dan. The distro-patch matches upstream and the resulting package builds fine.

LGTM. Sponsored into Noble and unsubscribed ~ubuntu-sponsors.

[Timo Aaltonen (tjaalton)]

Hello David, or anyone else affected,

Accepted libvirt into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/10.0.0-2ubuntu8.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted libvirt (10.0.0-2ubuntu8.4) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

libvirt/10.0.0-2ubuntu8.4 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#libvirt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[David Negreira (dnegreira)]

The regression doesn't seem related to the change that we are introducing, tests have been re-triggered by Dariusz Gadomski as per my request.

[David Negreira (dnegreira)]

armhf tests have now passed [1] and there isn't any other regression reported

[1] https://autopkgtest.ubuntu.com/run/66f2d865-76ba-4ebd-936a-d79a8660608e

[David Negreira (dnegreira)]

Hi,

I have verified that the patch is indeed now functioning and enabling max_age_days does not output any error, and the log files are cleared.

I did not capture the output of `ls /var/log/libvirt/*` log at the time of VM creation, but I did see the files there, and now they are gone.

lp-2072647-proposed-install-setup.txt has the installation and setup steps of virtlogd/libvirt

lp-2072647-verification.txt has the verification that there are no errors and logs are cleared.

[Launchpad Janitor (janitor)]

This bug was fixed in the package libvirt - 10.0.0-2ubuntu8.4

---------------
libvirt (10.0.0-2ubuntu8.4) noble; urgency=medium

  * d/p/u/lp-2072647-log_cleaner-Detect-rotated-filenames-properly.patch:
    Fix virtlogd bug "internal error: Failed to parse rotated index", this
    happens when max_age_days parameter is enabled. (LP: #2072647)

 -- David Negreira <email address hidden> Wed, 10 Jul 2024 13:23:06 +0000

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for libvirt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.