nova.task_log database records are never deleted

[melanie witt]

nova.task_log database records are related to the server usage audit log API [1] which is dependent on the [DEFAULT]/instance_usage_audit config option being set to True (it defaults to False). Enablement of the config option causes nova-compute to emit notifications which are consumed by OpenStack Telemetry. When the config option is enabled, records will be periodically created in the nova.task_log table. And there is no cleanup of the records in nova.

I started a ML thread about the server usage audit log API awhile back [2] and the conclusion was that deprecating or removing the API isn't practical as there are systems in the wild still using it, so the best action to take for now is to create a nova-manage command for cleaning up old nova.task_log records.

I brought up the proposal in a nova meeting [3] and in the nova channel [4] and the consensus was to treat it as a Wishlist bug so that it can be backported upstream.

So this is a bug for the work to add a new 'nova-manage db purge_task_log' command that offers a --before <datetime> option.

[1] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-09-06.log.html#t2019-09-06T14:19:50
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009245.html
[3] http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-10-17-14.00.log.html#l-306
[4] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-10-17.log.html#t2019-10-17T14:54:39

[melanie witt (melwitt)]

https://review.opendev.org/c/openstack/nova/+/780395 has been proposed

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/780395
Committed: https://opendev.org/openstack/nova/commit/bc71331bcaa92718fce06ac3d2ecda6a4c9d2d20
Submitter: "Zuul (22348)"
Branch: master

commit bc71331bcaa92718fce06ac3d2ecda6a4c9d2d20
Author: melanie witt <email address hidden>
Date: Thu Mar 11 23:47:05 2021 +0000

    Add --task-log option to nova-manage db archive_deleted_rows

    The task_log table contains instance usage audit records if
    nova-compute has been configured with [DEFAULT]instance_usage_audit =
    True. This will be the case if OpenStack Telemetry is being used in the
    deployment, as the option causes nova to generate audit information
    that Telemetry then retrieves from the server usage audit log API [1].

    Historically, there has been no way to delete task_log table records
    other than manual database modification. Because of this, task_log
    records could pile up over time and operators are forced to perform
    manual steps to periodically truncate the table.

    This adds a --task-log option to the 'nova-manage db
    archive_deleted_rows' CLI to also archive task_log records while
    archiving the database. --task-log works in conjunction with --before
    if operators desire archving only records that are older than <date>.
    The 'updated_at' field is used by --task-log --before <date> to
    determine the age of a task_log record for archival.

    Closes-Bug: #1877189

    [1] https://docs.openstack.org/api-ref/compute/#server-usage-audit-log-os-instance-usage-audit-log

    Change-Id: Ibed67854a693c930effd4dba7aca6cd03b65bd92

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/nova/+/801144

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/nova/+/801144
Committed: https://opendev.org/openstack/nova/commit/d093849c3c70e46bf409b1051454cbf70bc12a1f
Submitter: "Zuul (22348)"
Branch: master

commit d093849c3c70e46bf409b1051454cbf70bc12a1f
Author: melanie witt <email address hidden>
Date: Fri Jul 16 21:53:38 2021 +0000

    Correct reno for archiving task_log database records

    The release note incorrectly stated that the Telemetry service calls
    the /os-instance_usage_audit_log REST API to retrieve instance usage
    data. Telemetry rather consumes the notifications that are only generated
    when [DEFAULT]instance_usage_audit=True.

    Related-Bug: #1877189

    Change-Id: Iea358090c42877bedfdcc4a04019f6dc406176c2

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/802766

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 24.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/nova/+/802766
Reason: stable/wallaby branch of openstack/nova is about to be deleted. To be able to do that, all open patches need to be abandoned. Please cherry pick the patch to unmaintained/wallaby if you want to further work on this patch.