glance-manage db purge gives DBReferenceError

Bug #1702445 reported by Pooja Jadhav
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Glance
Fix Released
Undecided
Mitya Eremeev

Bug Description

Assume that 'image' table has 1 record and 120 image tags are added to a image. When you purge the db passing default parameters, it will give ‘IntegrityError’ error.

Steps to reproduce:

1. glance-manage db purge --age_in days 30 --max_rows 100

2. As per above command, max 100 records get purge from image_tags table

3. It fails with the below error when it tries to purge record from image table having reference in image_tags table because 20 records of image_tags still exists.

DBError detected when purging from images: (pymysql.err.IntegrityError) (1451, u'Cannot delete or update a parent row: a foreign key constraint fails (`glance`.`image_tags`, CONSTRAINT `image_tags_ibfk_1` FOREIGN KEY (`image_id`) REFERENCES `images` (`id`))') [SQL: u'DELETE FROM images WHERE images.id in (SELECT T1.id FROM (SELECT images.id \nFROM images \nWHERE images.deleted_at < %(deleted_at_1)s ORDER BY images.deleted_at \n LIMIT %(param_1)s) as T1)'] [parameters: {u'deleted_at_1': datetime.datetime(2017, 6, 4, 7, 25, 50, 16864), u'param_1': 100}]

Changed in glance:
assignee: nobody → Pooja Jadhav (poojajadhav)
Changed in glance:
status: New → Confirmed
description: updated
Revision history for this message
Pooja Jadhav (poojajadhav) wrote :

Hi all,

Instead of applying limit of max_rows to each and every table separately, we can apply on all the tables (similar to nova archive deleted rows logic).

For example, Images table has 2 records and image_tags table has 15 records, then if we pass max_rows as 10 then it will purge 10 records from image_tags table on first attempt of purge command and then on second attempt it will purge the 5 records from image_tags and 2 records from images table. Also records will be purged from child tables first before purging the records from parent table which will help to avoid ItegrityFailure issue.

Changed in glance:
status: Confirmed → In Progress
Changed in glance:
status: In Progress → Confirmed
Rajat Sharma (tajar29)
Changed in glance:
assignee: Pooja Jadhav (poojajadhav) → Rajat Sharma (tajar29)
Revision history for this message
Mitya Eremeev (mitos) wrote :

Hi, all !

I see no activity for 3 years.
I suppose Rajat is not against if I assign the bug to me.
Thank you.

Changed in glance:
assignee: Rajat Sharma (tajar29) → Mitya Eremeev (mitos)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to glance (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/glance/+/813691

Changed in glance:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to glance (master)

Reviewed: https://review.opendev.org/c/openstack/glance/+/813691
Committed: https://opendev.org/openstack/glance/commit/360dc45680df0c968994d60f87446e1a93e2d792
Submitter: "Zuul (22348)"
Branch: master

commit 360dc45680df0c968994d60f87446e1a93e2d792
Author: Mitya_Eremeev <email address hidden>
Date: Wed May 4 15:16:29 2022 +0300

    glance-manage can purge all deleted rows

    When we want to purge all deleted rows
    we use "glance-manage db purge" or
    "glance-manage purge_images_table"
    then we never know how many deleted rows are still in a table.
    So we need to launch the command many times until
    the command reports that 0 rows were deleted in every table.
    It's inconvenient.
    The patch introduced new valid value for "max_rows".
    If "max_rows" equals -1 all deleted rows are purged.

    Closes-Bug: 1702445
    Change-Id: Ia9811d93b1c0b67a1b29a9e8ee3e0fc098c57d4a

Changed in glance:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/glance 25.0.0.0b2

This issue was fixed in the openstack/glance 25.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.