Unknown SQLite 3.x database swift files

Bug #1864166 reported by Vlastimil Mikes
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
In Progress
Critical
Oleksiy Molchanov

Bug Description

Detailed bug description:
  /var/lib/glance is being filled due to frequent temp file(file type - SQLite 3.x database) generation in /var/lib/glance/node/1/tmp & /var/lib/glance/node/2/tmp.

 root@cic-2:~# du -h --max-depth 1 /var/lib/glance/node/1
 236G /var/lib/glance/node/1/tmp
 232M /var/lib/glance/node/1/containers
 224M /var/lib/glance/node/1/accounts
 39G /var/lib/glance/node/1/objects
 0 /var/lib/glance/node/1/async_pending
 82M /var/lib/glance/node/1/quarantined
 275G /var/lib/glance/node/1
 root@cic-2:~# du -h --max-depth 1 /var/lib/glance/node/2
 289G /var/lib/glance/node/2/tmp
 234M /var/lib/glance/node/2/containers
 207M /var/lib/glance/node/2/accounts
 40G /var/lib/glance/node/2/objects
 0 /var/lib/glance/node/2/async_pending
 1.1G /var/lib/glance/node/2/quarantined
 330G /var/lib/glance/node/2

 root@cic-2:~/temp# file /var/lib/glance/node/1/tmp/001fc080-8e61-4a33-9407-28a2cbceeb06
 /var/lib/glance/node/1/tmp/001fc080-8e61-4a33-9407-28a2cbceeb06: SQLite 3.x database
 root@cic-2:~/temp#
 root@cic-2:~/temp# sqlite3 /var/lib/glance/node/1/tmp/001fc080-8e61-4a33-9407-28a2cbceeb06
 SQLite version 3.8.2 2013-12-06 14:53:30
 Enter ".help" for instructions
 Enter SQL statements terminated with a ";"
 sqlite> .databases
 seq name file
 --- --------------- ----------------------------------------------------------
 0 main /var/lib/glance/node/1/tmp/001fc080-8e61-4a33-9407-28a2cbc
 sqlite> .tables
 container_info incoming_sync outgoing_sync
 container_stat object policy_stat
 sqlite>
 sqlite>
 sqlite> .dump
 PRAGMA foreign_keys=OFF;
 BEGIN TRANSACTION;
 CREATE TABLE outgoing_sync (
 remote_id TEXT UNIQUE,
 sync_point INTEGER,
 updated_at TEXT DEFAULT 0
 );
 CREATE TABLE incoming_sync (
 remote_id TEXT UNIQUE,
 sync_point INTEGER,
 updated_at TEXT DEFAULT 0
 );
 INSERT INTO "incoming_sync" VALUES('b0f296b0-287f-4d49-95ee-34dbb3244ed5',276,'1573295748');
 INSERT INTO "incoming_sync" VALUES('6bee0853-5c40-4ef9-a784-b23815c6c577',276,'1573295777');
 INSERT INTO "incoming_sync" VALUES('5a925e82-3ade-4e0e-9590-d69555dc7e63',276,'1573295809');
 INSERT INTO "incoming_sync" VALUES('ac1d64fd-4455-4771-af03-fa55792d5637',276,'1573295822');
 INSERT INTO "incoming_sync" VALUES('cc1a1c4f-efc1-4533-9a81-1cac510b0782',276,'1573295837');
 INSERT INTO "incoming_sync" VALUES('9f53cd44-4522-4fdc-96af-85cf5ffb482d',276,'1573295852');
 INSERT INTO "incoming_sync" VALUES('ea328e84-56ad-4071-a7d5-d1c4f5bee78d',276,'1573295867');
 INSERT INTO "incoming_sync" VALUES('12915aa8-00c9-453b-9be1-3bacc2f4dcca',276,'1573295897');
 INSERT INTO "incoming_sync" VALUES('9592623e-6c9c-4943-a913-fd64c2fe5a17',276,'1573295928');
 INSERT INTO "incoming_sync" VALUES('923e7465-b641-4aba-8bcf-4f266a82c997',276,'1573295957');
 INSERT INTO "incoming_sync" VALUES('c3ff4ff1-4ddb-4690-9e86-4d7c8b55684e',276,'1573295988');

Steps to reproduce:
 unknown
Expected results:
 These files should clear by itself.
Actual result:
 Files keeps creating until disk is completely full.
Reproducibility:
 unknown
Workaround:
 Delete tmp files (without certainty that it is safe).
Impact:
 No space left on device
Description of the environment:
 Operation system: Ubuntu 14.04
 Versions of components: MOS 9+ (code snapshot of August 2018), Mitaka
 Network model: Neutron + OVS

Changed in fuel:
milestone: none → 9.x-updates
assignee: nobody → MOS Maintenance (mos-maintenance)
importance: Undecided → Critical
status: New → Confirmed
tags: added: customer-found sla2
Changed in fuel:
assignee: MOS Maintenance (mos-maintenance) → Oleksiy Molchanov (omolchanov)
status: Confirmed → In Progress
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Might be connected with this one: https://bugs.launchpad.net/swift/+bug/1691566

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/swift (9.0/mitaka)

Fix proposed to branch: 9.0/mitaka
Change author: Denis Meltsaykin <email address hidden>
Review: https://review.fuel-infra.org/41575

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.