Trigger garbage collection on patch upload to free rootfs disk space

Bug #1797977 reported by Ghada Khalil
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Don Penney

Bug Description

Brief Description
-----------------
When attempting to upload & apply a large number of patches before initial commissioning, the patches were left in a 'Partial-Apply' instead of being in 'Applied' state.

In this case, the rootfs was completely full when the install-local was attempted and eventually it failed with the below error

error: installing package qemu-kvm-tools-ev-10:2.6.0-28.el7_3.9.1.tis.67.x86_64 needs 2MB on the / filesystem
error: installing package nfv-1.0-61.tis.x86_64 needs 2MB on the / filesystem
error: installing package TC_17.06_PATCH_0002-1.0-0.tis.x86_64 needs 2MB on the / filesystem
error: installing package puppet-drbd-0.3.1-rc0.tis.9.noarch needs 2MB on the / filesystem
error: installing package TC_17.06_PATCH_0008-1.0-0.tis.x86_64 needs 2MB on the / filesystem
error: installing package cgts-client-1.0-31.tis.x86_64 needs 4MB on the / filesystem
error: installing package kernel-tools-libs-3.10.0-693.21.1.el7.32.tis.x86_64 needs 23MB on the / filesystem
error: installing package TC_17.06_PATCH_0019-1.0-0.tis.x86_64 needs 23MB on the / filesystem
error: installing package python-ceilometer-1:7.0.0-1.tis.47.noarch needs 29MB on the / filesystem

Severity
--------
Minor - the likelihood of this occurring is pretty low

Steps to Reproduce
------------------
Apply a large number of patches as part of initial commissioning

Expected Behavior
------------------
Patch application passes

Actual Behavior
----------------
Patch application failed

Reproducibility
---------------
This is dependent on the number of patches being applied and is not easily reproducible

System Configuration
--------------------
Any

Branch/Pull Time/Commit
-----------------------
This can happen with any starlingx build

Timestamp/Logs
--------------
Not required

Ghada Khalil (gkhalil)
Changed in starlingx:
assignee: nobody → Don Penney (dpenney)
importance: Undecided → Low
status: New → Triaged
tags: added: stx.2019.03 stx.update
Ken Young (kenyis)
tags: added: stx.2019.05
removed: stx.2019.03
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to stx-update (master)

Fix proposed to branch: master
Review: https://review.openstack.org/642526

Changed in starlingx:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to stx-update (master)

Reviewed: https://review.openstack.org/642526
Committed: https://git.openstack.org/cgit/openstack/stx-update/commit/?id=0ea37b36a11fc556f4e7ac9c56fde33e150a75ed
Submitter: Zuul
Branch: master

commit 0ea37b36a11fc556f4e7ac9c56fde33e150a75ed
Author: Don Penney <email address hidden>
Date: Mon Mar 11 10:52:00 2019 -0400

    Trigger garbage collection to free WSGI resources

    The wsgiref.simple_server is used by patching for handling
    API requests. When uploading files, the server opens a file
    handle for the temporary resource, but does not close it.
    Instead, it's left to periodic garbage collection to free the
    resources. Until garbage collection, however, this means
    disk space is still in use for the deleted temporary file,
    as the handle is left open.

    This update adds a call to gc.collect() after the call
    to simple_server.handle_request() to immediately free all
    unused resources.

    Change-Id: Ie39213dad540448cede46cc8e580d31582419dcc
    Closes-Bug: 1797977
    Signed-off-by: Don Penney <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Ken Young (kenyis)
tags: added: stx.2.0
removed: stx.2019.05
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.