Not enough space for generating the backup using 'dockerctl backup'

Bug #1540331 reported by Vladimir Khlyunev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Matthew Mosesohn
8.0.x
Fix Released
High
Matthew Mosesohn
Mitaka
Invalid
High
Matthew Mosesohn

Bug Description

ISO 9.0-164

Steps to reproduce:
1) Deploy 9.0 fuel
2) (optional) - Deploy any cluster
3) On master node run "dockerctl backup" without args
OR
3) On master node run "dockerctl backup /root"

Result:
Saving containers to combined archive...
time="2016-02-01T12:03:20Z" level=debug msg="[stream] End of stdout"
write /dev/stdout: no space left on device
OR
tar: /var/log/fuel_backup_2016-02-01_1032.tar: Wrote only 2048 of 10240 bytes

It looks like there is not enough space for generating the archive:
df -h before backup:
/dev/mapper/os-root 9.5G 1.4G 7.6G 16% /
/dev/mapper/os-var 9.5G 5.3G 3.8G 59% /var

But when I used /var/log for generating the backup - all is fine.
/dev/mapper/os-varlog 32G 4.1G 27G 14% /var/log

Snapshot is not available due to exhausted space in /var directory.

no longer affects: fuel/future
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/274657

Changed in fuel:
status: New → In Progress
tags: added: team-bugfix
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/274896

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/274657
Committed: https://git.openstack.org/cgit/openstack/fuel-main/commit/?id=10b609078e81b3fc704ac8aa39f41c463c56af76
Submitter: Jenkins
Branch: master

commit 10b609078e81b3fc704ac8aa39f41c463c56af76
Author: Matthew Mosesohn <email address hidden>
Date: Mon Feb 1 16:36:49 2016 +0300

    Balance /var and /var/log disk partitions

    Prior to Fuel 8.0, there was a balance of the
    following directories for remaining space:
    /var, /var/lib/docker, /var/log

    Now docker has its own pv, and /var/log was
    incorrectly given all the remaining space,
    leaving /var fixed at 10GB.

    The new balance is as follows:
    docker pv: 20gb
    /: 10gb fixed size
    /var: 40% of remaining space
    /var/log: 60% of remaining space

    Also raised minimum disk space requirement to
    45GB. This reflects the actual minimum sizes
    so that a user receives the error before partitioning
    and is properly informed.

    Change-Id: Id283e7dd4b94d5f6e1eb8397af33fea2b9f985f7
    Closes-Bug: #1540331

Andrey Maximov (maximov)
tags: added: hit-hcf
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

This bug will be merged for HCF tomorrow.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (stable/8.0)

Reviewed: https://review.openstack.org/274896
Committed: https://git.openstack.org/cgit/openstack/fuel-main/commit/?id=c5aac6c0b8d2a37147fdd6e6b78df4ad2ad90e31
Submitter: Jenkins
Branch: stable/8.0

commit c5aac6c0b8d2a37147fdd6e6b78df4ad2ad90e31
Author: Matthew Mosesohn <email address hidden>
Date: Mon Feb 1 16:36:49 2016 +0300

    Balance /var and /var/log disk partitions

    Prior to Fuel 8.0, there was a balance of the
    following directories for remaining space:
    /var, /var/lib/docker, /var/log

    Now docker has its own pv, and /var/log was
    incorrectly given all the remaining space,
    leaving /var fixed at 10GB.

    The new balance is as follows:
    docker pv: 20gb
    /: 10gb fixed size
    /var: 40% of remaining space
    /var/log: 60% of remaining space

    Also raised minimum disk space requirement to
    45GB. This reflects the actual minimum sizes
    so that a user receives the error before partitioning
    and is properly informed.

    Change-Id: Id283e7dd4b94d5f6e1eb8397af33fea2b9f985f7
    Closes-Bug: #1540331

tags: added: on-verification
Revision history for this message
Dmitriy Kruglov (dkruglov) wrote :

Docker removal in 9.0 makes the original scenario not applicable for MOS 9.0.

tags: removed: on-verification
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.