v11.0 - Repo Containers - No space left on device

[Byron McCollum]

LVM backed repo containers run out of space during rsync from upstream repo. The default container size is 5G, which is an insufficient amount of space for the ever growing upstream repo.

[Nolan Brubaker (nolan-brubaker)]

Is this specifically on an all-in-one install in your case?

[Byron McCollum (byron-mccollum)]

Not an AIO.

I destroyed the repo container and re-ran the install multiple times, each resulting in running out of space inside the container. The entire upstream repo seem to be rsync'd, which i don't know if that is by design, but the total sum of the upstream repo now surpasses the available space in the repo container.

Again, this is for LXC backed containers. The default size is 5G. After updated env.d/pkg_repo.yml and bumping container_fs_size to 10G, rsync was able to finish...

[Byron McCollum (byron-mccollum)]

s/LXC/LVM

[Kevin Carter (kevin-carter)]

This issue was also reported by gus (IRC nic) and is related to how the rsync clone method is being implemented. To resolve this immediately we need to change the exclude stanzas such that they are truly excluding files / directories as intended however we may also want to take this a step further in the near future to remove the clone process from upstream and only rely on building the wheels as needed. A purpose built repo is < 2GB of space while the repo clone will be a never ending expansion of storage as the releases pile on (as mentioned by @byron-mccollum) and filtering is only going to get us so far.

In the mean time being that you are using lvm backed containers you can simply resize the logical volume to a more appropriate size IE:# lvresize -L+5G /dev/$LVNAME which will ensure that the clone operation is successful.

[He Weiwei (heww0205)]

For deploy OpenStack, only os-releases dir in repo is required, and now I disable the repo containers and provide python packages under os-releases by Nginx myself.

[Kevin Carter (kevin-carter)]

This PR fixes this issue and is presently in review https://review.openstack.org/#/c/195399/

[Nolan Brubaker (nolan-brubaker)]

Do we need to also rotate logs on the repo containers, too? This patch fixes initial deployment, but how about as the container ages?

[Nolan Brubaker (nolan-brubaker)]

Nevermind on the log rotation - the logs are shipped out through rsyslog and thus should not be in the root partition for the container.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: kilo
Review: https://review.openstack.org/198014

[Kevin Carter (kevin-carter)]

Kilo BP in progress https://review.openstack.org/#/c/198014/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/198014
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=2f602e5cb1e969c5de260e36b930d531ac60fa21
Submitter: Jenkins
Branch: kilo

commit 2f602e5cb1e969c5de260e36b930d531ac60fa21
Author: kevin <email address hidden>
Date: Wed Jun 24 22:42:41 2015 -0500

    Updated the clone process to exclude unneeded files

    This commit changes the clone process to properly exclude files from the repo
    clone play/task. This also ensures that when this play is used that all local
    repos are an exact match to what is upstream.

    Change-Id: I840997d6778f40a9007bce18df792c6b0c69c5eb
    Closes-Bug: #1468567
    (cherry picked from commit 955c851cddcc6876fc2b98e9cdf423688280d31c)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.