swift data is stored in root partition

Bug #1508482 reported by Matthew Mosesohn on 2015-10-21
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
High
Ivan Berezovskiy
7.0.x
High
Michael Polenchuk

Bug Description

Swift data is in /srv/node and should be located on a separate partition. Probably combined with glance (as we did in the past) is the best option. Using /srv/node on root partition could easily lead to root partition filling up and breaking the controller.

Changed in fuel:
assignee: nobody → Fuel Library Team (fuel-library)
status: New → Confirmed
importance: Undecided → High
milestone: none → 8.0
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Kyrylo Galanov (kgalanov)
Dmitry Pyzhov (dpyzhov) on 2015-10-27
tags: added: area-library
Changed in fuel:
assignee: Kyrylo Galanov (kgalanov) → Ivan Berezovskiy (iberezovskiy)

Fix proposed to branch: master
Review: https://review.openstack.org/239928

Changed in fuel:
status: Confirmed → In Progress

Reviewed: https://review.openstack.org/239928
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=77316ef2849437cc2842bd7ddf1322841524dbc2
Submitter: Jenkins
Branch: master

commit 77316ef2849437cc2842bd7ddf1322841524dbc2
Author: iberezovskiy <email address hidden>
Date: Wed Oct 28 16:07:13 2015 +0300

    Set swift_partition to /var/lib/glance/node by default

    We had an ability to set swift partition in modular
    script, but it was removed in this commit:
    Ia2428b49a7192a2d7d56ffdc61a75de9ebc816c9

    We should use /var/lib/glance/node as swift partition
    by default as it was before and provide access
    to this folder for swift user.
    Also noop tests were added to cover this parameter.

    Change-Id: Ic38c726d5ae9fff1815e6a8b4158549c09cec84c
    Closes-bug: #1508482

Changed in fuel:
status: In Progress → Fix Committed
tags: added: customer-found
tags: added: sla1
Anton Matveev (amatveev) wrote :

above tags relevant for 7.0 version

Alexander Rubtsov (arubtsov) wrote :

For the Maintenance team regarding 7.0 MU

Steps to reproduce
1. Deploy MOS 7.0 environment (HA, 3 Controller nodes, without Ceph)
2. Create a 3Gb size file on a Controller node
dd if=/dev/urandom of=/root/test.img bs=128M count=24
3. Check free space on the "/" partition on Controller nodes. Run from the Fuel master node:
for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
For example:
node-5
/dev/mapper/os-root 15G 6.3G 7.7G 46% /
node-1
/dev/mapper/os-root 15G 7.4G 6.6G 53% /
node-2
/dev/mapper/os-root 15G 4.3G 9.7G 31% /
4. Upload the file to Swift
swift upload mycont /root/test.img

Expected result
Free space on the "/" partition on Controller nodes is not changed
for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
For example:
node-5
/dev/mapper/os-root 15G 6.3G 7.7G 46% /
node-1
/dev/mapper/os-root 15G 7.4G 6.6G 53% /
node-2
/dev/mapper/os-root 15G 4.3G 9.7G 31% /

Actual result
Free space on the "/" partition on Controller nodes is reduced
For example:
node-5
/dev/mapper/os-root 15G 9.3G 4.7G 67% /
node-1
/dev/mapper/os-root 15G 11G 3.6G 75% /
node-2
/dev/mapper/os-root 15G 7.3G 6.7G 53% /

Reviewed: https://review.openstack.org/254208
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=6afb21a863282f654b4360e4be4be7e072b3a75d
Submitter: Jenkins
Branch: stable/7.0

commit 6afb21a863282f654b4360e4be4be7e072b3a75d
Author: Michael Polenchuk <email address hidden>
Date: Mon Dec 7 15:45:39 2015 +0300

    Set swift_partition to /var/lib/glance/node by default

    We should use /var/lib/glance/node as swift partition
    by default as it was before and provide access
    to this folder for swift user.
    Also noop tests were added to cover this parameter.

    Closes-bug: #1508482
    Change-Id: I928a87c0ad623f4d5a5dbcaaf8a71e3290d68d01
    (cherry picked from commit 77316ef2849437cc2842bd7ddf1322841524dbc2)

tags: added: on-verification
Ksenia Svechnikova (kdemina) wrote :

Verify with ISO#284 MOS 8.0

Steps:

Steps to reproduce
1. Deploy MOS 8.0 environment (HA, 3 Controller nodes, without Ceph)
2. Create a 3Gb size file on a Controller node
dd if=/dev/urandom of=/root/test.img bs=128M count=24
3. Check free space on the "/" partition on Controller nodes. Run from the Fuel master node:

[root@nailgun ~]# for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
node-3
Warning: Permanently added 'node-3' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.2G 12G 16% /
node-1
Warning: Permanently added 'node-1' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.3G 12G 17% /
node-2
Warning: Permanently added 'node-2' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 5.2G 8.7G 38% /

4. Upload the file to Swift
swift upload mycont /root/test.img

5. Check free space on the "/" partition again

[root@nailgun ~]# for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
node-3
Warning: Permanently added 'node-3' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.2G 12G 16% /
node-1
Warning: Permanently added 'node-1' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.3G 12G 17% /
node-2
Warning: Permanently added 'node-2' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 5.2G 8.7G 38% /

Changed in fuel:
status: Fix Committed → Fix Released
tags: removed: on-verification
Denis Puchkin (dpuchkin) on 2015-12-28
tags: added: on-verification
Denis Puchkin (dpuchkin) wrote :

Verified on 7.0

with fuel-library7.0-7.0.0-7251.2.gerrit241304.4.git2626eee

tags: removed: on-verification
Roman Podoliaka (rpodolyaka) wrote :

Release notes:

Swift data is now stored in a separate disk partition, not in the root one.

tags: added: release-notes
tags: added: 8.0 release-notes-done
removed: release-notes
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers