swift data is stored in root partition

Bug #1508482 reported by Matthew Mosesohn
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Ivan Berezovskiy
7.0.x
Fix Released
High
Michael Polenchuk

Bug Description

Swift data is in /srv/node and should be located on a separate partition. Probably combined with glance (as we did in the past) is the best option. Using /srv/node on root partition could easily lead to root partition filling up and breaking the controller.

Changed in fuel:
assignee: nobody → Fuel Library Team (fuel-library)
status: New → Confirmed
importance: Undecided → High
milestone: none → 8.0
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Kyrylo Galanov (kgalanov)
Dmitry Pyzhov (dpyzhov)
tags: added: area-library
Changed in fuel:
assignee: Kyrylo Galanov (kgalanov) → Ivan Berezovskiy (iberezovskiy)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/239928

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/239928
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=77316ef2849437cc2842bd7ddf1322841524dbc2
Submitter: Jenkins
Branch: master

commit 77316ef2849437cc2842bd7ddf1322841524dbc2
Author: iberezovskiy <email address hidden>
Date: Wed Oct 28 16:07:13 2015 +0300

    Set swift_partition to /var/lib/glance/node by default

    We had an ability to set swift partition in modular
    script, but it was removed in this commit:
    Ia2428b49a7192a2d7d56ffdc61a75de9ebc816c9

    We should use /var/lib/glance/node as swift partition
    by default as it was before and provide access
    to this folder for swift user.
    Also noop tests were added to cover this parameter.

    Change-Id: Ic38c726d5ae9fff1815e6a8b4158549c09cec84c
    Closes-bug: #1508482

Changed in fuel:
status: In Progress → Fix Committed
tags: added: customer-found
tags: added: sla1
Revision history for this message
Anton Matveev (amatveev) wrote :

above tags relevant for 7.0 version

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/7.0)

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/254208

Revision history for this message
Alexander Rubtsov (arubtsov) wrote :

For the Maintenance team regarding 7.0 MU

Steps to reproduce
1. Deploy MOS 7.0 environment (HA, 3 Controller nodes, without Ceph)
2. Create a 3Gb size file on a Controller node
dd if=/dev/urandom of=/root/test.img bs=128M count=24
3. Check free space on the "/" partition on Controller nodes. Run from the Fuel master node:
for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
For example:
node-5
/dev/mapper/os-root 15G 6.3G 7.7G 46% /
node-1
/dev/mapper/os-root 15G 7.4G 6.6G 53% /
node-2
/dev/mapper/os-root 15G 4.3G 9.7G 31% /
4. Upload the file to Swift
swift upload mycont /root/test.img

Expected result
Free space on the "/" partition on Controller nodes is not changed
for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
For example:
node-5
/dev/mapper/os-root 15G 6.3G 7.7G 46% /
node-1
/dev/mapper/os-root 15G 7.4G 6.6G 53% /
node-2
/dev/mapper/os-root 15G 4.3G 9.7G 31% /

Actual result
Free space on the "/" partition on Controller nodes is reduced
For example:
node-5
/dev/mapper/os-root 15G 9.3G 4.7G 67% /
node-1
/dev/mapper/os-root 15G 11G 3.6G 75% /
node-2
/dev/mapper/os-root 15G 7.3G 6.7G 53% /

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/7.0)

Reviewed: https://review.openstack.org/254208
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=6afb21a863282f654b4360e4be4be7e072b3a75d
Submitter: Jenkins
Branch: stable/7.0

commit 6afb21a863282f654b4360e4be4be7e072b3a75d
Author: Michael Polenchuk <email address hidden>
Date: Mon Dec 7 15:45:39 2015 +0300

    Set swift_partition to /var/lib/glance/node by default

    We should use /var/lib/glance/node as swift partition
    by default as it was before and provide access
    to this folder for swift user.
    Also noop tests were added to cover this parameter.

    Closes-bug: #1508482
    Change-Id: I928a87c0ad623f4d5a5dbcaaf8a71e3290d68d01
    (cherry picked from commit 77316ef2849437cc2842bd7ddf1322841524dbc2)

tags: added: on-verification
Revision history for this message
Ksenia Svechnikova (kdemina) wrote :

Verify with ISO#284 MOS 8.0

Steps:

Steps to reproduce
1. Deploy MOS 8.0 environment (HA, 3 Controller nodes, without Ceph)
2. Create a 3Gb size file on a Controller node
dd if=/dev/urandom of=/root/test.img bs=128M count=24
3. Check free space on the "/" partition on Controller nodes. Run from the Fuel master node:

[root@nailgun ~]# for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
node-3
Warning: Permanently added 'node-3' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.2G 12G 16% /
node-1
Warning: Permanently added 'node-1' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.3G 12G 17% /
node-2
Warning: Permanently added 'node-2' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 5.2G 8.7G 38% /

4. Upload the file to Swift
swift upload mycont /root/test.img

5. Check free space on the "/" partition again

[root@nailgun ~]# for i in `fuel node | grep "controller" | awk '{print $1}'`; do echo node-${i}; ssh node-${i} "df -h | grep /$"; done
node-3
Warning: Permanently added 'node-3' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.2G 12G 16% /
node-1
Warning: Permanently added 'node-1' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 2.3G 12G 17% /
node-2
Warning: Permanently added 'node-2' (ECDSA) to the list of known hosts.
/dev/dm-3 15G 5.2G 8.7G 38% /

Changed in fuel:
status: Fix Committed → Fix Released
tags: removed: on-verification
Denis Puchkin (dpuchkin)
tags: added: on-verification
Revision history for this message
Denis Puchkin (dpuchkin) wrote :

Verified on 7.0

with fuel-library7.0-7.0.0-7251.2.gerrit241304.4.git2626eee

tags: removed: on-verification
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Release notes:

Swift data is now stored in a separate disk partition, not in the root one.

tags: added: release-notes
tags: added: 8.0 release-notes-done
removed: release-notes
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.