Nova should reserve some memory when sharing node with Ceph/OSD role

Bug #1507306 reported by Eugene Korekin
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Medium
Michael Polenchuk

Bug Description

We always have value reserved_host_memory_mb in /etc/nova/nova.conf set to 512.
We probably should reserve more memory if compute node has additional roles.
For example, Ceph/OSD hosts are recommended to have 1GB of RAM per OSD daemon instance.

As it is now Ceph can have a very negative influence on compute node performance.

Changed in fuel:
assignee: nobody → Fuel Library Team (fuel-library)
importance: Undecided → Medium
tags: added: feature
Changed in fuel:
milestone: none → 8.0
status: New → Confirmed
Dmitry Pyzhov (dpyzhov)
tags: added: area-library
Changed in fuel:
assignee: Fuel Library Team (fuel-library) → Michael Polenchuk (mpolenchuk)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/240204

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-library (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/251366

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/240204
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=5199ed004c7e99ce9c847fcba2563bd57525b4cb
Submitter: Jenkins
Branch: master

commit 5199ed004c7e99ce9c847fcba2563bd57525b4cb
Author: Michael Polenchuk <email address hidden>
Date: Thu Oct 29 14:42:57 2015 +0300

    Get reserved host memory for nova-compute

    Get reserved host memory straight value for nova-compute if we have Ceph
    OSD neighbor resides on the same node.
    The formula is:
      * min default value = 512Mb
      * max value = 1.5Gb
      * straight value = 20% of total memory

    Closes-Bug: #1507306
    Change-Id: I6a58095825d248ba40d6cecd983812dd3e17abb7

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/251366
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=15564db03e60205934f8d7fa1f4bf143c9a67c06
Submitter: Jenkins
Branch: master

commit 15564db03e60205934f8d7fa1f4bf143c9a67c06
Author: Michael Polenchuk <email address hidden>
Date: Mon Nov 30 15:13:37 2015 +0300

    Align method to get value within the range

    Use min(max()) method instead of sort[1].

    Change-Id: I25e7379b92d5627a79b6f4a5f67cc77152e43480
    Related-Bug: #1507306

tags: added: on-verification
Revision history for this message
Mikhail Samoylov (msamoylov) wrote :

Steps for verification:
1. Create env
2. Add to env 1 controller, 2 compute with ceph-osd and 1 compute
3. Verify networks
4. Deploy cluster
5. On compute nodes check /etc/nova/nova.conf | grep reserved_host_memory_mb

Expected result for step 5: at least 512 Mb memory

root@node-7:~# cat /etc/nova/nova.conf | grep reserved_host_memory_mb
reserved_host_memory_mb=592

root@node-8:~# cat /etc/nova/nova.conf | grep reserved_host_memory_mb
reserved_host_memory_mb=512

root@node-6:~# cat /etc/nova/nova.conf | grep reserved_host_memory_mb
reserved_host_memory_mb=592

Verification on fuel version:
[root@nailgun ~]# cat /etc/fuel/version.yaml
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  openstack_version: "2015.1.0-8.0"
  api: "1.0"
  build_number: "269"
  build_id: "269"
  fuel-nailgun_sha: "218e8a2ab3dc95367fa3e6de79fe506c06b7c861"
  python-fuelclient_sha: "f033192b84263f0e699458a4274289a5198ae7e4"
  fuel-agent_sha: "660c6514caa8f5fcd482f1cc4008a6028243e009"
  fuel-nailgun-agent_sha: "a33a58d378c117c0f509b0e7badc6f0910364154"
  astute_sha: "48fd58676debcc85951db68df6d77c22daa55e52"
  fuel-library_sha: "6a5f6c0afbe6a9bcd2ef40657de371ada5f966ee"
  fuel-ostf_sha: "23b7ae2a1a57de5a3e1861ffb7805394ca339cc2"
  fuel-mirror_sha: "c66309976db831a4081e5e0064028d7c67014b2f"
  fuelmenu_sha: "761685b566af981479e7e0f3c3a5eede5a40400d"
  shotgun_sha: "a0bd06508067935f2ae9be2523ed0d1717b995ce"
  network-checker_sha: "a3534f8885246afb15609c54f91d3b23d599a5b1"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "d4d93fec7f0c35b18c50431e8bf3eae70a6deb27"

[root@nailgun ~]# fuel node
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---|----------|------------------|---------|-------------|-------------------|-------------------|---------------|--------|---------
13 | discover | Untitled (c1:29) | None | 10.109.5.10 | 64:be:e6:9e:c1:29 | | | True | None
11 | discover | Untitled (d5:d9) | None | 10.109.5.6 | 64:fd:36:5d:d5:d9 | | | True | None
6 | ready | Untitled (ee:50) | 3 | 10.109.5.8 | 3e:66:af:0b:ca:bb | ceph-osd, compute | | True | 3
7 | ready | Untitled (d1:7a) | 3 | 10.109.5.4 | 3a:76:bf:0a:7c:78 | ceph-osd, compute | | True | 3
9 | ready | Untitled (24:a3) | 3 | 10.109.5.7 | 64:e1:22:f9:24:a3 | controller | | True | 3
14 | discover | Untitled (a2:6d) | None | 10.109.5.5 | 64:74:52:3b:a2:6d | | | True | None
8 | ready | Untitled (a9:08) | 3 | 10.109.5.9 | 3e:af:6f:1a:56:0c | compute | | True | 3
10 | discover | Untitled (69:34) | None | 10.109.5.11 | 64:f1:d1:9b:69:34 | | | True | None
12 | discover | Untitled (a0:c0) | None | 10.109.5.3 | 64:bc:32:c3:a0:c0 | | | True | None

tags: removed: on-verification
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.