Need to increase size of RAM for virtual machines to deploy_bonding_neutron_vlan

Bug #1587466 reported by Alexandr Kostrikov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Alexandr Kostrikov
Mitaka
Fix Released
High
Alexandr Kostrikov

Bug Description

During swarm test run I found that job has been failed.

After revert I found that virtual machine with controller is under high load (even show as offline).

Before ssh to node:
[root@nailgun ~]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---|--------|---------------------|---------|-------------|-------------------|------------|---------------|--------|---------
5 | ready | slave-05_cinder | 1 | 10.109.12.7 | 64:16:93:f3:ce:8c | cinder | | True | 1
2 | ready | slave-01_controller | 1 | 10.109.12.3 | 64:32:ce:3a:b7:69 | controller | | False | 1
3 | ready | slave-03_controller | 1 | 10.109.12.5 | 64:dd:45:92:d6:3f | controller | | True | 1
1 | ready | slave-02_controller | 1 | 10.109.12.4 | 64:00:5b:2f:23:3c | controller | | True | 1
4 | ready | slave-04_compute | 1 | 10.109.12.6 | 64:9a:0b:f6:27:26 | compute | | True | 1

When I did ssh to node - it took a long time and htop shown high load.
After some time it has been shown as online.
[root@nailgun ~]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---|--------|---------------------|---------|-------------|-------------------|------------|---------------|--------|---------
5 | ready | slave-05_cinder | 1 | 10.109.12.7 | 64:16:93:f3:ce:8c | cinder | | True | 1
2 | ready | slave-01_controller | 1 | 10.109.12.3 | 64:32:ce:3a:b7:69 | controller | | True | 1
3 | ready | slave-03_controller | 1 | 10.109.12.5 | 64:dd:45:92:d6:3f | controller | | True | 1
1 | ready | slave-02_controller | 1 | 10.109.12.4 | 64:00:5b:2f:23:3c | controller | | True | 1
4 | ready | slave-04_compute | 1 | 10.109.12.6 | 64:9a:0b:f6:27:26 | compute | | True | 1

Revision history for this message
Alexandr Kostrikov (akostrikov-mirantis) wrote :
tags: removed: swarm-blocker
Revision history for this message
Alexandr Kostrikov (akostrikov-mirantis) wrote :

That affected only one test, removing swarm-blocker.

Ilya Kutukov (ikutukov)
Changed in fuel:
status: New → Confirmed
tags: added: swarm-fail
Revision history for this message
Alexandr Kostrikov (akostrikov-mirantis) wrote :

I have run following command: `ps -eLo pid,ppid,rss,cmd | sort -rnk3`
Output is at http://paste.openstack.org/show/508562/

Please, nova team, prove that such resource consumption by nova-api is acceptable and we should increase memory on that job. Or it is abnormal and nova-api should be tweaked.

Changed in fuel:
assignee: Alexandr Kostrikov (akostrikov-mirantis) → MOS Nova (mos-nova)
Revision history for this message
Alexandr Kostrikov (akostrikov-mirantis) wrote :

The failure is reproducible, feel free to request for env at https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.bonding_ha/

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Alenxandr, I'd say anything below ~200 Mb is perfectly normal - this is what you see after nova-api imports all the modules used. Short spikes of memory usage above 200 Mb is also ok, as long as it goes back to the previous level after processing of incoming requests.

Changed in fuel:
assignee: MOS Nova (mos-nova) → Alexandr Kostrikov (akostrikov-mirantis)
Revision history for this message
Andrey Sledzinskiy (asledzinskiy) wrote :

Set to won't fix for 9.0, we won't be able to make a decision about updating ram before HCF

Changed in fuel:
milestone: 9.0 → 10.0
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to fuel-infra/jenkins-jobs (master)

Fix proposed to branch: master
Change author: Alexandr Kostrikov <email address hidden>
Review: https://review.fuel-infra.org/21936

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to fuel-infra/jenkins-jobs (master)

Reviewed: https://review.fuel-infra.org/21936
Submitter: Lesya Novaselskaya <email address hidden>
Branch: master

Commit: 842b387bd5c571571822f2d9890cf6934ba00e02
Author: Alexandr Kostrikov <email address hidden>
Date: Fri Jun 10 11:22:35 2016

Increase memory for bonding tests

There is lack of resources on controllers
to check ha with bonding.

Change-Id: Icd0eefce98967ce09bc4d5cba501986659bb1ede
Closes-bug: #1587466

Changed in fuel:
status: Confirmed → Fix Committed
Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.