Performance degradation of scenario "create and delete subnets"

Bug #1623009 reported by Andriy Kurilin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Expired
High
Unassigned

Bug Description

Rally has a scenario NeutronNetworks.create_and_delete_subnets .
Logic of setUp method of scenario:
 - create network

Logic of scenario:
 - create 2 subnets in network
 - delete subnets

We configured this scenario to be launched with concurrency 10. Yesturday, we got first failure(like [1]).
First failed results of jenkins was posted at Sep 12 8:08 PM (UTC +3).
Last found success jobs was started at Sep 12 3:32 PM (UTC +3)

Recheck did not help(example https://review.openstack.org/#/c/362924/ gate-rally-dsvm-neutron-rally job)

Workaround: Reduce concurrency to 5.

[1] - http://logs.openstack.org/24/362924/2/check/gate-rally-dsvm-neutron-rally/39d985b/rally-plot/results.html.gz#/NeutronNetworks.create_and_delete_subnets/overview

Tags: loadimpact
Changed in neutron:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/369410

description: updated
Changed in neutron:
status: New → Confirmed
Revision history for this message
Khushbuparakh (khushbuparakh) wrote :

Why is the failure is it because the new subnets that are being created matches with the existing subnets in the list.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (master)

Change abandoned by Armando Migliaccio (<email address hidden>) on branch: master
Review: https://review.openstack.org/369410
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :

I recall that Kevin filed a bunch of fixes to bring rally back to sanity right about the time for feature freeze [2]. The rally job is a bit of a roller coaster so it's difficult to confirm this without further evidence.

[1] https://launchpad.net/neutron/+milestone/newton-rc1
[2] http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=11&fullscreen

tags: added: loadimpact
Changed in neutron:
status: Confirmed → Incomplete
Revision history for this message
Khushbuparakh (khushbuparakh) wrote :

I would like to work on it getting more details about reasons for failure and new fixes.

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for neutron because there has been no activity for 60 days.]

Changed in neutron:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.