NovaFlavors reset quota to 0

[Stéphane NGUYEN]

When running a nova create_and_list_flavor_access and create_flavor scenarios, Rally's cleanup resets the quota to 0 of existing users. Before the run of Rally, the quota is OK.

Log :

2016-08-17 08:53:32.786 28 RALLYDEBUG rally.task.runner [-] Starting the constant runner (task UUID: 0be1976b-e144-4e93-82ec-85dcfce0ed56).
 timeout: 0
 concurrency: 2
 concurrency_per_worker: 1
 times: 5
 concurrency_overhead: 0
 processes_to_start: 2
 max_cpu_used: 4
2016-08-17 08:53:32.792 48 RALLYDEBUG rally.task.runner [-] Starting a worker.
 concurrency: 1
 args: {'vcpus': 1, 'disk': 1, 'ram': 500}
 times: 5
 method_name: create_flavor
 timeout: 0
 cls: <class 'rally.plugins.openstack.scenarios.nova.flavors.NovaFlavors'>
2016-08-17 08:53:32.795 49 RALLYDEBUG rally.task.runner [-] Starting a worker.
 concurrency: 1
 args: {'vcpus': 1, 'disk': 1, 'ram': 500}
 times: 5
 method_name: create_flavor
 timeout: 0
 cls: <class 'rally.plugins.openstack.scenarios.nova.flavors.NovaFlavors'>
2016-08-17 08:53:32.797 48 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 0 START
2016-08-17 08:53:32.804 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 1 START
2016-08-17 08:53:33.358 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 1 END: OK
2016-08-17 08:53:33.366 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 2 START
2016-08-17 08:53:33.406 48 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 0 END: OK
2016-08-17 08:53:33.412 48 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 3 START
2016-08-17 08:53:33.701 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 2 END: OK
2016-08-17 08:53:33.710 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 4 START
2016-08-17 08:53:33.777 48 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 3 END: OK
2016-08-17 08:53:34.075 49 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 4 END: OK
2016-08-17 08:53:34.084 28 INFO rally.plugins.openstack.context.cleanup.admin [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | Starting: admin resources cleanup
2016-08-17 08:53:34.085 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Cleaning up nova quotas objects
2016-08-17 08:53:34.086 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Deleting nova quotas object (7753f51a98224a6db2e6750ba3dd0276)
2016-08-17 08:53:34.093 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Deleting nova quotas object (27fdf4fea26343e3922291e7e00f89ad)
[...]
2016-08-17 09:03:38.949 28 RALLYDEBUG rally.task.runner [-] Starting the constant runner (task UUID: 0be1976b-e144-4e93-82ec-85dcfce0ed56).
 timeout: 0
 concurrency: 2
 concurrency_per_worker: 1
 times: 5
 concurrency_overhead: 0
 processes_to_start: 2
 max_cpu_used: 4
2016-08-17 09:03:38.956 140 RALLYDEBUG rally.task.runner [-] Starting a worker.
 concurrency: 1
 args: {'vcpus': 1, 'disk': 1, 'ram': 500}
 times: 5
 method_name: create_and_list_flavor_access
 timeout: 0
 cls: <class 'rally.plugins.openstack.scenarios.nova.flavors.NovaFlavors'>
2016-08-17 09:03:38.956 139 RALLYDEBUG rally.task.runner [-] Starting a worker.
 concurrency: 1
 args: {'vcpus': 1, 'disk': 1, 'ram': 500}
 times: 5
 method_name: create_and_list_flavor_access
 timeout: 0
 cls: <class 'rally.plugins.openstack.scenarios.nova.flavors.NovaFlavors'>
2016-08-17 09:03:38.963 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 0 START
2016-08-17 09:03:38.969 139 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 1 START
2016-08-17 09:03:39.515 139 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 1 END: OK
2016-08-17 09:03:39.515 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 0 END: OK
2016-08-17 09:03:39.521 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 2 START
2016-08-17 09:03:39.525 139 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 3 START
2016-08-17 09:03:39.813 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 2 END: OK
2016-08-17 09:03:39.818 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 4 START
2016-08-17 09:03:39.875 139 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 3 END: OK
2016-08-17 09:03:40.107 140 INFO rally.task.runner [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | ITER: 4 END: OK
2016-08-17 09:03:40.117 28 INFO rally.plugins.openstack.context.cleanup.admin [-] Task 0be1976b-e144-4e93-82ec-85dcfce0ed56 | Starting: admin resources cleanup
2016-08-17 09:03:40.118 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Cleaning up nova quotas objects
2016-08-17 09:03:40.120 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Deleting nova quotas object (7753f51a98224a6db2e6750ba3dd0276)
2016-08-17 09:03:40.127 28 RALLYDEBUG rally.plugins.openstack.cleanup.manager [-] Deleting nova quotas object (27fdf4fea26343e3922291e7e00f89ad)

[Andriy Kurilin (andreykurilin)]

Which release of Rally was used? Also, please share task config

[Stéphane NGUYEN (stephane1-nguyen)]

Hi Andrey,

rally@32a0d1d0a4f5:/home/rally$ rally --version
0.5.1~dev95

This is my task :
  NovaFlavors.create_flavor:
    -
      args:
        ram: 500
        vcpus: 1
        disk: 1
      runner:
        type: "constant"
        times: 5
        concurrency: 2
      sla:
        failure_rate:
          max: 0

  NovaFlavors.create_and_list_flavor_access:
    -
      args:
        ram: 500
        vcpus: 1
        disk: 1
      runner:
        type: "constant"
        times: 5
        concurrency: 2
      sla:
        failure_rate:
          max: 0

Thx !

[Andriy Kurilin (andreykurilin)]

For a first look, I haven't understood why it so. Unfortunately, I hadn't enough time to investigate more this issue before.

Today, I find why it so, but I have not a quick solution now :( will raise this topic at our weekly meeting.

[zhangzhihui (zhangzhang)]

i don't find this bug, my rally version is 0.7.1

[Andriy Kurilin (andreykurilin)]

It looks like a bug is valid... In case of "nova" cleanup, "quotas" resource is triaged and it will cause changing quotas to default ones. I have an idea now how it can be fixed

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/450874

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/450874
Committed: https://git.openstack.org/cgit/openstack/rally/commit/?id=c341ff73698e08b0597370b708068b9c56f3178b
Submitter: Jenkins
Branch: master

commit c341ff73698e08b0597370b708068b9c56f3178b
Author: Andrey Kurilin <email address hidden>
Date: Tue Mar 28 19:20:45 2017 +0300

    [cleanup] Modify quotas resources

    ID of quotas resources were not ported to return NoName object, it causes
    to not-removed quotas.
    To fix an issue, let's use project object as a raw_resource. It has name
    (which can be used while matching) and id (which should be used while
    delete request).

    Change-Id: I66601978a5ac0b0c22935893f2247a5da8752419
    Closes-Bug: #1614142

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/0.9
Review: https://review.openstack.org/453687

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/453687
Committed: https://git.openstack.org/cgit/openstack/rally/commit/?id=f8598b6942c7f4a040f8f155f4da6752b637ccce
Submitter: Jenkins
Branch: stable/0.9

commit f8598b6942c7f4a040f8f155f4da6752b637ccce
Author: Andrey Kurilin <email address hidden>
Date: Tue Mar 28 19:20:45 2017 +0300

    [cleanup] Modify quotas resources

    ID of quotas resources were not ported to return NoName object, it causes
    to not-removed quotas.
    To fix an issue, let's use project object as a raw_resource. It has name
    (which can be used while matching) and id (which should be used while
    delete request).

    Change-Id: I66601978a5ac0b0c22935893f2247a5da8752419
    Closes-Bug: #1614142
    (cherry picked from commit c341ff73698e08b0597370b708068b9c56f3178b)