Add support for handling different error codes for different roles

[Cliff Parsons]

Patrole currently cannot handle the scenario where two possible error codes can returned by Neutron policy enforcement for a failed policy check (403 Forbidden and 404 NotFound), depending on what role is being tested. Patrole framework can only handle one expected_exception. Example below:

Roles:
  admin
  role1
  role2
  role3

Rules:
  “show_action”: “role:admin or role:role1”
  “update_action”: “role:admin”

In the enforcement of “update_action”, Neutron would return a 403 Forbidden for role1, but a 404 NotFound for role2 and role3 (because policy check fails on “show_action” for role2/role3). The Patrole test case would pass for roles admin and role1, but would always fail for role2 and role3 even though Neutron is producing the correct/expected result.

[Ghanshyam Mann (ghanshyammann)]

in-progress - https://review.openstack.org/#/c/570262/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/570262
Committed: https://git.openstack.org/cgit/openstack/patrole/commit/?id=35a77113fccfd5659c123adb2cc142836cdd764d
Submitter: Zuul
Branch: master

commit 35a77113fccfd5659c123adb2cc142836cdd764d
Author: Cliff Parsons <email address hidden>
Date: Mon May 7 14:03:40 2018 -0500

    Add support for handling multiple error codes

    Patrole currently cannot handle the scenario where two possible
    error codes can returned by Neutron policy enforcement for a
    failed policy check (403 Forbidden and 404 NotFound), depending
    on what role is being tested. Patrole framework can only handle
    one expected_exception.

    This change builds upon the recent multi-policy support to allow
    the tester to specify multiple policy actions for one API test.
    For each policy action, the tester would need to specify an
    error code that is expected if the action should fail. If multiple
    policy actions fail, the error code for the first policy action
    that fails will be expected to be returned from the service.

    This handles the cases in Neutron where Neutron may use a second
    policy rule to determine whether or not to return a 403 error
    code or a 404 error code. The tester is expected to list out
    which policy rules are being tested by the API endpoint test.

    Change-Id: I5cd861e184da90bb27f8ba454c94fa4d4f99c269
    Closes-Bug: #1772710

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/patrole 0.4.0 release.