"Rule already exists" error when using non-existent named parameter

[Eric K]

$ openstack congress policy rule create classification '
> a :- w(id=y)'
Rule already exists::An unknown exception occurred. (HTTP 409) (Request-ID: req-ee5062e6-0121-4a5d-8bc8-c2644d27cda4)

[comment: incorrect/misleading error message.]

$ openstack congress policy rule create classification '
> w(y) :- t(y)'
+---------+--------------------------------------+
| Field | Value |
+---------+--------------------------------------+
| comment | |
| id | 1047e0f9-9e1c-4d79-83e7-22f473922481 |
| name | None |
| rule | w(y) :- |
| | t(y) |
+---------+--------------------------------------+

[comment: make the table w known.]

$ openstack congress policy rule create classification '
> a :- w(id=y)'
Syntax error for rule::In literal w(id=y) column name id does not exist (HTTP 400) (Request-ID: req-26278518-eb7f-4c14-a1d5-9688e7359da2)

[comment: then the error message is correct.]

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/289150

[OpenStack Infra (hudson-openstack)]

Change abandoned by Anusha Ramineni (<email address hidden>) on branch: master
Review: https://review.openstack.org/289150
Reason: Alternate fix proposed

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/289650
Committed: https://git.openstack.org/cgit/openstack/congress/commit/?id=34d6e58d15c80e1b419c325dcf13f3445a0c0e8b
Submitter: Jenkins
Branch: master

commit 34d6e58d15c80e1b419c325dcf13f3445a0c0e8b
Author: Tim Hinrichs <email address hidden>
Date: Mon Mar 7 17:35:50 2016 -0800

    Reject rules with column refs on non-datasources

    Previously, the policy engine would accept rules that had
    column references on tables in policies that were not
    datasource policies and disable them. Unfortunately,
    there was no way to enable those same rules because
    there is no way to set the schema other than for
    datasource policies.

    This change throws an exception when someone tries
    to insert a rule that has a column reference
    for a table within a non-datasource policy.

    Also, fixed bug where eliminate_column_references
    for Literal and Rule were not fully polymorphic.
    Literal and Rule expected different arguments, but
    the callers were providing the same arguments.

    Change-Id: Ia0f1a9980befb0189d5727dfcfacad376fab94fe
    Closes-Bug:#1553433

[Eric K (ekcs)]

The bug hasn't been fixed completely. Indeed, the error message now makes sense for column-name-references on known policy tables, but the old message shows up for something that's not known to be either a policy table or a datasource table.

$ openstack congress policy rule create classification 'a :- w(id=y)'
Syntax error for rule::Literal w(id=y) uses column references, but 'classification' does not reference a datasource policy (HTTP 400) (Request-ID: req-30b0f225-0dfe-428a-967d-27ab9a2ff722)
[correct]

$ openstack congress policy rule create classification 'a :- action:w(id=y)'
Syntax error for rule::Literal action:w(id=y) uses column references, but 'action' does not reference a datasource policy (HTTP 400) (Request-ID: req-b34ef4c2-4710-4263-8ba4-832a38a43716)
[correct]

$ openstack congress policy rule create classification 'a :- blah:w(id=y)'
Rule already exists::(1008) Rule already exists (HTTP 409) (Request-ID: req-f5b5f25a-40cc-4278-ad7b-cc49d9a72779)
[misleading, old message]

[Eric K (ekcs)]

I think I found the main problem (among others):
At the end of Runtime.persistent_insert_rule (agnostic.py), the case of having no-changes-made-but-some-disabled is conflated with no-changes-made-and-none-disabled.

Code snippet:
    changes = self._safe_process_policy_update(rule, policy_name)
    if len(changes) > 0:
        ...
        ...

    # change not accepted means it was already there
    raise exception.PolicyRuntimeException(
        name='rule_already_exists')

[Eric K (ekcs)]

My question is: what is the proper response when a rule insert is permitted but disabled?
Warning message?
No message?

[OpenStack Infra (hudson-openstack)]

Change abandoned by Tim Hinrichs (<email address hidden>) on branch: master
Review: https://review.openstack.org/294839
Reason: Decided to go with a different fix for Mitaka.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/294389
Committed: https://git.openstack.org/cgit/openstack/congress/commit/?id=c7e394d4a43b1108a1a4c9c389f993cee4a3bed4
Submitter: Jenkins
Branch: master

commit c7e394d4a43b1108a1a4c9c389f993cee4a3bed4
Author: Eric K <email address hidden>
Date: Thu Mar 17 18:35:48 2016 -0700

    Make persistent_insert_rule error on IncompleteSchemaException

    Before this patch, persistent insert of rules with unknown table with
    column-name references leads to unexpected behavior:
    - misleading 'Rule already exists' error
    - limbos the rule
    - but never persists it in the DB, even if the rule is enabled by
      new schema.

    With this patch, the rule is outright rejected with syntax error.
    The rule is not limboed.

    Additional notes:
    This patch largely preserves the existing behavior on rule insertion.

    But it may be helpful to review the the behavior:

    When inserting a rule that refers to a table belonging to a policy,
    we have the following four cases of interest:

    1. policy is unknown (could be a datasource policy in the future)
    2. policy is a known datasource policy, but the schema is not
       yet available
    3. policy is a known datasource policy, and the schema is
       available, but table is not in the schema
    4. policy is known not to be a datasource policy.

    Pre-existing behavior:
    For rule without column-name references, all four cases are accepted.
    For rule with column-name references:
    a. For non-persistent insert, (1-3) are limboed and (4) is rejected.
    b. For persistent insert, (1-3) are limboed (with extraneous error)
       and (4) is rejected.

    With the patch, all four cases are rejected in (b).

    Neither the pre-existing behavior nor the patched behavior is
    particularly consistent.
    It may be appropriate to re-design the behavior in the future.

    Terminology notes:
    Accepted means rule added and put in effect.
    Rejected means rule not added and error shown.
    Limboed means rule is added to disabled events, to come into effect
    under the right conditions (eg. matching schema received).

    Closes-Bug: #1553433

    Change-Id: I5b377cd7868004d7891263a0649a31621845668c

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/congress 3.0.0.0rc1 release candidate.