keystone-manage mapping_engine tester problems

[John Dennis]

[Impact]

 * A bug in keystone-manage tool prohibits the use of the mapping_engine command for testing federation rules.

 * Users of Keystone Federation will not be able to verify their mapping rules before pushing these to production.

 * Not being able to test rules before pushing to production is a major operational challenge for our users.

 * The proposed upload fixes this by backporting a fix for this issue from upstream stable/ocata.

[Test Case]

 * Deploy keystone using Juju with this bundle:
   http://pastebin.ubuntu.com/24855409/

 * ssh to keystone unit, grab artifacts and run command:
   - mapping.json: http://pastebin.ubuntu.com/24855419/
   - input.txt: http://pastebin.ubuntu.com/24855420/
   - command:
   'keystone-manage mapping_engine --rules mapping.json --input input.txt'

 * Observe that command provides no output and that a Python Traceback is printed in /var/log/keystone/keystone.log

 * Install the proposed package, repeat the above steps and observe that the command now outputs its interpretation and effect of the rules.

[Regression Potential]

 * keystone-manage mapping_engine is a operational test tool and is solely used by the operator to test their rules.

 * The distributed version of this command in Xenial and Yakkety does currently not work at all.

 * The change will make the command work as our users expect it to.

[Original bug description]
There are several problems with keystone-manage mapping_engine

* It aborts with a backtrace because of wrong number of arguments
  passed to the RuleProcessor

* The --engine-debug option does not work.

* Error messages related to input data are cryptic and inprecise.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/418165

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/418165
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=f2d0f8c9ab38172a6e37b02339eac59da911435c
Submitter: Jenkins
Branch: master

commit f2d0f8c9ab38172a6e37b02339eac59da911435c
Author: John Dennis <email address hidden>
Date: Tue Nov 29 11:36:32 2016 -0500

    Fix keystone-manage mapping_engine tester

    There were several problems with keystone-manage mapping_engine

    * It aborts with a backtrace because of wrong number of arguments
      passed to the RuleProcessor, it was missing the mapping_id
      parameter.

    * Error messages related to input data were cryptic and inprecise.

    * The --engine-debug option did not work.

    A fake mapping_id is now generated and passed to the RuleProcessor.

    If there was invalid data passed it was nearly impossible to determine
    what was causing the error, the command takes 2 input files, but which
    file contained the error? At what line? Why? For example I was
    consistently getting this error:

    Error while parsing line: '{': need more than 1 value to unpack

    and had no idea of what was wrong, the JSON looked valid to me. Turns
    out the assertion file is not formatted as JSON (yes this is
    documented in the help message but given the rules are JSON formatted
    and the RuleProcessor expects a dict for the assertion_data it's
    reasonsable to assume the data in the assertion file is formatted as a
    JSON object).

    The documentation in mapping_combinations.rst added a note in the
    section suggesting the use of the keystone-manage mapping_engine
    tester alerting the reader to the expected file formats.

    The MappingEngineTester class was refactored slighly to allow each
    method to know what file it was operating on and emit error messages
    that identify the file. The error message in addition to the pathname
    now includes the offending line number as well. As a bonus it doesn't
    fail if there is a blank line. The error message now looks like this:

    assertion file input.txt at line 4 expected 'key: value' but found 'foo' see help for file format

    The mapping_engine.LOG.logger level is now explictily set to DEBUG
    when --engine-debug is passed instead of (mistakenly assuming it
    defaulted to DEBUG) otherwise it's set to WARN.

    Closes-Bug: 1655182
    Signed-off-by: John Dennis <email address hidden>
    Change-Id: I2dea0f38b127ec185b79bfe06dd6a212da75cbca

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 11.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/466873

[Frode Nordahl (fnordahl)]

As bug 1677730 demonstrates, this is also a bug that renders the mapping_engine command unusable on stable/newton and stable/mitaka.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/466873
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8726573940c435ec75ca72a9ec20d9744561c07c
Submitter: Jenkins
Branch: stable/newton

commit 8726573940c435ec75ca72a9ec20d9744561c07c
Author: John Dennis <email address hidden>
Date: Tue Nov 29 11:36:32 2016 -0500

    Fix keystone-manage mapping_engine tester

    There were several problems with keystone-manage mapping_engine

    * It aborts with a backtrace because of wrong number of arguments
      passed to the RuleProcessor, it was missing the mapping_id
      parameter.

    * Error messages related to input data were cryptic and inprecise.

    * The --engine-debug option did not work.

    A fake mapping_id is now generated and passed to the RuleProcessor.

    If there was invalid data passed it was nearly impossible to determine
    what was causing the error, the command takes 2 input files, but which
    file contained the error? At what line? Why? For example I was
    consistently getting this error:

    Error while parsing line: '{': need more than 1 value to unpack

    and had no idea of what was wrong, the JSON looked valid to me. Turns
    out the assertion file is not formatted as JSON (yes this is
    documented in the help message but given the rules are JSON formatted
    and the RuleProcessor expects a dict for the assertion_data it's
    reasonsable to assume the data in the assertion file is formatted as a
    JSON object).

    The documentation in mapping_combinations.rst added a note in the
    section suggesting the use of the keystone-manage mapping_engine
    tester alerting the reader to the expected file formats.

    The MappingEngineTester class was refactored slighly to allow each
    method to know what file it was operating on and emit error messages
    that identify the file. The error message in addition to the pathname
    now includes the offending line number as well. As a bonus it doesn't
    fail if there is a blank line. The error message now looks like this:

    assertion file input.txt at line 4 expected 'key: value' but found 'foo' see help for file format

    The mapping_engine.LOG.logger level is now explictily set to DEBUG
    when --engine-debug is passed instead of (mistakenly assuming it
    defaulted to DEBUG) otherwise it's set to WARN.

    Closes-Bug: 1655182
    Signed-off-by: John Dennis <email address hidden>
    Change-Id: I2dea0f38b127ec185b79bfe06dd6a212da75cbca
    (cherry picked from commit f2d0f8c9ab38172a6e37b02339eac59da911435c)

[James Page (james-page)]

Fixes for mitaka and newton pushed to git repository for keystone package

[James Page (james-page)]

Hello John, or anyone else affected,

Accepted keystone into newton-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:newton-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-newton-needed to verification-newton-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-newton-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 10.0.3 release.

[Łukasz Zemczak (sil2100)]

Hello John, or anyone else affected,

Accepted keystone into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/keystone/2:9.3.0-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Edward Hope-Morley (hopem)]

I have verified this fix using the steps detailed in the test case and can confirm that the fix is good:

root@juju-58b2b9-default-7:~# keystone-manage mapping_engine --rules mapping.json --input input.txt
{
  "group_ids": [],
  "user": {
    "domain": {
      "id": "default"
    },
    "type": "local",
    "name": "abc"
  },
  "group_names": []
}

[Edward Hope-Morley (hopem)]

^^ is for xenial-mitaka-proposed. I will test xenial-newton-proposed next.

[James Page (james-page)]

Hello John, or anyone else affected,

Accepted keystone into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Edward Hope-Morley (hopem)]

Verified for xenial-newton.

[Frode Nordahl (fnordahl)]

Verification done for trusty/mitaka.

Before upgrade:
root@juju-8feceb-1:~# keystone-manage mapping_engine --rules mapping.json --input input.txt
root@juju-8feceb-1:~# tail /var/log/keystone/keystone.log
Traceback (most recent call last):
  File "/usr/bin/keystone-manage", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/dist-packages/keystone/cmd/manage.py", line 47, in main
    cli.main(argv=sys.argv, config_files=config_files)
  File "/usr/lib/python2.7/dist-packages/keystone/cmd/cli.py", line 1024, in main
    CONF.command.cmd_class.main()
  File "/usr/lib/python2.7/dist-packages/keystone/cmd/cli.py", line 949, in main
    rp = mapping_engine.RuleProcessor(rules['rules'])
TypeError: __init__() takes exactly 3 arguments (2 given)

After upgrade:
root@juju-8feceb-1:~# keystone-manage mapping_engine --rules mapping.json --input input.txt
{
  "group_ids": [],
  "user": {
    "domain": {
      "id": "default"
    },
    "type": "local",
    "name": "abc"
  },
  "group_names": []
}

[Launchpad Janitor (janitor)]

This bug was fixed in the package keystone - 2:9.3.0-0ubuntu3

---------------
keystone (2:9.3.0-0ubuntu3) xenial; urgency=medium

  * d/p/keystone_manage_mapping_engine_fix.patch: Fix keystone-manage
    mapping_engine usability issues (LP: #1655182).

 -- Frode Nordahl <email address hidden> Tue, 25 Jul 2017 17:53:12 +0100

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for keystone has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

The verification of the Stable Release Update for keystone has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package keystone - 2:9.3.0-0ubuntu3~cloud0
---------------

 keystone (2:9.3.0-0ubuntu3~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 keystone (2:9.3.0-0ubuntu3) xenial; urgency=medium
 .
   * d/p/keystone_manage_mapping_engine_fix.patch: Fix keystone-manage
     mapping_engine usability issues (LP: #1655182).

[James Page (james-page)]

The verification of the Stable Release Update for keystone has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[James Page (james-page)]

This bug was fixed in the package keystone - 2:10.0.2-0ubuntu1~cloud0
---------------

 keystone (2:10.0.2-0ubuntu1~cloud0) xenial-newton; urgency=medium
 .
   [ Frode Nordahl ]
   * d/p/keystone_manage_mapping_engine_fix.patch: Fix keystone-manage
     mapping_engine usability issues (LP: #1655182).
 .
   [ James Page ]
   * New upstream point release for OpenStack Newton (LP: #1705176).