Context is reset to by ClientRouter in nova/rpc.py

[Kenneth Burger]

The following line in the constructor of ClientRouter in nova/rpc.py is resetting the context on the threading.local to an invalid context.

self.run_periodic_tasks(nova.context.RequestContext())

Rather than creating a new context here, I believe it should either be:

self.run_periodic_tasks(nova.context.RequestContext(overwrite=False))
or
from oslo_context import context
curr_context = context.get_current()
self.run_periodic_tasks(curr_context)

Before the call

<Context {'domain': None, 'project_name': u'default', 'project_domain': None, 'timestamp': '2016-09-26T19:00:37.292388', 'auth_token': u'gAAAAABX6XAxNbPfOJvmNZqCGd3kws4l92qrbvL_JTCpTVv-6Sk8fVE8F4VklDunreU', 'remote_address': u'127.0.0.1', 'quota_class': None, 'resource_uuid': None, 'is_admin': True, 'user': u'0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9', 'service_catalog': [{u'endpoints': [{u'adminURL': u'https://ip9-114-192-147.pok.stglabs.ibm.com:9000/v2/cd65f1b0d9f6410c92101ed1211b74ea', u'region': u'RegionOne', u'internalURL': u'http://127.0.0.1:9000/v2/cd65f1b0d9f6410c92101ed1211b74ea', u'publicURL': u'https://XXXXX:9000/v2/cd65f1b0d9f6410c92101ed1211b74ea'}], u'type': u'volume', u'name': u'cinder'}], 'tenant': u'cd65f1b0d9f6410c92101ed1211b74ea', 'read_only': False, 'project_id': u'cd65f1b0d9f6410c92101ed1211b74ea', 'user_id': u'0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9', 'show_deleted': False, 'roles': [u'admin'], 'user_identity': u'0688b01e6439ca32d698d20789d52169126fb41fb1a4ddafcebb97d854e836c9 cd65f1b0d9f6410c92101ed1211b74ea - - -', 'is_admin_project': True, 'read_deleted': u'no', 'request_id': u'req-f5b85c48-a3b5-4fc0-bf36-3cf7c66182ed', 'instance_lock_checked': False, 'user_domain': None, 'user_name': u'root'}>

After the call (auth_token, user props, etc. are gone )

<Context {'domain': None, 'project_name': None, 'project_domain': None, 'timestamp': '2016-09-26T19:00:44.442372', 'auth_token': None, 'remote_address': None, 'quota_class': None, 'resource_uuid': None, 'is_admin': False, 'user': None, 'service_catalog': [], 'tenant': None, 'read_only': False, 'project_id': None, 'user_id': None, 'show_deleted': False, 'roles': [], 'user_identity': u'- - - - -', 'is_admin_project': True, 'read_deleted': 'no', 'request_id': 'req-44d322cd-2e0c-4dba-bb65-c9962b8ac713', 'instance_lock_checked': False, 'user_domain': None, 'user_name': None}>

[Matt Riedemann (mriedem)]

Looks like this was the regression:

https://github.com/openstack/nova/commit/4df0869e00f4316ebb41ed9cdf4d6a433227f4d0

[Matt Riedemann (mriedem)]

https://review.openstack.org/#/c/377093/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/378751

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/377093
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=d218f0df65729e2818a63a22f9eb476d376e168e
Submitter: Jenkins
Branch: master

commit d218f0df65729e2818a63a22f9eb476d376e168e
Author: melanie witt <email address hidden>
Date: Mon Sep 26 20:08:33 2016 +0000

    Stop overwriting thread local context in ClientRouter

    In commit 4df0869, a ClientRouter was added to cache client connections
    to cell message queues. The periodic task for removing stale clients
    is called with an empty RequestContext, but this overwrites the copy
    in thread local storage unless overwrite=False is specified.

    This adds overwrite=False to the empty RequestContext to prevent it
    from overwriting the thread local context. All of the other periodic
    tasks use get_admin_context() which also creates a RequestContext with
    overwrite=False.

    Closes-Bug: #1627838

    Change-Id: I48024952865fe017e0cb786567b5b445b05e7659

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/378751
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=35f7942ff9a6dd972994dfa4162c96eaf0b21e02
Submitter: Jenkins
Branch: stable/newton

commit 35f7942ff9a6dd972994dfa4162c96eaf0b21e02
Author: melanie witt <email address hidden>
Date: Mon Sep 26 20:08:33 2016 +0000

    Stop overwriting thread local context in ClientRouter

    In commit 4df0869, a ClientRouter was added to cache client connections
    to cell message queues. The periodic task for removing stale clients
    is called with an empty RequestContext, but this overwrites the copy
    in thread local storage unless overwrite=False is specified.

    This adds overwrite=False to the empty RequestContext to prevent it
    from overwriting the thread local context. All of the other periodic
    tasks use get_admin_context() which also creates a RequestContext with
    overwrite=False.

    Closes-Bug: #1627838

    Change-Id: I48024952865fe017e0cb786567b5b445b05e7659
    (cherry picked from commit d218f0df65729e2818a63a22f9eb476d376e168e)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 14.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/554381
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2f7df9417bd79c00deb7acbd9febc1b730c3301a
Submitter: Zuul
Branch: master

commit 2f7df9417bd79c00deb7acbd9febc1b730c3301a
Author: melanie witt <email address hidden>
Date: Mon Mar 19 23:10:47 2018 +0000

    Remove useless run_periodic_tasks call in ClientRouter

    The only periodic task (_remove_stale_clients) in ClientRouter was
    removed in change I10f374adca672576058c4dbab708c040d166df47 and this
    isn't the correct way to run periodic tasks repeatedly anyway (this
    will only run them once upon ClientRouter.__init__), so remove the
    useless call.

    This also removes a functional regression test which was testing only
    this specific code path, where the intended periodic task ran only
    once upon init of the compute service. All other periodic tasks in
    compute are disabled by default in functional tests.

    Related-Bug: #1627838

    Change-Id: I3079c5ae4bd60de44f04e0136978a67d13e7a809