consoleauth cannot be run in HA configuration without external memcache

[Stanislaw Pitucha]

Running more than one consoleauth service causes silent failures where tokens simply don't get authenticated, because only one of the processes has it cached.

There are two ways to fix this:
- process sending the new token has to use the fanout queue rather than a direct message, so that all consoleauth services are updated
- token can be sent to the database, rather than consoleauth directly - this allows restarting services and adding new ones without creating new problems

Ideally both ways could be implemented at the same time.

[Michael H Wilson (geekinutah)]

I don't this think this is a bug at all, you can already use a shared memcache instance so that tokens are shared between consoleauth services. You just need to define memcached_servers for each consoleauth process ie.

memcached_server=sharedhost1:11211, sharedhost2:11211

The memcache client will take care of hashing keys to the different memcache instances.

[Stanislaw Pitucha (stanislaw-pitucha)]

This is a solution for sharding the tokens. That's something completely different from HA.

By setting up two memcache servers, you have 2 SPOFs - one is each of the memcache servers (because tokens are not shared between them), another one is the consoleauth (if that server goes down, you can't access any server). If you setup another consoleauth server, you're just ending up with 1 SPOF again (token stored in one memcache).

This bug is about removing that SPOF - make sure that the information itself is replicated in case of multiple servers. There are also reasons not to rely on external memcache (as mentioned in the bug title).

[Michael H Wilson (geekinutah)]

And I should read the title of the bug better..... disregard my previous comment.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/75140

[Joe Gordon (jogo)]

Is this still valid, patch was abandoned.

[Stanislaw Pitucha (stanislaw-pitucha)]

I'm not working with Nova an daily basis anymore, handed this patch to someone else. I'll make sure the bug is reassigned.

[Bjørnar Ness (bjornar-ness)]

What is progress on this.

I would like to see fanout + memory cache (local/memcached). Dont see the reason to stress database
with this, as worst case scenario is "open console again"

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sean Dague (<email address hidden>) on branch: master
Review: https://review.openstack.org/75140
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.

[Davanum Srinivas (DIMS) (dims-v)]

Removing "In Progress" status and assignee as change is abandoned.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/177078

[OpenStack Infra (hudson-openstack)]

Change abandoned by Eli Qiao (<email address hidden>) on branch: master
Review: https://review.openstack.org/177078

[Eli Qiao (taget-9)]

there was a nova sepc to address this bug. https://review.openstack.org/#/c/165838

but it has been postpone to M release.

[Markus Zoeller (markus_z) (mzoeller)]

wrt to comment #12

The bug report looks still valid. A blueprint which also intents to
solve this bug shouldn't be a reason to put it to "incomplete" IMO.
That's why I change it back to "confirmed".
As there is no open patch available, I remove the assignee too.

[Markus Zoeller (markus_z) (mzoeller)]

In addition to comment #13:

This bug report is pretty old. I'm closing it.
Please re-test this issue when [1] has merged. If it hasn't then reopen this report.

References:
[1] https://review.openstack.org/#/c/301158/