Elasticsearch plugin doesn't work with URL-based access control

[Aaron Wells]

We changed the Elasticsearch plugin to use the _bulk command to load data into Elasticsearch. But this poses problems for hosting setups that are using URL-based access control, as described here: https://www.elastic.co/guide/en/elasticsearch/reference/1.x/url-access-control.html

This is because the way Elastica handles the _bulk command, it puts the index name into the request body: localhost/_bulk, instead of localhost/index/_bulk

Elasticsearch *can* handle the indexname in the URL for _bulk commands. But we'll need to patch Elastica to handle it.

[Mahara Bot (dev-mahara)]

Patch for "master" branch: https://reviews.mahara.org/4788

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/4788
Committed: https://git.nzoss.org.nz/mahara/mahara/commit/278889b5bd6e33cadbd5e28f77b94cc256ab6a17
Submitter: Robert Lyon (<email address hidden>)
Branch: master

commit 278889b5bd6e33cadbd5e28f77b94cc256ab6a17
Author: Aaron Wells <email address hidden>
Date: Thu May 21 14:36:32 2015 +1200

elasticsearch: Add index name to _bulk command path

Bug 1457709: We only do this when the command comes
from an Index object itself, which means that all the
data in the bulk request will have the same index anyway.

You'll still need to set rest.action.multi.allow_explicit_index:false
in your config.yml, or else the data will be able to override the
index name in the URL.

See https://www.elastic.co/guide/en/elasticsearch/reference/current/url-access-control.html

Change-Id: I2f441bd3613e3c3195c88609989d122e6e26de76

[Mahara Bot (dev-mahara)]

Patch for "master" branch: https://reviews.mahara.org/4959

[Aaron Wells (u-aaronw)]

After deploying this to some client sites, I found that I was still having problems. So I've made a further tweak to simply remove the index name from the document itself. If you do this, then Elasticsearch will use the index from the request path instead.

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/4959
Committed: https://git.nzoss.org.nz/mahara/mahara/commit/a02be21a1d70a6866ae2964b647440682a7af2d5
Submitter: Robert Lyon (<email address hidden>)
Branch: master

commit a02be21a1d70a6866ae2964b647440682a7af2d5
Author: Aaron Wells <email address hidden>
Date: Wed Jul 15 13:26:24 2015 +1200

Remove the index name from elasticsearch JSON data (Bug 1457709)

If we remove the index name from the JSON data, then Elasticsearch
will just use the index name in the URL path, which we're already
setting correctly.

Doing it this way, we don't have to rely on the Elasticsearch server
setting the "rest.action.multi.allow_explicit_index: false" flag.

Change-Id: Ib6194b89e68d539a7959b9e12ec1ac9e3528dd27

[Mahara Bot (dev-mahara)]

Patch for "master" branch: https://reviews.mahara.org/6847

[Mahara Bot (dev-mahara)]

Reviewed: https://reviews.mahara.org/6847
Committed: https://git.mahara.org/mahara/mahara/commit/3c7e24e7defe3c3d1a8ec59026d8d05e0cd026b8
Submitter: Aaron Wells (<email address hidden>)
Branch: master

commit 3c7e24e7defe3c3d1a8ec59026d8d05e0cd026b8
Author: Aaron Wells <email address hidden>
Date: Thu May 21 14:36:32 2015 +1200

elasticsearch: Add index name to _bulk command path

Bug 1457709: We only do this when the command comes
from an Index object itself, which means that all the
data in the bulk request will have the same index anyway.

You'll still need to set rest.action.multi.allow_explicit_index:false
in your config.yml, or else the data will be able to override the
index name in the URL.

See https://www.elastic.co/guide/en/elasticsearch/reference/current/url-access-control.html

Conflicts:
 htdocs/lib/elastica/README.Mahara
 htdocs/lib/elastica/lib/Elastica/Client.php

behatnotneeded: Can't be tested in behat

Change-Id: Ide22a0026aeb480a130e39c3ea2f327ef2de167f