python elasticsearch must match server version

[Steve McLellan]

Elasticsearch 2.0.0 was released recently and the python client documentation notes that the major version of the python client must match the server version (https://github.com/elastic/elasticsearch-py).

Currently we specify elasticsearch>=1.3.0 which now will mean elasticsearch 2.0.0 will get installed. We either need to make a choice in requirements.txt (i.e. elasticsearch>=1.3.0,<2.0.0 OR elasticsearch>=2.0.0) and mandate a server version OR not include it in requirements.txt and make it an installation step to install one version or the other. The former is more friendly, the latter less error prone.

It may be that there aren't many breaking changes that affect us in any case; it might be worth identifying if there's a reason to be concerned either way before taking action.

[Liyingjun (liyingjun)]

There will be an error when using 2.x elasticsearch client when syncing index, and 2.x is installed by default.

[Liyingjun (liyingjun)]

i prefer restricting version less than 2.0.0 currently, and when we do have 2.x support, we can bump the version then.

[Steve McLellan (sjmc7)]

I'm ok with restricting the client to <=2.0 for now until we update the API calls to support 2.0. We'll need a similar note in the documentation.

[Steve McLellan (sjmc7)]

global-requirements currently does not cap the version of elasticsearch, so we may not be able to get away with doing that. I'll take a look to see how problematic this is.

[Steve McLellan (sjmc7)]

The only problem i've actually found in terms of cross compatibility so far is that deleting a document type is no longer supported (you have to delete an entire index) because it causes performance problems. This might be something better addressed with the zero downtime indexing.

[Liyingjun (liyingjun)]

I uploaded a patch to requirements: https://review.openstack.org/#/c/251213/

[Steve McLellan (sjmc7)]

I apologize, i should have noted this on the bug ticket. I proposed https://review.openstack.org/#/c/245222/ a couple of days ago because this is a pretty urgent issue; can you abandon yours and i'll add Davanum to the existing review? I'd already sent an email to the ceilometer and infra teams about this.

[Steve McLellan (sjmc7)]

I'm going to submit a temporary patch that will allow operation with the v2 client for now to ease the path for new developers.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/253177

[Steve McLellan (sjmc7)]

https://review.openstack.org/253177 as a temporary warning if the v2 client is installed.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/253177
Committed: https://git.openstack.org/cgit/openstack/searchlight/commit/?id=3ff4115d9160501f43d55ffb402ce15c699ba80a
Submitter: Jenkins
Branch: master

commit 3ff4115d9160501f43d55ffb402ce15c699ba80a
Author: Steve McLellan <email address hidden>
Date: Thu Dec 3 14:04:48 2015 -0600

    Emit warning if indexing with elasticsearch v2

    If the v2 client is installed, emit a warning when attempting to index
    without --no-delete (the default) instructing the user to consult the
    documentation to manually delete the index and rerun with --no-delete.

    This can be removed after global-requirements is capped (see
    https://review.openstack.org/#/c/252075/) or once reindexing no longer
    requires an in-place clearing of data.

    Change-Id: I8ad3efb0addf6be516b0f8bbf18543b19175ac5d
    Partial-Bug: #1515412

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/256219
Committed: https://git.openstack.org/cgit/openstack/searchlight/commit/?id=567051cefcdd3922a43f5bed8b7e84a2c85bd099
Submitter: Jenkins
Branch: master

commit 567051cefcdd3922a43f5bed8b7e84a2c85bd099
Author: Travis Tripp <email address hidden>
Date: Thu Dec 10 22:11:08 2015 -0700

    Cap elasticsearch client <2.0

    The elasticsearch 2.0 python client is not backwards compatible
    with 1.x, nor with server versions prior to 2.0 (and similarly, the
    1.x client cannot be used with server versions after 1.x).

    See https://elasticsearch-py.readthedocs.org/en/master/ for details.
    Since devstack installs elasticsearch 1.x it seems sensible to cap
    the client version. Deployers using version 2.0 will need to install
    that client version in projects requiring it.

    Note: Autoproposal bot isn't enabled yet or this would have trickled
    from Steve McLellan's patch:

    https://review.openstack.org/#/c/252075/

    Change-Id: I7f5a6a74968980636c340e2d059a70b508ece0fc
    Partial-Bug: #1515412
    Co-Authored-By: Steve McLellan <email address hidden>

[Travis Tripp (travis-tripp)]

Still need to actually support 2.0. Right now we just set max supported to 1.x