kafka-python version bump not compatible

[Andy McCrae]

The following patch increased the requirement for kafka-python:

https://review.openstack.org/#/c/392637/

This is incompatible with the global requirements for OpenStack, as other projects have a hard requirement on the previous version that was listed in requirements for ceilometer.

As a result OpenStack-Ansible (and I imagine other deployment projects) are running into compatibility issues. We were unable to bump to the latest SHA of ceilometer and have pinned ceilometer at the SHA just before the above patch merged.

[Julien Danjou (jdanjou)]

Which other projects are using Kafka?

Clearly, it's hard to state that the problem is in Ceilometer at this point. Other projects should update their requirements.

[gordon chung (chungg)]

monasca... ... ... https://review.openstack.org/#/c/355258/

[Andy McCrae (andrew-mccrae)]

@jdanjou - agree with that, but wasn't too sure the best place to start :)

I also noted the review itself had comments about removing the publisher and utilising oslo.messaging.

"I think we should drop this publisher and leverage the oslo.messaging kafka driver..."

[Julien Danjou (jdanjou)]

Sure Andy :)

I have added Monasca to the bug as they need to fix that on their side. The fix for Ceilometer as you mentionned is probably also to switch to oslo.messaging.

[Andy McCrae (andrew-mccrae)]

Adding Global-reqs, As it stands now we can't SHA bump ceilometer in master branch - so we're stuck at ceilometer from 2/11/2016 (which isn't a huge drama right now, but we'd need a resolution before release, so we can successfully use the released version of ceilometer).

[Tony Breeds (o-tony)]

There is a long'ish mailing list thread discussion why monasca won't/can't just bump the python-kafka requirement which may resolve this year but I'm not holding my breath.

With that in mind I'd urge the Telemetry to consider the oslo.messaging route so they're once again co-installable.

[Julien Danjou (jdanjou)]

oslo.messaging is also blocked by Monasca, so I am not sure I see the point.

[Julien Danjou (jdanjou)]

(Though TBH I'm sure Ceilometer would love to have someone porting its code to oslo.messaging if that's feasible as I'm sure it'd be a better mutualization of resources)

[gordon chung (chungg)]

ceilometer work items is to deprecate kafka specific publisher and reuse oslo.messaging (whenever it's oslo.messaging compatible)

[gordon chung (chungg)]

ceilometer specific: bug 1649947

related requirements changes:
https://review.openstack.org/#/c/424721/
https://review.openstack.org/#/c/425014/

closing this in ceilometer.

[Matthew Thode (prometheanfire)]

with the ceilometer fixes we've been able to move forward with this, fixes were merged to requirements a while ago.