[Feature Request] Add support for customizable Type value in heka encoders

Bug #1629463 reported by Matt Simonin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Invalid
Wishlist
Matt Simonin

Bug Description

In the current implementation, Kolla uses heka to sends parsed log lines to elasticsearch output.
If one sends differents Openstack instances logs into the same elasticsearch instance, I don't see any obvious way to differentiate the source of the logs.

I've slightly modified all the decoders to allow the Type to be passed through the configuration like this :

[openstack_log_decoder]
type = "SandboxDecoder"
filename = "lua_decoders/os_openstack_log.lua"

[openstack_log_decoder.config]
type = "my custom type"

Each decoder have to be slighly modified to read the value from the config. The value can be defaulted to the current hard-coded value to ensure backward compatibility [1].

Let me know, if you think it's worth a review on gerrit.

Otherwise if there's another way to differentiate log provenance in a single elasticsearch instance, I'll be happy to learn it.

[1]: https://github.com/openstack/kolla/blob/789f50b2f3bc733015d11287054b63db7c01c03a/docker/heka/plugins/decoders/os_openstack_log.lua#L24

Duong Ha-Quang (duonghq)
Changed in kolla:
importance: Undecided → Wishlist
status: New → Opinion
Revision history for this message
Duong Ha-Quang (duonghq) wrote :

Due to deprecation of Heka, Kolla is moving to another logging engine. So any improvement is still welcome but not we should not put much effort on this.

Anyway, if you are not sure about a review on gerrit, it's always worth it (don't forget tag it with appropriate bug id)

Changed in kolla:
assignee: nobody → Matt Simonin (matthieu-simonin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/381764

Changed in kolla:
status: Opinion → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Matthieu Simonin (<email address hidden>) on branch: master
Review: https://review.openstack.org/381764
Reason: Heka will be deprecated in ocata, i'm withdrawing this patch.

Revision history for this message
Christian Berendt (berendt) wrote :

Heka will be deprecated in Ocata.

Changed in kolla:
status: In Progress → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.