When fluentd pushes documents into elasticsearch it does not generate _id for each one, leaving that to elasticsearch instead.
When request to ES times out (either on fluentd side, or by some proxy) fluentd will retry the request but ES can't tell it's processing the same documents (as the _id differs) and so will try to save them all, potentially leading to another timeout.
If that happens it's very easy to have kolla's fluentd DDos the entire cluster by filling up the entire available space.
It's possible to generate _id before sending documents to elasticsearch, which will let cluster know that some documents are sent twice and update them instead of creating duplicates - see https://github.com/uken/fluent-plugin-elasticsearch#generate-hash-id for the configuration.
Fix proposed to branch: master
Review: https:/