Logs sent to Monasca can be lost in Fluent
Bug #1855700 reported by
Doug Szumski
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| kolla-ansible |
Fix Released
|
Medium
|
Doug Szumski | ||
| Rocky |
Won't Fix
|
Medium
|
Mark Goddard | ||
| Stein |
Won't Fix
|
Medium
|
Mark Goddard | ||
| Train |
Won't Fix
|
Medium
|
Mark Goddard | ||
| Ussuri |
Fix Released
|
Medium
|
Doug Szumski | ||
Bug Description
Monasca Fluentd output plugin is configured to use an in memory buffer. If a Fluentd container is shut down, killed, crashes, etc. then logs buffered to memory may not be sent to the Monasca Log API. We should buffer to file to fix this.
| Changed in kolla-ansible: | |
| assignee: | nobody → Doug Szumski (dszumski) |
| status: | New → In Progress |
| Changed in kolla-ansible: | |
| importance: | Undecided → Medium |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Fix merged to kolla-ansible (master) | #1 |
| Changed in kolla-ansible: | |
| status: | In Progress → Fix Released |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Fix proposed to kolla-ansible (stable/train) | #2 |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Fix proposed to kolla-ansible (stable/rocky) | #3 |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Change abandoned on kolla-ansible (stable/train) | #4 |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Change abandoned on kolla-ansible (stable/rocky) | #5 |
Revision history for this message
| OpenStack Infra (hudson-openstack) wrote Change abandoned on kolla-ansible (stable/stein) | #6 |
To post a comment you must log in.
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 5293b1294f5a671
Author: Doug Szumski <email address hidden>
Date: Wed Jun 19 11:27:14 2019 +0000
Enable buffering to file for Monasca logs
This enables buffering to file, rather than memory for Monasca logs.
A dedicated docker volume is used for the file buffer. If a post
to the Monasca Log API fails, retries will be made using an exponential
backoff algorithm with a maximum retry interval of 30mins. The maximum
interval is set relatively low to try and reduce the risk of large
buffers accumulating, and therefore the risk of overloading the Monasca
Log API.
Closes-Bug: #1855700
Change-Id: Ib5286e9dbaf2bc