New charm proposal: Filebeat

[Charles Butler]

A lightweight, open source shipper for log file data. As the next-generation Logstash Forwarder, Filebeat tails logs and quickly sends this information to Logstash for further parsing and enrichment or to Elasticsearch for centralized storage and analysis.

Let me know if there's anything in here that needs cleaning up! Ta

[Review Queue (review-queue)]

The results (PASS) are in and available here: http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3281/

[Review Queue (review-queue)]

This item has failed automated testing! Results available here http://juju-ci.vapour.ws:8080/job/charm-bundle-test-lxc/3265/

[Konstantinos Tsakalozos (kos.tsakalozos)]

Hi Charles,

thank you for your work here as well! The charm looks good, but i've spotted a couple of issues that need your attention:

======================================================================
FAIL: test_config_options (__main__.TestCharm)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./tests/20-deploy-with-elasticsearch", line 34, in test_config_options
    self.assertTrue('/tmp/amulet.log' in config)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 2 tests in 404.663s

I believe this is because in your reactive layer you are using @when('config.changed') to set a state set_state('beat.render').
Looking at the logs setting that state is not triggering the hook to actually do the rendering of the template file, but even if it did it may have been delayed. The hooks (including render_filebeat_template()) are checked and executed on the next status update.

Easy fixes:
- Missing "repo: https://github.com/juju-solutions/layer-filebeat.git" from layers.yaml
- 20-deploy-with-elasticsearch test is not marked as executable

Cheers,
Konstantinos

[Charles Butler (lazypower)]

When the net review comes in, i'd like to move from that LP branch that once existed as attached to this bug, to the upstream repository here: - "repo: https://github.com/juju-solutions/layer-filebeat.git"

The associated review comments have been patched and this is ready for re-review

[Kevin W Monroe (kwmonroe)]

+1 LGTM Chuck!

I tested filebeat and topbeat with AWS and both seem to be working as expected (that is, after loading the beats dashboard on kibana). I've promulgated this charm:

https://jujucharms.com/filebeat/

One note is that interface-elastic-beats is not published, so 'charm build' on the layered source fails unless you have the elastic-beats repo cloned locally. Please consider adding this interface to the registry so charm building is a bit smoother in the future:

http://interfaces.juju.solutions/

Thanks for the work! I really do like both topbeat and filebeat as alternatives to logstash forwarders. And the kibana tie-in is awesome!