Allow plugins to extend diagnostic snapshot rules

[Denis Klepikov]

Diagnostic snapshot from environments with the plugins installed does not contain any logs what generated by plugins.

(found on environment with:

# fuel plugins
DEPRECATION WARNING: /etc/fuel/client/config.yaml exists and will be used as the source for settings. This behavior is deprecated. Please specify the path to your custom settings file in the FUELCLIENT_CUSTOM_SETTINGS environment variable.
id | name | version | package_version
---|-----------------------------|---------|----------------
1 | elasticsearch_kibana | 0.8.0 | 3.0.0
2 | influxdb_grafana | 0.8.0 | 3.0.0
3 | lma_collector | 0.8.0 | 2.0.0
4 | lma_infrastructure_alerting | 0.8.0 | 3.0.0

)

# fuel nodes
DEPRECATION WARNING: /etc/fuel/client/config.yaml exists and will be used as the source for settings. This behavior is deprecated. Please specify the path to your custom settings file in the FUELCLIENT_CUSTOM_SETTINGS environment variable.
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---|----------|------------------|---------|-------------|-------------------|-------------------------|---------------|--------|---------
4 | ready | Untitled (76:8c) | 1 | 10.220.0.7 | 64:73:aa:77:76:8c | controller | | True | 1
10 | discover | Untitled (e1:e8) | None | 10.220.0.4 | 64:f9:ab:0f:e1:e8 | | | True | None
2 | ready | Untitled (67:88) | 1 | 10.220.0.8 | 64:10:3b:0e:67:88 | elasticsearch_kibana | | True | 1
9 | ready | Untitled (89:26) | 1 | 10.220.0.6 | 64:d6:92:5f:89:26 | influxdb_grafana | | True | 1
6 | ready | Untitled (c0:13) | 1 | 10.220.0.10 | 64:10:75:3c:c0:13 | ceph-osd, compute | | True | 1
1 | ready | Untitled (81:53) | 1 | 10.220.0.3 | 64:b2:43:db:81:53 | infrastructure_alerting | | True | 1
8 | ready | Untitled (ff:83) | 1 | 10.220.0.9 | 64:de:5a:31:ff:83 | ceph-osd, compute | | True | 1
3 | ready | Untitled (4c:ad) | 1 | 10.220.0.11 | 64:4d:03:47:4c:ad | controller | | True | 1
5 | ready | Untitled (60:85) | 1 | 10.220.0.5 | 64:5f:02:2f:60:85 | controller | | True | 1
7 | ready | Untitled (a5:89) | 1 | 10.220.0.12 | 64:a0:31:b9:a5:89 | ceph-osd, compute | | True | 1

root@node-1:/var/log# ls -la | grep heka
-rw-r--r-- 1 root root 1141 Feb 1 15:38 hekad.log
-rw-rw---- 1 heka heka 1879 Feb 1 17:06 lma_collector.log
root@node-1:/var/log# ls -la | grep lma

:~/ssd/TEMP1$ find . -name 'puppet.log'
./fuel-snapshot-2016-02-11_13-19-39/node-8.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-9.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-3.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-5.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-4.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-6.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-7.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-2.domain.local/var/log/puppet.log
./fuel-snapshot-2016-02-11_13-19-39/node-1.domain.local/var/log/puppet.log
:~/ssd/TEMP1$ find . -name 'hekad.log'
:~/ssd/TEMP1$ find . -name 'influxd.log'
:~/ssd/TEMP1$ find . -name 'lma_collector.log'
:~/ssd/TEMP1$ ls -la
total 12
drwxrwxr-x 3 dklepikov dklepikov 4096 Feb 11 19:01 .
drwxr-xr-x 26 dklepikov dklepikov 4096 Feb 11 19:01 ..
drwxr-xr-x 13 dklepikov dklepikov 4096 Feb 11 15:31 fuel-snapshot-2016-02-11_13-19-39

Please check all plugins for this issue.

[Maciej Kwiek (maciej-iai)]

Please provide the diagnostic snapshot that. It may be useful to have shotgun.log to see what happened.

[Denis Klepikov (dklepikov)]

the diagnostic snapshot:
https://drive.google.com/open?id=0B9BrZVVbs7L0OFpVLVpiM3Z1RzQ

[Denis Klepikov (dklepikov)]

# cat /etc/fuel/version.yaml
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "301"
  build_id: "301"
  nailgun_sha: "4162b0c15adb425b37608c787944d1983f543aa8"
  python-fuelclient_sha: "486bde57cda1badb68f915f66c61b544108606f3"
  fuel-agent_sha: "50e90af6e3d560e9085ff71d2950cfbcca91af67"
  fuel-nailgun-agent_sha: "d7027952870a35db8dc52f185bb1158cdd3d1ebd"
  astute_sha: "6c5b73f93e24cc781c809db9159927655ced5012"
  fuel-library_sha: "5d50055aeca1dd0dc53b43825dc4c8f7780be9dd"
  fuel-ostf_sha: "2cd967dccd66cfc3a0abd6af9f31e5b4d150a11c"
  fuelmain_sha: "a65d453215edb0284a2e4761be7a156bb5627677"

[Maciej Kwiek (maciej-iai)]

Need to verify that it's in 9.0

[Matthew Mosesohn (raytrac3r)]

My interpretation of this bug report is these 3 features don't exist:
1 - nailgun ingest plugin logs (or removal) showing details about plugin metadata parsing
2 - a feature to allow a plugin to add to settings.LOGS so their logs are captured from the new deployed service
3 - fuel plugins from fuel master should be listed in snapshot somewhere

[Maciej Kwiek (maciej-iai)]

Matthew hit the nail in the head - we need a feature that will allow user to add logs from slave nodes to master. Or maybe we should just download whole /var/log from slaves?

[Dmitry Pyzhov (dpyzhov)]

This is a feature request. Moving to the 10.0 release.

[Dmitry Pyzhov (dpyzhov)]

I agree that it is a required piece of functionality. However this kind of changes cannot be distributed with our maintenance updates.