Deploying telegraf configs from etc/kayobe impossible?

[Martin Ananda Boeker]

Env: 2023.1

I am trying to deploy a telegraf config just called global.conf to all nodes that have telegraf, in this case my controller.

For prometheus, I can create a file etc/kolla/config/prometheus/prometheus.yml.d/scrape.yml and that gets deployed on the nodes with the prometheus server. In venvs/kolla-ansible/share/kolla-ansible/ansible/roles/prometheus/tasks/config.yml:

- name: Find prometheus common config overrides
  find:
    # NOTE(wszumski): Non-existent paths don't produce a failure
    paths:
      - "{{ node_custom_config }}/prometheus/prometheus.yml.d"
    patterns: "*.yml"
  delegate_to: localhost
  register: prometheus_common_config_overrides_result

and then later:

- name: Copying over prometheus config file
  become: true
  vars:
    service: "{{ prometheus_services['prometheus-server'] }}"
    common_overrides: "{{ prometheus_common_config_overrides_result.files | map(attribute='path') | list }}"

That's great, because then I can have the config files in etc/kayobe/kolla/..

For telegraf, there is a task for copying telegraf plugin files, but the source path seems to be in venvs/kolla-ansible/ansible/roles/telegraf/templates/config/*.conf:

- name: Copying over telegraf plugin files
  vars:
    service: "{{ telegraf_services['telegraf'] }}"
  copy:
    src: "{{ item }}"
    dest: "{{ node_config_directory }}/telegraf/config"
    mode: "0660"
  become: true
  when:
    - inventory_hostname in groups[service.group]
    - item.value.enabled | bool
  with_fileglob:
    - "{{ role_path }}/templates/config/*.conf"
  notify:
    - Restart telegraf container

Am I misunderstanding what's going on here, or do I really need to copy my telegraf config files into the kolla-ansible venv?

Maybe I'm missing something and there is a functionality for this? I've tried putting the .conf under etc/kolla/config/telegraf, etc/kolla/config/telegraf/config, etc, but no luck.

[Martin Ananda Boeker (mboeker)]

I just tried putting the global.conf in venvs/.../telegraf/templates/config/ and got this error:

TASK [telegraf : Copying over telegraf plugin files] ***********************************************************************************************************************************************************************************
Friday 09 February 2024 16:03:46 +0000 (0:00:00.595) 0:00:08.677 *******
fatal: [control0]: FAILED! =>
  msg: |-
    The conditional check 'item.value.enabled | bool' failed. The error was: error while evaluating conditional (item.value.enabled | bool): 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'value'. 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'value'

    The error appears to be in '/home/ubuntu/venvs/kolla-ansible/share/kolla-ansible/ansible/roles/telegraf/tasks/config.yml': line 50, column 3, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

    - name: Copying over telegraf plugin files
      ^ here
fatal: [control1]: FAILED! =>
  msg: |-
    The conditional check 'item.value.enabled | bool' failed. The error was: error while evaluating conditional (item.value.enabled | bool): 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'value'. 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'value'

    The error appears to be in '/home/ubuntu/venvs/kolla-ansible/share/kolla-ansible/ansible/roles/telegraf/tasks/config.yml': line 50, column 3, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

    - name: Copying over telegraf plugin files
      ^ here

Any ideas or guidance would be very appreciated. Thanks!

[Martin Ananda Boeker (mboeker)]

In that block about copying telegraf.conf in the telegraf role:

- name: Copying over telegraf config file
  vars:
    service: "{{ telegraf_services['telegraf'] }}"
  template:
    src: "{{ item }}"
    dest: "{{ node_config_directory }}/telegraf/telegraf.conf"
    mode: "0660"
  become: true
  when:
    - inventory_hostname in groups[service.group]
    - service.enabled | bool
  with_first_found:
    - "{{ node_custom_config }}/telegraf/{{ inventory_hostname }}/telegraf.conf"
    - "{{ node_custom_config }}/telegraf/telegraf.conf"
    - "telegraf.conf.j2"
  notify:
    - Restart telegraf container

This makes me think that if I have a custom etc/kayobe/kolla/config/telegraf/telegraf.conf that it should be the one deployed to all nodes in the telegraf group, but that was ignored, instead it took the one from the venv.

EDIT: I think it's because there's no entry for /etc/kolla/config/telegraf on the Ensure the Kolla OpenStack configuration directories exist:

TASK [kolla-openstack : Ensure the Kolla OpenStack configuration directories exist] ****************************************************************************************************************************************************
Saturday 10 February 2024 14:36:35 +0000 (0:00:00.091) 0:00:09.832 *****
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/aodh', 'dest': '/etc/kolla/config/aodh', 'patterns': '*', 'enabled': 'no'})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/barbican', 'dest': '/etc/kolla/config/barbican', 'patterns': '*', 'enabled': 'no'})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/blazar', 'dest': '/etc/kolla/config/blazar', 'patterns': '*', 'enabled': 'no'})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/ceilometer', 'dest': '/etc/kolla/config/ceilometer', 'patterns': '*', 'enabled': 'no'})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config/cinder', 'dest': '/etc/kolla/config/cinder', 'patterns': '*', 'enabled': True})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/cloudkitty', 'dest': '/etc/kolla/config/cloudkitty', 'patterns': '*', 'enabled': 'no'})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/designate', 'dest': '/etc/kolla/config/designate', 'patterns': '*', 'enabled': 'no'})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config//fluentd/filter', 'dest': '/etc/kolla/config/fluentd/filter', 'patterns': '*.conf', 'enabled': True})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config//fluentd/input', 'dest': '/etc/kolla/config/fluentd/input', 'patterns': '*.conf', 'enabled': True})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config/fluentd/output', 'dest': '/etc/kolla/config/fluentd/output', 'patterns': '*.conf', 'enabled': True})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config/glance', 'dest': '/etc/kolla/config/glance', 'patterns': '*', 'enabled': True})
skipping: [localhost] => (item={'src': '/etc/kayobe/kolla/config/gnocchi', 'dest': '/etc/kolla/config/gnocchi', 'patterns': '*', 'enabled': 'no'})
ok: [localhost] => (item={'src': '/etc/kayobe/kolla/config/grafana', 'dest': '/etc/kolla/config/grafana', 'patt...

[Martin Ananda Boeker (mboeker)]

So while kayobe doesn't seem to synchronize etc/kayobe/kolla/config/telegraf to etc/kolla/config/telegraf, it does deploy the configuration of etc/kolla/config/telegraf/telegraf.conf.

It does not however deploy the configuration of etc/kolla/config/telegraf/config/*.conf.

[Will Szumski (willjs)]

Good spot. We don't often deploy telegraf, so haven't noticed this one. I assume this is because we are missing telegraf in the list of globs defined here: https://github.com/openstack/kayobe/blob/bb51db15febc195384a48406cb37571776988ee3/ansible/roles/kolla-openstack/defaults/main.yml#L32

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/kayobe/+/909600

[Will Szumski (willjs)]

Is it possible to try out https://review.opendev.org/c/openstack/kayobe/+/909600 to see if fixes your issue?

You can use Download patch option from the UI to cherry-pick the change to your local checkout.

EDIT: corrected broken link

[Martin Ananda Boeker (mboeker)]

Hi Will,

Getting this:

An error occurred
You might have not enough privileges. Error 404 (Not Found): Not found:
openstack/kayobe~90960 Endpoint: /changes/*~*/drafts

I signed in with my launchpad/ubuntu one login.

Sincerely | Mit freundlichen Grüßen

Martin Ananda Boeker
EDV-Consult IT GmbH

On Tue, Feb 20, 2024 at 6:15 PM Will Szumski <email address hidden>
wrote:

> Is it possible to try out
> https://review.opendev.org/c/openstack/kayobe/+/90960 to see if fixes
> your issue?
>
> You can use Download patch option from the UI to cherry-pick the change
> to your local checkout.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2052819
>
> Title:
> Deploying telegraf configs from etc/kayobe impossible?
>
> Status in kayobe:
> In Progress
>
> Bug description:
> Env: 2023.1
>
> I am trying to deploy a telegraf config just called global.conf to all
> nodes that have telegraf, in this case my controller.
>
> For prometheus, I can create a file
> etc/kolla/config/prometheus/prometheus.yml.d/scrape.yml and that gets
> deployed on the nodes with the prometheus server. In venvs/kolla-
> ansible/share/kolla-ansible/ansible/roles/prometheus/tasks/config.yml:
>
> - name: Find prometheus common config overrides
> find:
> # NOTE(wszumski): Non-existent paths don't produce a failure
> paths:
> - "{{ node_custom_config }}/prometheus/prometheus.yml.d"
> patterns: "*.yml"
> delegate_to: localhost
> register: prometheus_common_config_overrides_result
>
> and then later:
>
> - name: Copying over prometheus config file
> become: true
> vars:
> service: "{{ prometheus_services['prometheus-server'] }}"
> common_overrides: "{{
> prometheus_common_config_overrides_result.files | map(attribute='path') |
> list }}"
>
>
> That's great, because then I can have the config files in
> etc/kayobe/kolla/..
>
> For telegraf, there is a task for copying telegraf plugin files, but
> the source path seems to be in venvs/kolla-
> ansible/ansible/roles/telegraf/templates/config/*.conf:
>
>
> - name: Copying over telegraf plugin files
> vars:
> service: "{{ telegraf_services['telegraf'] }}"
> copy:
> src: "{{ item }}"
> dest: "{{ node_config_directory }}/telegraf/config"
> mode: "0660"
> become: true
> when:
> - inventory_hostname in groups[service.group]
> - item.value.enabled | bool
> with_fileglob:
> - "{{ role_path }}/templates/config/*.conf"
> notify:
> - Restart telegraf container
>
>
> Am I misunderstanding what's going on here, or do I really need to
> copy my telegraf config files into the kolla-ansible venv?
>
> Maybe I'm missing something and there is a functionality for this?
> I've tried putting the .conf under etc/kolla/config/telegraf,
> etc/kolla/config/telegraf/config, etc, but no luck.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/kayobe/+bug/2052819/+subscriptions
>
>

[Will Szumski (willjs)]

Sorry, link was missing some digits. This is the patch:

https://review.opendev.org/c/openstack/kayobe/+/909600

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/kayobe/+/909600
Committed: https://opendev.org/openstack/kayobe/commit/e39b9a94cdfd3f4ff6c59499c08d179ae710f4fd
Submitter: "Zuul (22348)"
Branch: master

commit e39b9a94cdfd3f4ff6c59499c08d179ae710f4fd
Author: Will Szumski <email address hidden>
Date: Tue Feb 20 16:22:37 2024 +0000

    Support customising telegraf configuration files

    Kayobe was not copying the telegraf configuration files in the generated
    kolla config directory.

    Closes-Bug: #2052819
    Change-Id: I93296b0671aeab9f2cd01de73da42071448a9f2a