Graylog 3 - properly configure 'http_publish_uri'

[Jeremy Lounder]

For a better understanding of the Graylog configs, and how they are set by the charm today, please refer to this FAQ article: https://answers.launchpad.net/charm-graylog/+faq/3215

We have temporarily removed the ability for the charm to configure `http_publish_uri` because it was being mis-used.

`http_publish_uri` needs to be set to a unique value on each node, and by default the charm will attempt to deduct the network interface that is used for connections to the default gateway and use the associated IP address.

The correct way to configure this would be to either specific a network, or space, or subnet (or something similar) where you want the Graylog communication to occur. The charm would then determine which IP address of the active node can access the specified network and set that IP as the value of `http_publish_uri`

[Jeff Hillman (jhillman)]

This is actually causing a problem on a current customer deploy. This environment does NOT have a gateway configured for OAM (all nodes proxy through MAAS), and thus it fails to find the correct interface.

from unit logs

```
2021-06-08 18:19:31 ERROR juju-log Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-graylog-0/charm/reactive/graylog.py", line 210, in configure_graylog
    validate_api_uri("{}://0.0.0.0:{}/".format(get_api_protocol(), api_port)),
  File "lib/charms/layer/graylog/utils.py", line 63, in validate_api_uri
    netloc="{}:{}".format(get_ip_address(), parsed_url.port)
  File "lib/charms/layer/graylog/utils.py", line 80, in get_ip_address
    iface = netifaces.gateways()["default"][netifaces.AF_INET][1]
KeyError: 2

2021-06-08 18:19:31 WARNING install Traceback (most recent call last):
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/charm/hooks/install", line 22, in <module>
2021-06-08 18:19:31 WARNING install main()
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
2021-06-08 18:19:31 WARNING install bus.dispatch(restricted=restricted_mode)
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
2021-06-08 18:19:31 WARNING install _invoke(other_handlers)
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
2021-06-08 18:19:31 WARNING install handler.invoke()
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
2021-06-08 18:19:31 WARNING install self._action(*args)
2021-06-08 18:19:31 WARNING install File "/var/lib/juju/agents/unit-graylog-0/charm/reactive/graylog.py", line 210, in configure_graylog
2021-06-08 18:19:31 WARNING install validate_api_uri("{}://0.0.0.0:{}/".format(get_api_protocol(), api_port)),
2021-06-08 18:19:31 WARNING install File "lib/charms/layer/graylog/utils.py", line 63, in validate_api_uri
2021-06-08 18:19:31 WARNING install netloc="{}:{}".format(get_ip_address(), parsed_url.port)
2021-06-08 18:19:31 WARNING install File "lib/charms/layer/graylog/utils.py", line 80, in get_ip_address
2021-06-08 18:19:31 WARNING install iface = netifaces.gateways()["default"][netifaces.AF_INET]...

[Jeff Hillman (jhillman)]

subscribed field-high

[Xav Paice (xavpaice)]

https://bugs.launchpad.net/juju/+bug/1863916 is relevant also, planned to land in Juju 2.9.5

[Jeff Hillman (jhillman)]

2.9.5 is now out in the wild

[Eric Chen (eric-chen)]

This issue was pending long time. Is it still valid?
Furthermore, soon or later, the monitoring system will migrate to COS. Telegraf will be replaced by grafana-agent. Therefore, we won't follow up this issue. But welcome to reopen it when anyone encounter this issue again.
(https://charmhub.io/topics/canonical-observability-stack)