AMQP server is unreachable

[Marian Gasparovic]

Deploying k8son top of Openstack.
Juju controller on top of openstack deploys fine, then all k8s machines stays in allocating state for hours until our CI eventually times out.

In n-c-c unit, nova-api-wsgi.log I can see
```
2021-11-21 17:41:40.869 179112 ERROR oslo.messaging._drivers.impl_rabbit [-] [0f98250f-94e9-4495-ae22-db16a5e27899] AMQP server on 192.168.33.203:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 s
econds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error>
2021-11-21 17:41:40.870 179112 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer
2021-11-21 17:41:40.879 179112 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer
2021-11-21 17:41:40.887 179112 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer

```
It is after 17:30 when k8s deployment started. I can see similar messages also earlier, but only sporadically, after 17:30 there is a lot of them and they don't seem to be recovering

Logs and artifacts
https://oil-jenkins.canonical.com/artifacts/bede36d6-d7ce-492b-ad17-563ff436fd92/index.html

openstack juju-crashdump
https://oil-jenkins.canonical.com/artifacts/bede36d6-d7ce-492b-ad17-563ff436fd92/generated/generated/openstack/juju-crashdump-openstack-2021-11-21-21.38.47.tar.gz

[Alex Kavanagh (ajkavanagh)]

Without the rabbit logs it's really difficult to see what is going on unfortunately. It could just have been a very busy system and rabbit was dropping its connections?

[Launchpad Janitor (janitor)]

[Expired for OpenStack RabbitMQ Server Charm because there has been no activity for 60 days.]