Murano-agent does not get rabbit settings

[Sergey Murashov]

ISO: {"build_id": "2014-07-24_00-31-14", "mirantis": "yes", "build_number": "145", "ostf_sha": "09b6bccf7d476771ac859bb3c76c9ebec9da9e1f", "nailgun_sha": "17444180b7e8c0c454488e63a05693881168f76a", "production": "docker", "api": "1.0", "fuelmain_sha": "4c90c6941889992858daa8417d6bc1759135ea4e", "astute_sha": "6db5f5031b74e67b92fcac1f7998eaa296d68025", "release": "5.0.1", "fuellib_sha": "a24ed802806fd39368f9ca8e043867b3c322d37e"}

Steps to reproduce:
1. Centos+GRE+3 Controller+1Compute+3 Ceph+Murano
2. Create environment
3. Add service
4. Send to deploy

Actual result:
Deployment doesn't finish, murano-agent doesn't get rabbit settings

[Igor Yozhikov (iyozhikov)]

metadata user_data unreachable -> murano-agent does not receive proper configuration

[Serg Melikyan (smelikyan)]

Murano configures murano-agent component via cloud_init. CloudInit works through metadata, that is not available on VM in deployed environment.

[Igor Yozhikov (iyozhikov)]

I see this in nova-all.log
<179>Jul 24 13:34:56 node-1 nova-oslo.messaging._drivers.impl_rabbit ERROR: Failed to publish message to topic 'conductor': [Errno 32] Broken pipe
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 630, in ensure
    return method(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 726, in _publish
    publisher = cls(self.conf, self.channel, topic, **kwargs)
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 383, in __init__
    **options)
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 330, in __init__
    self.reconnect(channel)
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 338, in reconnect
    routing_key=self.routing_key)
  File "/usr/lib/python2.6/site-packages/kombu/messaging.py", line 84, in __init__
    self.revive(self._channel)
  File "/usr/lib/python2.6/site-packages/kombu/messaging.py", line 218, in revive
    self.declare()
  File "/usr/lib/python2.6/site-packages/kombu/messaging.py", line 104, in declare
    self.exchange.declare()
  File "/usr/lib/python2.6/site-packages/kombu/entity.py", line 166, in declare
    nowait=nowait, passive=passive,
  File "/usr/lib/python2.6/site-packages/amqp/channel.py", line 613, in exchange_declare
    self._send_method((40, 10), args)
  File "/usr/lib/python2.6/site-packages/amqp/abstract_channel.py", line 56, in _send_method
    self.channel_id, method_sig, args, content,
  File "/usr/lib/python2.6/site-packages/amqp/method_framing.py", line 221, in write_method
    write_frame(1, channel, payload)
  File "/usr/lib/python2.6/site-packages/amqp/transport.py", line 177, in write_frame
    frame_type, channel, size, payload, 0xce,
  File "/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 309, in sendall
    tail = self.send(data, flags)
  File "/usr/lib/python2.6/site-packages/eventlet/greenio.py", line 295, in send
    total_sent += fd.send(data[total_sent:], flags)
error: [Errno 32] Broken pipe
....
<179>Jul 24 13:35:01 node-1 nova-oslo.messaging._drivers.impl_rabbit ERROR: Failed to consume message from queue: Socket closed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 630, in ensure
    return method(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/oslo/messaging/_drivers/impl_rabbit.py", line 710, in _consume
    return self.connection.drain_events(timeout=timeout)
  File "/usr/lib/python2.6/site-packages/kombu/connection.py", line 279, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
  File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqp.py", line 91, in drain_events
    return connection.drain_events(**kwargs)
  File "/usr/lib/python2.6/site-packages/amqp/connection.py", line 299, in drain_events
    chanmap, None, timeout=timeout,
  File "/usr/lib/python2.6/site-packages/amqp/connection.py", line 362, in _wait_multiple
    channel, method_sig, args, content = read_timeout(timeout)
  File "...

[Igor Yozhikov (iyozhikov)]

after restarting of nova-api service, user-data from nova-metadata became accessible for some period of time

[Igor Yozhikov (iyozhikov)]

During analysis of nova-all.log files was found errors related to oslo-messaging and rabbitmq.

[Serg Melikyan (smelikyan)]

Please, see this issues for more details about issue with oslo.messaging in MOS:
https://bugs.launchpad.net/bugs/856764
https://bugs.launchpad.net/bugs/1340711
https://bugs.launchpad.net/bugs/1348082