Comment 35 for bug 1800957

Revision history for this message
Antonio Ojea (aojea) wrote :

@kgiusti I think that I'm able to reproduce the issue only with the tools/simulator.py script that's in the oslo.messaging repository.

These are the steps that I followed

* Install a rabbitmq server with ssl enable

docker run -d --rm --name rabbitssl -p 5672:5672 -p 5671:5671 -p 61613:61613 -p 61614:61614 -p 15672:15672 mehdijrgr/docker-rabbitmq-ssl

* Open a shell on the container to configure a test user ...
$ docker exec -it rabbitssl bash

root@ef4c4bce08a9:/#
root@ef4c4bce08a9:/# rabbitmqctl add_user test test
Creating user "test"
root@ef4c4bce08a9:/# rabbitmqctl set_user_tags test administrator
Setting tags for user "test" to [administrator]
root@ef4c4bce08a9:/# rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
Setting permissions for user "test" in vhost "/"

* ... and start monitoring the queues

root@ef4c4bce08a9:/# watch -c "rabbitmqctl list_queues name messages_unacknowledged"

* Install oslo-messaging out of the container

$ git clone git://git.openstack.org/openstack/oslo.messaging
$ cd oslo.messaging/
$ git checkout stable/pike
$ virtualenv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ python setup.py install

* configure oslo.messaging.conf to use the SSL connection (I had to generate a config file to be able to use SSL)

$ oslo-config-generator --namespace oslo.messaging > oslo.messaging.conf

[DEFAULT]
transport_url = rabbit://test:test@localhost:5671
[oslo_messaging_rabbit]
ssl = True

* Modify the simulator.py to generate a large message with a fixed size

diff --git a/tools/simulator.py b/tools/simulator.py
index f952df0a..8de50ac8 100755
--- a/tools/simulator.py
+++ b/tools/simulator.py
@@ -472,9 +472,8 @@ def generate_messages(messages_count):
     LOG.info("Generating %d random messages", messages_count)
     generator = init_random_generator()
     for i in six.moves.range(messages_count):
- length = generator()
- msg = ''.join(random.choice(
- string.ascii_lowercase) for x in six.moves.range(length))
+ length = 2000000
+ msg = "A"*length
         MESSAGES.append(msg)

     LOG.info("Messages has been prepared")

* without SSL it works without errors

$ python tools/simulator.py --url rabbit://localhost:5672 rpc-client -p 1 -w 1 -m 200 --timeout 10
$ python tools/simulator.py --url rabbit://localhost:5672 rpc-server

2019-01-11 13:47:58,728 INFO root =================================== summary ===================================
2019-01-11 13:47:58,728 INFO root client: duration: 223.15 count: 200 (0.9 msg/sec) bytes: 400000000 (1792476 bps)
2019-01-11 13:47:58,729 INFO root error: duration: 0.00 count: 0 (0.0 msg/sec) bytes: 0 (0 bps)
2019-01-11 13:47:58,729 INFO root round-trip: duration: 222.20 count: 200 (0.9 msg/sec) bytes: 400000000 (1800187 bps) latency: 0.113 min: 0.045 max: 1.492

* start a test using SSL

$ python tools/simulator.py --config-file oslo.messaging.conf rpc-client -p 1 -w 1 -m 200 --timeout 10
$ python tools/simulator.py --config-file oslo.messaging.conf rpc-server

With SSL one of the queues starts to have unacknowledged messages, not always in the same queue (sometimes profiler_topic.profiler_server, others the reply_ queue) neither after the same time, but if I send a high number of messages (200 msgs in my tests) I always hit the problem.

Every 2.0s: rabbitmqctl list_queues name messages_unacknowledged Fri Jan 11 12:59:55 2019

Listing queues
profiler_topic_fanout_9c487c073bc345af9414695da7a47dc8 0
reply_dc2387abfc814b3e92f7b93669c2862f 7
profiler_topic.profiler_server 0
profiler_topic 0

The latencies are higher than without SSL but not close to the timeout

2019-01-11 14:06:55,319 INFO root server : seq: 1014 count: 1 bytes: 2000000 latency: 0.115 min: 0.115 max: 0.115