oslo.messaging

Bug #1800957
Comment #39

Comment 39 for bug 1800957

Revision history for this message

Ken Giusti (kgiusti) wrote on 2019-01-14:

#39

Here's a description of my environment and the results of a few test runs:

I am running rabbitmq in the container as described in your previous
post.

I am running the clients on my fedora 28 x86_64 machine:

$ python --version
Python 2.7.15

$ uname -a
Linux localhost.localdomain 4.19.13-200.fc28.x86_64 #1 SMP Sat Dec 29 23:10:35 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ openssl version
OpenSSL 1.1.0i-fips 14 Aug 2018

In my local oslo.messaging repo I've checked out stable/pike and
modified the simulator.py as you've described:

$ git diff
diff --git a/tools/simulator.py b/tools/simulator.py
index f952df0a..871ea33c 100755
--- a/tools/simulator.py
+++ b/tools/simulator.py
@@ -472,10 +472,12 @@ def generate_messages(messages_count):
     LOG.info("Generating %d random messages", messages_count)
     generator = init_random_generator()
     for i in six.moves.range(messages_count):
- length = generator()
- msg = ''.join(random.choice(
- string.ascii_lowercase) for x in six.moves.range(length))
- MESSAGES.append(msg)
+ # length = generator()
+ # msg = ''.join(random.choice(
+ # string.ascii_lowercase) for x in
+ # six.moves.range(length))
+ length = 2000000
+ MESSAGES.append("A"*length)

LOG.info("Messages has been prepared")

I've created the SSL configuration file:

$ cat test.conf
[DEFAULT]
transport_url = rabbit://test:test@localhost:5671
[oslo_messaging_rabbit]
ssl = True

And installed oslo.messaging + deps in a virtualenv:

$ pip list | grep oslo
oslo.concurrency 3.29.0
oslo.config 6.7.0
oslo.context 2.22.0
oslo.i18n 3.23.0
oslo.log 3.42.2
oslo.messaging 5.30.7.dev1
oslo.middleware 3.37.0
oslo.serialization 2.28.1
oslo.service 1.34.0
oslo.utils 3.39.1

I'm running the SSL test clients like this:
$ python simulator.py --config-file test.conf rpc-server &
$ python simulator.py --config-file test.conf rpc-client -p 1 -w 1 -m 200 --timeout 10

Verified all traffic passed via TLS1.2 (wireshark dump)

Output of three runs (no unacked msgs seen):

2019-01-14 11:28:50,453 INFO root =================================== summary ===================================
2019-01-14 11:28:50,453 INFO root client: duration: 254.24 count: 200 (0.8 msg/sec) bytes: 400000000 (1573303 bps)
2019-01-14 11:28:50,453 INFO root error: duration: 0.00 count: 0 (0.0 msg/sec) bytes: 0 (0 bps)
2019-01-14 11:28:50,453 INFO root round-trip: duration: 254.24 count: 200 (0.8 msg/sec) bytes: 400000000 (1573303 bps) latency: 0.271 min: 0.192 max: 1.335

2019-01-14 11:41:10,798 INFO root =================================== summary ===================================
2019-01-14 11:41:10,799 INFO root client: duration: 259.28 count: 200 (0.8 msg/sec) bytes: 400000000 (1542724 bps)
2019-01-14 11:41:10,799 INFO root error: duration: 0.00 count: 0 (0.0 msg/sec) bytes: 0 (0 bps)
2019-01-14 11:41:10,800 INFO root round-trip: duration: 258.28 count: 200 (0.8 msg/sec) bytes: 400000000 (1548706 bps) latency: 0.293 min: 0.190 max: 1.374

2019-01-14 11:41:22,700 INFO root =================================== summary ===================================
2019-01-14 11:41:22,701 INFO root server: duration: 258.31 count: 200 (0.8 msg/sec) bytes: 400000000 (1548510 bps) latency: 0.145 min: 0.091 max: 1.250

A third run under heavy load: one message gets dropped (call times out), but no unacked msgs on queues:

2019-01-14 12:06:32,619 INFO root =================================== summary ===================================
2019-01-14 12:06:32,620 INFO root client: duration: 831.16 count: 200 (0.2 msg/sec) bytes: 400000000 (481258 bps)
2019-01-14 12:06:32,620 INFO root error: duration: 1.00 count: 1 (1.0 msg/sec) bytes: 2000000 (2000000 bps)
2019-01-14 12:06:32,620 INFO root round-trip: duration: 830.15 count: 199 (0.2 msg/sec) bytes: 398000000 (479429 bps) latency: 0.293 min: 0.186 max: 2.352

2019-01-14 12:07:26,186 INFO root =================================== summary ===================================
2019-01-14 12:07:26,187 INFO root server: duration: 830.23 count: 199 (0.2 msg/sec) bytes: 398000000 (479387 bps) latency: 0.143 min: 0.090 max: 1.168

I'm running the clients in non-ssl mode like this:
$ python simulator.py --url rabbit://localhost:5672 rpc-server &
$ python simulator.py --url rabbit://localhost:5672 rpc-client -p 1 -w 1 -m 200 --timeout 10

Output of two runs

2019-01-14 11:34:14,968 INFO root =================================== summary ===================================
2019-01-14 11:34:14,968 INFO root client: duration: 217.29 count: 200 (0.9 msg/sec) bytes: 400000000 (1840823 bps)
2019-01-14 11:34:14,968 INFO root error: duration: 0.00 count: 0 (0.0 msg/sec) bytes: 0 (0 bps)
2019-01-14 11:34:14,969 INFO root round-trip: duration: 216.23 count: 200 (0.9 msg/sec) bytes: 400000000 (1849909 bps) latency: 0.084 min: 0.047 max: 0.810

2019-01-14 11:34:52,388 INFO root =================================== summary ===================================
2019-01-14 11:34:52,389 INFO root server: duration: 216.13 count: 200 (0.9 msg/sec) bytes: 400000000 (1850762 bps) latency: 0.046 min: 0.020 max: 0.773

2019-01-14 11:49:33,486 INFO root =================================== summary ===================================
2019-01-14 11:49:33,486 INFO root client: duration: 218.21 count: 200 (0.9 msg/sec) bytes: 400000000 (1833069 bps)
2019-01-14 11:49:33,486 INFO root error: duration: 0.00 count: 0 (0.0 msg/sec) bytes: 0 (0 bps)
2019-01-14 11:49:33,486 INFO root round-trip: duration: 217.21 count: 200 (0.9 msg/sec) bytes: 400000000 (1841516 bps) latency: 0.086 min: 0.049 max: 1.173

2019-01-14 11:49:45,468 INFO root =================================== summary ===================================
2019-01-14 11:49:45,468 INFO root server: duration: 217.14 count: 200 (0.9 msg/sec) bytes: 400000000 (1842114 bps) latency: 0.051 min: 0.023 max: 1.133

Here's the rabbitmq queue listing after 4 test runs:

Every 2.0s: rabbitmqctl list_queues name messages_u... Mon Jan 14 16:49:44 2019

Listing queues
reply_10de6283cce047769e2830c487e809d8 0
reply_82c3983ed54241b88f3941e692bfd9aa 0
reply_ea93a026556b409183eda9273f1111ae 0
reply_ef2e491a12774864af8e470b13c40b01 0
profiler_topic_fanout_cb5d5e4903d643659b05ebc96763520f 0
profiler_topic.profiler_server 0
profiler_topic 0

Here's a description of my environment and the results of a few test runs:

I am running rabbitmq in the container as described in your previous
post.

I am running the clients on my fedora 28 x86_64 machine:

$ python --version
Python 2.7.15

$ uname -a
Linux localhost.localdomain 4.19.13-200.fc28.x86_64 #1 SMP Sat Dec 29 23:10:35 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ openssl version
OpenSSL 1.1.0i-fips  14 Aug 2018

In my local oslo.messaging repo I've checked out stable/pike and
modified the simulator.py as you've described:

$ git diff
diff --git a/tools/simulator.py b/tools/simulator.py
index f952df0a..871ea33c 100755
--- a/tools/simulator.py
+++ b/tools/simulator.py
@@ -472,10 +472,12 @@ def generate_messages(messages_count):
     LOG.info("Generating %d random messages", messages_count)
     generator = init_random_generator()
     for i in six.moves.range(messages_count):
-        length = generator()
-        msg = ''.join(random.choice(
-                      string.ascii_lowercase) for x in six.moves.range(length))
-        MESSAGES.append(msg)
+        # length = generator()
+        # msg = ''.join(random.choice(
+        #               string.ascii_lowercase) for x in
+        #               six.moves.range(length))
+        length = 2000000
+        MESSAGES.append("A"*length)
 
     LOG.info("Messages has been prepared")

I've created the SSL configuration file:

$ cat test.conf
[DEFAULT]
transport_url = rabbit://test:test@localhost:5671
[oslo_messaging_rabbit]
ssl = True

And installed oslo.messaging + deps in a virtualenv:

$ pip list | grep oslo
oslo.concurrency   3.29.0     
oslo.config        6.7.0      
oslo.context       2.22.0     
oslo.i18n          3.23.0     
oslo.log           3.42.2     
oslo.messaging     5.30.7.dev1
oslo.middleware    3.37.0     
oslo.serialization 2.28.1     
oslo.service       1.34.0     
oslo.utils         3.39.1

I'm running the SSL test clients like this:
$ python simulator.py --config-file test.conf rpc-server &
$ python simulator.py --config-file test.conf rpc-client -p 1 -w 1 -m 200 --timeout 10

Verified all traffic passed via TLS1.2 (wireshark dump)

Output of three runs (no unacked msgs seen):

A third run under heavy load: one message gets dropped (call times out), but no unacked msgs on queues:

Output of two runs

Here's the rabbitmq queue listing after 4 test runs:

Every 2.0s: rabbitmqctl list_queues name messages_u...  Mon Jan 14 16:49:44 2019

Listing queues
reply_10de6283cce047769e2830c487e809d8  0
reply_82c3983ed54241b88f3941e692bfd9aa  0
reply_ea93a026556b409183eda9273f1111ae  0
reply_ef2e491a12774864af8e470b13c40b01  0
profiler_topic_fanout_cb5d5e4903d643659b05ebc96763520f  0
profiler_topic.profiler_server  0
profiler_topic  0