push runtime in benchmark seems off

[gordon chung]

not sure reasoning but there are a few times when push runtime is the same or higher than another benchmark even though the push latency mean/median/95%/etc... are all significantly lower than other benchmark mark run.

i am running 32 workers client against 32 workers server api on separate 48CPU machines

[gordon chung (chungg)]

it also doesn't change in performance if i change client workers from 32 -> 24 -> 12 -> 8. it will push the same rate for all the workers. it does push slower when i drop workers to 2 though.

[gordon chung (chungg)]

just some more info. i tried switching it to ProcessPoolExecutor -> ThreadPoolExecutor and it performed better but still had same performance between 12->48 threads.

i then used 12 threads and ran 3 separate gnocchiclient processes using bash (aka 36 threads) and got consistent performance as running 12 threads with single process. so it seems like either max_workers isn't being honoured, or we process things too quickly that it never needs more than a 6-12 processes to get through 5K batch POST

[Julien Danjou (jdanjou)]

I had trouble to scaling the benchmark client. I tend to blame Python, but doing some profiling might help.