Rabbitmq crashes when tokens are rapidly requested

Bug #1748042 reported by Eugene Nikanorov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Incomplete
Critical
Eugene Nikanorov

Bug Description

Fuel 8.x

When radosgw works with several clients uploading small files, rabbitmq may crash.

The following crash is available in /var/lib/rabbitmq/erl_crash.dmp

Slogan: Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k
System version: Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:24:24] [async-threads:10] [kernel-poll:false]
Compiled: Tue Jan 26 04:34:23 2016
Taints:
Atoms: 4809

tags: added: customer-found
tags: added: support
description: updated
Changed in fuel:
milestone: none → 8.0-updates
assignee: nobody → MOS Maintenance (mos-maintenance)
importance: Undecided → High
importance: High → Medium
importance: Medium → Critical
status: New → Confirmed
milestone: 8.0-updates → 8.0-mu-7
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Unfortunately this crash is not informative by itself, could you please provide more logs? A diagnostic snapshot would be even better.

Changed in fuel:
assignee: MOS Maintenance (mos-maintenance) → Eugene Nikanorov (enikanorov)
status: Confirmed → Incomplete
Revision history for this message
Dmitry Sutyagin (dsutyagin) wrote :

Asked Anatolii to fetch and provide data to you

Revision history for this message
Anatolii Neliubin (aneliubin) wrote :

Folks, Diag snapshot is useless. @Denis, let me know what exact logs is necessary

Just in short. RabbitMQ does not work well under load:
root@<censored>:~# grep -c 7-Feb-2018::19:47:22 /var/log/rabbitmq/rabbit@messaging-<censored>.log
0
root@<censored>:~# grep -c 7-Feb-2018::19:47:23 /var/log/rabbitmq/rabbit@messaging-<censored>.log
297
root@<censored>:~# grep -c 7-Feb-2018::19:47:24 /var/log/rabbitmq/rabbit@messaging-<censored>.log
2210
root@<censored>:~# grep -c 7-Feb-2018::19:47:25 /var/log/rabbitmq/rabbit@messaging-<censored>.log
0
root@<censored>:~# grep -c 7-Feb-2018::19:47:26 /var/log/rabbitmq/rabbit@messaging-<censored>.log
0
root@<censored>:~#

Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

If you say it's under load then you should provide at least rabbimqctl report. Even better would also be screenshots from management plugin with messages rate graphics.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers