Rabbitmq crashes when tokens are rapidly requested

Bug #1748042 reported by Eugene Nikanorov
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Eugene Nikanorov

Bug Description

Fuel 8.x

When radosgw works with several clients uploading small files, rabbitmq may crash.

The following crash is available in /var/lib/rabbitmq/erl_crash.dmp

Slogan: Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k
System version: Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:24:24] [async-threads:10] [kernel-poll:false]
Compiled: Tue Jan 26 04:34:23 2016
Atoms: 4809

tags: added: customer-found
tags: added: support
description: updated
Changed in fuel:
milestone: none → 8.0-updates
assignee: nobody → MOS Maintenance (mos-maintenance)
importance: Undecided → High
importance: High → Medium
importance: Medium → Critical
status: New → Confirmed
milestone: 8.0-updates → 8.0-mu-7
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Unfortunately this crash is not informative by itself, could you please provide more logs? A diagnostic snapshot would be even better.

Changed in fuel:
assignee: MOS Maintenance (mos-maintenance) → Eugene Nikanorov (enikanorov)
status: Confirmed → Incomplete
Revision history for this message
Dmitry Sutyagin (dsutyagin) wrote :

Asked Anatolii to fetch and provide data to you

Revision history for this message
Anatolii Neliubin (aneliubin) wrote :

Folks, Diag snapshot is useless. @Denis, let me know what exact logs is necessary

Just in short. RabbitMQ does not work well under load:
root@<censored>:~# grep -c 7-Feb-2018::19:47:22 /var/log/rabbitmq/rabbit@messaging-<censored>.log
root@<censored>:~# grep -c 7-Feb-2018::19:47:23 /var/log/rabbitmq/rabbit@messaging-<censored>.log
root@<censored>:~# grep -c 7-Feb-2018::19:47:24 /var/log/rabbitmq/rabbit@messaging-<censored>.log
root@<censored>:~# grep -c 7-Feb-2018::19:47:25 /var/log/rabbitmq/rabbit@messaging-<censored>.log
root@<censored>:~# grep -c 7-Feb-2018::19:47:26 /var/log/rabbitmq/rabbit@messaging-<censored>.log

Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

If you say it's under load then you should provide at least rabbimqctl report. Even better would also be screenshots from management plugin with messages rate graphics.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.