Cassandra exited with error 'Too many open files in system'
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Juniper Openstack | Status tracked in Trunk | |||||
R2.20 |
Fix Committed
|
High
|
Megh Bhatt | |||
R2.21.x |
Fix Committed
|
High
|
Megh Bhatt | |||
R2.22.x |
Fix Committed
|
High
|
Megh Bhatt | |||
R3.0 |
Fix Committed
|
High
|
Megh Bhatt | |||
Trunk |
Fix Committed
|
High
|
Megh Bhatt |
Bug Description
Logs saved at :http://
Build:3.02 35 kilo
Had the setup as below
Setup was running around 140 vms with traffic.
The cassandra got exited in all the 3 database nodes with the below error
org.apache.
Logs
====
WARN [Thread-13] 2016-05-14 21:41:20,331 CustomTThreadPo
org.apache.
at org.apache.
at org.apache.
at org.apache.
at org.apache.
at org.apache.
Caused by: java.net.
at java.net.
at java.net.
at java.net.
at java.net.
at org.apache.
... 4 common frames omitted
root@nodei27:~#
root@nodei27:~#
root@nodei27:~# uptime
12:31:58 up 5 days, 2:33, 1 user, load average: 3.08, 2.11, 1.87
setup
======
host1 = 'root@10.
host2 = 'root@10.
host3 = 'root@10.
host4 = 'root@10.
host5 = 'root@10.
host6 = 'root@10.
host7 = 'root@10.
host8 = 'root@10.
host9 = 'root@10.
host10 = 'root@10.
host11 = 'root@10.
host12 = 'root@10.
host13 = 'root@10.
host14 = 'root@10.
host15 = 'root@10.
host16 = 'root@10.
host17 = 'root@10.
host18 = 'root@10.
host19 = 'root@10.
host20 = 'root@10.
host21 = 'root@10.
host22 = 'root@10.
host23 = 'root@10.
ext_routers = [('blr-mx2', '10.204.216.245')]
router_asn = 64512
public_vn_rtgt = 30001
#public_vn_subnet = "10.204.219.72/29"
host_build = 'vjoshi@
env.roledefs = {
'all': [host1, host2, host3, host4, host5, host6,host7, host8, host9, host10, host11, host12, host13, host14, host15,
'cfgm': [host1, host2, host3],
'openstack': [host4, host5, host6],
'webui': [host1, host2, host3],
'control': [host1, host2, host3],
'compute': [host7, host8, host9, host10, host11, host12, host13, host14, host15,
'collector': [host1, host2, host3],
'database': [host1, host2, host3],
'build': [host_build],
}
env.hostnames = {
'all': ['nodei27', 'nodei28', 'nodei35', 'nodei32', 'nodei35', 'nodei36', 'nodei37', 'nodei38', 'nodel4', 'nodel7', 'nodel8', 'nodel9', 'nodel10', 'nodel11', 'nodel12'
}
Changed in juniperopenstack: | |
assignee: | Raj Reddy (rajreddy) → Megh Bhatt (meghb) |
importance: | Undecided → High |
tags: | added: blocker |
Changed in juniperopenstack: | |
milestone: | none → r3.1.0.0-fcs |
no longer affects: | juniperopenstack/r2.0 |
tags: | added: releasenote |
information type: | Proprietary → Public |
Review in progress for https:/ /review. opencontrail. org/20495
Submitter: Megh Bhatt (<email address hidden>)