Different errors "query failed: [2006] MySQL server has gone away" in zabbix.log

Bug #1400733 reported by Anastasia Palkina
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Medium
Stanislaw Bogatkin
6.0.x
Won't Fix
Medium
Stanislaw Bogatkin
6.1.x
Fix Released
Medium
Stanislaw Bogatkin

Bug Description

"build_id": "2014-12-08_22-41-06",
"ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4",
"build_number": "46",
"auth_required": true,
"api": "1.0",
"nailgun_sha": "73a2fa3aa669ca8d95788e80dc052dfa16eca24a",
"production": "docker",
"fuelmain_sha": "2040303660ca19ae7aba1ab16cc75485d9c944d6",
"astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91",
"feature_groups": ["mirantis", "experimental"],
"release": "6.0",
"release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-12-08_22-41-06", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "46", "api": "1.0", "nailgun_sha": "73a2fa3aa669ca8d95788e80dc052dfa16eca24a", "production": "docker", "fuelmain_sha": "2040303660ca19ae7aba1ab16cc75485d9c944d6", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "c8df66ac50f90ed7708ecd5ee4cf1cefe9fd1b70"}}},
"fuellib_sha": "c8df66ac50f90ed7708ecd5ee4cf1cefe9fd1b70"

1. Create new environment (Ubuntu, simple mode)
2. Enable Zabbix
3. Choose neutron, GRE
4. Add controller, compute, cinder, zabbix
5. Start deployment. It was successful
6. There are many errors with "query failed: [2006] MySQL server has gone away" part on zabbix node (node-3)

2014-12-09 14:29:48 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [select hostid,status from hosts where host='node-1.domain.tld' and status in (0,1) and flags<>2 and proxy_hostid is null]

2014-12-09 14:21:54 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [begin;]

2014-12-09 14:06:50 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [select hostid,key_,state,filter,error,lifetime from items where itemid=410]

2014-12-09 14:02:18 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [select hostid,status from hosts where host='Zabbix server' and status in (0,1) and flags<>2 and proxy_hostid is null]

2014-12-09 14:02:00 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [select m.maintenanceid,m.maintenance_type,m.active_since,tp.timeperiod_type,tp.every,tp.month,tp.dayofweek,tp.day,tp.start_time,tp.period,tp.start_date from maintenances m,maintenances_windows mw,timeperiods tp where m.maintenanceid=mw.maintenanceid and mw.timeperiodid=tp.timeperiodid and m.active_since<=1418133720 and m.active_till>1418133720]

2014-12-09 14:01:58 ERR

 [Z3005] query failed: [2006] MySQL server has gone away [select distinct r.druleid,r.iprange,r.name,c.dcheckid from drules r left join dchecks c on c.druleid=r.druleid and uniq=1 where r.proxy_hostid is null and r.status=0 and (r.nextcheck<=1418133718 or r.nextcheck>1418133718+r.delay) and mod(r.druleid,1)=0]

Revision history for this message
Anastasia Palkina (apalkina) wrote :
Revision history for this message
Anastasia Palkina (apalkina) wrote :

The same errors for CentOS. Zabbix node is node-7

description: updated
Changed in fuel:
milestone: 6.0 → 6.0.1
Revision history for this message
Stanislaw Bogatkin (sbogatkin) wrote :

Problem happens due to mysql server which closes long-live connections as it point by "wait_timeout" option in mysql config file. Zabbix prior to version 2.5 doesn't able to send keepalives, so we can use workaround to it by set "wait_timeout" parameter to big enough values, e.g. 86400. Then we will see that errors very rarely.
But it, as I said, not big deal. It just how mysql works.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/150808

Changed in fuel:
status: Triaged → In Progress
Revision history for this message
Ryan Moe (rmoe) wrote :

Medium priority bugs are not backported to stable releases.

Changed in fuel:
assignee: Stanislaw Bogatkin (sbogatkin) → Vladimir Kuklin (vkuklin)
Changed in fuel:
assignee: Vladimir Kuklin (vkuklin) → Stanislaw Bogatkin (sbogatkin)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/150808
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=031fcb6815a341253b57b9e67070f433305a977f
Submitter: Jenkins
Branch: master

commit 031fcb6815a341253b57b9e67070f433305a977f
Author: Stanislaw Bogatkin <email address hidden>
Date: Wed Jan 28 13:33:30 2015 +0300

    Change wait_timeout in MySQL for zabbix node

    Closes-Bug: 1400733
    Change-Id: Id14b05c1a07cd7729bd562759da998cb32cece9f

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Anastasia Palkina (apalkina) wrote :

Verified on ISO #295

"build_id": "2015-04-08_22-54-31", "ostf_sha": "c3b06dba5c96d225882e9f1a465f74eaa6374fbf", "build_number": "295", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-04-08_22-54-31", "ostf_sha": "c3b06dba5c96d225882e9f1a465f74eaa6374fbf", "build_number": "295", "api": "1.0", "nailgun_sha": "e5f101635455e94415969907deca510d2ded6b73", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "5c94b59bafc8dc1cbecb088020f4ef14ce62044a", "astute_sha": "5041b2fb508e6860c3cb96474ca31ec97e549e8b", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "2ca546b86e651d5638dbb1be9bae44b86c84a893", "fuellib_sha": "dee81f2be4d9063808a6755271ee818314997006"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "e5f101635455e94415969907deca510d2ded6b73", "openstack_version": "2014.2-6.1", "production": "docker", "python-fuelclient_sha": "5c94b59bafc8dc1cbecb088020f4ef14ce62044a", "astute_sha": "5041b2fb508e6860c3cb96474ca31ec97e549e8b", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "2ca546b86e651d5638dbb1be9bae44b86c84a893", "fuellib_sha": "dee81f2be4d9063808a6755271ee818314997006"

Quantity of warnings decreased

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.