oslo-incubator

MySQL 2013 lost connection is being raised

Bug #1276510 reported by Flavio Percoco on 2014-02-05

This bug affects 2 people

	Status	Importance	Assigned to	Milestone
Ceilometer	Fix Released	Undecided	Xurong Yang	Ceilometer 2014.2 "juno"
Cinder	Fix Released	High	Flavio Percoco
Havana	Fix Released	High	Flavio Percoco	Cinder 2013.2.3
Ironic	Fix Released	Undecided	Xurong Yang
OpenStack Compute (nova)	Fix Released	Undecided	Xurong Yang	OpenStack Compute (nova) 2014.2 "juno"
Havana	Fix Released	Undecided	Unassigned	OpenStack Compute (nova) 2013.2.3
OpenStack Identity (keystone)	Invalid	Undecided	Unassigned
Sahara	Fix Released	Low	Xurong Yang	Sahara 2014.1 "icehouse"
neutron	Fix Released	Undecided	Xurong Yang
oslo-incubator	Fix Released	High	Flavio Percoco	oslo-incubator 2014.1 "icehouse"
Havana	Fix Released	High	Flavio Percoco
tuskar	Fix Released	Medium	Xurong Yang

Bug Description

MySQL's 2013 code error is not in the list of connection lost issues. This causes the reconnect loop to raise this error and stop retrying.

[database]
max_retries = -1
retry_interval = 1

mysql down:

==> scheduler.log <==
2014-02-03 16:51:50.956 16184 CRITICAL cinder [-] (OperationalError) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") None None

Tags:

Revision history for this message

Flavio Percoco (flaper87) wrote on 2014-02-05:

https://review.openstack.org/#/c/71225/

Changed in oslo:
importance:	Undecided → High
assignee:	nobody → Flavio Percoco (flaper87)
milestone:	none → icehouse-3
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-05: Fix merged to oslo-incubator (master)

Reviewed: https://review.openstack.org/71225
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=b4f72b2987d53bb434e3a833f06d70a2167f5efc
Submitter: Jenkins
Branch: master

commit b4f72b2987d53bb434e3a833f06d70a2167f5efc
Author: Flavio Percoco <email address hidden>
Date: Wed Feb 5 11:06:12 2014 +0100

Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db

Changed in oslo:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-06: Fix proposed to oslo-incubator (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/71473

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-06: Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/71477

Changed in cinder:
assignee:	nobody → Flavio Percoco (flaper87)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-09: Fix merged to oslo-incubator (stable/havana)

Reviewed: https://review.openstack.org/71473
Committed: https://git.openstack.org/cgit/openstack/oslo-incubator/commit/?id=5007a01cc4438a54845043e2cd724e8b2de4e833
Submitter: Jenkins
Branch: stable/havana

commit 5007a01cc4438a54845043e2cd724e8b2de4e833
Author: Flavio Percoco <email address hidden>
Date: Wed Feb 5 11:06:12 2014 +0100

Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db
(cherry picked from commit b4f72b2987d53bb434e3a833f06d70a2167f5efc)

Alan Pevec (apevec) on 2014-02-09

Changed in cinder:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-11: Fix proposed to cinder (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/72665

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-12: Fix proposed to nova (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/72924

Xurong Yang (idopra) on 2014-02-15

Changed in neutron:
assignee:	nobody → Xurong Yang (idopra)
status:	New → In Progress
Changed in nova:
assignee:	nobody → Xurong Yang (idopra)

Xurong Yang (idopra) on 2014-02-15

Changed in ceilometer:
assignee:	nobody → Xurong Yang (idopra)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-15: Fix proposed to ceilometer (master)

Fix proposed to branch: master
Review: https://review.openstack.org/73791

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-15: Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/73793

Xurong Yang (idopra) on 2014-02-15

Changed in keystone:
assignee:	nobody → Xurong Yang (idopra)
status:	New → In Progress

Xurong Yang (idopra) on 2014-02-15

Changed in ironic:
assignee:	nobody → Xurong Yang (idopra)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-15: Fix proposed to ironic (master)

#10

Fix proposed to branch: master
Review: https://review.openstack.org/73794

Changed in ironic:
status:	New → In Progress

Xurong Yang (idopra) on 2014-02-15

Changed in savanna:
assignee:	nobody → Xurong Yang (idopra)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-15: Fix proposed to savanna (master)

#11

Fix proposed to branch: master
Review: https://review.openstack.org/73796

Xurong Yang (idopra) on 2014-02-15

Changed in tuskar:
assignee:	nobody → Xurong Yang (idopra)
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-15: Fix proposed to tuskar (master)

#12

Fix proposed to branch: master
Review: https://review.openstack.org/73799

Sergey Lukjanov (slukjanov) on 2014-02-15

Changed in savanna:
importance:	Undecided → Low
milestone:	none → icehouse-3

Jiří Stránský (jistr) on 2014-02-17

Changed in tuskar:
importance:	Undecided → Medium

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-18: Fix merged to tuskar (master)

#13

Reviewed: https://review.openstack.org/73799
Committed: https://git.openstack.org/cgit/openstack/tuskar/commit/?id=efaa924a9f1fb3d222c4060dd71fdaad5f14ddea
Submitter: Jenkins
Branch: master

commit efaa924a9f1fb3d222c4060dd71fdaad5f14ddea
Author: yangxurong <email address hidden>
Date: Sat Feb 15 17:11:47 2014 +0800

Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: Id7cb3c8a757fa72ed5403662c3f184146c23398b

Changed in tuskar:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-19: Fix merged to savanna (master)

#14

Reviewed: https://review.openstack.org/73796
Committed: https://git.openstack.org/cgit/openstack/savanna/commit/?id=83331c465838d9587d91f9bb8642a2f1816ea919
Submitter: Jenkins
Branch: master

commit 83331c465838d9587d91f9bb8642a2f1816ea919
Author: yangxurong <email address hidden>
Date: Sat Feb 15 17:05:35 2014 +0800

Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: I006fd3ae88dd7ab61f7f8811562fb78ce4a331ad

Changed in savanna:
status:	In Progress → Fix Committed

Matt Riedemann (mriedem) on 2014-02-22

tags:

added: db oslo

Roman Podoliaka (rpodolyaka) on 2014-02-24

Changed in tuskar:
status:	Fix Committed → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-02-25: Fix merged to ironic (master)

#15

Reviewed: https://review.openstack.org/73794
Committed: https://git.openstack.org/cgit/openstack/ironic/commit/?id=6a207d4b4652e368967650d64446e850fe0fec32
Submitter: Jenkins
Branch: master

commit 6a207d4b4652e368967650d64446e850fe0fec32
Author: yangxurong <email address hidden>
Date: Sat Feb 15 16:56:14 2014 +0800

Don't raise MySQL 2013 'Lost connection' errors

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Closes-bug: #1276510
Change-Id: I27cad999b30a480ae23250150bfcba2668acabd7

Changed in ironic:
status:	In Progress → Fix Committed

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-03: Fix merged to cinder (stable/havana)

#16

Reviewed: https://review.openstack.org/72665
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=d043874cef9a783b89226e8462e50465de31d117
Submitter: Jenkins
Branch: stable/havana

commit d043874cef9a783b89226e8462e50465de31d117
Author: Flavio Percoco <email address hidden>
Date: Wed Feb 5 11:06:12 2014 +0100

Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db

Allison Randal (allison) on 2014-03-03

Changed in nova:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2014-03-04: Fix merged to nova (stable/havana)

#17

Reviewed: https://review.openstack.org/72924
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b09ccd55bd7e67f4351d0cc4f6c6bbcef40f0c41
Submitter: Jenkins
Branch: stable/havana

commit b09ccd55bd7e67f4351d0cc4f6c6bbcef40f0c41
Author: Xavier Queralt <email address hidden>
Date: Wed Feb 12 12:49:26 2014 +0100

Don't raise MySQL 2013 'Lost connection' errors

Although this code error is part of the `ping_listener` tests, it
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Closes-bug: #1276510
Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db

Thierry Carrez (ttx) on 2014-03-05

Changed in savanna:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-03-05

Changed in ironic:
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-03-06

Changed in oslo:
status:	Fix Committed → Fix Released

Revision history for this message

Dolph Mathews (dolph) wrote on 2014-03-28:

#18

This appears to have been fixed in 96be7449 as part of a larger sync with oslo.db

https://review.openstack.org/#/c/71311/

Changed in keystone:
assignee:	Xurong Yang (idopra) → nobody
status:	In Progress → Invalid

Revision history for this message

gordon chung (chungg) wrote on 2014-04-10:

#19

fix merged as part of this patch: https://review.openstack.org/#/c/78132/

Changed in ceilometer:
status:	In Progress → Fix Committed

Thierry Carrez (ttx) on 2014-04-17

Changed in oslo:
milestone:	icehouse-3 → 2014.1

Thierry Carrez (ttx) on 2014-04-17

Changed in sahara:
milestone:	icehouse-3 → 2014.1

Thierry Carrez (ttx) on 2014-06-12

Changed in ceilometer:
status:	Fix Committed → Fix Released
milestone:	none → juno-1

Revision history for this message

David Medberry (med) wrote on 2014-08-22:

#20

@idopra ~idopra @ttx ~ttx it looks (as I read this) that this fix has never been propagated to neutron (not havana, not icehouse) and the change was abandoned as it was waiting for OSLO (which happened long long ago). Do I need to go revive that icehouse neutron gerrit? https://review.openstack.org/#/c/73793/

Revision history for this message

Davanum Srinivas (DIMS) (dims-v) wrote on 2014-09-06:

#21

we can close this as fixed for Nova when https://review.openstack.org/#/c/101901/ merges

Revision history for this message

Davanum Srinivas (DIMS) (dims-v) wrote on 2014-09-07:

#22

101901 is merged

Changed in nova:
status:	In Progress → Fix Committed

Revision history for this message

haruka tanizawa (h-tanizawa) wrote on 2014-09-24:

#23

Hi David!
I think below commit contains synchronization in Neutron from oslo-incubator.
So Neutron's status of this bug report should be set 'Fix Released' from 'In Progress' .

commit 9a20c0819074f17e3c607522aee4020b811e00e3
Author: Julia Varlamova <email address hidden>
Date: Tue Feb 11 18:04:01 2014 +0400

Sync latest oslo.db code into neutron

Changes that were ported from oslo:

    b4f72b2 Don't raise MySQL 2013 'Lost connection' errors ★here
    271adfb Format sql in db.sqlalchemy.session docstring
    0334cb3 Handle exception messages with six.text_type
    …

Thierry Carrez (ttx) on 2014-10-01

Changed in nova:
milestone:	none → juno-rc1
status:	Fix Committed → Fix Released

Thierry Carrez (ttx) on 2014-10-16

Changed in nova:
milestone:	juno-rc1 → 2014.2

Thierry Carrez (ttx) on 2014-10-16

Changed in ceilometer:
milestone:	juno-1 → 2014.2

Huang Zhiteng (zhiteng-huang) on 2015-03-18

Changed in cinder:
status:	In Progress → Fix Released

Revision history for this message

Ihar Hrachyshka (ihar-hrachyshka) wrote on 2015-11-09:

#24

We switched to oslo.db.

Changed in neutron:
status:	In Progress → Fix Released

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.