init script timeout is too short when resizing logs
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| Percona Server moved to https://jira.percona.com/projects/PS |
Fix Released
|
High
|
Tomislav Plavcic | |
| 5.1 |
Fix Released
|
High
|
Tomislav Plavcic | |
| 5.5 |
Fix Released
|
High
|
Tomislav Plavcic | |
| 5.6 |
Fix Released
|
High
|
Tomislav Plavcic |
Bug Description
Currently the init script debian/
In MySQL 5.6 the redo logs may be resized during starup, which may take longer than 14 seconds.
In my testing (on SSDs), 30 seconds was a sufficient timeout to wait for 2GB logs to be generated.
Related branches
- Laurynas Biveinis (community): Approve on 2014-12-10
-
Diff: 22 lines (+8/-2)1 file modifiedbuild/debian/percona-server-server-5.1.mysql.init (+8/-2)
- Laurynas Biveinis (community): Approve on 2014-12-09
-
Diff: 22 lines (+8/-2)1 file modifiedbuild-ps/debian/percona-server-server-5.5.mysql.init (+8/-2)
- Laurynas Biveinis (community): Approve on 2014-12-09
-
Diff: 22 lines (+8/-2)1 file modifiedbuild-ps/debian/percona-server-server-5.6.mysql.init (+8/-2)
Nilnandan Joshi (nilnandan-joshi) wrote : | #2 |
Hi,
Can you provide the exact Debian and OS versions so we can try to reproduce the same? Thanks.
Nilnandan Joshi (nilnandan-joshi) wrote : | #3 |
Sorry, Exact Debian and PS (Percona Server) versions. i.e 5.6.*
Tom Manville (tdmanville) wrote : | #4 |
We've reproed this on Ubuntu 13.10, PS 5.6.17 and Ubuntu 12.04, PS 5.6.16
Attached is our fix.
Nilnandan Joshi (nilnandan-joshi) wrote : | #5 |
Confirmed with 5.1, 5.5 and 5.6 versions. Everywhere, 14 seconds hardcoded timeout.
/usr/bin/
# 6s was reported in #352070 to be too few when using ndbcluster
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14; do
if mysqld_status check_alive nowarn ; then break; fi
log_progress_msg "."
done
Tom Manville (tdmanville) wrote : | #6 |
Even 30 seconds timed out occasionally when resizing logs (crash recovery may be worse, but we haven't tested it).
We've bumped it up to 300 seconds without incident.
Vladimir Lazarenko (favoretti) wrote : | #7 |
In the past we've asked Percona PS to rebuild our packages in such a way that we could override it from /etc/default/mysql, like:
xxx@db003:
# Default timeout for the server to start
MYSQL_START_
# Default timeout for the server to stop
MYSQL_STOP_
If this could be ported upstream - this would be awesome. We have MySQL servers with 300GB RAM, just a startup (or a shutdown for that matter) takes longer than 5 minutes. Hence, whatever value you hardcode won't cut it for edge cases - so why not make it configurable?
Tom Manville (tdmanville) wrote : Re: [Bug 1328262] Re: init script timeout is too short when resizing logs | #8 |
Thanks for the email. Making it configurable does seem like a good way to
go. If you propose this in the bug, I'll add support for the idea.
On Wed, Sep 17, 2014 at 5:21 AM, Favoretti <email address hidden> wrote:
> In the past we've asked Percona PS to rebuild our packages in such a way
> that we could override it from /etc/default/mysql, like:
>
> xxx@db003:
> # Default timeout for the server to start
> MYSQL_START_
>
> # Default timeout for the server to stop
> MYSQL_STOP_
>
> If this could be ported upstream - this would be awesome. We have MySQL
> servers with 300GB RAM, just a startup (or a shutdown for that matter)
> takes longer than 5 minutes. Hence, whatever value you hardcode won't
> cut it for edge cases - so why not make it configurable?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> init script timeout is too short when resizing logs
>
> Status in Percona Server with XtraDB:
> Triaged
> Status in Percona Server 5.1 series:
> Triaged
> Status in Percona Server 5.5 series:
> Triaged
> Status in Percona Server 5.6 series:
> Triaged
>
> Bug description:
> Currently the init script debian/
> starts mysqld, then pings it it until it starts. If mysql is not
> accessable after 14 seconds, then the init script will fail.
>
> In MySQL 5.6 the redo logs may be resized during starup, which may
> take longer than 14 seconds.
>
> In my testing (on SSDs), 30 seconds was a sufficient timeout to wait
> for 2GB logs to be generated.
>
> To manage notifications about this bug go to:
> https:/
>
Vladimir Lazarenko (favoretti) wrote : | #9 |
Hey.
I did? :) or you want it as an explicit proposal?
Cheers,
Vlad
Text by me, typos by iPhone
> On 17 sep. 2014, at 19:35, Tom Manville <email address hidden> wrote:
>
> Thanks for the email. Making it configurable does seem like a good way to
> go. If you propose this in the bug, I'll add support for the idea.
>
>> On Wed, Sep 17, 2014 at 5:21 AM, Favoretti <email address hidden> wrote:
>>
>> In the past we've asked Percona PS to rebuild our packages in such a way
>> that we could override it from /etc/default/mysql, like:
>>
>> xxx@db003:
>> # Default timeout for the server to start
>> MYSQL_START_
>>
>> # Default timeout for the server to stop
>> MYSQL_STOP_
>>
>> If this could be ported upstream - this would be awesome. We have MySQL
>> servers with 300GB RAM, just a startup (or a shutdown for that matter)
>> takes longer than 5 minutes. Hence, whatever value you hardcode won't
>> cut it for edge cases - so why not make it configurable?
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https:/
>>
>> Title:
>> init script timeout is too short when resizing logs
>>
>> Status in Percona Server with XtraDB:
>> Triaged
>> Status in Percona Server 5.1 series:
>> Triaged
>> Status in Percona Server 5.5 series:
>> Triaged
>> Status in Percona Server 5.6 series:
>> Triaged
>>
>> Bug description:
>> Currently the init script debian/
>> starts mysqld, then pings it it until it starts. If mysql is not
>> accessable after 14 seconds, then the init script will fail.
>>
>> In MySQL 5.6 the redo logs may be resized during starup, which may
>> take longer than 14 seconds.
>>
>> In my testing (on SSDs), 30 seconds was a sufficient timeout to wait
>> for 2GB logs to be generated.
>>
>> To manage notifications about this bug go to:
>> https:/
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https:/
>
> Title:
> init script timeout is too short when resizing logs
>
> Status in Percona Server with XtraDB:
> Triaged
> Status in Percona Server 5.1 series:
> Triaged
> Status in Percona Server 5.5 series:
> Triaged
> Status in Percona Server 5.6 series:
> Triaged
>
> Bug description:
> Currently the init script debian/
> starts mysqld, then pings it it until it starts. If mysql is not
> accessable after 14 seconds, then the init script will fail.
>
> In MySQL 5.6 the redo logs may be resized during starup, which may
> take longer than 14 seconds.
>
> In my testing (on SSDs), 30 seconds was a sufficient timeout to wait
> for 2GB logs to be generated.
>
> To manage notifications about this bug go to:
> https:/
Percona now uses JIRA for bug reports so this bug report is migrated to: https:/
Related bug 1072538.