Add signal handler for SIGXFSZ to log 'max file size exceeded' properly

[Jay Janssen]

Caught a PS 5.5.20 crashing with no log error, had to strace to find a SIGXFSZ. (ext3 filesystem with a 2TB file).

[Laurynas Biveinis (laurynas-biveinis)]

Looks like a good idea, too bad the signal handler won't be able to tell which of the open files exceeded it.

In the case of SIGXFSZ the last I/O call may or may not have failed too - can you attach that strace()?

[Jay Janssen (jay-janssen)]

I don't have access to the machine any more, and unfortunately, I neglected to get a copy of the strace, sorry.

Even just having the message with the name of the SIGNAL and/or the number would be better than what it is.

[Stewart Smith (stewart)]

We should probably just explode on this anyway, but with a sensible error message.

[Raghavendra D Prabhu (raghavendra-prabhu)]

Another way to trigger is to use limits.

"""
RLIMIT_FSIZE
              This is the maximum size of a file, in bytes, that
              may be created by a process. If a write or truncate
              operation would cause this limit to be exceeded,
              SIGXFSZ shall be generated for the
              thread. If the thread is blocking, or the process
              is catching or ignoring SIGXFSZ, continued attempts
              to increase the size of a file from end-of-file to
              beyond the limit shall fail with errno
               set to [EFBIG].

"""

So, SIGXFSZ is delivered and the errno should be EFBIG in this
case.

As per the report, there is no error printed and/or not handled
gracefully.

Added
"* hard fsize 1000000" to limits file

and then set innodb_log_file_size to 2048M.

Following is the gdb backtrace:

========================================================================

sudo gdb --args mysqld --user=mysql
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-56.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/mysqld...Reading symbols from /usr/lib/debug/usr/sbin/mysqld.debug...done.
done.
r(gdb) run
Starting program: /usr/sbin/mysqld --user=mysql
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff7fea700 (LWP 2700)]
[Thread 0x7ffff7fea700 (LWP 2700) exited]
121210 15:29:15 [Warning] The syntax '--log-slow-queries' is deprecated and will be removed in a future release. Please use '--slow-query-log'/'--slow-query-log-file' instead.
121210 15:29:15 [Note] Flashcache bypass: disabled
121210 15:29:15 [Note] Flashcache setup error is : ioctl failed

121210 15:29:15 [Warning] option 'innodb-log-files-in-group': signed value 1 adjusted to 2
121210 15:29:15 [Note] Plugin 'FEDERATED' is disabled.
121210 15:29:15 InnoDB: The InnoDB memory heap is disabled
121210 15:29:15 InnoDB: Mutexes and rw_locks use GCC atomic builtins
121210 15:29:15 InnoDB: Compressed tables use zlib 1.2.3
121210 15:29:15 InnoDB: Using Linux native AIO
121210 15:29:15 InnoDB: Initializing buffer pool, size = 500.0M
121210 15:29:15 InnoDB: Completed initialization of buffer pool
[New Thread 0x7ffff7fea700 (LWP 2701)]
[New Thread 0x7fffcb7ee700 (LWP 2702)]
[New Thread 0x7fffcaded700 (LWP 2703)]
[New Thread 0x7fffca3ec700 (LWP 2704)]
[New Thread 0x7fffc99eb700 (LWP 2705)]
[New Thread 0x7fffc8fea700 (LWP 2706)]
[New Thread 0x7fffc85e9700 (LWP 2707)]
[New Thread 0x7fffc7be8700 (LWP 2708)]
[New Thread 0x7fffc71e7700 (LWP 2709)]
[New Thread 0x7fffc67e6700 (LWP 2710)]
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
121210 15:29:15 InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the ...

[Laurynas Biveinis (laurynas-biveinis)]

Jay, would you mind reporting this upstream as well?

[Jay Janssen (jay-janssen)]

Submitted here: https://bugs.mysql.com/bug.php?id=80849

On Thu, Mar 24, 2016 at 1:40 AM, Laurynas Biveinis <
<email address hidden>> wrote:

> Jay, would you mind reporting this upstream as well?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1013829
>
> Title:
> Add signal handler for SIGXFSZ to log 'max file size exceeded'
> properly
>
> Status in Percona Server:
> Triaged
> Status in Percona Server 5.1 series:
> Triaged
> Status in Percona Server 5.5 series:
> Triaged
> Status in Percona Server 5.6 series:
> Triaged
> Status in Percona Server 5.7 series:
> Triaged
>
> Bug description:
> Caught a PS 5.5.20 crashing with no log error, had to strace to find a
> SIGXFSZ. (ext3 filesystem with a 2TB file).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/percona-server/+bug/1013829/+subscriptions
>

--
Jay Janssen
Managing Consultant, Percona
https://about.me/jay.janssen

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-566