slurmdbd segfaults on armhf
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
slurm-wlm (Ubuntu) |
New
|
High
|
Unassigned |
Bug Description
The slurmdbd segfaults on armhf.
This is from the autopkgtest logs:
613s × slurmdbd.service - Slurm DBD accounting daemon
613s Loaded: loaded (/usr/lib/
613s Active: failed (Result: core-dump) since Tue 2024-03-26 09:13:24 UTC; 19s ago
613s Duration: 627ms
613s Docs: man:slurmdbd(8)
613s Process: 3801 ExecStart=
613s Main PID: 3801 (code=dumped, signal=SEGV)
613s CPU: 28ms
613s
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:23 autopkgtest-
613s Mar 26 09:13:24 autopkgtest-
613s Mar 26 09:13:24 autopkgtest-
614s autopkgtest [09:13:44]: test sacct: -------
617s sacct FAIL non-zero exit status 3
Trying to run the binary in an armhf LXD container also fails:
root@autopkgtes
slurmdbd: accounting_
slurmdbd: error: Database settings not recommended values: innodb_
Segmentation fault
The database settings error seems to not be related as they are also happening on other archtectures.
Running on gdb I get the following stack trace:
(gdb) bt
#0 __GI_strlen () at ../sysdeps/
#1 0xf7d8927a in __printf_buffer (buf=buf@
#2 0xf7d9cd26 in __vsnprintf_
#3 0xf7e02bba in ___vsnprintf_chk (s=<optimized out>, maxlen=
#4 0xf7f62f2a in vsnprintf (__ap=..., __fmt=0xf7c6dc4c "insert into %s (creation_time, mod_time, table_name, definition) values (%ld, %ld, '%s', '%s') on duplicate key update definition='%s', mod_time=%ld;",
__n=100, __s=<optimized out>) at /usr/include/
#5 _xstrdup_vprintf (str=str@
fmt=
at ../../.
#6 0xf7f6384c in xstrdup_printf (fmt=0xf7c6dc4c "insert into %s (creation_time, mod_time, table_name, definition) values (%ld, %ld, '%s', '%s') on duplicate key update definition='%s', mod_time=%ld;")
at ../../.
#7 0xf7c58ed4 in _mysql_
#8 mysql_db_
at ../../.
#9 0xf7c238f6 in _as_mysql_
#10 init () at ../../.
#11 0xf7eebcae in plugin_
#12 0xf7eebec6 in plugin_
#13 0xf7eec038 in plugin_
names_
#14 0xf7f64114 in acct_storage_g_init () at ../../.
#15 0x00404460 in main (argc=<optimized out>, argv=0xfffef5a4) at ../../.
It crashes inside strlen.S.
This is the initial parameters passed to vsnprintf:
(gdb) frame 7
#7 0xf7c58ed4 in _mysql_
667 query2 = xstrdup_
(gdb) l
662 if (mysql_
663 xfree(query);
664 return SLURM_ERROR;
665 }
666 quoted = slurm_add_
667 query2 = xstrdup_
668 "mod_time, table_name, definition) "
669 "values (%ld, %ld, '%s', '%s') "
670 "on duplicate key update "
671 "definition='%s', mod_time=%ld;",
(gdb) l
672 table_defs_table, now, now,
673 table_name, quoted,
674 quoted, now);
675 xfree(quoted);
676 debug3("query\n%s", query2);
677 if (mysql_
678 xfree(query2);
679 return SLURM_ERROR;
680 }
681 xfree(query2);
The final string is partially generated:
(gdb) frame 5
#5 _xstrdup_vprintf (str=str@
fmt=
at ../../.
799 n = vsnprintf(p, size, fmt, our_ap);
(gdb) p p
$1 = 0x447a10 "insert into table_defs_table (creation_time, mod_time, table_name, definition) values (1711462045, "
It crashes at this offset in the strlen.S code:
(gdb) frame 0
Download failed: Invalid argument. Continuing without source file ./string/
#0 __GI_strlen () at ../sysdeps/
warning: 126 ../sysdeps/
(gdb) i r pc
pc 0xf7db752e 0xf7db752e <__GI_strlen+173>
(gdb) disas
Dump of assembler code for function __GI_strlen:
...
0xf7db7525 <+163>: ldrd r4, r5, [sp], #8
0xf7db7529 <+167>: add.w r0, r0, r2, lsr #3
0xf7db752d <+171>: bx lr
0xf7db752f <+173>: ldrd r2, r3, [r1]
0xf7db7533 <+177>: and.w r5, r4, #3
0xf7db7537 <+181>: rsb r0, r4, #0
...
It tries to load from whatever address r1 points to, but r1 seems to contain a timestamp that is supposed to be part of the final string (the timestamp seems to be 1711462045 actually):
(gdb) i r r1
r1 0x6602d698 1711462040
(gdb) x 0x6602d698
0x6602d698: Cannot access memory at address 0x6602d698
tags: | added: time-t update-excuse |
The variable "now" used in xstrdup_printf is defined as "time_t now = time(NULL);" hmmmm