cupsd crashes with SIGSEGV on ubuntu 17.04 on start

[guysoft]

Hey,
I upgraded to Ubuntu 17.04, but CUPS crashes on boot, and when I run cupsd it segfaults. Attaching strace ouput.
---
ApportVersion: 2.20.4-0ubuntu4.4
Architecture: amd64
CupsErrorLog:

CurrentDesktop: KDE
DistroRelease: Ubuntu 17.04
InstallationDate: Installed on 2012-12-20 (1698 days ago)
InstallationMedia: Kubuntu 12.10 "Quantal Quetzal" - Release amd64 (20121017.1)
Lpstat: Error: command ['lpstat', '-v'] failed with exit code 1: lpstat: Bad file descriptor
MachineType: ASUSTeK COMPUTER INC. UX303LB
NonfreeKernelModules: nvidia_uvm nvidia_drm nvidia_modeset nvidia
Package: cups 2.2.2-1ubuntu1
PackageArchitecture: amd64
Papersize: letter
ProcCmdline: BOOT_IMAGE=/boot/vmlinuz-4.10.0-28-generic root=UUID=a8690ccc-4675-4e21-896f-eadcc45159f4 ro quiet splash vga=791 pcie_aspm=force drm.vblankoffdelay=1
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.10.0-28-generic root=UUID=a8690ccc-4675-4e21-896f-eadcc45159f4 ro quiet splash vga=791 pcie_aspm=force drm.vblankoffdelay=1
ProcVersionSignature: Ubuntu 4.10.0-28.32-generic 4.10.17
Tags: zesty
Uname: Linux 4.10.0-28-generic x86_64
UpgradeStatus: Upgraded to zesty on 2017-04-19 (117 days ago)
UserGroups: adm bluetooth bumblebee cdrom dialout dip lp lpadmin netdev nvidia-persistenced plugdev sambashare sudo usrp
_MarkForUpload: True
dmi.bios.date: 03/17/2015
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: UX303LB.203
dmi.board.asset.tag: ATN12345678901234567
dmi.board.name: UX303LB
dmi.board.vendor: ASUSTeK COMPUTER INC.
dmi.board.version: 1.0
dmi.chassis.asset.tag: ATN12345678901234567
dmi.chassis.type: 10
dmi.chassis.vendor: ASUSTeK COMPUTER INC.
dmi.chassis.version: 1.0
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrUX303LB.203:bd03/17/2015:svnASUSTeKCOMPUTERINC.:pnUX303LB:pvr1.0:rvnASUSTeKCOMPUTERINC.:rnUX303LB:rvr1.0:cvnASUSTeKCOMPUTERINC.:ct10:cvr1.0:
dmi.product.name: UX303LB
dmi.product.version: 1.0
dmi.sys.vendor: ASUSTeK COMPUTER INC.
mtime.conffile..etc.cups.cupsd.conf: 2015-10-21T13:38:49.740915

[Till Kamppeter (till-kamppeter)]

If you start cupsd from the command line and it reproducably crashes (or it at least crashes often enough so that you can easily get it to crash), please do the following:

Run

ulimit -c unlimited

Then check whether there is no file named "core" in the current directory. If so, delete or rename the file or change to another directory.

Then start cupsd. If it crashes, you should get a file named "core" in the current directory. Attach the file to this bug report and also run

gdb -c core /usr/sbin/cupsd

and on the gdb prompt enter the command

bt

and after that

quit.

Post all the screen out put of your gdb session here.

[guysoft (guysoft)]

Attaching, the output from gdb is:

GNU gdb (Ubuntu 7.12.50.20170314-0ubuntu1) 7.12.50.20170314-git
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/cupsd...(no debugging symbols found)...done.
[New LWP 12373]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `cupsd'.
Program terminated with signal SIGSEGV, Segmentation fault.
£0 __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
165 ../sysdeps/x86_64/multiarch/strcmp-sse42.S: No such file or directory.

[Till Kamppeter (till-kamppeter)]

Thanks for the reply, but if you run the gdb command from comment #2, it does not exit, but it stays running with its own command prompt.

At this command, please enter

bt

and post the output here.

Enter

quit

to close gdb and get back to the command prompt.

[guysoft (guysoft)]

Sorry, missed that last part.
Here is the full output:

guy@golem4:~$ sudo gdb -c core /usr/sbin/cupsd
GNU gdb (Ubuntu 7.12.50.20170314-0ubuntu1) 7.12.50.20170314-git
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/cupsd...(no debugging symbols found)...done.
[New LWP 13673]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `cupsd'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
165 ../sysdeps/x86_64/multiarch/strcmp-sse42.S: No such file or directory.
(gdb) bt
#0 __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
#1 0x00007fa072279fe5 in lsp_initialize () from /lib/lib/x86_64-linux-gnu/liblsp.so
#2 0x00007fa07248d9ca in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fff01cb5728,
    env=env@entry=0x7fff01cb5738) at dl-init.c:72
#3 0x00007fa07248dadb in call_init (env=0x7fff01cb5738, argv=0x7fff01cb5728, argc=1, l=<optimized out>)
    at dl-init.c:30
#4 _dl_init (main_map=0x7fa0726a4168, argc=1, argv=0x7fff01cb5728, env=0x7fff01cb5738) at dl-init.c:120
#5 0x00007fa07247dc5a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#6 0x0000000000000001 in ?? ()
#7 0x00007fff01cb78af in ?? ()
#8 0x0000000000000000 in ?? ()
(gdb) quit
guy@golem4:

[guysoft (guysoft)]

Hey, I updated the bug report, can you please update it and change it from incomplete?

[Till Kamppeter (till-kamppeter)]

Unfortunately, your gdb backtrace is touching nowhere the code of CUPS, probably due to being a sub thread which was created somewhere in the libraries.

You need to get this crash registered in the Ubuntu Error Tracker. To do so, you need to follow the instructions in the crash report pop-ups. So if some window is popping up on your machine telling you that some problem has occurred, do not click it away, please follow the instructions.

[Till Kamppeter (till-kamppeter)]

Another posibility is to check whether there is already a crash report file. You can attach it to this bug report.

These files are in the /var/crash directory and their names contain path and name of the executable file which has crashed. In your case /usr/sbin/cupsd has crashed, so the name looks like:

/var/crash/_usr_sbin_cupsd.1000.crash

where the number is different with each crash (probably it is the process ID of the process which has crashed).

List the files with date

ls -l /var/crash/*cups*

and create a bug report with the newest file

apport-bug /var/crash/_usr_sbin_cupsd.1000.crash

or simply try whether the system automatically finds a crash report which fits to this bug:

apport-collect 1706052

[guysoft (guysoft)]

Well the good news there is a /var/crash/_usr_sbin_cupsd.1000.crash file.
The bad (not so bad) news is, well this is what apport-collect 1706052 outputs this amazing question dialog (what do I pick?).

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

apport information

[guysoft (guysoft)]

Ok, so if I pick in the "Text" dialog "yes", it sends a report, and it also gives the following error inside "lpstat: Bad file descriptor". Indeed, if I run /usr/bin/lpstat I get the error:
lpstat: Bad file descriptor

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[HuaiDan (dhutchison69)]

From /var/crash

[HuaiDan (dhutchison69)]

Ubuntu 17.04 4.10.0-35-generic amd64 (upgraded from Wiley->Xenial->Zesty)

gbd output:

(gdb) bt
#0 __strcasecmp_l_avx () at ../sysdeps/x86_64/multiarch/strcmp-sse42.S:165
#1 0x00007f699c907fe5 in lsp_initialize ()
   from /lib/lib/x86_64-linux-gnu/liblsp.so
#2 0x00007f699cb1b9ca in call_init (l=<optimized out>, argc=argc@entry=1,
    argv=argv@entry=0x7fffc1969d68, env=env@entry=0x7fffc1969d78)
    at dl-init.c:72
#3 0x00007f699cb1badb in call_init (env=0x7fffc1969d78, argv=0x7fffc1969d68,
    argc=1, l=<optimized out>) at dl-init.c:30
#4 _dl_init (main_map=0x7f699cd32168, argc=1, argv=0x7fffc1969d68,
    env=0x7fffc1969d78) at dl-init.c:120
#5 0x00007f699cb0bc5a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#6 0x0000000000000001 in ?? ()
#7 0x00007fffc196a27f in ?? ()
#8 0x0000000000000000 in ?? ()
(gdb) quit

I've been sending the reports from the crash notifications whenever prompted.

[HuaiDan (dhutchison69)]

Workaround:

sudo aa-complain /usr/sbin/cupsd

[Till Kamppeter (till-kamppeter)]

HuaiDan (dhutchison69), if this works for you, please post all the CUPS-related "audit" lines from your /var/log/syslog file.

[guysoft (guysoft)]

Ok really strange, after running once with aa-complain, it seems to solve the problem.

In syslog I see this:

golem4 kernel: [188942.996377] audit: type=1400 audit(1507148174.867:125): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/20273/cmdline" pid=20273 comm="cupsd" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

Before that I see entries that look like this:
Oct 4 12:50:54 golem4 kernel: [151422.393607] audit: type=1400 audit(1507110654.087:74): apparmor="DENIED" operation="open" profile="/usr/sbin/cupsd" name="/proc/2399/cmdline" pid=2399 comm="cupsd" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Oct 4 12:50:54 golem4 kernel: [151422.393706] cupsd[2399]: segfault at 0 ip 00007fb9ba1f7715 sp 00007ffce931b968 error 4 in libc-2.24.so[7fb9ba0aa000+1be000]
Oct 4 12:50:54 golem4 systemd[1]: cups.service: Main process exited, code=dumped, status=11/SEGV

Note it says apparmor="DENIED"

[HuaiDan (dhutchison69)]

Here's one set:
[ 189.457095] audit: type=1400 audit(1507146357.726:48): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cups-browsed" pid=2784 comm="apparmor_parser"
[ 216.802135] audit: type=1400 audit(1507146385.075:49): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cups-browsed" name="/proc/2864/cmdline" pid=2864 comm="cups-browsed" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 217.974857] audit: type=1400 audit(1507146386.247:50): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/2871/cmdline" pid=2871 comm="cups-exec" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 217.978903] audit: type=1400 audit(1507146386.251:51): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/2871/cmdline" pid=2871 comm="dbus" requested_mask="r" denied_mask="r" fsuid=7 ouid=7
[ 358.429024] audit: type=1400 audit(1507146526.583:52): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/3294/cmdline" pid=3294 comm="cups-exec" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 358.429102] audit: type=1400 audit(1507146526.583:53): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/3295/cmdline" pid=3295 comm="cups-exec" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 358.429204] audit: type=1400 audit(1507146526.583:54): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/3296/cmdline" pid=3296 comm="cups-exec" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 358.429339] audit: type=1400 audit(1507146526.583:55): apparmor="ALLOWED" operation="open" profile="/usr/sbin/cupsd" name="/proc/3297/cmdline" pid=3297 comm="cups-exec" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

[HuaiDan (dhutchison69)]

I also did
aa-complain /usr/sbin/cups-browsed

but printing worked before I set that.

[HuaiDan (dhutchison69)]

Here's something else you may want to look at: The apparmor profile on my machine for cups.

[Till Kamppeter (till-kamppeter)]

Everyone with this problem, can you please add a line

@{PROC}/*/cmdline r,

after the line

@{PROC}/*/auxv r,

in your /etc/apparmor.d/usr.sbin.cupsd file, then run

sudo aa-enforce /usr/sbin/cupsd

and reboot.

Does this solve your problem?

[HuaiDan (dhutchison69)]

Here's what we've got so far:
Cups is back online and working in enforce mode. No errors are thrown, no crash reports.
HPLIP toolbox shows the printer and the right information.
I am able to print.

So far, so good.

However, hplip-systray is not playing along. The icon appears, but it is functionless, nothing appears on click. Apparmor's not complaining, but /var/log is getting flooded with hundreds of messages originating from hplip-systray. Is this a separate or related issue?

[HuaiDan (dhutchison69)]

Thanks BTW, good work.

[Till Kamppeter (till-kamppeter)]

Thanks, so the adding of said line to usr.sbin.cupsd helps?

Can you test Artful (on a virtual machine or as a live system) to see whether it has the same problem and whether it can be fixed the same way?

Note that the problems of HPLIP you mention do not have anything to do with the CUPS package or the AppArmor profile of CUPS. Please report a bug on hplip for them.

[HuaiDan (dhutchison69)]

Yes,
@{PROC}/*/cmdline r,
and resetting to enforce fixed the printing problem. For the record, the same solution also worked for a problem I was having with the dhclient apparmor profile that was stopping NetworkManager from connecting.

As for 17.10 Artful, I'm not entirely willing to do that just yet. I upgraded to Zesty only to attempt to fix a Bluetooth audio problem (it didn't) without testing the waters first, and got spanked for it. It's taken me a week just to root out the critical bugs.

As for HPLIP systray, I feel it might be related, since systray will work with apparmor set to complain on cups (both with and without the profile edit), but not when set to enforce. Nonetheless, I will file a bug with HPLIP.