Bus error

[Stephan Ferraro]

Binary package hint: lighttpd

Lighttpd crashes with a "Bus error" every hour on my server.
# lighttpd -f /etc/lighttpd/lighttpd.conf -D
Bus error

Can't use GDB to find out more because I get "generic error":
# gdb /usr/sbin/lighttpd
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/lighttpd...(no debugging symbols found)...done.
(gdb) r -D -f /etc/lighttpd/lighttpd.conf
Starting program: /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf
[Thread debugging using libthread_db enabled]
Cannot find new threads: generic error

Should I recompile the source code with debugging options to provide more useful informations?

[Stephan Ferraro (ferraro)]

Here are my system information:
# cat /etc/issue
Ubuntu 10.04.1 LTS \n \l

# uname -ar
Linux trashmail 2.6.32-23-server #37-Ubuntu SMP Fri Jun 11 09:11:11 UTC 2010 x86_64 GNU/Linux

[Stephan Ferraro (ferraro)]

lighttpd version:
lighttpd/1.4.26 (ssl)

I could use strace to trace when it bus error occured:
ioctl(14, FIONREAD, [0]) = 0
read(14, 0x298dd70, 4159) = -1 EAGAIN (Resource temporarily unavailable)
epoll_ctl(12, EPOLL_CTL_MOD, 14, {EPOLLIN|EPOLLERR|EPOLLHUP, {u32=14, u64=14}}) = 0
epoll_ctl(12, EPOLL_CTL_MOD, 14, {EPOLLIN|EPOLLERR|EPOLLHUP, {u32=14, u64=14}}) = 0
epoll_wait(12, {{EPOLLIN, {u32=13, u64=13}}}, 10001, 1000) = 1
ioctl(13, FIONREAD, [613]) = 0
read(13, "GET /stats HTTP/1.1\r\nHost: www.t"..., 4159) = 613
setsockopt(13, SOL_TCP, TCP_CORK, [1], 4) = 0
writev(13, [{"HTTP/1.1 200 OK\r\nContent-Type: a"..., 163}], 1) = 163
open("/var/www/trashmail/htdocs/stats", O_RDONLY) = 24
fcntl(24, F_SETFD, FD_CLOEXEC) = 0
mmap(NULL, 26, PROT_READ, MAP_SHARED, 24, 0) = 0x7ff6a72bc000
write(13, "221744625:0.00:1222:36391\n", 26) = -1 EFAULT (Bad address)
--- SIGBUS (Bus error) @ 0 (0) ---
Process 332 detached

[Stephan Ferraro (ferraro)]

Its possible that the content of "/var/www/trashmail/htdocs/stats" changed between the "mmap" and "write" from outside on the filesystem.
Then the data addresses provided by mmap could be invalid.
Should lighttpd not lock the file after open and before executing mmap?
Then unlock after reading it?

[Stephan Ferraro (ferraro)]

locking would probably uses too much resources for a high-speed webserver.
The error comes from my side: I change a file while lighttpd is using it. The solution would be to change it atomically without that lighttpd loss its file descriptor. I will use "rename" for this.
So if I change the contents of file "stats", I will create "stats.tmp" and then rename it to "stats".
I will try this out to see if I will still get a bus error.

[Stephan Ferraro (ferraro)]

Since I no more changed the file "stats", lighttpd no more crashes with bus error.
So: Do not change files while lighttpd is reading them with nmap under Linux.
Solution: Change document root files only with UNIX "rename" syscall.

This bug will not happen if files will be changed properly.
You can close the bug.

[Daniel Hahler (blueyed)]

Thank you for your bug report and investigating the root cause / issue.

I do not really agree that it is not a bug; a web server should not crash in this case, but handle it more gracefully.
Therefore I am marking it Triaged for now.

Do you like to report it to the Lighttpd developers upstream?
If they believe it is invalid, I would agree - but they are probably interested to know about this.

[Stephan Ferraro (ferraro)]

Yes I will ask lighttpd development team if its normal or not.
From my gut feeling a server software should never crash with Bus error.
What is interessting, I didn't notice to have this problem with OpenSolaris and ZFS. Maybe nmap has a different behaviour on Linux than Solaris.

[Stephan Ferraro (ferraro)]

I posted my question in the lighttpd development forum here:
http://redmine.lighttpd.net/boards/3/topics/3623

[gstrauss (gstrauss)]

FYI: fixed in lighttpd 1.4.40, released July 2016.

Latest lighttpd release (at time this is being written) is lighttpd 1.4.44