apt: segmentation fault [ubuntu 8.04]

[rafalm]

admin_user=$(getent group admin|cut -d: -f4|cut -d, -f1)

sudo -u "$admin_user" echo "dd"
Segmentation fault

for example:

sudo -u rafal echo "dd"
Segmentation fault

/etc/cron.daily# ./apt
a1
a2
a3
a4
a5
a5a
Segmentation fault
Segmentation fault
Segmentation fault
a6

"/etc/cron.daily/apt:"

echo "a5a"

if [ -n "$admin_user" ] && [ -x /usr/bin/sudo ] && [ -z "$http_proxy" ] && [ -x /usr/bin/gconftool ]; then
 use=$(sudo -u "$admin_user" gconftool --get /system/http_proxy/use_http_proxy)
 host=$(sudo -u "$admin_user" gconftool --get /system/http_proxy/host)
 port=$(sudo -u "$admin_user" gconftool --get /system/http_proxy/port)
 if [ "$use" = "true" ] && [ -n "$host" ] && [ -n "$port" ]; then
  export http_proxy="http://$host:$port/"
 fi
fi

echo "a6"

[rafalm (rafalm1980)]

no comment

[Arthur Archnix (arthur-archnix)]

I was attempting to update my Hardy system, and had just done "sudo apt-get update", then "sudo apt-get dist-upgrade". My internet connections dropped out while downloading packages. I could not sudo anything after that. All attempts to use sudo resulted in the error "Segmentation fault".

I rebooted into single user mode, did apt-get clean, apt-get update, apt-get dist-upgrade, and then rebooted. Sudo was still broken, and this was in my dmesg:

sudo[5357]: segfault at bf380a74 eip b7ea8b79 esp bf380a58 error

I then, just for kicks, rebooted into single user mode again and deleted everything in /var/cache /tmp /lost+found but that had no effect. I am no looking for the tires on this thing so I can kick them. I don't understand what the original bug report is saying, but it seems to be similar to my problem so I wanted to add this information.

If I can add more please let me know. I don't know how anyone could reproduce this, because I don't understand what has happened. Other than I was running apt at the time, I have no idea what has done this to sudo.

[rafalm (rafalm1980)]

sudo -u rafal echo "a"
Segmentation fault

strace sudo -u rafal echo "a"
[...]
close(3) = 0
open("/usr/lib/locale/en_US.UTF-8/LC_NUMERIC", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en_US.utf8/LC_NUMERIC", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0
mmap2(NULL, 54, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ed4000
close(3)
open("/usr/lib/locale/en_US.UTF-8/LC_CTYPE", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/en_US.utf8/LC_CTYPE", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=254076, ...}) = 0
mmap2(NULL, 254076, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7c55000
close(3) = 0
geteuid32() = 1000
write(2, "sudo: ", 6sudo: ) = 6
write(2, "must be setuid root", 19must be setuid root) = 19
write(2, "\n", 1
) = 1
exit_group(1) = ?
Process 9903 detached

[Tom_OConnor (tom-twinhelix)]

open("/etc/ld.so.cache", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0644, st_size=74784, ...}) = 0
mmap(NULL, 74784, PROT_READ, MAP_PRIVATE, 6, 0) = 0x7f933043f000
close(6) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/libselinux.so.1", O_RDONLY) = 6
read(6, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240Q\0\0"..., 832) = 832
fstat(6, {st_mode=S_IFREG|0644, st_size=109368, ...}) = 0
mmap(NULL, 2209176, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) = 0x7f932da35000
mprotect(0x7f932da4e000, 2097152, PROT_NONE) = 0
mmap(0x7f932dc4e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 6, 0x19000) = 0x7f932dc4e000
mmap(0x7f932dc50000, 1432, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f932dc50000
close(6) = 0
open("/etc/selinux/config", O_RDONLY) = -1 ENOENT (No such file or directory)
statfs("/selinux", 0x7fff385f3490) = -1 ENOENT (No such file or directory)
open("/proc/mounts", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f93305e2000
read(6, "rootfs / rootfs rw 0 0\nnone /sys"..., 1024) = 1024
read(6, ",nodev,noexec 0 0\ntmpfs /var/run"..., 1024) = 545
read(6, "", 1024) = 0
close(6) = 0
munmap(0x7f93305e2000, 4096) = 0
munmap(0x7f933043f000, 74784) = 0
read(5, "", 4096) = 0
close(5) = 0
munmap(0x7f9330452000, 4096) = 0
stat("/etc/pam.d", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/pam.d/common-account", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=491, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f93305e2000
read(5, "#\n# /etc/pam.d/common-account - "..., 4096) = 491
read(5, "", 4096) = 0
close(5) = 0
munmap(0x7f93305e2000, 4096) = 0
read(4, "", 4096) = 0
close(4) = 0
munmap(0x7f9330453000, 4096) = 0
open("/etc/pam.d/other", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=520, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f93305e2000
read(4, "#\n# /etc/pam.d/other - specify t"..., 4096) = 520
stat("/etc/pam.d", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/pam.d/common-auth", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=639, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f93305e1000
read(5, "#\n# /etc/pam.d/common-auth - aut"..., 4096) = 639
read(5, "", 4096) = 0
close(5) = 0
munmap(0x7f93305e1000, 4096) = 0
stat("/etc/pam.d", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/pam.d/common-account", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=491, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f93305e1000
rea...

[Tom_OConnor (tom-twinhelix)]

This only occurs on a system where the users' home directory is mounted on a remote server.
 The result of the "admin_user=$(getent group admin|cut -d: -f4|cut -d, -f1)" command is "instuser" which exists on the local machine, but no such home directory exists in the NFS /home export.

With a working user, ie one with a true home directory:
 stat("/home/toconnor", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
 getuid() = 0

With the non-working instuser user.
stat("/home/instuser", 0x7fffc4e41ef0) = -1 ENOENT (No such file or directory)

And all subsequent system calls fail.
The process then exits with SIGSEGV.

After creating the home directory /home/instuser
strace shows:
stat("/home/instuser", {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
getuid()
and the process nolonger segfaults.

While this fixes the bug for me, I'm in two minds over whether a similar fix is a good idea for other users.
I'm not so convinced this is a bug at all.

[Francis GUDIN (fgudin)]

Hi,

this is unrelated: my user's and the target user's home directories both exist locally, and neither are mounted from or exported to elsewhere.

Furthermore, sudo *should not segfault*: exceptional conditions must be handled properly. Especially when we talk about SUID to root programs.

--
Francis

[Tom_OConnor (tom-twinhelix)]

Francis,
What command are you trying to get sudo to execute? are you specifiying a user to run as?
Can you provide any more information about the specifics of where you encounter the bug, environment, commands, strace,
is there anything in /var/log/syslog at the time of failure? Or in /var/log/auth.log too?

[Francis GUDIN (fgudin)]

Tom,

I discovered the bug reading cron.daily/apt reports, in the same conditions as rafalm.

Until now, I didn't pay attention to those lines (they belong to auth.log):
Sep 10 11:54:31 pc07 sudo: pam_unix(sudo:session): session opened for user fgudin by (uid=0)
Sep 10 11:54:31 pc07 sudo: pam_mount(pam_mount.c:512) error trying to retrieve authtok from auth code

Any command wrapped by "sudo -u another_user " segfaults, *except* when targetted at "-u root".

With these two clues, I tried to disable pam_mount temporarily: bingo! The culprit is the "hooking" of pam_mount.so from /etc/pam.d/common-session, as per:
"session optional pam_mount.so"
Removing this entry alone works around the problem.
I don't know enough about PAM to investigate any further without help, though.

I guess you're also using PAM to mount the homedirs from NFS, aren't you ?

--
Francis

[Tom_OConnor (tom-twinhelix)]

open("/etc/pam.d/common-session", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=570, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc
5bce2d000
read(5, "#\n# /etc/pam.d/common-session - "..., 4096) = 570
open("/lib/security/pam_mkhomedir.so", O_RDONLY) = 6

Yeah, looks like that's the case on my system.
I also don't know enough about PAM, it's a bit too cryptic for me i think.

Good investigation Francis!

[rafalm (rafalm1980)]

I have pam_mount + encrypted /home/user directory

Tom_OConnor wrote:
> Francis,
> What command are you trying to get sudo to execute? are you specifiying a user to run as?
> Can you provide any more information about the specifics of where you encounter the bug, environment, commands, strace,
> is there anything in /var/log/syslog at the time of failure? Or in /var/log/auth.log too?
>

[Tom_OConnor (tom-twinhelix)]

getent group admin|cut -d: -f4|cut -d, -f1
On my system, returns 'instuser'
on my home system, returns 'tom'

This is the first user who set up the system, and this script automatically assumes that they're the first valid user in the admin group. I'm seeing this sudo error on systems where the local installation user, ie instuser; exists in a local context only, or not at all, and userdel has failed to remove them from the system.
that getent group command might not always return a valid user with a valid home directory.

[D J Gardner (djgardner)]

On my system I can confirm the pam_mount link (encrypted dir), as sudo -u works fine for a normal user (no enc. directory), just not for me (with enc. directory). However: I've just run:
# ltrace sudo -u david ls
the last part is:

[cut]
setregid(-1, 1877, 2, 0xbfbba668, 0x806e2b0) = 0
vasprintf(0xbfbba658, 0x8058dd4, 0xbfbba618, 2, 0xb7f62190) = 58
strlen("david") = 5
strlen("TTY=pts/2 ; PWD=/home/david ; US"...) = 58
openlog("sudo", 0, 80) = <void>
vsnprintf(" david : TTY=pts/2 ; PWD=/home"..., 961, "%8s : %s", 0xbfbba5e8) = 69
syslog(5, "%s", " david : TTY=pts/2 ; PWD=/home"...) = <void>
closelog() = <void>
strlen("david") = 5
free(0x806e998) = <void>
umask(022) = 022
pam_start(0x8060406, 0x8069b14, 0x806426c, 0x8064278, 1) = 0
pam_set_item(0x806e9d8, 3, 0x8067f58, 0x8064278, 1) = 0
pam_set_item(0x806e9d8, 2, 0x8069b14, 1, 0xb7e1e85c) = 0
pam_set_item(0x806e9d8, 8, 0x806e964, 1, 0xb7e1e85c) = 0
pam_set_item(0x806e9d8, 4, 0x8067eb8, 1, 0xb7e1e85c) = 0
pam_setcred(0x806e9d8, 2, 0x8067eb8, 1, 0xb7e1e85c) = 0
pam_open_session(0x806e9d8, 0, 0x8067eb8, 1, 0xb7e1e85c <unfinished ...>
malloc(8) = 0x807d080
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++
Now, what I see here is that there's a free() on a region that's then sent to pam_set_item, pam_setcred, and pam_open_session.

I'd guess this is a good place to look!
David

[D J Gardner (djgardner)]

I wrote:
> Now, what I see here is that there's a free() on a region that's then sent to ....

Only it seems I can't read. 9d8 != 998 :-( Sorry. That's no help then!

:-(

David

[D J Gardner (djgardner)]

I've tracked it down! Patch attached.

 There's a call-back from pam_open_session to sudo_conv, and sudo_conv calls strncmp on a null pointer. If the user's not using something that pam_mount needs a password for, then there's no call, and so no crash.

Problem is, that solving this bug reveals another couple of issues, because using sudo -u to a user affected by pam_mount requires a password for the session.
1) This would allow pam_mount to mount something, but the session isn't handled fully in sudo - it's opened then immediatly closed. This probably means that there's no benefit to the program being run of getting that password in the first place, haven't confirmed this though.
2) It also means that every init/cron script that uses sudo -u to that user will ask for a password. This may not be useful!

David.

[Martin Pitt (pitti)]

David, thanks for tracking this down. So it seems that there are really two independent issues here: (1) pam_verify() being called with a NULL prompt, triggering the crash; and (2) sudo -u not asking for the user's password in cases where you use libpam_mount? Or did you mean that sudo -u *does* ask for the password, but in vain, since it immediately closes the PAM session again (which would then be the real bug)?

BTW, using sudo in init scripts is not common practice or recommended at all. They should use su, and only to system users, which should not ever have an encrypted home directory. So I don't think that this bit is a problem.

Thanks for clarification! Once this is settled, this should be forwarded upstream (first the crash, which is easy enough, and then also your followup problem).

[David Tomaschik (matir)]

As a related issue: I know that /etc/cron.daily/apt is trying to get http proxy information when it does this, but there really ought to be a mechanism that doesn't involve becoming a standard user to get this information. It's bad to assume that the first account has this information. It's also bad to become a user account, if that can be avoided. Does apt have its own proxy settings?

[David Tomaschik (matir)]

This bug still persists in the Jaunty Alpha 4.

[rafalm (rafalm1980)]

David Tomaschik wrote:
> This bug still persists in the Jaunty Alpha 4.
>

hardy also

[Jan Engelhardt (jengelh)]

Tracked on http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=492333 (and who would have thought - initially posted the same day)

[Lorenzo De Liso (blackz)]

Looks like this bug has been fixed since at least (about) Lucid Lynx. Feel free to reopen this bug report if you can still reproduce the problem.

[Andy Brody (abrody)]

I can reproduce this bug with sudo 1.6.9p10-1ubuntu3.8 and a user whose home directory does not exist. It appears to happen only when sudo does not prompt for a password. It happens on several servers with similar configuration, but not on another 8.04.4 server that doesn't use ldap.

abrody@cato:~$ getent passwd nobody
nobody:x:65534:65534:nobody:/nonexistent:/bin/sh
abrody@cato:~$ ls /nonexistent
ls: cannot access /nonexistent: No such file or directory
abrody@cato:~$ sudo -u nobody true
[sudo] password for abrody:
Creating directory '/nonexistent'.
Creating directory '/nonexistent/web'.
abrody@cato:~$ sudo rm -r /nonexistent/
abrody@cato:~$ sudo -u nobody true
Segmentation fault
abrody@cato:~$

# ltrace sudo -u nobody true
[snip]
free(0x629380) = <void>
umask(022) = 022
pam_start(0x41970a, 0x6221d0, 0x61eae0, 0x61eaf8, 0x629380) = 0
pam_set_item(0x629380, 3, 0x621f20, 1, 0x629850) = 0
pam_set_item(0x629380, 2, 0x6221d0, 0, 0xfefefefefefefeff) = 0
pam_set_item(0x629380, 8, 0x629340, 49, 0x6246c0) = 0
pam_set_item(0x629380, 4, 0x621ee0, 0, 0xfefefefefefefeff) = 0
pam_setcred(0x629380, 2, 0x6295f6, 0, 0xfefefefefefefeff) = 0
pam_open_session(0x629380, 0, 0xfffffffe, 0, 1 <unfinished ...>
malloc(16) = 0x6298c0
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++