Samba crashes repeatedly to assert_uid

[Zds]

Binary package hint: samba

My Samba has been working fine for years now, month or two of that on Hardy binaries and out of sudden it started to crash. It has now crashed six times in 20 minutes with following stack dump:

[Thread debugging using libthread_db enabled]
[New Thread 0xb7af76d0 (LWP 7307)]
0xffffe410 in __kernel_vsyscall ()
#0 0xffffe410 in __kernel_vsyscall ()
#1 0xb7c133c3 in waitpid () from /lib/tls/i686/cmov/libc.so.6
#2 0xb7bb6533 in ?? () from /lib/tls/i686/cmov/libc.so.6
#3 0xb7d7ed7d in system () from /lib/tls/i686/cmov/libpthread.so.0
#4 0x0828aab8 in smb_panic (why=0x83bee31 "failed to set uid\n")
    at lib/util.c:1639
#5 0x08290cee in assert_uid (ruid=4294967295, euid=65534)
    at lib/util_sec.c:102
#6 0x080ffe9c in become_id (uid=65534, gid=65534) at smbd/sec_ctx.c:57
#7 0x08100245 in set_sec_ctx (uid=65534, gid=65534, ngroups=0, groups=0x0,
    token=0x850fe68) at smbd/sec_ctx.c:280
#8 0x080f3f38 in change_to_user (conn=0x851eac8, vuid=101) at smbd/uid.c:259
#9 0x08113221 in make_connection_snum (snum=6, vuser=0x851d818, password=
      {data = 0x84f9970 "", length = 1, free = 0x8287760 <free_data_blob>},
    pdev=0xbfc79058 "?????", status=0xbfc792e8) at smbd/service.c:926
#10 0x08114bf7 in make_connection (service_in=0xbfc797ec "IPC$", password=
      {data = 0x84f9970 "", length = 1, free = 0x8287760 <free_data_blob>},
    pdev=0xbfc796ec "?????", vuid=101, status=0xbfc792e8)
    at smbd/service.c:1207
#11 0x080d5657 in reply_tcon_and_X (conn=0x0, inbuf=0x85233f8 "",
    outbuf=0x8543840 "", length=80, bufsize=131072) at smbd/reply.c:508
#12 0x0810f89e in switch_message (type=117, inbuf=0x85233f8 "",
    outbuf=0x8543840 "", size=80, bufsize=131072) at smbd/process.c:1003
#13 0x08110cfb in smbd_process () at smbd/process.c:1030
#14 0x0835e8f8 in main (argc=-1212455959, argv=0xbfc79d54)
    at smbd/server.c:1120
The program is running. Quit anyway (and detach it)? (y or n) [answered Y; input not from terminal]

[Steve Langasek (vorlon)]

Thank you for taking the time to report this issue and help to improve Ubuntu.

Hitting this assertion error should also generate log entries. Could you please check /var/log/samba/log.smbd (or the per-host logfile) for entries of the form "Failure to set uid privileges to [...]"?

Does this problem persist after restarting the samba service?

[Zds (zds)]

It did persist, but as there seemed to be something wrong in the OS altogether, I gave it a reboot and after that the problem has not occurred, at least not yet.

The log does not seem to reveal much of anything:

[2008/04/10 19:56:43, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/10 20:28:43, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/10 21:00:43, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/10 21:32:43, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/10 23:40:43, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/12 15:48:17, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/12 18:56:21, 0] smbd/server.c:main(944)
  smbd version 3.0.28a started.
  Copyright Andrew Tridgell and the Samba Team 1992-2008
[2008/04/12 18:56:21, 1] param/loadparm.c:lp_do_parameter(3541)
  WARNING: The "printer admin" option is deprecated
[2008/04/12 19:00:39, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected
[2008/04/12 23:16:39, 0] lib/util_sock.c:get_peer_addr(1232)
  getpeername failed. Error was Transport endpoint is not connected

I am almost ready to blame this for hardware problem now. Before the reboot load was above 9.00, while top still said "99% idle" and showed no process taking up CPU..

But the said hardware (sans the hard drives) has run Windows for two years and memtest for days without any problems, so HW failure is not that likely either.

All in all, if the problem does not come back in few weeks, lets close this bug. I am sorry for the inconvenience.

[berni123 (bernd-bernd--zimmermann)]

I can confirm this kind of bug!
On a new installed Hardy, Samab crashes not everytime but often:

Log from the per Host logfile:
[2008/10/04 20:09:57, 0] lib/util_sec.c:set_effective_uid(205)
  setresuid failed with EAGAIN. uid(1001) might be over its NPROC limit
[2008/10/04 20:09:57, 0] lib/util_sec.c:assert_uid(101)
  Failed to set uid privileges to (-1,1001) now set to (0,0)
[2008/10/04 20:09:57, 0] lib/util.c:smb_panic(1633)
  PANIC (pid 23366): failed to set uid
[2008/10/04 20:09:57, 0] lib/util.c:log_stack_trace(1737)
  BACKTRACE: 20 stack frames:
   #0 /usr/sbin/smbd(log_stack_trace+0x1c) [0x613b1c]
   #1 /usr/sbin/smbd(smb_panic+0x43) [0x613c03]
   #2 /usr/sbin/smbd [0x618cc1]
   #3 /usr/sbin/smbd [0x4ba57e]
   #4 /usr/sbin/smbd(pop_sec_ctx+0x96) [0x4ba6f6]
   #5 /usr/sbin/smbd(unbecome_root+0x9) [0x4afec9]
   #6 /usr/sbin/smbd(gid_to_sid+0x168) [0x5d36b8]
   #7 /usr/sbin/smbd(get_nt_acl+0x44a) [0x4c432a]
   #8 /usr/sbin/smbd(is_visible_file+0x26e) [0x46e06e]
   #9 /usr/sbin/smbd [0x46e5f0]
   #10 /usr/sbin/smbd(dptr_ReadDirName+0x54) [0x46e664]
   #11 /usr/sbin/smbd [0x4a54b4]
   #12 /usr/sbin/smbd [0x4a8ae3]
   #13 /usr/sbin/smbd(handle_trans2+0x1be) [0x4a927e]
   #14 /usr/sbin/smbd(reply_trans2+0x6ea) [0x4afc3a]
   #15 /usr/sbin/smbd [0x4c879e]
   #16 /usr/sbin/smbd(smbd_process+0x7e2) [0x4c9b92]
   #17 /usr/sbin/smbd(main+0x8cd) [0x6c5f6d]
   #18 /lib/libc.so.6(__libc_start_main+0xf4) [0x7f02cab011c4]
   #19 /usr/sbin/smbd [0x45a869]
[2008/10/04 20:09:57, 0] lib/util.c:smb_panic(1638)
  smb_panic(): calling panic action [/usr/share/samba/panic-action 23366]
[2008/10/04 20:09:59, 0] lib/util.c:smb_panic(1646)
  smb_panic(): action returned status 0
[2008/10/04 20:09:59, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/smbd

Panic-Action output:
[Thread debugging using libthread_db enabled]
[New Thread 0x7f02cd972700 (LWP 23366)]
0x00007f02cab804a5 in waitpid () from /lib/libc.so.6
#0 0x00007f02cab804a5 in waitpid () from /lib/libc.so.6
#1 0x00007f02cab21461 in ?? () from /lib/libc.so.6
#2 0x0000000000613c4b in smb_panic (why=<value optimized out>)
    at lib/util.c:1639
#3 0x0000000000618cc1 in assert_uid (ruid=4294967295, euid=1001)
    at lib/util_sec.c:102
#4 0x00000000004ba57e in become_id (uid=1001, gid=100) at smbd/sec_ctx.c:57
#5 0x00000000004ba6f6 in pop_sec_ctx () at smbd/sec_ctx.c:345
#6 0x00000000004afec9 in unbecome_root () at smbd/uid.c:400
#7 0x00000000005d36b8 in gid_to_sid (psid=0x7fffd5990990, gid=100)
    at passdb/lookup_sid.c:1202
#8 0x00000000004c432a in get_nt_acl (fsp=0xb407a0, security_info=7,
    ppdesc=0x7fffd5990ac8) at smbd/posix_acls.c:2809
#9 0x000000000046e06e in is_visible_file (conn=0xaa6da0,
    dir_path=0xa72060 "./", name=<value optimized out>, pst=0x7fffd5991490,
    use_veto=1) at smbd/dir.c:897
#10 0x000000000046e5f0 in dptr_normal_ReadDirName (dptr=0xa05440,
    poffset=0x7fffd5991558, pst=0x7fffd5991490) at smbd/dir.c:562
#11 0x000000000046e664 in dptr_ReadDirName (dptr=0xa05440,
    poffset=0x7fffd5991558, pst=0x7fffd5991490) at smbd/dir.c:642
#12 0x00000000004a54b4 in get_lanman2_dir_entry (conn=0xaa6da0,
    inbuf=<value optimized out>, outbuf=0xadf130...

[Steve Langasek (vorlon)]

berni123,

Thanks for providing this log. Now that I see the log, the crash makes perfect sense. :)

But I have to say that it's not a samba bug. What we're looking at is a failed assertion because the OS wouldn't let the process change UIDs; as the log says, this may be because the user has reached his limit on the number of processes allowed.

So to fix this, you should do one of two things:

- raise the process limit for your user (for instance, via /etc/security/limits.conf) if you want these connections to succeed
- or, if you think the process limit is already set correctly, you can comment out the 'panic action' line in /etc/samba/smb.conf to suppress the email output.

[berni123 (bernd-bernd--zimmermann)]

Steve,

this could not be normal - in /etc/security/limits.conf there is no limit configured, and I dont know,
but could it be unlimited as default in Ubuntuty Hardy ?

Commenting out panic-action does not solve the problem, because it only prevents the mail
and not the action. smbd still crashes and the users got the windows messages that their
network drives are not available.

Just a few minuts ago I had crashes after crashes and have to shut down samba and restart it,
so I commented out the panic action but it smbs still crashes.

Memtest shows now error with the memory and the rest of the system is running fine.
There is still this strange samba behavior.

[berni123 (bernd-bernd--zimmermann)]

So again here is the last panic action output, a little longer, but still the NPROC limit error in the host logfile:

[2008/10/06 08:49:27, 0] lib/util_sec.c:set_effective_uid(205)
  setresuid failed with EAGAIN. uid(1001) might be over its NPROC limit
[2008/10/06 08:49:27, 0] lib/util_sec.c:assert_uid(101)
  Failed to set uid privileges to (-1,1001) now set to (0,0)
[2008/10/06 08:49:27, 0] lib/util.c:smb_panic(1633)
  PANIC (pid 6517): failed to set uid

[2008/10/06 08:49:27, 0] lib/util.c:log_stack_trace(1737)
  BACKTRACE: 20 stack frames:
   #0 /usr/sbin/smbd(log_stack_trace+0x1c) [0x613b1c]
   #1 /usr/sbin/smbd(smb_panic+0x43) [0x613c03]
   #2 /usr/sbin/smbd [0x618cc1]
   #3 /usr/sbin/smbd [0x4ba57e]
   #4 /usr/sbin/smbd(pop_sec_ctx+0x96) [0x4ba6f6]
   #5 /usr/sbin/smbd(unbecome_root+0x9) [0x4afec9]
   #6 /usr/sbin/smbd(gid_to_sid+0x168) [0x5d36b8]
   #7 /usr/sbin/smbd(get_nt_acl+0x44a) [0x4c432a]
   #8 /usr/sbin/smbd(is_visible_file+0x26e) [0x46e06e]
   #9 /usr/sbin/smbd [0x46e5f0]
   #10 /usr/sbin/smbd(dptr_ReadDirName+0x54) [0x46e664]
   #11 /usr/sbin/smbd [0x4a54b4]
   #12 /usr/sbin/smbd [0x4a8ae3]
   #13 /usr/sbin/smbd(handle_trans2+0x1be) [0x4a927e]
   #14 /usr/sbin/smbd(reply_trans2+0x6ea) [0x4afc3a]
   #15 /usr/sbin/smbd [0x4c879e]
   #16 /usr/sbin/smbd(smbd_process+0x7e2) [0x4c9b92]
   #17 /usr/sbin/smbd(main+0x8cd) [0x6c5f6d]
   #18 /lib/libc.so.6(__libc_start_main+0xf4) [0x7f22689ce1c4]
   #19 /usr/sbin/smbd [0x45a869]
[2008/10/06 08:49:27, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/smbd

[Thread debugging using libthread_db enabled]
[New Thread 0x7f02cd972700 (LWP 6430)]
0x00007f02cab804a5 in waitpid () from /lib/libc.so.6
#0 0x00007f02cab804a5 in waitpid () from /lib/libc.so.6
#1 0x00007f02cab21461 in ?? () from /lib/libc.so.6
#2 0x0000000000613c4b in smb_panic (why=<value optimized out>)
    at lib/util.c:1639
#3 0x0000000000618cc1 in assert_uid (ruid=4294967295, euid=1001)
    at lib/util_sec.c:102
#4 0x00000000004ba57e in become_id (uid=1001, gid=100) at smbd/sec_ctx.c:57
#5 0x00000000004ba6f6 in pop_sec_ctx () at smbd/sec_ctx.c:345
#6 0x00000000004afec9 in unbecome_root () at smbd/uid.c:400
#7 0x00000000005d36b8 in gid_to_sid (psid=0x7fffd5990990, gid=100)
    at passdb/lookup_sid.c:1202
#8 0x00000000004c432a in get_nt_acl (fsp=0xb16fb0, security_info=7,
    ppdesc=0x7fffd5990ac8) at smbd/posix_acls.c:2809
#9 0x000000000046e06e in is_visible_file (conn=0xab9240,
    dir_path=0xa85ce0 "./", name=<value optimized out>, pst=0x7fffd5991490,
    use_veto=1) at smbd/dir.c:897
#10 0x000000000046e5f0 in dptr_normal_ReadDirName (dptr=0xa13ff0,
    poffset=0x7fffd5991558, pst=0x7fffd5991490) at smbd/dir.c:562
#11 0x000000000046e664 in dptr_ReadDirName (dptr=0xa13ff0,
    poffset=0x7fffd5991558, pst=0x7fffd5991490) at smbd/dir.c:642
#12 0x00000000004a54b4 in get_lanman2_dir_entry (conn=0xab9240,
    inbuf=<value optimized out>, outbuf=0xadefb0 "",
    path_mask=0x7fffd5992770 "*", dirtype=22, info_level=260,
    requires_resume_key=4, dont_descend=0, ppdata=0x7fffd5992740,
    base_data=0xaff400 "`", end_data=0xb043ff "", space_remaining=10504,
    out_of_space=0x7fffd5992764, got_ex...

[Steve Langasek (vorlon)]

Bernd,

Can you please provide the output of 'ps -u 1001' on this system?

I'm not sure how you've managed to hit the process limit during normal operation, without having set a lower limit in /etc/security/limits.conf.

If uid 1001 is your user (as confirmed by 'getent passwd 1001'), could you also run 'ulimit -S -u' and post that output?

[Paul Dufresne (paulduf)]

Bug #314657 and bug #229654 seems to be duplicates.

Well, I would make them duplicates of this one, but, as it was marked invalid...
hum, in fact looks more like Steve thinks his explanation for invaliding is doubtful.

Guess I'll mark those duplicate of this one, and change to Incomplete.

[Paul Dufresne (paulduf)]

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468296 suggests it could be linked to roaming profiles.

[Paul Dufresne (paulduf)]

I now tend to think this is caused by the -1 not being casted to appropriate type in:
setresgid(-1,gid,-1);
and in:
assert_uid(-1, uid);
in lib/sec_util.c as I said in bug #314657 causing problems on 64 bits machines.
But I am unsure.

[Steve Langasek (vorlon)]

On Sat, Mar 14, 2009 at 02:39:53AM -0000, Paul Dufresne wrote:
> I now tend to think this is caused by the -1 not being casted to appropriate type in:
> setresgid(-1,gid,-1);
> and in:
> assert_uid(-1, uid);
> in lib/sec_util.c as I said in bug #314657 causing problems on 64 bits
> machines.

Absolutely not. You don't have to cast integer types when passing them as
function arguments, you just have to have a proper function prototype -
which Samba has for all functions it uses. This can be confirmed by
grepping for warnings in the build logs.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Paul Dufresne (paulduf)]

Yeah, I began to realise I was wrong before reading you comment.

I have a new, I hope better hypothesis now.
It look a bit stupid, but maybe it is trying to become a deleted user.

For ZDS (original poster) it would have means he would have deleted the "nobody" user (normally nobody have
id 65534 in /etc/passwd, but I remember a time where I was deleting it for feeling my system more secure.
Maybe even the user nogroup in /etc/group.

For Berni, it would be not having a group 1001 in /etc/group
Based on "Failed to set uid privileges to (-1,1001) now set to (0,0)" info.
Try to look if "sudo /etc/group | grep 1001" returns something.

[Paul Dufresne (paulduf)]

For Berni, I meant:
"sudo cat /etc/group | grep 1001" of course.

[Steve Langasek (vorlon)]

On Sun, Mar 15, 2009 at 03:01:40AM -0000, Paul Dufresne wrote:
> It look a bit stupid, but maybe it is trying to become a deleted user.

> For ZDS (original poster) it would have means he would have deleted the
> "nobody" user (normally nobody have id 65534 in /etc/passwd, but I
> remember a time where I was deleting it for feeling my system more secure.
> Maybe even the user nogroup in /etc/group.

Sorry, not this either. The setuid() calls don't require that the uid has a
matching name on the system.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Paul Dufresne (paulduf)]

Ok, I think this time I have found.

taken from 3.0.34 samba changelog (latest in 3.0):
http://www.samba.org/samba/history/samba-3.0.34.html
o Andrew Tridgell
    * Avoid a race condition in glibc between AIO and setresuid().
    * Become root for AIO operations.

This would explain why https://bugs.launchpad.net/ubuntu/+source/samba/+bug/341816
say that Build the sernet samba packages from source (3.0.34):
http://ftp.sernet.de/pub/samba/old/src/debian/3.0.34 fix it.

[Paul Dufresne (paulduf)]

In fact I believe it comes from that guy:
http://webui.sourcelabs.com/rhel/issues/459901
he seems to suggest that the real problem is with the kernel not exactly implementing Posix correctly.

I still have a doubt because he speak of setresuid changing the value of AIO, not changing the value of
setresuid itself. OTOH, previous comment suggest that this fix the bug with ASSERT_UID.
Maybe this also change the value returned by setresuid.

[Thierry Carrez (ttx)]

Avoid a race condition in glibc between AIO and setresuid():
http://gitweb.samba.org/?p=samba.git;a=commit;h=c1ef3b8e28777f70086c8d2ee851e3a0b5b4ca4c

Become root for AIO operations:
http://gitweb.samba.org/?p=samba.git;a=commit;h=c60631879dbe6e87bc03c0412a4c8857b2f4339b

[Thierry Carrez (ttx)]

Proposed patch for Hardy

That fix was included in 3.0.34, 3.2.4 and 3.3.0 so Jaunty would be already fixed.

Please confirm that it really fixes the issue by testing the hardy samba packages from my PPA (samba_3.0.28a-1ubuntu4.9~ppa1) at https://launchpad.net/~ttx/+archive/ppa

[Thierry Carrez (ttx)]

Proposed patch for Intrepid

Packages also available for testing in my PPA (2:3.2.3-1ubuntu3.6~ppa1) at https://launchpad.net/~ttx/+archive/ppa

I can't really reproduce the issue in the first place, so I'd welcome some testing to make sure this is indeed the right fix :)

[Paul Dufresne (paulduf)]

Well, I don't really can test it either, so it would have to be tested by others.

I don't really understand, how this race between AIO and setresuid would affect the result of setresuid.
Seems it's fix the result of the (Asynchronous IO), not of setresuid.

But I have an even much bigger problem with this.
My understanding of the problem (which could well be wrong), is that Linux have effective user ID by thread,
rather than by process (this is one of the big guess I make that are often wrong).
But POSIX works with uid on process rather than threads (yes, an other guess).

So, glibc guys, think that they should change effective uid of other threads of the process to respect the POSIX standard.
Which does make sense to me (even if it may cause problems like race condition with AIO, and maybe other stuff).

Now, my really big problem with all this, is that glibc seems to have made a big error.
They seems to think that setresuid is a syscall in the POSIX standards, and setreuid is not.
After all, they seems to have made setresuid ('being smart' about changing other threads of the process).
But it is reverse, as far as I know: setresuid is not part of any POSIX standard, but setreuid is.
Need to find back the pdf where I have read that.

Now, what would seems like a better patch for me, would be to simply make a loop to retry 3 to 5 times,
the setresuid or setreuid, when the result is EAGAIN, which does means after all, "can not do your requested
operation now, please try again".

[Paul Dufresne (paulduf)]

Sigh. It is even a bit more complex than that:
"In Unix-like systems, user-level activities are implemented by running processes. Most Unix systems support a ``thread'' as a separate concept; threads share memory inside a process, and the system scheduler actually schedules threads. Linux does this differently (and in my opinion uses a better approach): there is no essential difference between a thread and a process. Instead, in Linux, when a process creates another process it can choose what resources are shared (e.g., memory can be shared). The Linux kernel then performs optimizations to get thread-level speeds; see clone(2) for more information. It's worth noting that the Linux kernel developers tend to use the word ``task'', not ``thread'' or ``process'', but the external documentation tends to use the word process (so I'll use the term ``process'' here). When programming a multi-threaded application, it's usually better to use one of the standard thread libraries that hide these differences."
Taken from: http://linux.com/base/ldp/howto/Secure-Programs-HOWTO/processes.html

[Anders Paulsen (anders-sce)]

I've installed the files from https://launchpad.net/~ttx/+archive/ppa and restarted the machine. It's running Hardy, and one of my shares are completely inaccessible from my Ubuntu desktop running gnome. Are there any other race conditions that should be avoided?

My trace seems a bit different from the original bug post, as there are a few other system calls involved, namely unbecome_root(), gid_to_sid() and maybe more:

[Thread debugging using libthread_db enabled]
[New Thread 0xb7af76d0 (LWP 5391)]
0xb7f7d410 in __kernel_vsyscall ()
#0 0xb7f7d410 in __kernel_vsyscall ()
#1 0xb7c134d3 in waitpid () from /lib/tls/i686/cmov/libc.so.6
#2 0xb7bb6643 in ?? () from /lib/tls/i686/cmov/libc.so.6
#3 0xb7d7ed7d in system () from /lib/tls/i686/cmov/libpthread.so.0
#4 0x0828ad48 in smb_panic (why=0x83bf191 "failed to set uid\n")
    at lib/util.c:1639
#5 0x08290f5e in assert_uid (ruid=4294967295, euid=1000) at lib/util_sec.c:102
#6 0x080fff3c in become_id (uid=1000, gid=1000) at smbd/sec_ctx.c:57
#7 0x081000d2 in pop_sec_ctx () at smbd/sec_ctx.c:345
#8 0x080f39d7 in unbecome_root () at smbd/uid.c:400
#9 0x082412bc in gid_to_sid (psid=0xbf803eb8, gid=1000)
    at passdb/lookup_sid.c:1202
#10 0x0810646c in create_file_sids (psbuf=0xbf803e58,
    powner_sid=<value optimized out>, pgroup_sid=0xbf803eb8)
    at smbd/posix_acls.c:669
#11 0x0810acbe in get_nt_acl (fsp=0x857dd50, security_info=7,
    ppdesc=0xbf803ff0) at smbd/posix_acls.c:2809
#12 0x08122e2b in vfswrap_fget_nt_acl (handle=0x851b988, fsp=0x857dd50, fd=-1,
    security_info=7, ppdesc=0xbf803ff0) at modules/vfs_default.c:921
#13 0x080ab18d in is_visible_file (conn=0x85602b8, dir_path=0x8503d30 "./",
    name=0x8579efb "ad39a.odt", pst=0xbf804974, use_veto=1)
    at smbd/dir.c:897
#14 0x080ab8bd in dptr_normal_ReadDirName (dptr=0x8433bc8, poffset=0xbf8049f0,
    pst=0xbf804974) at smbd/dir.c:562
#15 0x080ab921 in dptr_ReadDirName (dptr=0x8433bc8, poffset=0x0,
    pst=0xbf804974) at smbd/dir.c:578
#16 0x080e8e4f in get_lanman2_dir_entry (conn=0x85602b8,
    inbuf=<value optimized out>, outbuf=0x853efa8 "",
    path_mask=0xbf805b28 "*", dirtype=22, info_level=260,
    requires_resume_key=4, dont_descend=0, ppdata=0xbf805b24,
    base_data=0x858bcc0 "`", end_data=0x8590dc3 "", space_remaining=10680,
    out_of_space=0xbf805b18, got_exact_match=0xbf805b0c,
    last_entry_off=0xbf805b20, name_list=0x0, ea_ctx=0x0) at smbd/trans2.c:1149
#17 0x080ec30e in call_trans2findfirst (conn=0x85602b8, inbuf=0x851eb60 "",
    outbuf=0x853efa8 "", bufsize=131072, pparams=0x851c6e8, total_params=18,
    ppdata=0x851c6f0, total_data=0, max_data_bytes=16644) at smbd/trans2.c:1859
#18 0x080ecb24 in handle_trans2 (conn=0x85602b8, state=0x851c5b0,
    inbuf=0x851eb60 "", outbuf=0x853efa8 "", size=90, bufsize=131072)
    at smbd/trans2.c:6433
#19 0x080f367c in reply_trans2 (conn=0x85602b8, inbuf=0x851eb60 "",
    outbuf=0x853efa8 "", size=90, bufsize=131072) at smbd/trans2.c:6703
#20 0x0810f93e in switch_message (type=50, inbuf=0x851eb60 "",
    outbuf=0x853efa8 "", size=90, bufsize=131072) at smbd/process.c:1004
#21 0x08110d9b in smbd_process () at smbd/process.c:1031
#22 0x0835ec68 in main (...

[Paul Dufresne (paulduf)]

Thanks anders for that new piece of information.

Could you give the output of 'dpkg -l | grep samba' just to be sure that you are running the patch version?

This seems to confirmed that the new version does fix Asynchronous I/O result, but not the fact that setreSuid results in EAGAIN error. That said, like I said, stopping samba on an EAGAIN error without retrying seems wrong.

I don't think we can consider that getting an EAGAIN is a linux kernel bug.
OTOH I guess we could, because it did not seems to do this in the past.

[Thierry Carrez (ttx)]

Anders: thanks for testing !

Apparently my packages do not change anything :/

That might mean bug 341816 is not a duplicate of this one, I'll follow up on that one separately.
The problem here is probably a similar one, but does not involve AIO code.

[Thierry Carrez (ttx)]

Please let me know if all reporters run SELinux, so that I can comment on the upstream bug.

[Anders Paulsen (anders-sce)]

I posted this output in my previous post, but it was lost inside the "Read more..." link :)

~$ dpkg -l | grep samba
ii samba 3.0.28a-1ubuntu4.9~ppa1
ii samba-common 3.0.28a-1ubuntu4.9~ppa1
ii samba-dbg 3.0.28a-1ubuntu4.9~ppa1

I've looked through the emails i got with the samba panic/segfault messages, and as far as I can see, they haven't changed after i installed the package from the PPA, which leads me to believe this is a different issue than the one this bug was originally about.
The file included in the report, however, seems to be random, but is always a file in the root directory of the share (in this case ad3.odt).

#13 0x080ab18d in is_visible_file (conn=0x85602b8, dir_path=0x8503d30 "./",
    name=0x8579efb "ad39a.odt", pst=0xbf804974, use_veto=1)
    at smbd/dir.c:897

Is there any more information you need from me, or anything else i could do to help out more?

Anders

[Paul Dufresne (paulduf)]

Thanks Anders, I should have seen this "Read more..." link.

For me, this helps to know if samba logs have the following line:
setresuid failed with EAGAIN.

But the latest ppa1 version probably won't do that, you'd have to revert to plain samba first.

And the Samba log are many files by default because of the:
   log file = /var/log/samba/log.%m
inside /etc/samba/smb.conf file.

Removing .%m at the end should allow to put all messages in one file, which could make it easier to grep EAGAIN on it.

The idea is that this give a clue as if it have failed for permissions errors (no EAGAIN), or because of EAGAIN error, which is much more harder to understand why.

But as current code, only show this #if USE_SETRESUID
then not seeing EAGAIN message is not that much a clear information.

Slightly rewriting that part could help a lot I guess.
I am not too used to that kind of programming, but I guess I could try to write a little patch that would help to pinpoint the problem. (without really fixint it... although it could fix it).

[Anders Paulsen (anders-sce)]

I haven't reverted to plain samba, but I get output that has the line you mention in the logfile concerning my desktop computer (log.eos). I've attached the log.

[Keith Matthews (keith-matthews-themegroup)]

Thierry Carrez wrote:
> Please let me know if all reporters run SELinux, so that I can comment
> on the upstream bug.
>

Installed but default config (which I interpret to means 'effectively
disabled'.

I haven't managed to try the patched version yet - been at the UKUUG
conference.

Keith

--
Theme Group
3 & 4 Grove Park
Waltham Road
White Waltham
MAIDENHEAD
Berkshire
England
SL6 3LW

E: <email address hidden>
T: [phone]
F: [phone]
I: [phone]

http://www.themegroup.com <http://www.themegroup.com/>

----------------------------------------------------------------------------
Theme Group is a wholly owned trading style of This & That '95 Limited.
Registered in England No: 3092394 at 7/8 Eghams Court, Boston Drive, Bourne
End, Buckinghamshire. SL8 5YS.
----------------------------------------------------------------------------
This message and its attached file(s), is strictly confidential and intended
solely for the person whom it is addressed. It may contain personal and
confidential information and as such may be protected by the Data Protection
Act 1998. If you received this message in error, you must not copy,
distribute or take any action in reliance on it. Please notify us as soon
as possible and delete it and any attached files from your system. Any views
expressed in this communication may not necessarily be the views held by
Theme Group. As Internet communications are not secure we do not accept
legal responsibility for the contents of this message nor responsibility for
any change made to this message after the original sender sent it. Although
we have taken steps to ensure that this email and attachments are free from
any virus, we advise that in keeping with good computer practice the
recipient should ensure they are actually virus free. We advise you to
carry out your own virus check before opening any attachment, as Theme Group
is not liable for any loss or damage arising in any way from this message or
its attachments. Thank you for taking the time to read this as it may have
been a complete waste of your time. Very few people bother to read these
things but it proves how special you are 'cause you have, so thanks again.
Please take time to read our terms and conditions at
www.themegroup.co.uk/terms.pdf
<blocked::blocked::">http://www.themegroup.co.uk/terms.pdf>
<http://www.themegroup.co.uk/terms.pdf>

---------------------------------------------------------------------------

[Keith Matthews (keith-matthews-themegroup)]

We've got the updated debs in and the problems is appearing again. Twice in a couple of days.

Log entry from latest.
[2009/04/02 09:18:12, 0] lib/util_sec.c:set_effective_uid(205)
  setresuid failed with EAGAIN. uid(10000) might be over its NPROC limit
[2009/04/02 09:18:12, 0] lib/util_sec.c:assert_uid(101)
  Failed to set uid privileges to (-1,10000) now set to (0,0)
[2009/04/02 09:18:12, 0] lib/util.c:smb_panic(1633)
  PANIC (pid 7577): failed to set uid

[2009/04/02 09:18:12, 0] lib/util.c:log_stack_trace(1737)
  BACKTRACE: 20 stack frames:
   #0 /usr/sbin/smbd(log_stack_trace+0x1c) [0x613b4c]
   #1 /usr/sbin/smbd(smb_panic+0x43) [0x613c33]
   #2 /usr/sbin/smbd [0x618cf1]
   #3 /usr/sbin/smbd [0x4ba5ae]
   #4 /usr/sbin/smbd(pop_sec_ctx+0x96) [0x4ba726]
   #5 /usr/sbin/smbd(unbecome_root+0x9) [0x4afef9]
   #6 /usr/sbin/smbd(gid_to_sid+0x168) [0x5d36e8]
   #7 /usr/sbin/smbd(get_nt_acl+0x44a) [0x4c435a]
   #8 /usr/sbin/smbd(is_visible_file+0x26e) [0x46e09e]
   #9 /usr/sbin/smbd [0x46e620]
   #10 /usr/sbin/smbd(dptr_ReadDirName+0x54) [0x46e694]
   #11 /usr/sbin/smbd [0x4a54e4]
   #12 /usr/sbin/smbd [0x4a8b13]
   #13 /usr/sbin/smbd(handle_trans2+0x1be) [0x4a92ae]
   #14 /usr/sbin/smbd(reply_trans2+0x6ea) [0x4afc6a]
   #15 /usr/sbin/smbd [0x4c87ce]
   #16 /usr/sbin/smbd(smbd_process+0x7e2) [0x4c9bc2]
   #17 /usr/sbin/smbd(main+0x8cd) [0x6c5fad]
   #18 /lib/libc.so.6(__libc_start_main+0xf4) [0x7fe2ab5d11c4]
   #19 /usr/sbin/smbd [0x45a899]
[2009/04/02 09:18:12, 0] lib/util.c:smb_panic(1638)
  smb_panic(): calling panic action [/usr/share/samba/panic-action 7577]
[2009/04/02 09:18:15, 0] lib/util.c:smb_panic(1646)
  smb_panic(): action returned status 0
[2009/04/02 09:18:15, 0] lib/fault.c:dump_core(181)
  dumping core in /var/log/samba/cores/smbd
[2009/04/02 09:18:15, 1] smbd/service.c:make_connection_snum(1033)
  ldorsett (10.0.0.222) connect to service Admin initially as user ldorsett (uid=10000, gid=1009) (pid 7748)
[2009/04/02 09:18:16, 0] lib/util_sec.c:set_effective_uid(205)
  setresuid failed with EAGAIN. uid(10000) might be over its NPROC limit
[2009/04/02 09:18:16, 0] lib/util_sec.c:assert_uid(101)
  Failed to set uid privileges to (-1,10000) now set to (0,0)
[2009/04/02 09:18:16, 0] lib/util.c:smb_panic(1633)
  PANIC (pid 7748): failed to set uid

[2009/04/02 09:18:16, 0] lib/util.c:log_stack_trace(1737)
  BACKTRACE: 20 stack frames:
   #0 /usr/sbin/smbd(log_stack_trace+0x1c) [0x613b4c]
   #1 /usr/sbin/smbd(smb_panic+0x43) [0x613c33]
   #2 /usr/sbin/smbd [0x618cf1]
   #3 /usr/sbin/smbd [0x4ba5ae]
   #4 /usr/sbin/smbd(pop_sec_ctx+0x96) [0x4ba726]
   #5 /usr/sbin/smbd(unbecome_root+0x9) [0x4afef9]
   #6 /usr/sbin/smbd(gid_to_sid+0x168) [0x5d36e8]
   #7 /usr/sbin/smbd(get_nt_acl+0x44a) [0x4c435a]
   #8 /usr/sbin/smbd(is_visible_file+0x26e) [0x46e09e]
   #9 /usr/sbin/smbd [0x46e620]
   #10 /usr/sbin/smbd(dptr_ReadDirName+0x54) [0x46e694]
   #11 /usr/sbin/smbd [0x4a54e4]
   #12 /usr/sbin/smbd [0x4a8b13]
   #13 /usr/sbin/smbd(handle_trans2+0x1be) [0x4a92ae]
   #14 /usr/sbin/smbd(reply_trans2+0x6ea) [0x4afc6a]
   #15 /usr/sbin/smbd [0x4c87ce]
   #16 /usr/sbin/smbd(smbd_process+0x7e2) [0x4c9bc2]
   #17 /usr/sbin/smbd(main+0x8cd) [0x6c5fad]
  ...

[Brian Murray (brian-murray)]

I've removed the patch flags from the previous attachments that were flagged as patches since they didn't seem to fix the issue although they are still patches. This will prevent the bug from showing up in the patched bug workflow which seems most appropriate so no action is to be taken with the old patches. Feel free to contact me directly if you feel this was incorrect.

[Dirk Schaare (dirk-schaare)]

We are also randomly affected by this issue. I'm wondering if there is any progress in fixing these problems in Hardy.

Nov 30 12:29:59 hm-server smbd[9133]: [2009/11/30 12:29:59, 0] lib/util_sec.c:set_effective_uid(205)
Nov 30 12:29:59 hm-server smbd[9133]: setresuid failed with EAGAIN. uid(1027) might be over its NPROC limit
Nov 30 12:29:59 hm-server smbd[9133]: [2009/11/30 12:29:59, 0] lib/util_sec.c:assert_uid(101)
Nov 30 12:29:59 hm-server smbd[9133]: Failed to set uid privileges to (-1,1027) now set to (0,0)
Nov 30 12:29:59 hm-server smbd[9133]: [2009/11/30 12:29:59, 0] lib/util.c:smb_panic(1633)
Nov 30 12:29:59 hm-server smbd[9133]: PANIC (pid 9133): failed to set uid
Nov 30 12:29:59 hm-server smbd[9133]:
Nov 30 12:29:59 hm-server smbd[9133]: [2009/11/30 12:29:59, 0] lib/util.c:log_stack_trace(1737)
Nov 30 12:29:59 hm-server smbd[9133]: BACKTRACE: 21 stack frames:
Nov 30 12:29:59 hm-server smbd[9133]: #0 /usr/sbin/smbd(log_stack_trace+0x1c) [0x613c3c]
Nov 30 12:29:59 hm-server smbd[9133]: #1 /usr/sbin/smbd(smb_panic+0x43) [0x613d23]
Nov 30 12:29:59 hm-server smbd[9133]: #2 /usr/sbin/smbd [0x618de1]
Nov 30 12:29:59 hm-server smbd[9133]: #3 /usr/sbin/smbd [0x4ba5de]
Nov 30 12:29:59 hm-server smbd[9133]: #4 /usr/sbin/smbd(pop_sec_ctx+0x96) [0x4ba756]
Nov 30 12:29:59 hm-server smbd[9133]: #5 /usr/sbin/smbd(unbecome_root+0x9) [0x4aff29]
Nov 30 12:29:59 hm-server smbd[9133]: #6 /usr/sbin/smbd(uid_to_sid+0x171) [0x5d3f31]
Nov 30 12:29:59 hm-server smbd[9133]: #7 /usr/sbin/smbd [0x4bfd6f]
Nov 30 12:29:59 hm-server smbd[9133]: #8 /usr/sbin/smbd(get_nt_acl+0x44a) [0x4c438a]
Nov 30 12:29:59 hm-server smbd[9133]: #9 /usr/sbin/smbd(is_visible_file+0x26e) [0x46e0ce]
Nov 30 12:29:59 hm-server smbd[9133]: #10 /usr/sbin/smbd [0x46e650]
Nov 30 12:29:59 hm-server smbd[9133]: #11 /usr/sbin/smbd(dptr_ReadDirName+0x54) [0x46e6c4]
Nov 30 12:29:59 hm-server smbd[9133]: #12 /usr/sbin/smbd [0x4a5514]
Nov 30 12:29:59 hm-server smbd[9133]: #13 /usr/sbin/smbd [0x4a8b43]
Nov 30 12:29:59 hm-server smbd[9133]: #14 /usr/sbin/smbd(handle_trans2+0x1be) [0x4a92de]
Nov 30 12:29:59 hm-server smbd[9133]: #15 /usr/sbin/smbd(reply_trans2+0x6ea) [0x4afc9a]
Nov 30 12:29:59 hm-server smbd[9133]: #16 /usr/sbin/smbd [0x4c880e]
Nov 30 12:29:59 hm-server smbd[9133]: #17 /usr/sbin/smbd(smbd_process+0x7cd) [0x4c9eed]
Nov 30 12:29:59 hm-server smbd[9133]: #18 /usr/sbin/smbd(main+0x8cd) [0x6c609d]
Nov 30 12:29:59 hm-server smbd[9133]: #19 /lib/libc.so.6(__libc_start_main+0xf4) [0x7f73d42951c4]
Nov 30 12:29:59 hm-server smbd[9133]: #20 /usr/sbin/smbd [0x45a899]
Nov 30 12:29:59 hm-server smbd[9133]: [2009/11/30 12:29:59, 0] lib/fault.c:dump_core(181)
Nov 30 12:29:59 hm-server smbd[9133]: dumping core in /var/log/samba/cores/smbd

[Thierry Carrez (ttx)]

I haven't seen this one happening in karmic/lucid, could anyone confirm ?

[Chuck Short (zulcss)]

Intrepid has reached EOL so im closing this SRU.

[Alex Valavanis (valavanisalex)]

Intrepid Ibex reached end-of-life on 30 April 2010 so I am closing the
report. Can anyone confirm whether this issue still exists in later versions of Ubuntu?

[Maarten Bezemer (veger)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. It would help us a lot if you could test it on a currently supported Ubuntu version. When you test it and it is still an issue, kindly upload the updated logs by running apport-collect 216358 and any other logs that are relevant for this particular issue.