unionfs stack trace

[Chris Elford]

I updated my system and now am seeing the following kernel dmesg when I try to run my app. Looking at my strace in the user space, I appear to open a file in /var/tmp open(O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE,0600), unlink it so that it is temporary, then run assorted fcntl64, fstat64, _llseek, write to the file descriptor before calling ftruncate64 on the file descriptor which dies with a sigsegv).

I'm using 2.6.24-9-menlow-default but see the same issue with 2.4.22-29-menlow-default. Any ideas?

hda-intel: Invalid position buffer, using LPIB read method instead.
BUG: unable to handle kernel NULL pointer dereference at virtual address 0000006e
printing eip: c01ec76b *pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: rfcomm l2cap bluetooth g_file_storage nls_utf8 cifs psb drm agpgart firmware_class i2c_algo_bit af_packet ipv6 i2c_sch i2c_core snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore snd_page_alloc joydev cpufreq_ondemand

Pid: 3791, comm: runtime Not tainted (2.6.24-9-menlow-default #1)
EIP: 0060:[<c01ec76b>] EFLAGS: 00210246 CPU: 0
EIP is at unionfs_copy_attr_all+0x6/0x46
EAX: ec81b564 EBX: ec81b564 ECX: 00000000 EDX: 00000000
ESI: 00000000 EDI: 00000000 EBP: ec81b564 ESP: f3591eb8
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process runtime (pid: 3791, ti=f3590000 task=f72973f0 task.ti=f3590000)
Stack: 00000000 00000000 c01eded5 f359ec40 00000040 c0203aa4 f7c1c0c0 c017f378
       00000000 f3591f3c ec81c9d8 ec81c8d0 00000000 f3591f3c 00000000 f3591f3c
       00000000 00002068 ec81b564 c0186c9b ffffffff 7fffffff ec81c9d8 00000008
Call Trace:
 [<c01eded5>] unionfs_setattr+0x23c/0x300
 [<c0203aa4>] inode_has_perm+0x66/0x6e
 [<c017f378>] fasync_helper+0x3c/0xae
 [<c0186c9b>] notify_change+0x146/0x2cb
 [<c0174c6c>] do_truncate+0x5e/0x75
 [<c0174db6>] do_sys_ftruncate+0x133/0x14f
 [<c0174ded>] sys_ftruncate64+0x1b/0x1d
 [<c0104e66>] sysenter_past_esp+0x5f/0x89
 =======================
Code: 28 00 74 1e 0f b7 40 6e b1 01 25 00 f0 00 00 3d 00 40 00 00 74 0c 8b 42 d4 8b 52 fc 8b 04 82 8b 48 28 89 c8 c3 56 89 d6 53 89 c3 <66> 8b 42 6e 66 89 43 6e 8b 42 2c 89 43 2c 8b 42 30 89 43 30 8b
EIP: [<c01ec76b>] unionfs_copy_attr_all+0x6/0x46 SS:ESP 0068:f3591eb8
---[ end trace 923573de2573f922 ]---

[Chris Elford (chris-l-elford)]

note that the stack details above are in dmesg.... My userapp just crashes with segv.

[Chris Elford (chris-l-elford)]

This appears to be the same as #209547 but this is for another app not for openoffice.

[alek du (alek-du)]

Chris, could you share you app with me, then I could use it to debug this issue.

[Chris Elford (chris-l-elford)]

I cannot publically share the actual app which is quite large but I was able to create a simple reproducer which should be even better... :-)

When the file is on unionfs this basically hangs during the ftruncate call and dmesg reports the error. The process becomes pretty unkillable.

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define FNAME "/tmp/myfile"

main() {
  int f,s;
  f=open(FNAME,O_RDWR|O_CREAT|O_EXCL,0600);
  fprintf(stderr,"open f=%d\n",f);
  s=unlink(FNAME);
  fprintf(stderr,"unlink status=%d\n",s);
  s=write(f,"hi",3);
  fprintf(stderr,"write status=%d\n",s);
  s=ftruncate(f,0);
  fprintf(stderr,"truncate status=%d\n",s);
}

[alek du (alek-du)]

Jay, any progress on tracing this bug?

[Jay Chetty (jaychetty)]

No, I could only test the latest unionfs (upstream). I will continue working on this after I finish the "profiling of USB suspend/resume"

Thanks
+Jay

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of alek du
Sent: Sunday, April 13, 2008 7:31 PM
To: Chetty, Jay
Subject: [Bug 210581] Re: unionfs stack trace

Jay, any progress on tracing this bug?

** Changed in: moblin-kernel
   Importance: Undecided => Low

--
unionfs stack trace
https://bugs.launchpad.net/bugs/210581
You received this bug notification because you are a bug assignee.

[alek du (alek-du)]

Jay, since you are busy with other tasks, I tried to trace the case and root caused the bug:

in fs/unionfs/inode.c 1009 line

lower_inode = unionfs_lower_inode(inode);

here lower_inode has the possibility to be NULL !!!

Add lines:
if (!lower_inode) {
                err = -ENOENT;
                goto out;
        }

to fix that. Will do more test and commit to Moblin kernel.

[alek du (alek-du)]

Code committed to Moblin kernel but currently I have no way to release it.

 diff --git a/fs/unionfs/inode.c b/fs/unionfs/inode.c
 new file mode 100644
-index 0000000..6dee4ac
+index 0000000..ea10d77
 --- /dev/null
 +++ b/fs/unionfs/inode.c
-@@ -0,0 +1,1097 @@
+@@ -0,0 +1,1101 @@
 +/*
 + * Copyright (c) 2003-2007 Erez Zadok
 + * Copyright (c) 2003-2006 Charles P. Wright
@@ -5727,6 +5727,10 @@ index 0000000..6dee4ac
 +
 + lower_inode = unionfs_lower_inode(inode);
 +
++ if (!lower_inode || !lower_dentry) {
++ err = -ENOENT;
++ goto out;
++ }
 + /*
 + * If shrinking, first truncate upper level to cancel writing dirty
 + * pages beyond the new eof; and also if its' maxbytes is more

[Chris Elford (chris-l-elford)]

Excellent! Hopefully it will come thru whenever the next Moblin kernel update happens.

Thanks,
Chris

[alek du (alek-du)]

I need build server to build "master" branch of Moblin kernel -- otherwise I have no change to close this bug.

[May Xie (may-xie)]

There is no way to build moblin kernel, will this patch be applied to UME kernel? and you will submit it as kernel PPA?

[alek du (alek-du)]

I already sent the patch to Jay and Amit (Ubuntu kernel team), do not know if they apply my patch yet. Include Jay in the loop.

Thanks
Alek
-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of May Xie
Sent: 2008年5月19日 22:57
To: Du, Alek
Subject: [Bug 210581] Re: unionfs stack trace

There is no way to build moblin kernel, will this patch be applied to
UME kernel? and you will submit it as kernel PPA?

** Changed in: moblin-kernel
       Target: m11 => None

--
unionfs stack trace
https://bugs.launchpad.net/bugs/210581
You received this bug notification because you are a bug assignee.