Scrubber/attachment bug on NetBSD

[Ericding]

I'm using Mailman 2.1.1, and have encountered issues
with mail delivery of messages with attachments (having
to do, I think, with the recent change to use the
scrubber on such messages for digests).

I actually have ARCHIVE_SCRUBBER set to 0 on my system,
since I use MHonArc for archiving. Still, the scrubber
is called from ToDigest. As a result of this bug, ANY
message with an attachment is currently getting
shunted!! :(

Here's an example stacktrace from the error log:

Feb 18 11:20:55 2003 (7122) Uncaught runner exception:
[Errno 1] Operation not permitted:
'/home/mailman/archives/private/greengrass_coordinator/attachments/20030217
/d2897c2f'
Feb 18 11:20:55 2003 (7122) Traceback (most recent call
last):
  File "/home/mailman/Mailman/Queue/Runner.py", line
105, in _oneloop
    self._onefile(msg, msgdata)
  File "/home/mailman/Mailman/Queue/Runner.py", line
155, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/home/mailman/Mailman/Queue/IncomingRunner.py",
line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/home/mailman/Mailman/Queue/IncomingRunner.py",
line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/home/mailman/Mailman/Handlers/ToDigest.py",
line 82, in process
    send_digests(mlist, mboxfp)
  File "/home/mailman/Mailman/Handlers/ToDigest.py",
line 123, in send_digests
    send_i18n_digests(mlist, mboxfp)
  File "/home/mailman/Mailman/Handlers/ToDigest.py",
line 295, in send_i18n_digests
    msg = scrubber(mlist, msg)
  File "/home/mailman/Mailman/Handlers/Scrubber.py",
line 219, in process
    url = save_attachment(mlist, part, dir, filter_html=0)
  File "/home/mailman/Mailman/Handlers/Scrubber.py",
line 341, in save_attachment
    makedirs(fsdir)
  File "/home/mailman/Mailman/Handlers/Scrubber.py",
line 333, in makedirs
    os.path.walk(dir, twiddle, None)
  File "/usr/pkg/lib/python2.2/posixpath.py", line 279,
in walk
    func(arg, top, names)
  File "/home/mailman/Mailman/Handlers/Scrubber.py",
line 332, in twiddle
    os.chmod(dirname, 02775)
OSError: [Errno 1] Operation not permitted:
'/home/mailman/archives/private/greengrass_coordinator/attachments/20030217
/d2897c2f'

To work aorund this issue, I've currently commented out
lines 331-333 on my system.

[http://sourceforge.net/tracker/index.php?func=detail&aid=688751&group_id=103&atid=100103]

[Barry Warsaw (barry)]

I need more information for this one. Do you still get the
crash if you run check_perms first? I've never seen this on
Linux and I don't have a NetBSD box available to see if
there's some platform specific problem involved.

[Ericding (ericding)]

The only problems that check_perms reveals are all the
directories that do not have the setgid bit set. I installed
with README.BSD in mind, assuming that the setgid bit is
unnecessary. Running check_perms -f to turn on that bit has
no effect on the behavior.

[Dkhelmet (dkhelmet)]

I can confirm this on OpenBSD 3.3 running Mailman 2.1.3 and
Postfix 2.0.16. I get the same exact error. Perhaps I should
try reinstalling without following the setgid flag for BSD.

Just for confirmation, after each email is sent with an
attachment, check_perms reports it doesn't have 02775
permissions. After check_perms -f is run and another email
sent, the same thing occurs, but for a different directory.
Commenting out the offending lines seems to have fixed the
problem; even the archive is correct.

I'll try recompiling in a day or two after I get some sleep.

[Awrede (awrede)]

I can confirm the original problem. Unlike dkhelmet's
problem on OpenBSD with incorrect permissions, my check_perm
reports No Problem. The work-around of commenting out the
function twiddle() and the os.path.walk on lines 342ff works.

Enviroment is NetBSD 1.6.1, with python 2.1.3 and mailman
2.1.4, both installed from NetBSD's pkgsrc system (ie.
mailman gets installed with DIRSETGID)