mac osx socket.accept return None instead of ""

Bug #135108 reported by Paolo Losi
6
Affects Status Importance Assigned to Milestone
ZODB
Fix Released
High
Christian Theune
3.10
Fix Released
Undecided
Unassigned
3.6
Won't Fix
Undecided
Unassigned

Bug Description

ZODB 3.6.0 (should apply to trunk as well)

on OS X version:
hypermac:~ paolo$ uname -a
Darwin hypermac.local 8.10.1 Darwin Kernel Version 8.10.1: Wed May 23 16:33:00 PDT 2007; root:xnu-792.22.5~1/RELEASE_I386 i386 i386

python installed from mac ports:
Python 2.4.4 (#1, Jun 30 2007, 16:45:04)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin

unix socket accept return addr == None instead of addr == "".

--- StorageServer.py.old 2007-08-27 19:04:54.000000000 +0200
+++ StorageServer.py 2007-08-27 19:18:31.000000000 +0200
@@ -113,6 +113,8 @@
         addr = conn.addr
         if isinstance(addr, type("")):
             label = addr
+ elif addr is None:
+ label = ''
         else:
             host, port = addr
             label = str(host) + ":" + str(port)

Related branches

Revision history for this message
Christian Theune (ctheune) wrote :

What error happens in your case? Can you maybe provide a test case that shows this?

Changed in zodb:
importance: Undecided → Medium
status: New → Incomplete
Revision history for this message
Paolo Losi (paolo-enuan) wrote :

I've just reproduced it with ZODB3.8 beta4

OS: Mac OS X 10.4.10
python version: 2.4.4 (mac ports version)

You find a small test case attacched.

Here you have the ZEO log.

zeo log:
(zeotest)hypermac:~/schifo/zeotest/src paolo$ ./launch_zeo.sh
------
2007-11-12T09:54:34 INFO ZEO.runzeo (511) opening storage '1' using FileStorage
------
2007-11-12T09:54:34 INFO ZEO.StorageServer (511) StorageServer created RW with storages: 1:RW:test.fs
------
2007-11-12T09:54:34 INFO ZEO.zrpc (511) listening on ./test.sock
error: uncaptured python exception, closing channel <ZEO.zrpc.server.Dispatcher listening './test.sock' at 0x12015d0> (exceptions.TypeError:unpack non-sequence [/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/asyncore.py|read|69] [/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/asyncore.py|handle_read_event|385] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/zrpc/server.py|handle_accept|58] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/StorageServer.py|new_connection|957] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/zrpc/connection.py|__init__|863] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/StorageServer.py|notifyConnected|130])
------
2007-11-12T09:54:37 INFO ZEO.zrpc.Connection(S) () received handshake 'Z308'
------
2007-11-12T09:54:37 INFO ZEO.StorageServer (511) disconnected
------
2007-11-12T09:54:37 INFO ZEO.runzeo (511) closing storage '1'

Revision history for this message
Paolo Losi (paolo-enuan) wrote :
Revision history for this message
Christian Theune (ctheune) wrote :

Note to self: the bug is referring to a *unix* socket, not a tcp socket.

Revision history for this message
Christian Theune (ctheune) wrote :

I'll look at this today.

Changed in zodb:
assignee: nobody → ct-gocept
Revision history for this message
Christian Theune (ctheune) wrote :

The test applies neither to the trunk nor the 3.6 branch.

There is an API mismatch regarding the database interface in your test case: db.conn() does not exist and db.open() works.

Also, your testcase says DB.conn() which I'm not sure was ever working.

Please try to provide a test case that works directly and try to make it work against a current version of the release branch you are referring to.

Revision history for this message
Paolo Losi (paolo-enuan) wrote :

Christian,

I've updated the test case and still getting the same traceback.
Please note the problem is server side (zeo).

I'm using ZODB3.8b4.
Wich OS X version / python version are you using?
please note the problem is OS X specific AFAIK

Thanks
Paolo

2007-11-18T16:29:49 INFO ZEO.StorageServer (535) StorageServer created RW with storages: 1:RW:test.fs
------
2007-11-18T16:29:49 INFO ZEO.zrpc (535) listening on ./test.sock
error: uncaptured python exception, closing channel <ZEO.zrpc.server.Dispatcher listening './test.sock' at 0x1478580> (exceptions.TypeError:unpack non-sequence [/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/asyncore.py|read|69] [/opt/local/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/asyncore.py|handle_read_event|385] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/zrpc/server.py|handle_accept|58] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/StorageServer.py|new_connection|957] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/zrpc/connection.py|__init__|863] [/Users/paolo/schifo/zeotest/lib/python2.4/ZODB3-3.8.0b4-py2.4-macosx-10.3-i386.egg/ZEO/StorageServer.py|notifyConnected|130])

Changed in zodb:
status: Incomplete → New
Revision history for this message
Christian Theune (ctheune) wrote :

I'm using a Mac:

Darwin lrrr.whq.gocept.com 9.1.0 Darwin Kernel Version 9.1.0: Wed Oct 31 17:46:22 PDT 2007; root:xnu-1228.0.2~1/RELEASE_I386 i386

With Python:

Python 2.5.1 (r251:54863, Oct 5 2007, 21:08:09)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin

on ZODB trunk and 3.6 branch.

The test works fine for me.

Are you using a system python. Can you try using a custom-compiled python?

Changed in zodb:
status: New → Incomplete
Jim Fulton (jim-zope)
Changed in zodb:
importance: Medium → High
Changed in python:
assignee: nobody → Python Development (python-dev)
Revision history for this message
Jim Fulton (jim-zope) wrote :

I don't know that there's enough info here for the Python folks to
work on. Here's an explanation from a guy here at ZC:

"Setting the SO_LINGER option to off on a socket means that when the
 socket is closed, the call returns immediately, and a TCP RST packet
 is sent to the peer.

 Normally when a socket is closed it is handled gracefully, i.e., care
 is taken that all outstanding packets have been received, and a TCP
 FIN packet is sent.

 In a way this is a race condition. In Mac OS X, the sockaddr struct
 must be deallocated as soon as a RST packet is received (causing
 accept to return None). This may only happen if the RST packet is
 sent close enough to the initial TCP handshake."

It sounds like the implementation of socket.accept should check this
condition.

ZEO has a broader problem, with a broader solution, that it isn't
proprtly handling exceptions raised in it's application-level accept
logic.

Interestingly, I can't repoduce the symptom reported in this issue,
although I get a result, for ZEO, that is just as bad. The ZEO server
stops accepting new connections.

Revision history for this message
Jim Fulton (jim-zope) wrote :

This is for Python maintainers, if anyone is paying attention. :)

I was able to reproduce this problem with Python 2.6 on Mac OS X 10.6.

I've attached a demonstration script, 135108.py.

Run it with a "server" argument:

  python2.6 135108.py server

In a separate shell, run it with a client argument:

  python2.6 135108.py client

On my mac, this causes the server to print None and exit,
demonstrating that that accept returned None, when it probably should
have done something else. :)

Revision history for this message
Jim Fulton (jim-zope) wrote :

Bonus:

If getpeername is called on the returned socket, [Errno 22] Invalid
argument is raised, rather than errno.ENOTCONN, which is what asyncore
expects. :)

Jim Fulton (jim-zope)
Changed in zodb:
milestone: none → 3.9.6
status: Incomplete → Fix Committed
milestone: 3.9.6 → 3.10.0a2
milestone: 3.10.0a2 → 3.9.6
Revision history for this message
Tres Seaver (tseaver) wrote :
Changed in zodb:
status: Fix Committed → Fix Released
Curtis Hovey (sinzui)
affects: python → obsolete-junk
no longer affects: obsolete-junk
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.