--introspect-c-modules loads foreign extension modules

[Jean-Paul Calderone]

If I build Twisted's extension modules using PyPy and then run pydoctor using CPython, passing the --introspect-c-modules flag, pydoctor fails with this exception:

Traceback (most recent call last):
  File "/home/exarkun/Projects/pydoctor/trunk/bin/pydoctor", line 12, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/exarkun/Projects/pydoctor/trunk/pydoctor/driver.py", line 318, in main
    system.addPackage(path, prependedpackage)
  File "/home/exarkun/Projects/pydoctor/trunk/pydoctor/model.py", line 618, in addPackage
    self.addPackage(fullname, package)
  File "/home/exarkun/Projects/pydoctor/trunk/pydoctor/model.py", line 620, in addPackage
    self.addModuleFromPath(package, fullname)
  File "/home/exarkun/Projects/pydoctor/trunk/pydoctor/model.py", line 637, in addModuleFromPath
    (suffix, mode, type))
ImportError: /home/exarkun/Projects/Twisted/trunk/twisted/runner/portmap.pypy-14.so: undefined symbol: _Py_InitPyPyModule

It seems the logic for deciding whether a .so should be loaded as an extension or not needs a little more smarts.

[Michael Hudson-Doyle (mwhudson)]

Do you think just catching ImportError is enough? Clearly extension modules _can_ raise any error which loaded (the can also call abort() I guess).

[Jean-Paul Calderone (exarkun)]

Perhaps? It seems this particular case could be solved by recognizing the newer naming convention for extension modules and not loading extensions that the current Python runtime wouldn't load (perhaps with a warning?).

Actually, the misbehavior is a little worse. My tree actually has _both_ CPython _and_ PyPy builds of these extension modules. pydoctor should pick up the CPython version when running on CPython and ignore the PyPy version.

    exarkun@top:~/Projects/Twisted/trunk$ find -name '*.so'
    ./twisted/runner/portmap.pypy-14.so
    ./twisted/runner/portmap.so
    ./twisted/test/raiser.so
    ./twisted/python/_initgroups.so
    ./twisted/python/_initgroups.pypy-14.so
    ./twisted/python/sendmsg.pypy-14.so
    ./twisted/python/sendmsg.so
    ./twisted/internet/_sigchld.pypy-14.so
    ./twisted/internet/_sigchld.so
    exarkun@top:~/Projects/Twisted/trunk$