zope.security 3.8 does not compile during grokproject on macosx 10.6.6

[Ben Dadsetan]

When running grokproject for grok1.3 for the first time, it complains about missing .h files. As suggested by much older support tickets from other projects this happens whether I set CC=gcc-4.0 or not.

Blue:profile sysupbda$ grokproject profile
Enter user (Name of an initial administrator user): admin
Enter passwd (Password for the initial administrator user):
Determining current grok version...
Creating directory ./profile
Running /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Users/sysupbda/proj/profile/profile/bootstrap.py...
Creating directory '/Users/sysupbda/proj/profile/profile/bin'.
Creating directory '/Users/sysupbda/proj/profile/profile/parts'.
Creating directory '/Users/sysupbda/proj/profile/profile/develop-eggs'.
Generated script '/Users/sysupbda/proj/profile/profile/bin/buildout'.
Running /Users/sysupbda/proj/profile/profile/bin/buildout...
Develop: '/Users/sysupbda/proj/profile/profile/.'
Getting distribution for 'zope.security==3.8.0'.
In file included from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/unicodeobject.h:4,
                 from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:85,
                 from src/zope/security/_proxy.c:17:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory
In file included from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/unicodeobject.h:4,
                 from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:85,
                 from src/zope/security/_proxy.c:17:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/74/74y4zWDcHISyTOvG5rUySk+++TI/-Tmp-//ccWCUPmJ.out
error: Setup script exited with error: command 'gcc-4.0' failed with exit status 1
An error occurred when trying to install zope.security 3.8.0. Look above this message for any errors that were output by easy_install.
While:
  Installing.
  Getting section i18n.
  Initializing section i18n.
  Installing recipe z3c.recipe.i18n.
  Getting distribution for 'zope.security==3.8.0'.
Error: Couldn't install: zope.security 3.8.0
*************** PICKED VERSIONS ****************
[versions]

*************** /PICKED VERSIONS ***************

[Jan-Jaap Driessen (janjaapdriessen)]

I am a little bit wary of the MacOSX10.4 SDK in your traceback. Do you have the 10.5 and 10.6 SDK installed?

The buildslave that tests grok on OSX/python 2.7 [1] has the 10.4, 10.5 and 10.6 SDKs and CC=gcc-4.0

1) http://dev.thehealthagency.com/buildbot/builders/grok_4%20slave-osx

[Ben Dadsetan (bdadsetan)]

Hi Jan-Jaap,

  Since the last time I successfully created a grokproject using grok 1.0 I probably upgraded both MacOS and XCode but also back then I used python 2.6. I did not consciously change anything to the SDKs but I believe being pretty much up to date.

My simple test to answer your question is:

Blue:profile sysupbda$ ls /Developer/SDKs/
MacOSX10.4u.sdk MacOSX10.5.sdk MacOSX10.6.sdk

Do you think that is a sufficient test?

Thanks!

[Jan-Jaap Driessen (janjaapdriessen)]

I can see nothing wrong with the SDKs :(

In order to isolate this problem from grokproject, could you run the following buildout snippet:

'''
[buildout]
parts = zopesecurity

[zopesecurity]
recipe = z3c.recipe.scripts
eggs = zope.security
'''

This installs zope.security on my machine without problems.

Did you update xcode recently? Maybe that helps.

[Ben Dadsetan (bdadsetan)]

Hi Jan-Jaap,

  I have created a directory called a and created a file called security. I then called buildout with -c security. Note I am not that familiar with buildout so please correct me if I have done things wrong.

Blue:a sysupbda$ ../profile/bin/buildout -c security
Creating directory '/Users/sysupbda/proj/a/bin'.
Creating directory '/Users/sysupbda/proj/a/parts'.
Creating directory '/Users/sysupbda/proj/a/develop-eggs'.
Getting distribution for 'setuptools'.
Got setuptools 0.6c12dev-r88124.
Not upgrading because not running a local buildout command.
Installing zopesecurity.
Getting distribution for 'zope.security'.
In file included from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/unicodeobject.h:4,
                 from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:85,
                 from src/zope/security/_proxy.c:17:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory
In file included from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/unicodeobject.h:4,
                 from /Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:85,
                 from src/zope/security/_proxy.c:17:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory
lipo: can't figure out the architecture type of: /var/folders/74/74y4zWDcHISyTOvG5rUySk+++TI/-Tmp-//cc1wNL4n.out
error: Setup script exited with error: command 'gcc-4.0' failed with exit status 1
An error occurred when trying to install zope.security 3.8.0. Look above this message for any errors that were output by easy_install.
While:
  Installing zopesecurity.
  Getting distribution for 'zope.security'.
Error: Couldn't install: zope.security 3.8.0
Blue:a sysupbda$ cat security
[buildout]
parts = zopesecurity

[zopesecurity]
recipe = z3c.recipe.scripts
eggs = zope.security

[Jan-Jaap Driessen (janjaapdriessen)]

Did you try upgrading xcode?

[Ben Dadsetan (bdadsetan)]

Hi!

  If the way to do so is to run the Software Update... tool then I have no outstanding upgrade.
When I run Xcode and open the About... I get:
Xcode Version 3.2.4 64bit
Components have their own version:
Xcode IDE: 1708.0
Xcode Core: 1705.0
ToolSupport: 1591.0

When I look at the details, I get:
Verson: 3.2 (10M2309)
Applications:
Xcode: 3.2.4 (1708)
Interface Builder: 3.2.4 (804)
Instruments: 2.7 (3014)
Dashcode: 3.0.2 (333)

SDKs:
 Mac OS X:
   10.5 (9L31a)
   10.6 (10M2309)

I use the stock python freshly recently installed from the dmg provided on python.org.
The versions shown when starting python are the following:

Blue:~ sysupbda$ python
Python 2.7.1 (r271:86882M, Nov 30 2010, 09:39:13)
[GCC 4.0.1 (Apple Inc. build 5494)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Let me try to have a look at what others do in this case and update you with the findings. I am not that familiar with these Mac tools.

Thanks a lot

[Ben Dadsetan (bdadsetan)]

From the grok download page I see:
N O T E : XCode 3.2 which comes with Mac OS X 10.6 won't compile C-code from older Zope eggs out of the box. If you run into this problem, install XCode 3.0.

If I can't find the installation DVDs I guess it is simply a known problem and I can stop fighting it now? :)

[Ben Dadsetan (bdadsetan)]

Here are the findings I got by trying out random things. I downloaded the zip file for zope.security 3.8.0.
I tried to compile it with python setup.py install and got similar errors.

However it gave me much more messages including the command line run with gcc.
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch ppc -arch i386 -g -O2 -DNDEBUG -g -O3 -Iinclude -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/zope/security/_proxy.c -o build/temp.macosx-10.3-fat-2.7/src/zope/security/_proxy.o

was guilty of not compiling. I attempted instead

gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.5.sdk -arch ppc -arch i386 -g -O2 -DNDEBUG -g -O3 -Iinclude -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/zope/security/_proxy.c -o build/temp.macosx-10.3-fat-2.7/src/zope/security/_proxy.o

so basically just replacing MacOSX.10.4u.sdk with MacOSX10.5.sdk on the command line. This helped me compile proxy.c.

Because I am not familiar with these tools and I did not know how to get buildout to use MacOSX10.5 for testing, and I am a barbarian, I decided to rename /Developer/SDKs/MacOSX10.4 to some other name and create a symbolic link to MacOSX10.5.

It was looking good. I could compile all zope packages. I am not sure whether there are not some packages that were compiled with the old SDK and some with the new. Sadly, maybe due to the dodgy way I went ahead, my new error occurs when I try to startup the newly created grokproject.

Blue:profile sysupbda$ bin/paster serve parts/etc/deploy.ini
Traceback (most recent call last):
  File "bin/paster", line 20, in <module>
    paste.script.command.run()
  File "/Users/sysupbda/.buildout/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run
    invoke(command, command_name, options, args[1:])
  File "/Users/sysupbda/.buildout/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
  File "/Users/sysupbda/.buildout/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run
    result = self.command()
  File "/Users/sysupbda/.buildout/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 276, in command
    relative_to=base, global_conf=vars)
  File "/Users/sysupbda/.buildout/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 313, in loadapp
    **kw)
  File "/Users/sysupbda/.buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 203, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/Users/sysupbda/.buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 224, in loadobj
    return context.create()
  File "/Users/sysupbda/.buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 617, in create
    return self.object_type.invoke(self)
  File "/Users/sysupbda/.buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 163, in invoke
    app = context.app_context.create()
  File "/Users/sysupbda/.buildout/eggs/PasteDeploy-1.3.4-py2.7.egg/paste/deploy/loadwsgi.py", line 617, in create
    r...

[Jan-Jaap Driessen (janjaapdriessen)]

Moving the 10.4 SDK out of the way seems to have helped. The error you are running into now means that the packages have been installed correctly.

Unfortunately I can not reproduce the configure.zcml error on my machine, but my guess that it is related to z3c.autoinclude. Could you remove the line

  <includeDependencies package="." />

from the configure.zcml in your package and run the paster command again?

[Ben Dadsetan (bdadsetan)]

Hi!

  Sadly it produced the same error. I also tried removing my entire .buildout, I tried to look at the code mentioned in the traceback and tweaking it around to understand it better. I was not successful with any of these attempts.

  I did have the feeling that the problem is something somewhere believes it should find a configure.zcml in the wrong place. Is that accurate?

  Also according to the Download page my system (with my version of XCode) is not supported isn't it?

Thanks

[Ben Dadsetan (bdadsetan)]

Hi all,

  I finally got it by accident. It clearly was not enough to hit one issue so I hit two issues. I logged another bug for the other issue under 711279. The last problem was just that we do not support any project that is called "profile". So call your projects differently.

  This bug is therefore limited to the compilation issue and has the workaround of symbolic linking MacOSX10.4u to MacOSX10.5 in the SDKs directory as I described above.

  If we can find a way so buildout uses the right SDK then we can close this bug. Alternatively we can continue to not support my scenario as described on the Download page. Finally we also could suggest the terrible workaround I used.

Thanks for your patience and help!

Ben.

[Jan-Jaap Driessen (janjaapdriessen)]

The name you chose for your project explains the configure.zcml troubles. I'll work on grokproject and get back to you.