December 27, 2009 2:06 am

Python 2.6.4 and Twisted 9 on OS X 10.6 Snow Leopard

4 comments
by Mahmoud    in programming, python, software

I just recently purchased a MacBook Pro, which comes with Snow Leopard installed, and I noticed that it comes with python 2.6.1 installed. I wanted to upgrade to the latest python release of 2.6.4, so I tried installing the official python Mac OS distribution from python.org. After installation, I wanted to install Twisted and I kept getting this error below:

creating build/temp.macosx-10.3-fat-2.6
creating build/temp.macosx-10.3-fat-2.6/twisted
creating build/temp.macosx-10.3-fat-2.6/twisted/runner
gcc-4.0 -arch ppc -arch i386 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c twisted/runner/portmap.c -o build/temp.macosx-10.3-fat-2.6/twisted/runner/portmap.o
In file included from /usr/include/architecture/i386/math.h:626,
 from /usr/include/math.h:28,
 from /Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/pyport.h:235,
 from /Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:58,
 from twisted/runner/portmap.c:10:
/usr/include/AvailabilityMacros.h:108:14: warning: #warning Building for Intel with Mac OS X Deployment Target < 10.4 is invalid.
Compiling with an SDK that doesn't seem to exist: /Developer/SDKs/MacOSX10.4u.sdk
Please check your Xcode installation
gcc-4.0 -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.6/twisted/runner/portmap.o -o build/lib.macosx-10.3-fat-2.6/twisted/runner/portmap.so
ld: library not found for -lbundle1.o
ld: library not found for -lbundle1.o
collect2: ld returned 1 exit status
collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/T6/T6diKRiFGJSwsabKP4864E+++TI/-Tmp-//ccIK1c3K.out (No such file or directory)
error: command 'gcc-4.0' failed with exit status 1

Something’s not right — setuptools is detecting that I’m using macosx-10.3, but I’m using Mac OS X 10.6.  Why is it that setuptools also wants to use /Developer/SDKs/MacOS10.4u.sdk to build python extensions? I’m currently using the SDK for Mac OS X 10.6, and I don’t want to install another SDK.

Well, I did a little bit of research and I learned that PSF’s 2.6.4 python package for a Mac is built with an option called –enable-universalsdk which, according to the readme, defaults to /Developer/SDKs/MacOSX.10.4u.sdk. This is why building third-party extensions tries to reference the 10.4 SDK.

I was able to build python successfully using the following:

./configure --enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk/ --with-universal-archs=intel
make && make test
sudo make install

You’ll notice that the following 3 tests failed when attempting to run the unit tests:

  • asyncore
  • test_platform
  • test_macostools

The test that should really put you on alert is asyncore. After doing some research, it turns out the asyncore module is using some variant of select.poll(), which isn’t supported by the FreeBSD kernel. FreeBSD uses something called kqueue, which is what the test doesn’t take into account. To fix this, I pulled the asyncore.py module from the trunk and overwrote /Lib/asyncore.py. The tests passed then.

You don’t need to fix the other two, as they only pertain to fixing the actual tests themselves instead of having to actually change a module. If you’re interested though, fixing “test_platform” follows the same pattern as asyncore. Brett Cannon actually filed a bug for this test and submitted a patch, but you will still need to replace the entire test_platform.py module to get it working. Apparently, this patch is for python v2.7, v3.1, and v3.2. To fix it, download the patched test_platform.py module and replace it with the one in /Lib/test/test_platform.py. Make sure you also delete /Lib/test/test_platform.pyc.

The last test, test_macostools, is actually quite interesting. Apparently, Apple does not supply 64-bit versions of the Carbon frameworks used by these modules. This is why this test is failing. Looks like there might not be a way to fix this test until Apple upgrades the Carbon frameworks to 64-bit usage.

After fixing these bugs, make sure you run the following command:

sudo make install

Needless to say, after I installed python with the options above, and after I fixed all these modules, the Twisted 9.0 installation was successful.

I hope this helps some of you in case you run into this problem!

tags: , , ,
4 comments!
July 6, 2009 11:24 am

Verifying Python64 builds

2 comments
by mahmoud    in linux, programming, python

At work, I’m migrating over python to our 64bit machines and one thing that I’ve noticed was that there really was no standard python 64bit verification method to ensure the build was really 64bit or not. I’ve read somewhere previously, especially for the Mac OS X crowd, that the LDFLAGS=”-arch x86_64″ flag had to be passed in before building on a 64bit machine.

It looks like python2.6 changed the way it was required to build respective 64bit binaries. To build on standard linux x86_64 architecture, the following standard steps to installing on a 64bit machine worked for me:

./configure
make && make test
make install

Surprisingly, I received a segmentation fault when building as well as testing. I’ve never seen this before, but for those of you who are interested, the error message was:

Parser/pgen ./Grammar/Grammar ./Include/graminit.h ./Python/graminit.c
make: *** [Include/graminit.h] Segmentation fault
Parser/pgen ./Grammar/Grammar ./Include/graminit.h ./Python/graminit.c
make: *** [Python/graminit.c] Segmentation fault

The verification step is actually pretty intuitive. An easy test to verify that you’re on a 64bit machine is to find the size of the MAX_INT. Luckily for us, python makes this a very easy verification.

To verify the build, I went on a regular python 32bit machine and I did:

h[1] >>> import sys
h[1] >>> sys.maxint
2147483647

On a 64bit machine, I did:

h[2] >>> import sys
h[2] >>> sys.maxint
9223372036854775807

Clearly, my 64bit installation worked:)

Hope this helps some of you.

tags: , , , ,
2 comments!
July 4, 2009 1:19 pm

python -c ‘print “hello world!” ‘

no comments
by mahmoud    in algorithms, engineering, general, ide, linux, math, programming, python, software, ubuntu, vim

And so, we meet again, world. I’ve finally gotten around to registering a home online, installing Wordpress, and ready to share my ideas with the world. I’ve given a lot of topics some thought, and I think I might be able to influence and/or help others with my various migrations.

First, I’d like to thank Canonical for Ubuntu and making my migration from Windows to Linux desktop. Some kinks here, and there, but overall I think that it was pretty flawless. I’ve also started a large push towards using vim as my primary editor, instead of constantly switching between IDEs. I believe Netbeans 6.7 came out, I haven’t had the opportunity to play around with it, but if it was anything like Netbeans 6.5, then hey, that’s +1 for them! What a great IDE, especially with the fantastic jVI extension.

I’ll update various posts here and there with some musings about python (what a language), some software releases, mathematical musings, natural language processing tidbits (including really cool algorithms to generate domain names), and various interesting ideas that I’ve had some time to play around with.

A primary reason for starting up this blog is to share with the world some of my thoughts, improve my writing, and try to contribute to the open source world. I think there’s a lot of work to do and I can NOT wait to start. Well world, I’ll hope to speak to you soon.

tags: , , , , , , , , , , , , , , , ,
No comment?