středa 8. prosince 2010

How to build Python libraries on 64bit Windows

The other day I needed to try some Python libraries at work. Being mostly .NET shop, we usually use Windows machines for development which sometimes causes problems for Linux-primary applications.
I'm running Windows 7 64bit and Python 2.7 64bit. Of course, the libraries were not built for this combination so I had to build them from source. Normally, this is a simple task of unzipping and running python setup.py install, but it was not that easy on this new machine.
After a lot of errors and pointless googling, I finally found a solution in an article Building Python Extensions in a Modern Windows Environment by Matt Schmidt. So if you have the same problem, you can just head there and read the article while I rant about the useless paths.

Fruitless paths


The problem manifested in a few different ways. Firstly, trying to build the libraries like usual threw error

raise ValueError(str(list(result.keys())))
ValueError: [u'path']


Not very useful message, mind you. There is a reported issue in Python for this exact exception during building libraries but it relates to different problem (using Visual Studio Express for building).
The problem appeared to be in Python's usage of Visual Studio compiler so I tried to use MinGW instead. What I did not verify was whether MinGW supports 64bit applications, or just 32bit. So instead of a built library I just had another error:

collect2: ld returned 1 exit status


I eventually got to the information that base MinGW can only be used for 32bit applications and even found a fork that should work for 64bits. I don't remember which fork it was, I only remember it not working. Fortunately I didn't have to fight with the libraries much longer because I finally stumbled on Matt's article.


Solution


This is a quick summary of Matt's article above:

  1. Install Microsoft Visual C++ 2008 (sadly, VS 2010 is not supported at the moment)
  2. Install the Windows 7 Platform SDK
  3. Create directory C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\
  4. Create file C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat with following content:

    call "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64 /Release

  5. Patch Python Visual studio compiler - add the following line to distutils/msvc9compiler.py after line 648 (this is in method link after line ld_args.append('/MANIFESTFILE:' + temp_manifest):

    ld_args.append('/MANIFEST')

  6. That's it. Now you can run python setup.py install to install your brand new Python libraries


I hope this will be of use if you have similar problems, at least as a point to Matt's article.