Freitag, 20. Februar 2009

Never shout at your hard disks...

...you will only slow them down (via Federico Mena-Quintero). Amazing stuff. Funny analogy in regard to leadership principles: Never shout at your staff, you will not make them work faster!

Donnerstag, 19. Februar 2009

Have your python toolchain in $HOME

In my previous post I explained how to install scipy from source on openSUSE.

What makes it particularly nice is that I can now carry most of my toolchain in my $HOME. I make python include modules from within my $HOME by setting $PYTHONPATH to something like
/home/micha/mypython/lib64/site-packages
. The good thing is that you can install any python package in your $HOME by using the --prefix option to setup.py:
python setup.py install --prefix=$HOME/mypython


For even more python goodness I tell easy_install to put everything there by having a file called .pydistutils.cfg in my $HOME with the contents

[install]
prefix=/home/micha/mypython


So everytime I easy_install a package, it is automatically put into my $HOME directory. That makes it much easier to reinstall or upgrade the system. Since most python-related stuff is now in my $HOME and not in the system, rebuilding my python-toolchain basically consists of installing python and distutils. Isn't that great :)

Update: I learnt from Brandon Rhodes that virtualenv will set up everything for you automatically. Awesome python goodness.

Installing scipy 0.7.0 on openSUSE

I had to rebuild parts of my toolchain because I messed up my OS and needed to reinstall. In the process of searching for nice numpy and scipy packages for openSuSE (which failed), I discovered that now its actually possible to do
easy_install numpy
easy_install scipy

(provided that you have the python-distutils package installed). That's great. But... it doesn't work! At least not on openSUSE. I could convince numpy to install somehow. I don't remember exactly, I think I at least needed to install gfortran, maybe also blas and lapack from the scientificlinux-repository in the build service.

For scipy then it was a little bit more work. It kept complaining that it did not find BLAS and LAPACK, even though I edited numpy's site.cfg file so that it should be aware of the location of the shared libs.

It turned out that to install scipy I had to:

  1. Download BLAS sources and unpack them, e.g. to $HOME/Apps/BLAS
  2. edit make.inc in that directory, changing the FORTRAN line to
    FORTRAN  = gfortran

  3. build BLAS by calling make in the BLAS dir
  4. DL and unpack LAPACK to $HOME/Apps/lapack-3.2
  5. edit make.inc.example in that dir, changing the BLASLIB line to
    BLASLIB = $(HOME)/Apps/BLAS/blas$(PLAT).a
    and saving that file as make.inc
  6. build LAPACK by typing make in the lapack dir.
  7. Download scipy, unpack it and start the build process:
    python setup.py install



That takes quite a while. It seems it builds LAPACK and BLAS again, so maybe you don't have to build it first, but I guess you need at least to make the appropriate modifications to the respective make.inc files. Comments on that are welcome.

But most important: it finally worked :)

Update: At least I thought it worked. It didn't :D Problem was that import scipy produced a symbol not found error. Maybe adjusting ldconfig's path could fix this, but I don't have time to look into this. Installed numpy & scipy from the scientificLinux repo (link see above).