Re: Building things with setup.py



Martin v. Löwis wrote:
James Stroud schrieb:
I think I would like to know how to avoid or correct these sort of
issues in the future, which seem to be limited, for me at least, to
scipy and numpy, with the possible exception of MySQLdb and its
dependency on zlib. Ideally, I would like to understand exactly what
causes these errors in addition to knowing what I can do to correct them
in the future.

Let's take a specific failure, namely the line

/auto_nfs/data10/users/jstroud/Programs/bin/g77
-L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o
-L/data10/users/jstroud/Programs/lib -lblas -lg2c -o
build/lib.linux-i686-2.5/numpy/core/_dotblas.so

This gives errors like

build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o(.text+0x758):numpy/core/blasdot/_dotblas.c:226:
undefined reference to `PyTuple_New'

That's not too surprising: this line tries to link the input
*as an executable program*, despite calling the output
_dotblas.so. In an executable program, all symbols need to
be defined; that's why it it complains about so many missing
symbols (including MAIN__ - which should never be missing
in a library). Even though adding python2.5.a to the linker
link makes these symbols appear, the result still won't
work, as you can't use an executable program file as if
it were a shared library.

Now, compare this to a succeeding build of a C extension
module,

gcc -pthread -shared -L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
-I/data10/users/jstroud/Programs/include
-I/data10/users/jstroud/Programs/qt/include -I/usr/include
-I/data10/users/jstroud/Programs/include/freetype2/freetype
build/temp.linux-i686-2.5/build/src.linux-i686-2.5/numpy/core/src/umathmodule.o
-lm -o build/lib.linux-i686-2.5/numpy/core/umath.so

Notice that this passes -shared to the compiler, requesting
construction of a shared library. This is the problem with
the g77 linker line; try invoking

/auto_nfs/data10/users/jstroud/Programs/bin/g77 -shared
-L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o
-L/data10/users/jstroud/Programs/lib -lblas -lg2c -o
build/lib.linux-i686-2.5/numpy/core/_dotblas.so

manually (whether or not -pthread is also necessary
or supported for g77, I don't know). This should at least
make the complaints about missing symbols go away; you
should then see whether the resulting module can be
imported in Python.

If that solves the problem, the question is why the
-shared option isn't passed automatically; your setting
LDFLAGS might indeed be a cause.

Regards,
Martin

As per your and Robert Kern's suggestions, unsetenv'ing $LDFLAGS and $CPPFLAGS indeed did the trick for numpy, scipy, and mysqldb. What a tough lesson in distutils!

<COMMENTARY>Though great for self development, I'm not so sure such lessons should be necessary to build these tools.</COMMENTARY>

James

James
.



Relevant Pages

  • Re: Building things with setup.py
    ... Please see my last message to Robert Kern. ... I think I would like to know how to avoid or correct these sort of issues in the future, which seem to be limited, for me at least, to scipy and numpy, with the possible exception of MySQLdb and its dependency on zlib. ... James Stroud ...
    (comp.lang.python)
  • NumPy 1.0.3.1 and SciPy 0.5.2.1 released
    ... I'm pleased to announce the release of NumPy 1.0.3.1 and SciPy 0.5.2.1 ... These are minor bug fix releases, which mainly addresses some build issues. ... NumPy 1.0.3.1 Bug-fixes ...
    (comp.lang.python.announce)
  • Re: Numpy array index handling
    ... Also I suggest you don't use SciPy and Matplotlib unless you ... Import from numpy instead. ... Get the latest installer from: ... At least that is what makes me as a scientist happy with Python:) ...
    (comp.lang.python)
  • Re: Numeric Soup
    ... There is SciPy, NumPy, NumArray, Numeric... ... "I have come to believe that the whole world is an enigma, a harmless enigma ...
    (comp.lang.python)
  • Trouble with numpy-0.9.4 and numpy-0.9.5
    ... I use the Python Numeric package extensively, ... transition to the new scipy core. ... I am running Python 2.4.2 on a Debian box, ... When building numpy everything seems to ...
    (comp.lang.python)