You must log in to use subscribtions.

Clear message

Table of Contents

Introduction

These are simple NumPy and SWIG examples which use the numpy.i interface file. There is also a MinGW section for people who may want to use these in a Win32 environment. The following code is C, rather than C++.

The information contained here was first made available by Bill Spotz in his article numpy.i: a SWIG Interface File for NumPy, and the NumPy SVN which can be checked out using the following command:

svn co http://scipy.org/svn/numpy/trunk numpy

Initial setup

gcc and SWIG

Check that both gcc and SWIG are available (paths known):

swig -version

and

gcc -v

Both should output some text...

Modifying the pyfragments.swg file (MinGW only)

This is from my own tests, running SWIG Version 1.3.36 and gcc version 3.4.5 (mingw-vista special r3). I had to remove the 'static' statements from the source, otherwise your SWIGed sources won't compile. There are only two 'static' statements in the file, both will need removing. Here is my modified version: pyfragments.swg

Compilation and testing

A setup.py file specific to each module must be written first. I based mine on the reference setup.py available in http://scipy.org/svn/numpy/trunk/doc/swig/test/ with added automatic handling of swig.

On a un*x like system, the command-line is:

python setup.py build

In a Win32 environment (either cygwin or cmd), the setup command-line is (for use with MinGW):

python setup.py build --compiler=mingw32

The command handles both the SWIG process (generation of wrapper C and Python code) and gcc compilation. The resulting module (a pyd file) is built in the build\lib.XXX directory (e.g. for a Python 2.5 install and on a Win32 machine, the build\lib.win32-2.5 directory).

A simple ARGOUT_ARRAY1 example

This is a re-implementation of the range function. The module is called ezrange. One thing to remember with ARGOUT_ARRAY1 is that the dimension of the array must be passed from Python.

From Bill Spotz's article: The python user does not pass these arrays in, they simply get returned. For the case where a dimension is specified, the python user must provide that dimension as an argument.

This is useful for functions like numpy.arange(N), for which the size of the returned array is known in advance and passed to the C function.

For functions that follow array_out = function(array_in) where the size of array_out is not known in advance and depends on memory allocated in C, see the example given in Cookbook/SWIG Memory Deallocation.

The C source (ezrange.c and ezrange.h)

Here is the ezrange.h file:

   1 void range(int *rangevec, int n);

Here is the ezrange.c file:

   1 void range(int *rangevec, int n)
   2 {
   3     int i;
   4 
   5     for (i=0; i< n; i++)
   6         rangevec[i] = i;
   7 }

The interface file (ezrange.i)

Here is the ezrange.i file.

   1 %module ezrange
   2 
   3 %{
   4     #define SWIG_FILE_WITH_INIT
   5     #include "ezrange.h"
   6 %}
   7 
   8 %include "numpy.i"
   9 
  10 %init %{
  11     import_array();
  12 %}
  13 
  14 %apply (int* ARGOUT_ARRAY1, int DIM1) {(int* rangevec, int n)}
  15 
  16 %include "ezrange.h"

Don't forget that you will also need the numpy.i file in the same directory.

Setup file (setup.py)

This is my setup.py file:

   1 #! /usr/bin/env python
   2 
   3 # System imports
   4 from distutils.core import *
   5 from distutils      import sysconfig
   6 
   7 # Third-party modules - we depend on numpy for everything
   8 import numpy
   9 
  10 # Obtain the numpy include directory.  This logic works across numpy versions.
  11 try:
  12     numpy_include = numpy.get_include()
  13 except AttributeError:
  14     numpy_include = numpy.get_numpy_include()
  15 
  16 # ezrange extension module
  17 _ezrange = Extension("_ezrange",
  18                    ["ezrange.i","ezrange.c"],
  19                    include_dirs = [numpy_include],
  20                    )
  21 
  22 # ezrange setup
  23 setup(  name        = "range function",
  24         description = "range takes an integer and returns an n element int array where each element is equal to its index",
  25         author      = "Egor Zindy",
  26         version     = "1.0",
  27         ext_modules = [_ezrange]
  28         )

Compiling the module

The setup command-line is:

python setup.py build

or

python setup.py build --compiler=mingw32

depending on your environment.

Testing the module

If everything goes according to plan, there should be a _ezrange.pyd file available in the build\lib.XXX directory. You will need to copy the file in the directory where the ezrange.py file is (generated by swig), in which case, the following will work (in python):

>>> import ezrange
>>> ezrange.range(10)
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

A simple INPLACE_ARRAY1 example

This example doubles the elements of the 1-D array passed to it. The operation is done in-place, which means that the array passed to the function is changed.

The C source (inplace.c and inplace.h)

Here is the inplace.h file:

   1 void inplace(double *invec, int n);

Here is the inplace.c file:

   1 void inplace(double *invec, int n)
   2 {
   3     int i;
   4 
   5     for (i=0; i<n; i++)
   6     {
   7         invec[i] = 2*invec[i];
   8     }
   9 }

The interface file (inplace.i)

Here is the inplace.i interface file:

   1 %module inplace
   2 
   3 %{
   4     #define SWIG_FILE_WITH_INIT
   5     #include "inplace.h"
   6 %}
   7 
   8 %include "numpy.i"
   9 
  10 %init %{
  11     import_array();
  12 %}
  13 
  14 %apply (double* INPLACE_ARRAY1, int DIM1) {(double* invec, int n)}
  15 %include "inplace.h"
  16 

Setup file (setup.py)

This is my setup.py file:

   1 #! /usr/bin/env python
   2 
   3 # System imports
   4 from distutils.core import *
   5 from distutils      import sysconfig
   6 
   7 # Third-party modules - we depend on numpy for everything
   8 import numpy
   9 
  10 # Obtain the numpy include directory.  This logic works across numpy versions.
  11 try:
  12     numpy_include = numpy.get_include()
  13 except AttributeError:
  14     numpy_include = numpy.get_numpy_include()
  15 
  16 # inplace extension module
  17 _inplace = Extension("_inplace",
  18                    ["inplace.i","inplace.c"],
  19                    include_dirs = [numpy_include],
  20                    )
  21 
  22 # NumyTypemapTests setup
  23 setup(  name        = "inplace function",
  24         description = "inplace takes a double array and doubles each of its elements in-place.",
  25 
  26         author      = "Egor Zindy",
  27         version     = "1.0",
  28         ext_modules = [_inplace]
  29         )

Compiling the module

The setup command-line is:

python setup.py build

or

python setup.py build --compiler=mingw32

depending on your environment.

Testing the module

If everything goes according to plan, there should be a _inplace.pyd file available in the build\lib.XXX directory. You will need to copy the file in the directory where the inplace.py file is (generated by swig), in which case, the following will work (in python):

>>> import numpy
>>> import inplace
>>> a = numpy.array([1,2,3],'d')
>>> inplace.inplace(a)
>>> a
array([2., 4., 6.])

A simple ARGOUTVIEW_ARRAY1 example

Big fat multiple warnings

Please note, Bill Spotz advises against the use of argout_view arrays, unless absolutely necessary:

Python does not take care of memory de-allocation, as stated here by Travis Oliphant: http://blog.enthought.com/?p=62

Memory deallocation is also difficult to handle automatically as there is no easy way to do module "finalization". There is a Py_InitModule() function, but nothing to handle deletion/destruction/finalization (this will be addressed in Python 3000 as stated in PEP3121. In my example, I use the python module atexit but there must be a better way.

Having said all that, if you have no other choice, here is an example that uses ARGOUTVIEW_ARRAY1. As usual, comments welcome!

The module declares a block of memory and a couple of functions:

The C source (ezview.c and ezview.h)

Here is the ezview.h file:

   1 void set_ones(double *array, int n);

Here is the ezview.c file:

   1 #include <stdio.h>
   2 #include <stdlib.h> 
   3 
   4 #include "ezview.h"
   5 
   6 void set_ones(double *array, int n)
   7 {
   8     int i;
   9 
  10     if (array == NULL)
  11         return;
  12 
  13     for (i=0;i<n;i++)
  14         array[i] = 1.;
  15 }

The interface file (ezview.i)

Here is the ezview.i interface file:

   1 %module ezview
   2 
   3 %{
   4     #define SWIG_FILE_WITH_INIT
   5     #include "ezview.h"
   6 
   7     double *my_array = NULL;
   8     int my_n = 10;
   9 
  10     void __call_at_begining()
  11     {
  12         printf("__call_at_begining...\n");
  13         my_array = (double *)malloc(my_n*sizeof(double));
  14     }
  15 
  16     void __call_at_end(void)
  17     {
  18         printf("__call_at_end...\n");
  19         if (my_array != NULL)
  20             free(my_array);
  21     }
  22 %}
  23 
  24 %include "numpy.i"
  25 
  26 %init %{
  27     import_array();
  28     __call_at_begining();
  29 %}
  30 
  31 %apply (double** ARGOUTVIEW_ARRAY1, int *DIM1) {(double** vec, int* n)}
  32 
  33 %include "ezview.h"
  34 %rename (set_ones) my_set_ones;
  35 
  36 %inline %{
  37 void finalize(void){
  38     __call_at_end();
  39 }
  40 
  41 void get_view(double **vec, int* n) {
  42     *vec = my_array;
  43     *n = my_n;
  44 }
  45 
  46 void my_set_ones(double **vec, int* n) {
  47     set_ones(my_array,my_n);
  48     *vec = my_array;
  49     *n = my_n;
  50 }
  51 %}

Don't forget that you will also need the numpy.i file in the same directory.

Setup file (setup.py)

This is my setup.py file:

   1 #! /usr/bin/env python
   2 
   3 # System imports
   4 from distutils.core import *
   5 from distutils      import sysconfig
   6 
   7 # Third-party modules - we depend on numpy for everything
   8 import numpy
   9 
  10 # Obtain the numpy include directory.  This logic works across numpy versions.
  11 try:
  12     numpy_include = numpy.get_include()
  13 except AttributeError:
  14     numpy_include = numpy.get_numpy_include()
  15 
  16 # view extension module
  17 _ezview = Extension("_ezview",
  18                    ["ezview.i","ezview.c"],
  19                    include_dirs = [numpy_include],
  20                    )
  21 
  22 # NumyTypemapTests setup
  23 setup(  name        = "ezview module",
  24         description = "ezview provides 3 functions: set_ones(), get_view() and finalize(). set_ones() and get_view() provide a view on a memory block allocated in C, finalize() takes care of the memory deallocation.",
  25         author      = "Egor Zindy",
  26         version     = "1.0",
  27         ext_modules = [_ezview]
  28         )

Compiling the module

The setup command-line is:

python setup.py build

or

python setup.py build --compiler=mingw32

depending on your environment.

Testing the module

If everything goes according to plan, there should be a _ezview.pyd file available in the build\lib.XXX directory. You will need to copy the file in the directory where the ezview.py file is (generated by swig), in which case, the following will work (in python):

The test code test_ezview.py follows:

   1 import atexit
   2 import numpy
   3 print "first message is from __call_at_begining()"
   4 import ezview
   5 
   6 #There is no easy way to finalize the module (see PEP3121)
   7 atexit.register(ezview.finalize)
   8 
   9 a = ezview.set_ones()
  10 print "\ncalling ezview.set_ones() - now the memory block is all ones.\nReturned array (a view on the allocated memory block) is:"
  11 print a
  12 
  13 print "\nwe're setting the array using a[:]=arange(a.shape[0])\nThis changes the content of the allocated memory block:"
  14 a[:] = numpy.arange(a.shape[0])
  15 print a
  16 
  17 print "\nwe're now deleting the array  - this only deletes the view,\nnot the allocated memory!"
  18 del a
  19 
  20 print "\nlet's get a new view on the allocated memory, should STILL contain [0,1,2,3...]"
  21 b = ezview.get_view()
  22 print b
  23 
  24 print "\nnext message from __call_at_end() - finalize() registered via module atexit"

Launch test_ezview.py and the following will hopefully happen:

~> python test_ezview.py
first message is from __call_at_begining()
__call_at_begining...

calling ezview.set_ones() - now the memory block is all ones.
Returned array (a view on the allocated memory block) is:
[ 1.  1.  1.  1.  1.  1.  1.  1.  1.  1.]

we re setting the array using a[:]=arange(a.shape[0])
This changes the content of the allocated memory block:
[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9.]

we re now deleting the array  - this only deletes the view,
not the allocated memory!

let s get a new view on the allocated memory, should STILL contain [0,1,2,3...]
[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9.]

next message from __call_at_end() - finalize() registered via module atexit
__call_at_end...

Error handling using errno and python exceptions

I have been testing this for a few months now and this is the best I've come-up with. If anyone knows of a better way, please let me know.

From the opengroup website, the lvalue errno is used by many functions to return error values. The idea is that the global variable errno is set to 0 before a function is called (in swig parlance: $action), and checked afterwards. If errno is non-zero, a python exception with a meaningful message is generated, depending on the value of errno.

The following example comprises two examples: First example uses errno when checking whether an array index is valid. Second example uses errno to notify the user of a malloc() problem.

The C source (ezerr.c and ezerr.h)

Here is the ezerr.h file:

   1 int val(int *array, int n, int index);
   2 void alloc(int n);

Here is the ezerr.c file:

   1 #include <stdlib.h>
   2 #include <errno.h>
   3 
   4 #include "ezerr.h"
   5 
   6 //return the array element defined by index
   7 int val(int *array, int n, int index)
   8 {
   9     int value=0;
  10 
  11     if (index < 0 || index >=n)
  12     {
  13         errno = EPERM;
  14         goto end;
  15     }
  16 
  17     value = array[index];
  18 
  19 end:
  20     return value;
  21 }
  22 
  23 //allocate (and free) a char array of size n
  24 void alloc(int n)
  25 {
  26     char *array;
  27 
  28     array = (char *)malloc(n*sizeof(char));
  29     if (array == NULL)
  30     {
  31         errno = ENOMEM;
  32         goto end;
  33     }
  34 
  35     //don't keep the memory allocated...
  36     free(array);
  37 
  38 end:
  39     return;
  40 }

The interface file (ezerr.i)

Here is the ezerr.i interface file:

   1 %module ezerr
   2 %{
   3 #include <errno.h>
   4 #include "ezerr.h"
   5 
   6 #define SWIG_FILE_WITH_INIT
   7 %}
   8 
   9 %include "numpy.i"
  10 
  11 %init %{
  12     import_array();
  13 %}
  14 
  15 %exception
  16 {
  17     errno = 0;
  18     $action
  19 
  20     if (errno != 0)
  21     {
  22         switch(errno)
  23         {
  24             case EPERM:
  25                 PyErr_Format(PyExc_IndexError, "Index out of range");
  26                 break;
  27             case ENOMEM:
  28                 PyErr_Format(PyExc_MemoryError, "Failed malloc()");
  29                 break;
  30             default:
  31                 PyErr_Format(PyExc_Exception, "Unknown exception");
  32         }
  33         SWIG_fail;
  34     }
  35 }
  36 
  37 %apply (int* IN_ARRAY1, int DIM1) {(int *array, int n)}
  38 
  39 %include "ezerr.h"

Note the SWIG_fail, which is a macro for goto fail in case there is any other cleanup code to execute (thanks Bill!).

Don't forget that you will also need the numpy.i file in the same directory.

Setup file (setup.py)

This is my setup.py file:

   1 #! /usr/bin/env python
   2 
   3 # System imports
   4 from distutils.core import *
   5 from distutils      import sysconfig
   6 
   7 # Third-party modules - we depend on numpy for everything
   8 import numpy
   9 
  10 # Obtain the numpy include directory.  This logic works across numpy versions.
  11 try:
  12     numpy_include = numpy.get_include()
  13 except AttributeError:
  14     numpy_include = numpy.get_numpy_include()
  15 
  16 # err extension module
  17 ezerr = Extension("_ezerr",
  18                    ["ezerr.i","ezerr.c"],
  19                    include_dirs = [numpy_include],
  20 
  21                    extra_compile_args = ["--verbose"]
  22                    )
  23 
  24 # NumyTypemapTests setup
  25 setup(  name        = "err test",
  26         description = "A simple test to demonstrate the use of errno and python exceptions",
  27         author      = "Egor Zindy",
  28         version     = "1.0",
  29         ext_modules = [ezerr]
  30         )

Compiling the module

The setup command-line is:

python setup.py build

or

python setup.py build --compiler=mingw32

depending on your environment.

Testing the module

If everything goes according to plan, there should be a _ezerr.pyd file available in the build\lib.XXX directory. You will need to copy the file in the directory where the ezerr.py file is (generated by swig), in which case, the following will work (in python):

The test code test_err.py follows:

   1 
   2 import traceback,sys
   3 import numpy
   4 import ezerr
   5 
   6 print "\n--- testing ezerr.val() ---"
   7 a = numpy.arange(10)
   8 indexes = [5,20,-1]
   9 
  10 for i in indexes:
  11     try:
  12         value = ezerr.val(a,i)
  13     except:
  14         print ">> failed for index=%d" % i
  15         traceback.print_exc(file=sys.stdout)
  16     else:
  17         print "using ezerr.val() a[%d]=%d - should be %d" % (i,value,a[i])
  18 
  19 print "\n--- testing ezerr.alloc() ---"
  20 amounts = [1,-1] #1 byte, -1 byte
  21 
  22 for n in amounts:
  23     try:
  24         ezerr.alloc(n)
  25     except:
  26         print ">> could not allocate %d bytes" % n
  27         traceback.print_exc(file=sys.stdout)
  28     else:
  29         print "allocated (and deallocated) %d bytes" % n

Launch test_err.py and the following will hopefully happen:

~> python test_err.py

--- testing ezerr.val() ---
using ezerr.val() a[5]=5 - should be 5
>> failed for index=20
Traceback (most recent call last):
  File "test_err.py", line 11, in <module>
    value = ezerr.val(a,i)
IndexError: Index out of range
>> failed for index=-1
Traceback (most recent call last):
  File "test_err.py", line 11, in <module>
    value = ezerr.val(a,i)
IndexError: Index out of range

--- testing ezerr.alloc() ---
allocated (and deallocated) 1 bytes
>> could not allocate -1 bytes
Traceback (most recent call last):
  File "test_err.py", line 23, in <module>
    ezerr.alloc(n)
MemoryError: Failed malloc()

Dot product example (from Bill Spotz's article)

The last example given in Bill Spotz's artice is for a dot product function. Here is a fleshed-out version.

The C source (dot.c and dot.h)

Here is the dot.h file:

   1 double dot(int len, double* vec1, double* vec2);

Here is the dot.c file:

   1 #include <stdio.h>
   2 #include "dot.h"
   3 
   4 double dot(int len, double* vec1, double* vec2)
   5 {
   6     int i;
   7     double d;
   8 
   9     d = 0;
  10     for(i=0;i<len;i++)
  11         d += vec1[i]*vec2[i];
  12 
  13     return d;
  14 }

The interface files (dot.i and numpy.i)

Here is the complete dot.i file:

   1 %module dot
   2 
   3 %{
   4     #define SWIG_FILE_WITH_INIT
   5     #include "dot.h"
   6 %}
   7 
   8 %include "numpy.i"
   9 
  10 %init %{
  11     import_array();
  12 %}
  13 
  14 %apply (int DIM1, double* IN_ARRAY1) {(int len1, double* vec1), (int len2, double* vec2)}
  15 
  16 
  17 %include "dot.h"
  18 %rename (dot) my_dot;
  19 
  20 %inline %{
  21     double my_dot(int len1, double* vec1, int len2, double* vec2) {
  22     if (len1 != len2) {
  23         PyErr_Format(PyExc_ValueError, "Arrays of lengths (%d,%d) given", len1, len2);
  24         return 0.0;
  25     }
  26     return dot(len1, vec1, vec2);
  27 }
  28 %}

Setup file (setup.py)

This is the setup.py file:

   1 #! /usr/bin/env python
   2 
   3 # System imports
   4 from distutils.core import *
   5 from distutils      import sysconfig
   6 
   7 # Third-party modules - we depend on numpy for everything
   8 import numpy
   9 
  10 # Obtain the numpy include directory.  This logic works across numpy versions.
  11 try:
  12     numpy_include = numpy.get_include()
  13 except AttributeError:
  14     numpy_include = numpy.get_numpy_include()
  15 
  16 # dot extension module
  17 _dot = Extension("_dot",
  18                    ["dot.i","dot.c"],
  19                    include_dirs = [numpy_include],
  20                    )
  21 
  22 # dot setup
  23 setup(  name        = "Dot product",
  24         description = "Function that performs a dot product (numpy.i: a SWIG Interface File for NumPy)",
  25         author      = "Egor Zindy (based on the setup.py file available in the numpy tree)",
  26         version     = "1.0",
  27         ext_modules = [_dot]
  28         )

Compiling the module

The setup command-line is:

python setup.py build

or

python setup.py build --compiler=mingw32

depending on your environment.

Testing

If everything goes according to plan, there should be a _dot.pyd file available in the build\lib.XXX directory. You will need to copy the file in the directory where the dot.py file is (generated by swig), in which case, the following will work (in python):

>>> import dot
>>> dot.dot([1,2,3],[1,2,3])
14.0

Conclusion

That's all folks (for now)! As usual, comments welcome!

Regards, Egor


CategoryCookbook

Cookbook/SWIG NumPy examples (last edited 2009-08-11 06:33:15 by EgorZindy)