Attacking PyPy
Hey, more fractal fun tonight as I attack PyPy! I must admit, this effort was more challenging – PyPy is pretty big and scary, but I found the folks in #pypy to be quite helpful. exarkun helped me quite a bit in working out the details. (Apparently PyPy doesn’t use ctypes anymore, it uses rffi .. which is sparsely documented.) Anyway with exarkun’s help I got a crude SDL wrapper put together!
Initially I just implemented my julia demo again, but I figured it would make for a boring blog entry to have the same pictures two days in a row. As it turns out, the julia was about 30% faster in PyPy than in shed-skin. Both shed-skin and PyPy appear to have similar limitations. shed-skin’s error messages are less cryptic than PyPy’s. Both FFI’s were somewhat challenging to work with, though I think I prefer shed-skin’s.
If you want to give it a whirl, download PyPy and my junk. The NOTES.txt file lists the command I used to build the executable. Sorry if the mouse interface is a bit lousy, but it was the best I could do in 2 minutes.
November 30th, 2007 at 1:22 am
Cool!
When you say “implemented in PyPy”, do you mean you implemented it in RPython?
November 30th, 2007 at 7:53 am
Nice posts. To make the list (ShedSkin, PyPy) complete, you can also use Pyrex (http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/) to build amazingly fast extension modules. Some time ago I also played around with some Mandelbrot examples. Here is the Pyrex code for the inner loop:
def performIterations (double p_a, double p_b, long p_max):
cdef double r, i, x, y, d
r = i = x = y = d = 0.0
cdef long n
n = 0
while d < 4.0 and n < p_max:
x = r; y = i;
r = x * x – y * y + p_a
i = 2 * x * y + p_b
d = r * r + i * i
n = n + 1
return n
The advantage is, that Pyrex compiles the above code to C-Code, which can then be compiled to a .pyd. On Windows for example, the size of this .pyd is only 8 kB. No other library is needed in contrast to ShedSkin.
November 30th, 2007 at 8:57 am
Yay, that is extremely cool! We should steal that SDL wrapper :-). Can you test ShedSkin’s FFI on top of CPython without translating the program first?
November 30th, 2007 at 10:09 am
Markus:
Actually – you’ve got to consider all the SDL binding stuff too .. I did this ages ago, but thanks for the reminder – I think I’ll check pyrex out again and see how it suits me. Maybe I’ll do an article about that next 🙂
Carl:
Yeah, by PyPy I meant Rpython. Feel free to do whatever you like with the contents of that .zip file. public domain as far as I’m concerned.
You couldn’t use ShedSkin’s FFI on top of CPython .. ShedSkin’s FFI is literally C++ with a few helper classes + you have to write a bit of dummy code in python demonstrating how the functions will be used. You don’t actually have to write any ctypes / rffi -esque stuff. (Check out my example lib/_gfx.* from my previous post to see what it looks like.)
December 1st, 2007 at 5:18 am
hi phil,
any idea where the 30% difference stems from? possibly something minor I might fix?
mark dufour.
December 1st, 2007 at 11:28 am
Nothing comes to mind other than C vs C++ overhead.
December 1st, 2007 at 4:18 pm
There’s a major difference: ShedSkin is a general Python to C++ translator. PyPy’s translator translated your *R*Python to C. ShedSkin has to do a lot more work to make sure that it supports all the dynamism of Python, whereas the C backend just needs to support the very non-dynamic RPython.
December 1st, 2007 at 4:54 pm
Chris –
shed-skin only translates programs that are written in it’s version of Rpython. Both PyPy’s Rpython and shed-skin’s Rpython support roughly the same sub-set python.
The notable difference between the code outputted is:
shed-skin produces human readable C++ code (with C++ classes, namespaces, etc)
PyPy produces (fairly) unreadable C code.
Phil