64k version of python is 4234x faster* than c-python!
To satisfy some sick curiosity of mine, I decided to write a miniature version of python in less than 64k of code. The code includes a tokenizer (which works rather well) a parser (which sort of works ) a translator (which does the job). All of those bits are written in python. The last bit is the C library I wrote to handle all the grit. That part isn’t so good.
To sum up, my implementation does very little, and what it does, it does very slowly. Most of the speed is spent up in memory allocation (my best guess). I wish I had a julia screenshot to show you, but I don’t think I have the patience, nor the RAM, to provide you with the pleasure.
If you wish to witness the horrors, feel free to check out svn://www.imitationpickles.org/tinypy/trunk … Please don’t look too closely. I think if I really want to work on this sort of stuff my time would be better spent either:
- Learning the python C API or ctypes so I can do stuff without SWIG. (I’ve liked SWIG, but all the magic makes me nervous.)
- Contribute to shed-skin, and add some features I’d like to it
- Contribute to PyPy-Rpython, and add some features I’d like to it
- Wait for py3k, where hopefully someone will write some magic code that uses the swell function annotation features to make fast things happen.
Of my options, learning the C-Python API (or maybe ctypes) is probably the easiest answer – I’d get all the fun of python, and I’d get my C speed. Between working on shed-skin and PyPy — my feeling is that I really wish shed-skin generated C code or that PyPy-Rpython wasn’t so huge and scary. All that said, I’ll probably just wait for py3k and hope 😉
*today is opposite day!
December 23rd, 2007 at 1:34 am
ctypes is actually really easy, the bad thing about ctypes is that you’re going to a C-libraries ABI, not to its API, and this can be brittle. Using the C-Python API is much better, although it takes longer to use. You should also take a look at Pyrex, which lets you write very python-like code which gets compiled down to C – which might be what you want.
Also, take a look at: stackless python, psyco, libs like numpy/scipy.
And make sure to profile your apps before trying to fix a performance problem! Sometimes performance problems can be easily fixed without having to rewrite things in C.
I don’t think py3k is going to have massive performance improvements – it’s more of a backwards-incompatible cleaning-things-up release with a few extra funky features, nothing on the scale of the rewrites that ruby 2 and perl 6 are.
You could also try looking at other languages which might be faster but aren’t as painful as low-level C, such as: OCaml, haskell, one of the many common-lisp implementations, hell maybe even D/Java/C# (though I consider these lower-level than the other languages I’ve named). I guess Python has an awesome amount of libraries though.