64k tinypy – garbage collection is tough
For giggles I tried to write a garbage collector to replace libgc in tinypy. I tried doing a tri-color incremental collector. I couldn’t get it to work, so I ended up switching it to be more of a tri-color mark and sweep collector.
The result of my mark and sweep collector was a 40% reduction in speed. I’m guessing ol’ libgc was designed with a bit more cleverness than mine 😉 Anyway, for now I’ve moved my “tgc” development into a branch of tinypy svn://www.imitationpickles.org/tinypy/branches/tgc if you want to see it in action.
Stuff to read: Memory Management Reference and libgc.
February 5th, 2008 at 5:24 pm
Hi,
that’s pretty cool 🙂 Only 40% reduction in speed is pretty good for replacing a big mature library like libgc. I’m sure you can eventually make it faster, and getting rid of the libgc dependency is really good.
A couple of memory debuggering and profiling aids:
– valgrind http://valgrind.org/
– dmalloc http://dmalloc.com/
A memory pool of sorts should help speed things up. Figure out the sizes that get malloc’d often and create a pool for that stuff. Like a small cache of objects you can draw from.
Also reducing many small mallocs into one is another optimization. So instead of asking to create one object change your code to create many objects.
So if you do something like range(1000) it can do one malloc, and not 1000. http://rene.f0o.com/mywiki/BatchingApis
I’m sure profiling (probably with valgrind) will help you speed it up too.
cu,