64k tinypy – now with list comprehensions and fancy arguments
Well, I got two of my favorite python features added in – list comprehensions [x*x for x in range(1,5)] and fancy arguments test(1,2,a=3,*c,**d).
Adding list comprehensions was painless, took only a few minutes. The arguments change was rather difficult because I had to add more rules to the parser, change how the bytecode was outputted, and rework all the internal calling stuff in the VM. Now every function takes a single argument of type ‘params’ which can contain all the details of a call.
I’ve also added in better error handling. Errors print out a backtrace of the function calls and lines of code. This is making debugging everything much easier.
I’m also finding that the more features I add, well, the slower things get. I’m also coming up against my 64k limit, so I’ve moved my tests out of the core code into a separate tests.py file to ensure that I don’t “trim down the tests” to give myself more space 🙂 The tests I’ve made have been invaluable in making it possible to do big changes fairly quickly.
Next on the agenda – either bootstrapping, or making a game with it 🙂
For the brave: svn://www.imitationpickles.org/tinypy/trunk ; python tests.py ; ./run_julia_o3 (not very fast)
January 16th, 2008 at 1:00 pm
I really like what you’re doing here. The build system is a bit overtly dirty (but because you’re one guy and focusing on the vm, that’s nothing to scoff at).
I’m a game developer, and things like this are going to be motivational when writing a CPython implementation for Xbox360 or PS3.
January 16th, 2008 at 1:02 pm
Heh, yah, it’s all in the “just hacking it together” stage, of course 🙂
The main use-case I have for what I’m doing is games, so stuff like that is part of what I’m looking at doing, maybe.
Thanks!
Phil
January 16th, 2008 at 1:12 pm
Question, though. Why did you just not use the py_compile package to convert your python scripts into bytecode?
January 16th, 2008 at 2:19 pm
Chuck – I’m hoping to be able to bootstrap this project so that in a game you could compile and run scripts on the fly. Maybe 🙂
January 16th, 2008 at 2:44 pm
Have you seen pyvm?
January 16th, 2008 at 2:54 pm
No I hadn’t, that’s really cool! I’ll be looking into that stuff a bit more. I wonder if I could run Galcon with it …
January 16th, 2008 at 3:40 pm
Hmn .. just tried to compile pyvm and got this:
../lwc/objdir/lwc seg-malloc.c+ > cdir/seg-malloc.c
/usr/include/string.h:9 [Token 7283]: invalid declaration separator : `__asm__’
Somewhere in ### __errnum , char * __buf , size_t __buflen ) __asm__ ( “__xpg_strerror_r” ) __attribute__ ( ( __nothrow__ ) ) __attribute__ ###
make[1]: *** [cdir/seg-malloc.c] Error 1
Hmn ..
January 16th, 2008 at 4:04 pm
Congratulations. This is a *really* interesting project. Good luck and keep hacking. 🙂
Fuzzyman
January 16th, 2008 at 4:23 pm
Very cool!
Does this version have very fast image manipulation? Like with integers and stuff?
Maybe you could fast path the basic function call somehow? like f(x) ?
January 16th, 2008 at 4:45 pm
Rene – no such luck at this point, but if you roll back to my earlier versions, the python->C code does some interesting stuff. Now I’m targetting more just doing a good VM setup. However, it would be possible to write a JIT function that took a function and generated fast asm functions for doing image manipulation.
January 16th, 2008 at 5:05 pm
Hi Phil,
you just discovered rule number one of python reimplementations: everything is fast as long as you don’t have all the features :-). It doesn’t completely apply to you, since your goal is not to reach complete CPython compatibility. But for several Python reimplementation projects it’s some sort of recurring pattern. When the new implementation is started you get extreme speed claims, which are then silently retracted as more features are implemented which slows everything else down.
Cheers,
Carl Friedrich
January 16th, 2008 at 5:10 pm
Carl – Yeah, I’m certainly learning (first hand) what each bit of syntax costs 🙂 At present I’m just trying to keep the speed “reasonable”. I think the only unique thing this project might bring to the table is the small code footprint of 64k of readable code. For me that’s important, because working on much larger things just isn’t all that much fun!