Here it is: http://tinypy.googlecode.com/files/tinypy-1.1.tar.gz

tinypy is a minimalist implementation of python in 64k of code.  If you want to get involved, be sure to join the mailing list!

Thanks loads to everyone (allefant, Seth, Krzy, Rockins, Dean) who helped get this release together 🙂

The highlights are:

– works in VS now
– math module
– better setup.py
– a million bug fixes

Enjoy!
-Phil

Well, I developed pgu several years ago .. and I’ve sort of moved onto other things.  And now I really feel like I’m not doing a great job maintaining pgu anymore.  I get occasional contributions which I don’t handle and I don’t really have any direction I want to take pgu, so I think it’s time for me to step down as maintainer of that project.

I’ve got a project page on pygame.org that has a pretty good description of what it does.  I’ve also got a project page on sourceforge that has the mailing list.  Really, it would probably do better all hosted with google code and google groups.   (tinypy has been doing *great* with those!)

Anyway, it’s been fun, but it’s time for me to let someone else take the torch!  Good luck to whoever that ends up being!  I guess I’m sort of taking applications here, so .. tell me why you’d be the best maintainer for pgu 🙂

A notable draw-back to sitting in front of a screen all day is that I get bloated after a while.  Thus, I’ve built the corndog diet for when I wish to drop a few pounds.

• Breakfast: yogurt + 1 cup grape juice
• Lunch: one corndog + 1 cup water
• Afternoon: 20 minute workout *
• Dinner: something modest + 1 cup milk

Drops around 1/2 lb per day if I actually do it.  I usually don’t exactly do that, so I’m probably at a 1/4 lb rate right now.  At any rate, 10 down, 5 to go.  Gotta keep my body at least moderately comparable to my blog pics.

For fun, see the Corndog Patrol Flag.

* Usually about 150 jumping jacks, 7 pull-ups, 45 push-ups, and 120 sit-ups.  At a separate time Nan and I walk Cuzco the goat around the lake.

If I were going to OSCON, I’d certainly go to this talk. But I’m not planning on it, so …

… So I’ve got that tinypy project sort of taking off. It’s got a SOC student (thanks google & PSF) and a bit of interest. So far most of the open source projects I’ve done haven’t generated more than a couple patches — in their lifetime. With that in mind, I’m not entirely sure what is a good way to accept patches and run a project, since I don’t have loads of experience in it. I’m a “solo” dev in my day job as well. I want to do a decent job of managing the tinypy community without having to work too hard.

Do I take whatever people give me and patch it as-is and just figure, well, someone else will fix it?
Obviously not. Especially not for tinypy where “good” and “concise code” is of such large value to the project. And with the SOC coming up, “secure code” is also very important, as well as testing.

Do I be really strict and keep rejecting a patch until it’s “perfect”?
Maybe? I remember submitting a patch to one project ages ago, and it got rejected without much good explanation. I’ve also submitted others successfully. Not quite sure what the balance is.

Do I take the patch and apply it and then fix things up myself?
Maybe? If it is easy to fix … but then again, why should I have to do all the work? I don’t really have that much spare time (all evidence to the contrary.)

How do I grow the project (in general) without it turning into a monster?
Stick to my guns on the 64k code limit*? Maybe not, since things like VC support, security, etc, are going to take up *some* bytes. Not to mention, tinypy could use a few batteries … At the same time, I’ve gotta draw the line somewhere.

Obviously some common sense is needed here, but sometimes I run a bit short on that. Advice / links to good articles would be swell at this point. I could google this sort of stuff, but I think having the context of responses from a community I know would be much more valuable here.

-Phil

* on that point, I think having the 1.0 source always in the “featured downloads” might be a good idea. Even if the project gets bigger, people can still check out the “classic version.”

For the sake of this post, I’m going to pretend to know what metaprogramming* is.  Yeah, so tinypy** totally has that.  At least, since the parser and compiler in tinypy is written in tinypy, you are able to modify those modules on-the-fly and add new features into the tinypy language.  (Not that you’d want to, but certain other languages get so uppity about being able to do that, I figured I’d plug for tinypy here.)

For example, (at present) tinypy doesn’t have support for decorators.  I’ve always liked decorators, so I made this code (a zip of main.py, deco.py***, and test.py) so that if you have a main.py:

import deco
import test

When the deco module is loaded, it cleanly**** adds decorator support into the tokenize, parse, and encode modules of tinypy.  Then when the test module is loaded, it is able to use decorator syntax.  Yay!  This mostly thanks to the top down operator precedence implementation in tinypy.

So now, if say, you have some crazy idea for how the $ operator should be used in bigpy, you can go ahead and use metaprogramming to add it into tinypy and show all your friends how awful your new syntax looks and have a working proof-of-concept!  Yay!

* feel free to enlighten me
** it’s got a mailing list now, join in on all the fun!!
*** only 611 bytes :)  They were pretty simple to implement, since they really just mean: “given ‘@a \n def b …’ do ‘def b … \n b=a(b)'”
**** Since all the language features are stored in dictionaries, it’s “pretty easy” to add new symbols / operators.  (Or remove features, or whatever!)

$ history|awk ‘{a[$2]++} END{for(i in a){printf “%5d\t%s\n”,a[i],i}}’|sort -rn|head
269   ls
253   cd
182   kate
50   cp
48   svn
43   mkdir
23   mv
21   grep
16   gqview
13   python

1. The only bit I remember from that conversation is all the stuff about Canadian global domination.  Let me tell you, if Mike Fletcher has his way, we’re doomed!  (Maybe we’re doomed either way, but be sure your passport is ready when we have to head north!)
2. I remember peaking into the pycon-dev room at one point and Doug was giving a lecture on menu structures in websites.  He said the pycon navigation was stored completely independently of all the modules used by pycon.  I think that’s great.  I’ve been thinking about it ever since.
3. I do most all my dev at home on my linux system.  My laptop last year was loaded with XP and I found it painful to do work on at pycon.  So this year, the day before pycon, I loaded it up with Ubuntu.  Which was a mistake.  It *sort* of worked, in a kind of the network didn’t exactly always work and the video driver didn’t always work .. and then it got worse .. but I won’t name names 😉
4. Which brings me to two happier points, which are points 5 and 6 in this list.
5. Sean was quite swell and set me up a personal hard-line during the sprints when I needed to get some work done :)  Yay!
6. I was told about PyPE, which is a nifty python editor for windows :)  I’m having to do some win32 dev lately (see previous post), so having that on hand is great!  I like it.  I was able to do about 5 hours of dev on it this evening, and I think the main complaint I had was I was using a laptop keyboard.  If you like kate, this is the editor for you.

So I’m getting the feeling I should host tinypy somewhere greater than just in my blog. What a pity, the blog suited me so well. I dunno, I guess some people have unreasonable demands. (If I were feeling clever, I’d link all the words in that last sentence to other people’s blogs who’ve bugged me about this. But I don’t think there are actually that many.)

Anyway, my options are as follows:

• use the project management software I’ve written for my health-care company. Pros: I wrote it, so it must be better than anything else. Cons: I’ll have to maintain it when I find out otherwise.
• use google code. Pros: People know how to use it, and it’s pretty clean looking and they host it for me. Cons: I don’t get to host it. But I guess I could set up the first few pages of the tinypy site on my own server to make myself feel good.
• use trac or something. Pros: I’d get to host my whole project myself. Cons: I’d have to host the whole project myself and learn some new software and maybe set up some stuff.

So .. all my aimless thoughts aside, since this isn’t really about me. (If it were, I’d just blog about it every now and again and leave it at that.) It’s about you, since there is interest in the project it needs to be easier for people to poke at. What setup do you think* would be the best?

* it probably doesn’t matter what you think, because after writing this post, I think I’ve already made up my mind. Writing down stuff like this helps me think things through. But since you’ve bothered to read this far, you might as well put in your 2c.

Its been a long two months getting this project to the 1.0 state. Here it is:

tinypy1.zip | tinypy.tgz | svn://www.imitationpickles.org/tinypy/tags/1.0.0

$ python build.py
$ ./tinypy-sdl julia.py *
$ ./tinypy your-program-goes-here.py

The win32.zip includes the tinypy.exe and tinypy-sdl.exe binaries, so you can skip the building process.

Features include the ability to parse and compile to bytecode a pretty decent subset of python code. Its got a pretty simple API, incremental garbage collection, and a handful of builtin functions. The vm supports strings, dicts, lists, numbers, functions, methods, and custom data types. tinypy even has exotic features like list comprehensions, variable and named function arguments, inheritance, and exceptions with tracebacks. All this in just 64k** of code!!

I also had some fun this past weekend testing tinypy “in the field” making an OpenGL tetris knock-off. My apologies to Alexey Pajitnov for my misguided contribution of tinypy to the open source stack. win32.zip | svn://www.imitationpickles.org/ld105/trunk

I hope you enjoy checking out tinypy – I’ve had a pretty wild ride building it.

*julia.py included is a low-res “realtime” jula fractal demo. I felt like rendering a mandelbrot for this article.

**See README.txt for my definition of 64k. It’s close enough.

Updates – I’ve got it all fitting in 64k*. It’s amazing how many functions that don’t do anything you can come across if you look around long enough. Not to mention how many little things you can trim out that don’t actually do anything. I have no idea where all this cruft comes from, but having a nice suite of tests sure is helpful for re-working stuff. I also cut out a bunch of stupid features nobody would ever use**. I was able to reduce the number of native types from 9 to 7.

One of the challenges I faced was trying to fix up the incremental garbage collection. My initial implementation was rather inefficient and caused some odd problems with how I wanted to code things. I was using a dict to store all the “white” items, which caused loads of dict hash lookups.

So in my mind I crafted a grand vision on how to accomplish this goal. I would adjust all objects (sans numbers) to contain a pointer to some data which would have some header data for the GC to do some bookkeeping in. Great! However, when I implemented this, I found that a number of problems presented themselves: I had to perform a malloc for each and every string that I used, which killed performance, actually making things 2-3x slower. I also noticed that the weird struct I defined was maybe a bit less standards compliant. This attempt was a wash.

So I re-crafted my grand vision. This time I would do the same thing. Brilliant aren’t I? Anyway, the results were basically the same. Who’d’ve thunk? It was slower again, this time I was quite confused by it, since I had worked around some of the string issues. I also found that the API for creating new strings wasn’t quite as “clean” as my original simple one. This caused some issues in the exception handling mechanism. I had to toss this try as well.

At this point, having re-mangled the code twice and having poor results, I suspected something else might be wrong. My brain was turning into mush. Each time I had completely edited my “tp.h” with all my struct changes in one go. I decided to make a final attempt at reworking tinypy, this time *one* data type at a time. After each data type I added I was able to see if my changes caused any performance issues. I found that my function data type was the culprit. My hashing function (borrowed from lua) wasn’t getting enough entropy and was generating massive collision cases! A few tweaks later, this was resolved. I was able to also craft the string interface to be backwards compatible with the original string interface while also working with the new garbage collection. This “step by step” approach got me to my goal. All said and done, with a bit more tweaking, I was able to *double* the speed of tinypy 🙂

Lesson learned – even if it’s only 64k, it’s better to do changes step by step instead of in one big go. valgrind and callgrind are your friends. (Although I found that tinypy doesn’t entirely agree with callgrind … ideas anyone?)

To wrap up this excessively long post about me trying to get code to work — this weekend I’m hosting a Ludum Dare warmup compo. I’m going to give tinypy a run in the “real world”. Here’s to hoping! Next week I plan on releasing the 1.0 version of tinypy.

I’m also thinking about renaming some of my files. And although pylang, dumbparse, and dump2vm have a certain rustic charm, I wonder if I’d do better with names like goat, gorilla, and sausage. Or maybe more descriptive names like tokenator, parsalizer, and bytecodatron.

svn://www.imitationpickles.org/tinypy/trunk for the brave. If you want a zip or an exe, check back in a week. I’ll have all those and more (a game!) Note that I’ve split the SDL dependency out of the main tinypy code. tinypy-sdl.c lets you run my julia.py example. The bootstrapping process also has a final step of compiling with -O3, which I think might not work for everyone. It gives pretty good speed gains on my system, so if it works for you, great!

*python mk64k.py will do a bit of search-n-replace to cut it down to size. I’ve resisted doing anything really ghastly, the code is still indented and readable. See README.txt for more disgusting details on how I cheat to pretend this is 64k.

**Okay, I’ve used some of those features. But hey, this is a 64k implementation, I’ve got to trim the fat.