So, my mind has been poking away at the idea of making a “tinypy2” that would be a bit less like python, so probably even more confusing for end users. Being only one sentence into my proposal here, and it already sounds bad. Well, here goes:

-1 All objects would be assigned and passed by value instead of by reference
=0 There would be a dictionary of global variables
+1 The VM state could be saved to readable JSON
+1 The VM state could be loaded from JSON
=0 The VM would be written in C++
+1 The VM would have super-easy interoperability with C++ code
-1 The VM would be on the slow side

Tallying up the pros and cons, we get to a +1 for this idea. I don’t suppose there’s much way for anyone to comment on if this sounds like a good idea or not, but thos are some of the thoughts going around in my head right now. It seems like it would be an interesting rudimentary scripting language. The load / save of VM state and C++ interoperability are the key points.

#badideas ??

-Phil

Hey,

I’ve been spending some time checking out lua, because I’m thinking about using it for scripting in games.  I’m looking at it instead of tinypy because tinypy is slow and really needs some more TLC to be totally useful to me.  However, after looking at lua for a while, I’m finding the things that make it feel awkward to me.

1. List indexes starting at 1.
2. For loops use [a,b] intervals
3. No += operator.
4. do / end keywords instead of { }
5. List and Dict types combined into one

At the same time, I’ve found that lua is totally awesome.  I mean, wow.  Really slick stuff!  Here’s 5 awesome things:

1. Small codebase
2. Faster than other dynamic languages
3. Easy API integration
4. Coroutines <- these are neat!
5. State is serializable

So here are my thoughts:

• The first 4 awkward things are superficial, I could probably hack those into shape in a day or two.  The 5th item (List + Dict) as one item is probably not something I can fix, but I could probably live with it.
• If I changed the language like that, and provided it for users to create mods for my games I would not be able to give them a link to the lua documentation, since it would not be accurate anymore.

Not quite sure what to do here.  Lua seems great, though a few language design decisions seem awkward to me.  I could change the language easily enough, but then I would be breaking all the lua documentation for my users.

Anyone got some language-design thoughts on this one?

-Phil

So .. Yeah.  Check it out.  Realtime multi-player game .. in flash!

So on a more technical note .. The game involves quite a number of technologies!

• AS3 – of course – for the Flash client itself
• C++ – for the server
• PHP – for the web API and rankings system
• python – for the bots

If I did the project over, I’d probably do the server in python as well.  But all in all the project went pretty well.  After things were done, it only took me a morning to write up the python client.  There’s a very good chance I’ll release that code in a few weeks and let people try making bots for the game.

Anyway – have fun checking it out.  I’m a bit wiped out from wrapping all this up, so I’ll try and post some more interesting details later!

-Phil

So .. who thinks flash + Galcon = win?  I don’t know yet, but I’m giving it a try.  I’m not actually developing the flash version.  An awesome friend of mine has been doing the time on this one.  Though I’ll probably be doing the LAMP end of the deal.  At present the game is non-networked, but we’re been considering the possibilities.  A TCP/IP edition of the game could be made, but I wonder if that would end in tears.  TCP/IP isn’t ideal for gamedev IMO.  Flash doesn’t support UDP as-of-yet.

This isn’t the first time Galcon has reared it’s head from within the browser window.  (See my earlier blog posts …)

I haven’t decided exactly what I’m going to do with this once it’s completed.  Probably a few things.  Throwing it on facebook is an obvious one.  Throwing it on galcon.com is another one.  Maybe mixing it with something like The Maze of Madness is another one.  Or maybe something else!

One idea that I have that would be really cool would be if the tinypy vm were ported to ActionScript using Alchemy* .. and then somehow people would be able to script Galcon on the web and share it with their friends..

*I don’t know if you caught that .. but I think my weekend just got booked 🙂

Hey, so I just put up a new game on my website called “The Maze of MADNESS.”  Check it out.

The interesting thing about it is that I didn’t create any of the game.  I just created a web based interface for people to create this adventure game on.  I made it for a 48 hour contest this past weekend, and the other contest people have already created 45 rooms, 40 new items, and 152 actions.

It should be interesting to see how long the game lasts before it caves under the weight of itself.  I’m quite interested in trying out user-content games.  So if this even marginally works out, there is a good chance I’ll be doing more in the future.

On another tech note, I used AS3 for the pixel art editor in the game.  I decided on this instead of haxe, because it seemed there was a lot more random bits of information on the internet about AS3.  I managed to get it to work without using a .mxml file.  The tutorial I followed was this.  I also got lots of random snippets from the internet.

-Phil

For the sake of this post, I’m going to pretend to know what metaprogramming* is.  Yeah, so tinypy** totally has that.  At least, since the parser and compiler in tinypy is written in tinypy, you are able to modify those modules on-the-fly and add new features into the tinypy language.  (Not that you’d want to, but certain other languages get so uppity about being able to do that, I figured I’d plug for tinypy here.)

For example, (at present) tinypy doesn’t have support for decorators.  I’ve always liked decorators, so I made this code (a zip of main.py, deco.py***, and test.py) so that if you have a main.py:

import deco
import test

When the deco module is loaded, it cleanly**** adds decorator support into the tokenize, parse, and encode modules of tinypy.  Then when the test module is loaded, it is able to use decorator syntax.  Yay!  This mostly thanks to the top down operator precedence implementation in tinypy.

So now, if say, you have some crazy idea for how the $ operator should be used in bigpy, you can go ahead and use metaprogramming to add it into tinypy and show all your friends how awful your new syntax looks and have a working proof-of-concept!  Yay!

* feel free to enlighten me
** it’s got a mailing list now, join in on all the fun!!
*** only 611 bytes :)  They were pretty simple to implement, since they really just mean: “given ‘@a \n def b …’ do ‘def b … \n b=a(b)'”
**** Since all the language features are stored in dictionaries, it’s “pretty easy” to add new symbols / operators.  (Or remove features, or whatever!)

Its been a long two months getting this project to the 1.0 state. Here it is:

tinypy1.zip | tinypy.tgz | svn://www.imitationpickles.org/tinypy/tags/1.0.0

$ python build.py
$ ./tinypy-sdl julia.py *
$ ./tinypy your-program-goes-here.py

The win32.zip includes the tinypy.exe and tinypy-sdl.exe binaries, so you can skip the building process.

Features include the ability to parse and compile to bytecode a pretty decent subset of python code. Its got a pretty simple API, incremental garbage collection, and a handful of builtin functions. The vm supports strings, dicts, lists, numbers, functions, methods, and custom data types. tinypy even has exotic features like list comprehensions, variable and named function arguments, inheritance, and exceptions with tracebacks. All this in just 64k** of code!!

I also had some fun this past weekend testing tinypy “in the field” making an OpenGL tetris knock-off. My apologies to Alexey Pajitnov for my misguided contribution of tinypy to the open source stack. win32.zip | svn://www.imitationpickles.org/ld105/trunk

I hope you enjoy checking out tinypy – I’ve had a pretty wild ride building it.

*julia.py included is a low-res “realtime” jula fractal demo. I felt like rendering a mandelbrot for this article.

**See README.txt for my definition of 64k. It’s close enough.

Updates – I’ve got it all fitting in 64k*. It’s amazing how many functions that don’t do anything you can come across if you look around long enough. Not to mention how many little things you can trim out that don’t actually do anything. I have no idea where all this cruft comes from, but having a nice suite of tests sure is helpful for re-working stuff. I also cut out a bunch of stupid features nobody would ever use**. I was able to reduce the number of native types from 9 to 7.

One of the challenges I faced was trying to fix up the incremental garbage collection. My initial implementation was rather inefficient and caused some odd problems with how I wanted to code things. I was using a dict to store all the “white” items, which caused loads of dict hash lookups.

So in my mind I crafted a grand vision on how to accomplish this goal. I would adjust all objects (sans numbers) to contain a pointer to some data which would have some header data for the GC to do some bookkeeping in. Great! However, when I implemented this, I found that a number of problems presented themselves: I had to perform a malloc for each and every string that I used, which killed performance, actually making things 2-3x slower. I also noticed that the weird struct I defined was maybe a bit less standards compliant. This attempt was a wash.

So I re-crafted my grand vision. This time I would do the same thing. Brilliant aren’t I? Anyway, the results were basically the same. Who’d’ve thunk? It was slower again, this time I was quite confused by it, since I had worked around some of the string issues. I also found that the API for creating new strings wasn’t quite as “clean” as my original simple one. This caused some issues in the exception handling mechanism. I had to toss this try as well.

At this point, having re-mangled the code twice and having poor results, I suspected something else might be wrong. My brain was turning into mush. Each time I had completely edited my “tp.h” with all my struct changes in one go. I decided to make a final attempt at reworking tinypy, this time *one* data type at a time. After each data type I added I was able to see if my changes caused any performance issues. I found that my function data type was the culprit. My hashing function (borrowed from lua) wasn’t getting enough entropy and was generating massive collision cases! A few tweaks later, this was resolved. I was able to also craft the string interface to be backwards compatible with the original string interface while also working with the new garbage collection. This “step by step” approach got me to my goal. All said and done, with a bit more tweaking, I was able to *double* the speed of tinypy 🙂

Lesson learned – even if it’s only 64k, it’s better to do changes step by step instead of in one big go. valgrind and callgrind are your friends. (Although I found that tinypy doesn’t entirely agree with callgrind … ideas anyone?)

To wrap up this excessively long post about me trying to get code to work — this weekend I’m hosting a Ludum Dare warmup compo. I’m going to give tinypy a run in the “real world”. Here’s to hoping! Next week I plan on releasing the 1.0 version of tinypy.

I’m also thinking about renaming some of my files. And although pylang, dumbparse, and dump2vm have a certain rustic charm, I wonder if I’d do better with names like goat, gorilla, and sausage. Or maybe more descriptive names like tokenator, parsalizer, and bytecodatron.

svn://www.imitationpickles.org/tinypy/trunk for the brave. If you want a zip or an exe, check back in a week. I’ll have all those and more (a game!) Note that I’ve split the SDL dependency out of the main tinypy code. tinypy-sdl.c lets you run my julia.py example. The bootstrapping process also has a final step of compiling with -O3, which I think might not work for everyone. It gives pretty good speed gains on my system, so if it works for you, great!

*python mk64k.py will do a bit of search-n-replace to cut it down to size. I’ve resisted doing anything really ghastly, the code is still indented and readable. See README.txt for more disgusting details on how I cheat to pretend this is 64k.

**Okay, I’ve used some of those features. But hey, this is a 64k implementation, I’ve got to trim the fat.

So.. hey, it’s done. Basically. tinypy is a 64k implementation of a subset of python. It can bootstrap itself into a single executable that can compile python files to bytecode and run them on a VM. Thanks to everyone who gave feedback thus far on this project. Double thanks to allefant who listened to me blab about it endlessly on irc for the last month 🙂

I found all the stuff people told me about for parsing was a huge help. This article http://javascript.crockford.com/tdop/tdop.html was what I ended basing it on. It’s almost like magic, but it makes for a really simple easy to follow parser. The VM is based on stuff I read about the lua VM.

So what’s next? I need to let it sit around for a week and then I’ll do a “release” I guess. I’ve gotta pick a license or something for it (probably MIT? I’m open for suggestions.) I’m also mulling over possible names. Maybe “tinypy” .. or “wedge” .. or “cupcake” .. or “garter”. Hmmn.

Anyway – I’m sure I’ll be tweaking it a bit over time, but I’m pretty happy to have it to this point now. I probably won’t do much with it until I try making a game with it. Right now it depends on libgc for garbage collection. If someone clever out there can implement a garbage collector for it that works in like 2-4k, that’d be better. My brain is pretty spent.

For the brave: svn://www.imitationpickles.org/tinypy/trunk or tinypy.zip. The following is only tested under linux, but I bet it would work in any bash environment. Maybe.

$ python boot.py

Will run the 3-phase testing + bootstrapping process. It will first use python to generate the .tpc files for the compiler. Second phase uses the VM to generate those same files. Third phase uses the bootstrapped tinypy executable to “re-bootstrap” tinypy to get the final version. The -nopos option strips out debug info from the .tpc files.

$ ./tinypy julia.py

Run the julia demo without dependence on *anything* but the tinypy executable.

$ ./tinypy your_own_code.py

Will do something! Probably print out a pretty traceback about how you tried to use a python feature / module that tinypy doesn’t support 🙂 “batteries not included”

Well, I got two of my favorite python features added in – list comprehensions [x*x for x in range(1,5)] and fancy arguments test(1,2,a=3,*c,**d).

Adding list comprehensions was painless, took only a few minutes. The arguments change was rather difficult because I had to add more rules to the parser, change how the bytecode was outputted, and rework all the internal calling stuff in the VM. Now every function takes a single argument of type ‘params’ which can contain all the details of a call.

I’ve also added in better error handling. Errors print out a backtrace of the function calls and lines of code. This is making debugging everything much easier.

I’m also finding that the more features I add, well, the slower things get. I’m also coming up against my 64k limit, so I’ve moved my tests out of the core code into a separate tests.py file to ensure that I don’t “trim down the tests” to give myself more space 🙂 The tests I’ve made have been invaluable in making it possible to do big changes fairly quickly.

Next on the agenda – either bootstrapping, or making a game with it 🙂

For the brave: svn://www.imitationpickles.org/tinypy/trunk ; python tests.py ; ./run_julia_o3 (not very fast)