I’ve been a bit quiet the last couple weeks .. after my spree of “python->c” converter posts I got pretty busy working on the Ludum Dare 10 website.  Ludum Dare is a 48 hour game development competition.

I used WordPress as the basis for this project and I think it was a pretty good choice.  Generally anything I needed to do, there was a hook in the API to let me do it.  I was able to keep all my compo code (theme voting, trophies, rating of entries,  tag clouds, screenshot grids, security tweaks) all within a module I wrote without having to modify any of the core WordPress files 🙂

There were around 150 signups on the site and 50 people completed entries.  The theme was “Chain Reaction” which won even in the first round of voting .. and still won after the 2nd and 3rd rounds.  I spent most of the competition sitting around on IRC doing nothing and occasionally working on my game.

The one notable glitch in the compo was the announcement of the theme ceremony.  I switched the voting to closed so we could see the results and it showed several themes with almost nobody voting for them.  Turns out those were two themes which I initially had in the final round but removed (because they didn’t make the cut).  The results of a few people voting was stored in the database so they still showed up.  Anyway, that was easily fixed.

At the end of the compo it took me an hour or so to get the entry rating system set up.  I also added the ability to leave a comment along with your rating to encourage more people to leave comments.  (They could use the WordPress blog commenting system, but that would take a few extra clicks and thinking.)  This way seems much nicer.

I think the funnest feature I added to the site was the Trophy feature.  This feature lets users award each-other 64×64 pixel trophies at any time.  It’s a nice community feature because it lets people recognize cool things that people did out-side of the pre-set rating categories in the contest.  (For example, one entrant recorded a tuba solo for their game.  Although the compo has a sound category, several people felt that the tuba playing merited special tuba trophies.)

I don’t know if I’ll be running the Ludum Dare competition again or not, but I’m sure I’ll be hosting some others now that I’ve got this swell compo system written 🙂

My last post explained my frustration trying to get pyrex to do *anything*. Since then, several people posted solutions and e-mailed me with more ideas (thanks allefant and Filip Wasilewski). My review was focused on the usability of pyrex as a python->C translator, since that’s what I was working on with both shed-skin and pypy/rpython. It didn’t hold up so well. This is because … pyrex ISN’T a python->C translator.

pyrex is its own special language that contains both python and C stuff. pyrex does not do type inference. What pyrex provides is a way to mix both python and python-like C code together in the same files. The trick to understanding pyrex is to know where the line between python and C is in pyrex.

Here are some of the gotchas I experienced. I wasn’t able to detail them in my previous post because I didn’t really “get it”. Now I think I do, so I’ll explain some stuff so you can try out pyrex without getting confused the way I did:

Gotcha #1: float() is a python function

cdef int a
cdef float b
a = 3
b = float(a)

Solution: use pyrex’s casting to avoid massive object-conversion speed penalties

b = <float>a

Gotcha #2: pyrex doesn’t do type inference

def set_video(int w, int h):
    s = Surface()
    s.surf = SDL_SetVideoMode(w,h,32,0)
    return s

pyrex: Cannot convert 'SDL_Surface *' to Python object

Solution: to access your C-members you must declare the variable’s type first

def set_video(int w, int h):
    cdef Surface s
    s = Surface()
    s.surf = SDL_SetVideoMode(w,h,32,0)
    return s

Gotcha #3: def makes python functions – which are slow

def set_pixel(self,int x, int y, int color)

Solution: To be able to access a fast version of a method in C and be able to access the same method in python you must define two versions of it

cdef _set_pixel(self,int x, int y, int color):
    pass # do this really fast

def set_pixel(self,int x, int y, int color):
    return self._set_pixel(x,y,color)

All that said, I think pyrex is really neat for writing extensions!  One of the best features it has is properties (which are like descriptors, only easier).  Just be careful if you use it to not mistake it for python.  Lastly, by using those tips I was able to get my julia example to be full speed.

In my last blog post, someone said I should check out pyrex. So I did.

Okay .. I must say, I found it extremely difficult to use and rather slow – 700 ms per frame (my pure C implementation is around 65 ms per frame). I had quite a bit of trouble figuring out the magic of getting anything to work, a few things seem pretty inconsistent. Thankfully I found someone (allefant – thanks) who helped me get some stuff working, but even he was rather baffled by some of the stuff we had to do to get it going.

Unfortunately, even after all our efforts, it was still slow. pyrex does so many magic things, that even with everything defined out to be C-ish, it still wraps ’em up so we don’t get the C-speed we want. I think the largest confusion is that pyrex marries both python and a magic brand of c-python code together in the same place – making it hard to know what “context” you are in at any point. To be kind, it does have some *really* cool features that Shed-Skin and PyPy/Rpython don’t have (such as properties). But the feeling of inconsistency and confusion I got while working with it just took the fun out of it for me.

Hey, more fractal fun tonight as I attack PyPy! I must admit, this effort was more challenging – PyPy is pretty big and scary, but I found the folks in #pypy to be quite helpful. exarkun helped me quite a bit in working out the details. (Apparently PyPy doesn’t use ctypes anymore, it uses rffi .. which is sparsely documented.) Anyway with exarkun’s help I got a crude SDL wrapper put together!

Initially I just implemented my julia demo again, but I figured it would make for a boring blog entry to have the same pictures two days in a row. As it turns out, the julia was about 30% faster in PyPy than in shed-skin. Both shed-skin and PyPy appear to have similar limitations. shed-skin’s error messages are less cryptic than PyPy’s. Both FFI’s were somewhat challenging to work with, though I think I prefer shed-skin’s.

If you want to give it a whirl, download PyPy and my junk. The NOTES.txt file lists the command I used to build the executable. Sorry if the mouse interface is a bit lousy, but it was the best I could do in 2 minutes.

There’s this swell python to c++ compiler called shedskin. It’s really interesting. I took a few hours tonight to make it do something:

Yep those are real-time julia fractals 🙂 If you dare – check it out (make sure python and SDL-devel are all setup):

wget http://www.imitationpickles.org/tmp/juliashed.zip ; unzip juliashed.zip ; cd juliashed ; python ss.py test.py ; make ; ./test

I’ve sent Mark an excessively long e-mail with my comments. I’ll spare you the trouble and just give the highlights:

• My C++ skills don’t exist
• python descriptors are probably the feature I would like to see most – so I could implement pygame.Rect style Rects!
• misc packaging issues (see his previous blog post)
• The README says it “does not scale very well beyond a few hundred lines” – so I guess I’ll wait a while before I try to make a game with it. (Even simple games I make are around 1500 lines.)

Anyway, it’s a cool project. I plan to keep an eye on it.

This just in – I made a python+pygame equivalent. The shed-skin version is ~100x faster! Also, the black areas in the middle of the shapes are caused by my access of out-of-bounds palette entries. I added the -b (bounds check) to the command line and fixed the bug. The bounds check gave me no noticeable speed penalty.

As far as code goes, I’ve usually been a do-it-yourself kind of guy. However, I’ve been so impressed with WordPress I’ve actually used it to implement four of my sites in the last couple months. WordPress is an easy to use, smart piece of blogging software. It really seems to have just the right set of features in its default installation to be useful for most cases out of the box.

However, there comes a time when what’s given just isn’t enough. Thankfully, its got an extensive collection of plugins! Everything from blog aggregation to voting to forms to photo galleries. Not all plugins are great, but usually if you check out a few you can find one that will do what you want.

That is .. until you want something different! I might be hosting the 10th Ludum Dare compo. For that I needed some special features for collecting ratings of contestants entries, showing screenshot grids, and giving trophies to entrants.

WordPress comes with a fairly nice themes and plugins system which made it possible to add all those features to my blog without modifying the core-code of WordPress. Frequently I would implement a feature, and after learning more about WP internals, I was able to refactor it to be simpler by using more of the existing WP framework.

It wasn’t all fun and games, though, the learning curve was a bit painful for some features. A couple WordPress features (like table deltas) seemed a bit too clever (not to mention broken) for their own good. Fortunately, I was able to get away with not using those features.

The other challenge I had was when I came across a bug in WordPress. I did my best to figure out the bug, but it appears to be some strange javascripty thing which was beyond me. So I’ve reported the bug, and according to their schedule, it probably won’t be fixed for about six months. Ah well, at least it’s pretty minor.

All that said, it has been a fairly enjoyable process. I’ve been able to develop more site in less time by working with the WordPress plugin system. I have *considerably* less code to maintain, since I’m only responsible for the plugins I’ve made. Had I created this from scratch, I wouldn’t have gotten even half as far given the amount of time I invested.

This just in, the WordPress spell checker chokes on the word “with” .. weirdness!

One line PHP templating engine:

echo str_replace(array_keys($replace),
array_values($replace),$content);

I’ve been using open source software for well over 10 years. Here are some stories of times when I’ve had brief interactions with communities regarding “bugs”.

Back in 2001, I tried to submit a “bug report” to the php project – I had discovered that in certain cases a function worked slightly different than advertised. My report was dismissed with an RTFM and flagged as “Bogus”. I was a bit put-off by their response, as I had RTFM (and demonstrated that clearly in my report.) The problem with my report is that I was complaining about an edge case that wasn’t really a bug. It just wasn’t worth their time to really understand what I was getting at.

In 2003, I submitted a bug report to the dosbox project – I discovered several games that I wanted to work with the software that didn’t. I offered my help in fixing the bug. My report was greeted with open arms, my offer to help fix the bug lead the maintainers to give me tips on how to troubleshoot the issue. Not being an accomplished ASM debugger I wasn’t able to get very far in my work. They saw that I had given it “the ol’ college try” and were happy to fix it for me. (I decided to sweeten the deal for them with a small donation to the project.)

Earlier this year, I had some trouble with rsync. I was losing connections part way through the backups. I realized that it probably wasn’t an issue with rsync but with my connections. I read over all the help information and found nothing addressing my issue on the rsync website. After talking with a few of my sysadmin contacts, they were able to help me track the issue down to a problem with my ssh setup (which was using default settings.) I was then able to write up an informative message to the rsync mailing list suggesting an addition to their FAQ which I felt would save people in similar situations hours of research. They updated the FAQ with the information I provided.

A bit later in this year, I submitted a small patch to rsync fixing a trivial problem with building in a certain hosting environment. This message went ignored. Since it was a one line fix for a problem which likely only happens to me, it seems that may have been the right response on their part. (Especially since, thinking over it, my patch may very well be the wrong thing to do for most environments.)

All that to say, from my experience it really seems that the more I put into reports and patches the more likely I am to see results and a positive response.*

*read this if you haven’t: How to Report Bugs Effectively