Sunday, March 8, 2015

Test Driven Compiler Development


Some more (hard to screenshot) progress, some more features that turn up to be annoying in real development.

A while ago, I was trying to get aggregate types (e.g. structs, classes, etc) into my language so that I could represent the position of the ball in my pong game as a single variable. I had trouble with getting LLVMPY to assign to or read from members of an instance of an aggregate type.

I dug into the mailing list for LLVMPY and discovered that LLVMPY was no longer maintained, and the cool new thing is llvmlite, which is a thinner wrapper around LLVM. Ok, so that's what I need to take advantage of the full LLVM feature set, that's fine.

So, I set aside the work on aggregated types and began rewriting the compiler to use llvmlite. I'm using git, so this was as easy as creating a new branch on my local machine.

This being the recommended upgrade path, there is a LLVMPY emulation layer, which you can presumably just import instead of the old library. I figured switching all at once seemed the right thing to do, so at first I ignored the migration layer. Much of the migration was straightforward, but I bumped into some bits for instance of managing global variables, and looking at the migration layer was a really good reference to see how the migration might be implemented.


As I was making my changes, I compiled several of my earlier test cases, verifying that the compilation completed, and then eyeballing the output. A better approach would be to embed expected output into the tests themselves and then automatically verify the built output against the expected output.

One step in that direction that I did implement is a simple python script that verifies that all the binaries build without compilation error, which is what's in the screenshot here:


You can see "built 17/17". Can't do much better than that.

Along the way, I updated some of the pong code to start using floats for position and velocity, but then I started getting a lot of errors where I was comparing a floating point number to an integer. My language doesn't allow this, and it has no automatic type conversion. Indeed, it has no conversion at all, so pong wasn't compiling, and I had to wade through a bunch of these lines.

Worse than changing numerous lines of code was the difficulty of finding the lines where the problems were. Several of the errors reported a line number, but they were invariably way too high, which resulted from doing two passes over the source code.

I fixed that, and now my error messages are more useful, and all of my tests compile and run.


I proceeded to check in my llvmlite branch, which now needs to be merged with my work on aggregate types.

No comments:

Post a Comment