Back in college, before I knew much about anything, I gave a resume to a tiny little company called HDC Computer Corporation. That name kind of pegged them (constrained them?) to be a Microsoft Windows development house.
They managed to get some of their stuff bundled into Windows-286, if I recall correctly, including a little space shooter game called "HDC Rocks". Yes, it was about shooting rocks in space. Also, the title was a gag and tongue-in-cheek motto.
The Internet doesn't seem to remember that little nugget, but maybe it will now.
What you see here is the rebuilt version of as far as I had got on my "Space Rocks" demo - a little space ship moving around on the screen, rendering in OpenGL 2.1, with a window provided by SDL2.
Nothing's particularly exciting about it - the pathfinding, if you can call it that, is "move horizontally, then move vertically". There's no real obstacles. It doesn't have a lose condition.
But it's running again, OpenGL commands, as issued from my language, sent through a C shim to a vanilla OpenGL implementation.
As I mentioned in my previous post, I had lost a file (failed to check in my work) called gl_tools.c, which contained that layer, so the effort necessary to reproduce it was simply(?) a matter of exporting GL calls down into my library.
Sort of. The time management code is coming from SDL, so that took a little more attention. The initialization of an OpenGL context isn't quite as simple as I had recalled. I made use of Lazy Foo' Productions' SDL/OpenGL tutorial to figure out what bits I really had to rebuild inside my shim library.
And there you go, it's working again. And checked in, besides.
I even made myself a little convenience tool that launches the program, records frames of it rendering, then compresses that into an animated GIF all in one handy script. I could make it slightly cleverer, by forcing the window to open up at a particular point on the desktop, which would keep you from looking at my emacs window in the background. But maybe emacs makes you comfortable. I wouldn't want to deprive you of that.
The cursor, though, not so sure about that. I could easily suppress the cursor.
So. The demo is back up and running, and the thing I was running into back in July when I last checked stuff in (I don't remember this stuff, but GitHub does) was that I wanted to have a solid story about what I was doing with arrays of complex objects (like structs, say, like you might need to have for a bunch of asteroids floating across your screen). Would arrays have to be statically allocated? What's initialization like? Am I going to have pointers to these things? Python and Java don't have pointers, except that they do, but you don't see them. Is that what I want?
I want to avoid copying complex objects except by explicit deliberate action. It feels like there are a few other choices that can come as a result of that, but I haven't really meditated deeply enough to make it obvious how things ought to work. Perhaps the next step is to try some stuff, and be prepared to roll back if I don't like how things are working.
Musings, ramblings, rantings about technology, games, puzzles, and whatever else catches my attention.
Showing posts with label BDGL. Show all posts
Showing posts with label BDGL. Show all posts
Friday, February 5, 2016
Wednesday, February 3, 2016
Setbacks, and starting not from scratch
So, it's been a while since I've posted about my compiler project. Projects being what they are, you don't always have steam for everything.
And then, a few months ago, the laptop where I had been doing my development was taken. I can curse the darkness, but it's not as bad as it might have been - I had most of the project checked in to two offsite locations. So, that's nice.
At the risk of Pollyanna, this is an opportunity to verify that I don't have hidden dependencies that I wasn't aware of. So, I'm starting on a new machine, and hopefully, I'll be back to productivity soon.
I've added a Prerequisites page to my development Wiki. (Is that external? I forget. Probably not, or I'd link here.) It's the stuff you'd expect:
And then, a few months ago, the laptop where I had been doing my development was taken. I can curse the darkness, but it's not as bad as it might have been - I had most of the project checked in to two offsite locations. So, that's nice.
At the risk of Pollyanna, this is an opportunity to verify that I don't have hidden dependencies that I wasn't aware of. So, I'm starting on a new machine, and hopefully, I'll be back to productivity soon.
I've added a Prerequisites page to my development Wiki. (Is that external? I forget. Probably not, or I'd link here.) It's the stuff you'd expect:
- LLVM 3.6
- libedit-dev
- enum34
- llvmlite
- PLY
- emscripten
- SDL
Ok, maybe not all of that is expected. I think llvmlite's dependencies could be tidied up. And maybe I don't need SDL for everything, but for the Pong sample app, I did.
Once I got stuff installed and cloned the repo down onto my machine, I built the pong demo as a local executable, and it worked great.
I haven't yet rebuilt the web version, but you can play it here, if you like: http://bigdicegames.com/BDGRT/Pong/pong.html - I just link it because I keep forgetting where it is, so this is a bookmark for my own purposes. Also, I notice that that doesn't play nicely with my (current) version of the Chrome browser, but it does in Firefox.
Looking back through my checkin log, I see that I had been doing work on my Asteroids tribute sample, but when I try to build it, it looks like gl_tools.c never got checked in, so I get to rewrite that. How hard can that be? Just wrapping stuff like gl_begin and gl_color3f. I should be able to knock that out in short order, maybe tonight.
I'm also noticing that building emscripten targets (like the web version of Pong) is failing, probably due to not having installed certain bits of emscripten.
But, I'm delighted to say that my little test programs for computing factorial and Fibonacci stuff worked just fine.
Sunday, June 28, 2015
Smooth Animation, but lots of jaggy edges
You'd think a game loop wouldn't be a hard thing.
while(true) {
process_input();
update_game_logic();
render();
}
That can be adequate for a lot of simple games, but the cool thing to do is to decouple the update calls from the render calls, update on a fixed (sim) clock, and render as often as you can, using a very lightweight interpolation inside the render loop to smooth the trajectories between update calls.
In particular, I'm using a version from Robert Nystrom's "Game Programming Patterns", which looks like this in my language:
loop {
float currentTime = get_current_time();
float elapsedTime = currentTime - previousTime;
previousTime = currentTime;
lag = lag + elapsedTime;
sdl_tick_input();
if (sdl_quit_signalled() == 1) {
break;
}
while (lag >= SECONDS_PER_UPDATE) {
update();
lag = lag - SECONDS_PER_UPDATE;
updates = updates + 1;
}
float alpha = lag / SECONDS_PER_UPDATE;
draw(alpha);
renders = renders + 1;
}
The nice thing about this is that the updates are always at a fixed rate, so the simulations don't go unstable, and you still get to render as often as your machine is capable of. I could stick an artificial delay in there to keep the frame rate from going too high (rarely a problem, but if I wanted to conserve battery power on portable devices (phones, tablets, laptops) it might be worth considering.
Several bits that I ran into along the way that I need to clean up:
- I'm being sloppy about pointers within my compiler. I just throw an extra load() to dereference the pointer, and things seem to work out, but it feels super sloppy.
- I'm getting a segmentation fault about 13 seconds into the run. Don't know why that is, but it's reproducible. Which is a nice thing for a crash bug.
- I've got user-defined data types (let's call them "structs"). But if I create them at global scope, they don't get linked properly. Gotta figure that out.
I was hoping to get the ship to fly around with simulated intention by this point. That's soon, but not yet, I guess.
Wednesday, June 24, 2015
White Triangle
From a compiler/language perspective, this isn't an interesting post. All I've done is incorporate OpenGL into my existing SDL2 support from the Pong game, earlier.
It was a bit of a hassle merging the SDL2 OpenGL tutorials (which are, I guess, pretty new, and pretty spartan) with the NeHe "drawing a shape on the screen" tutorial (which has a SDL1 implementation). But, hey, there you go. A triangle, rendered in perspective, in my language.
If you're familiar with the NeHe tutorials, you might expect there to be a square on the screen, too. And the square does get drawn, just not at the point that you see here - I have a few different "game modes", and in the first state, I draw a triangle. After several seconds, I switch states, and then I draw the square. So that's even fancier than the NeHe tutorials. Take that!
For the next foreseeable bit, I'll be hooking up OpenGL to my language, so that's not pretty, but it's what needs to happen. I suppose I can write a Python script to grab GL.h and provide some amount of interface for free.
A few things that I should add to my TODO list:
- GLES support - I'm using the old OpenGL glBegin(GL_TRIANGLES) style. That won't fly on all the platforms I intend to hit, so I'll rewrite the triangle using a vertex array, which hopefully my language will be able to support natively.
- Strings - pretty soon, I'm going to want to accept user input (for a high score list?) and show text onscreen (on an about screen?) without the gross approach I used in Pong, which included a draw_char call that put a character at a position on the screen - no word wrapping, no language support for strings. Maybe that won't be too hard.
Monday, June 22, 2015
I have nothing to say, but it's OK, because now I can say it.
Apologies to The Beatles.
A bit ago, I copied the sourcecode for my Pong game into a new directory, as a way to start work on my next game in my language. I proceeded to comment out a large chunk of code.
Tangent: languages often don't have good facilities for commenting out chunks of code that might already include commented out code. In C++, say, you've got your single line comments
// like this
And there are multiple-line comments
/*
* that can go on for many
* lines, like this
*/
And if you want to disable a chunk of code that has a single-line comment, it's fine:
for (int i=0; i<10; i++) {
if (i < 3) {
/*
// show small numbers
printf("small number %d", i);
*/
process_data(i);
}
But it gets tricky if I wanted to disable that entire loop because */ terminates a comment, even if the comment was itself in a comment.
C and C++ have one more tool, the preprocessor, which helps some, so I could just do this:
#if 0
for (int i=0; i<10; i++) {
if (i < 3) {
/*
// show small numbers
printf("small number %d", i);
*/
process_data(i);
}
#endif //0
but that's still not terrific. Maybe disabling code isn't what a compiler is for. Still.
But that's not what I came here to talk about today. As I was saying at the top, I commented out a bunch of code from my Pong game, and stuff wouldn't compile. Ok, I thought, perhaps I was sloppy and commented out half of a function, and the remaining bits are causing problems. That may well have been part of it, but I cleaned up the obvious errors like that, and still I was running into issues.
One thing that doesn't get a lot of coverage in compilers tutorials is how to deal with errors - lexing errors, parse errors - there's perhaps a paragraph talking about inserting a special error token into your language and that's what gets called when things go wrong, which is a start, but then what do you do when that production gets called? How do you present enough information to the user of your compiler to fix the problem?
Right now, I print a warning with a line number and a little more information, which isn't currently sufficient. For one thing, the line number doesn't line up with the actual line number of the file being compiled. I think part of that comes from my compiler starting at line number 0 versus emacs, which starts at 1. Also, if I #include files, those throw off the numbering. Whoops. So, that's stuff to fix.
It'd also be nice to know what column the error was on, if I know that.
I've looked a little bit at pycparser, which seems to do a better job than I am for this sort of thing. TODO: roll in more of pycparser's approach to error handling.
And even that isn't what I meant to be talking about.
So, I waded through the output of my compiler, complaining about unexpected closing braces, and I ultimately determined that code like this was causing trouble:
void display_char(char c) {
# sdl_show_char(c);
}
Seems innocuous - a call to an external function that I commented out, since I'm rewriting my graphics library. Except, whoops, aha. The body of display_char is empty. And, whaddayaknow, my compiler doesn't like empty bodies for functions. Or, as it turns out for if/elif/else bodies, either:
if (phase == PLAY_GAME) {
# draw_game();
} elif (phase == SHOW_TITLE) {
# draw_title();
} elif (phase == SHOW_CREDITS) {
# ...
}
As it turned out, that was super easy to fix - I just extended my grammar to allow statementlists to be 0 or more statements, instead of 1 or more, which is how they were before.
So, now I can have functions that do nothing, and blocks of code that are empty. Progress!
A bit ago, I copied the sourcecode for my Pong game into a new directory, as a way to start work on my next game in my language. I proceeded to comment out a large chunk of code.
Tangent: languages often don't have good facilities for commenting out chunks of code that might already include commented out code. In C++, say, you've got your single line comments
// like this
And there are multiple-line comments
/*
* that can go on for many
* lines, like this
*/
And if you want to disable a chunk of code that has a single-line comment, it's fine:
for (int i=0; i<10; i++) {
if (i < 3) {
/*
// show small numbers
printf("small number %d", i);
*/
process_data(i);
}
But it gets tricky if I wanted to disable that entire loop because */ terminates a comment, even if the comment was itself in a comment.
C and C++ have one more tool, the preprocessor, which helps some, so I could just do this:
#if 0
for (int i=0; i<10; i++) {
if (i < 3) {
/*
// show small numbers
printf("small number %d", i);
*/
process_data(i);
}
#endif //0
but that's still not terrific. Maybe disabling code isn't what a compiler is for. Still.
But that's not what I came here to talk about today. As I was saying at the top, I commented out a bunch of code from my Pong game, and stuff wouldn't compile. Ok, I thought, perhaps I was sloppy and commented out half of a function, and the remaining bits are causing problems. That may well have been part of it, but I cleaned up the obvious errors like that, and still I was running into issues.
One thing that doesn't get a lot of coverage in compilers tutorials is how to deal with errors - lexing errors, parse errors - there's perhaps a paragraph talking about inserting a special error token into your language and that's what gets called when things go wrong, which is a start, but then what do you do when that production gets called? How do you present enough information to the user of your compiler to fix the problem?
Right now, I print a warning with a line number and a little more information, which isn't currently sufficient. For one thing, the line number doesn't line up with the actual line number of the file being compiled. I think part of that comes from my compiler starting at line number 0 versus emacs, which starts at 1. Also, if I #include files, those throw off the numbering. Whoops. So, that's stuff to fix.
It'd also be nice to know what column the error was on, if I know that.
I've looked a little bit at pycparser, which seems to do a better job than I am for this sort of thing. TODO: roll in more of pycparser's approach to error handling.
And even that isn't what I meant to be talking about.
So, I waded through the output of my compiler, complaining about unexpected closing braces, and I ultimately determined that code like this was causing trouble:
void display_char(char c) {
# sdl_show_char(c);
}
Seems innocuous - a call to an external function that I commented out, since I'm rewriting my graphics library. Except, whoops, aha. The body of display_char is empty. And, whaddayaknow, my compiler doesn't like empty bodies for functions. Or, as it turns out for if/elif/else bodies, either:
if (phase == PLAY_GAME) {
# draw_game();
} elif (phase == SHOW_TITLE) {
# draw_title();
} elif (phase == SHOW_CREDITS) {
# ...
}
As it turned out, that was super easy to fix - I just extended my grammar to allow statementlists to be 0 or more statements, instead of 1 or more, which is how they were before.
So, now I can have functions that do nothing, and blocks of code that are empty. Progress!
Saturday, June 13, 2015
Local arrays, arrays of objects
What you see here is an array of user-defined objects (Ship) like so:
class Ship {
Vec2f pos;
Vec2f vel;
}
in a locally-defined array. And you can see position and velocity being set, and then the position being updated using the ship's velocity. And the values are being read out correctly.
All working as you expect.
After the flailing to get the arrays working to begin with, this feels anticlimactic.
Next up: graphics, including using the above position and velocity data, vertex arrays, all on the way to an Asteroids-like game.
Friday, June 12, 2015
Reading/Writing ints/floats to global arrays
It doesn't look like a big thing, but the relevant bit of my code is this:
int intvals[3];
float floatvals[10];
int main() {
intvals[0] = 17;
intvals[1] = 19;
floatvals[0] = 3.14;
floatvals[1] = 1.5;
Here, I'm allocating two arrays in global space, assigning to them (and then reading from them, not pictured in the code snippet here).
A couple things I discovered along the way, which isn't well documented in the stuff I was looking at, was that array allocations need to have their "linkage" specified, which I haven't been doing for other global variables. Also, I wasn't able to get things working until I provided initializers for my arrays. That's a good thing, and I was considering providing default initialization anyway. I found it surprising that the LLVM assembler didn't want to accept my generated bytecode until I provided initialization values for my arrays.
Next up:
- arrays declared locally (top of function, inside loops)
- arrays of structs (e.g. Vec2f, Ship)
- *GL linkage, passing arrays through to OpenGL and WebGL
Thursday, June 4, 2015
Loading into, reading from, structs
I've found that in order to make progress on my language, it helps to have very specific lines of code that I want to make sure my compiler compiles, and focus on those.
The couple lines of code that I've been struggling with off and on lately have been:
newpos.x = pos.x + vel.x;
newpos.y = pos.y + vel.y;
Really simple stuff; retrieve member data from an aggregate type, do some arithmetic, store the results into member fields of an aggregate type. It so happens that this is simple physics / animation code, but the compiler doesn't care about that.
I had done some reading of the LLVMlite documentation and was sure that I needed to do some sort of insert_value and extract_value code. And then there's getelementptr, which is tricky, to the point that it's got its own FAQ page: http://llvm.org/releases/3.5.1/docs/GetElementPtr.html
In my previous post, I talked about using Clang as a reference - I might not have to use all of LLVM if Clang's got a way to accomplish what I'm doing, I can do the same thing that it does. And all it does is simple getelementptr instructions.
So, I started stripping out stuff that I had built to handle stuff on the left hand side of assignments differently from stuff on the right hand side. I put some getelementptr calls into my code. I started seeing errors about "gep not implemented on identified struct type", which I thought I maybe had to work around, but it turns out that somebody else had run into that bug and had submitted a fix for it. So, git pull and reinstall the LLVMlite module, and all of a sudden, things were working a lot better.
There's still some gross bits that are left over from my efforts to set up a complex solution to what was ultimately a simple problem - that's got to get cleaned up before I push this stuff up to my repository. I think I've also found some pointer stuff that now looks ripe for refactoring.
Next short term feature: (static?) array support - being able to create an array of objects, either simple or aggregate objects, and then reading and writing those objects.
Longer term ambition: some sort of Asteroids-ish clone. That'll put the above physics / animation code into context, and having a collection of asteroids floating around the screen will motivate the array support.
Getting away from language features, an Asteroids clone will provide pressure for more OpenGL support, which will be a good thing.
Sunday, March 15, 2015
Storing into aggregate types
I've been working on updating the grammar to handle aggregate types (structs, classes, that sort of thing). This weekend, I've adapted some of the C grammar and got things running again.
The LLVM code you see up there uses the "insertvalue" instruction, which is the way to write into a struct. The paired instruction is "extractvalue", which the compiler doesn't yet generate.
There are a few other issues with the generated code - the correct offset isn't being used. I think the "1" at the end of each line is an index to the element within the aggregate structure. I'm computing an offset for x and y members of my 2d vectors, but it's not piped through properly.
And then there's the loading and reloading of each of the aggregate instances. I know that LLVM has some aggressive optimization, but I'd rather not rely on it to clean this sort of thing up.
Friday, March 13, 2015
K&R 2e
I don't have any progress to report on the development of the language - I'm currently trying to get structs working, and that's involved some pretty substantial breakage of the grammar. I had a grammar rule for assignments which was something like <variablename> = <expression>, which was sufficient for a while - I could evaluate functions and bind them to variables, I could have mathematical expressions, I could take the contents of variables, and assign them, things were rosy.
But then I tried "velocity.x = 3", and my structure didn't hold up anymore. I had already created the velocity struct, that part of the work was pretty much working (see my earlier posts about migrating to llvmlite in order to get aggregate types being generated). I even knew that the "x" member of the velocity structure was the zeroeth element, because I was keeping that information around.
I just ran into a stumbling block in how to represent a struct member as a "L-value", a thing that can be assigned to.
I looked at some online references for the C grammar, and found an ANSI C grammar which sent me down a rabbit hole of rewriting my expression productions. I looked at either pycparser or maybe pycparser (honestly, probably the first one) and I see a bunch of ways that my code can be cleaner, but I'm still not parsing struct assignment correctly.
I stumbled across a LALR tutorial which makes everything seem simple.
And, I guess, for simple languages, simple parsers are simple. But things get hairy quick.
C is, in a lot of ways, a simple language. I learned it in maybe a week, probably much less than that. I bought myself the K&R "white book" (pictured above) around Christmas of 1989. Hm, I'm not sure all of the chronology of this story makes sense, because I'm pretty sure I bought myself an IBM PS/2 286 in January of 1990, and I taught myself C on that 286 with the clackety keyboard and the VGA monitor and the 20 MB hard drive.
C is an easy language to dive into, and it's pretty easy to understand how things work. I still smile at the characterization of C as having the power of assembly language with the readability of assembly language. Looking at C and assembly today, it's easy for me to read either one of them, but the difference is the level of abstraction - there's more "idioms per line" in C than there is in assembly.
And then, there's C++, which I taught myself bits of in 1992 and 1993. Again, the structure of the language is there to afford more idioms per line than C. But I never found C++ quite as satisfying as C was, and over time, I managed to ship plenty of working C++ code, and pass several programming interviews, and I knew about the "diamond of death" multiple inheritance issues, and I knew about vtables, and it was fine, but I kept looking for something else.
On the flip side, when you look at the grammar of C, it's hard to call it "clean". There are simpler languages out there (LISP and PostScript come to mind, as well as a few proprietary languages that I've had to use professionally). In each of these environments, there's a balance between the complexity of the language tools (parser, compiler, interpreter, etc) and the productivity of the people using these tools. At one company I worked at, an engineer insisted that a scripting language was important to the project, but he couldn't convince management to allocate time for it, so a simple PostScript(ish) interpreter was coded up over a weekend, and that engineer was happy. Management was happy, because people were still on-schedule, and then junior engineers were given the task of creating a coding culture in this stack-based language.
Toward the end of that project, I was approached to see if I wanted to add "regular expressions" onto the language, which didn't make any sense to me. I'm pretty sure it didn't make any sense to the person asking me, either. Perhaps he meant infix operators, which would have been a substantial change to the language.
All of this to say, I still have fondness for C. I dug around in the house and found my 25 year old copy of K&R, and I'll be using that as a reference as I do some more hacking with the grammar of my language this weekend. Hopefully, I'll move closer to that sweet spot of clean language design balanced against clean language implementation.
Sunday, March 8, 2015
Test Driven Compiler Development
A while ago, I was trying to get aggregate types (e.g. structs, classes, etc) into my language so that I could represent the position of the ball in my pong game as a single variable. I had trouble with getting LLVMPY to assign to or read from members of an instance of an aggregate type.
I dug into the mailing list for LLVMPY and discovered that LLVMPY was no longer maintained, and the cool new thing is llvmlite, which is a thinner wrapper around LLVM. Ok, so that's what I need to take advantage of the full LLVM feature set, that's fine.
So, I set aside the work on aggregated types and began rewriting the compiler to use llvmlite. I'm using git, so this was as easy as creating a new branch on my local machine.
This being the recommended upgrade path, there is a LLVMPY emulation layer, which you can presumably just import instead of the old library. I figured switching all at once seemed the right thing to do, so at first I ignored the migration layer. Much of the migration was straightforward, but I bumped into some bits for instance of managing global variables, and looking at the migration layer was a really good reference to see how the migration might be implemented.
As I was making my changes, I compiled several of my earlier test cases, verifying that the compilation completed, and then eyeballing the output. A better approach would be to embed expected output into the tests themselves and then automatically verify the built output against the expected output.
One step in that direction that I did implement is a simple python script that verifies that all the binaries build without compilation error, which is what's in the screenshot here:
You can see "built 17/17". Can't do much better than that.
Along the way, I updated some of the pong code to start using floats for position and velocity, but then I started getting a lot of errors where I was comparing a floating point number to an integer. My language doesn't allow this, and it has no automatic type conversion. Indeed, it has no conversion at all, so pong wasn't compiling, and I had to wade through a bunch of these lines.
Worse than changing numerous lines of code was the difficulty of finding the lines where the problems were. Several of the errors reported a line number, but they were invariably way too high, which resulted from doing two passes over the source code.
I fixed that, and now my error messages are more useful, and all of my tests compile and run.
I proceeded to check in my llvmlite branch, which now needs to be merged with my work on aggregate types.
Saturday, March 7, 2015
It prints 7
I've installed llvmlite, the successor to llvmpy, which is itself the successor to something else.
I've installed the prerequisites, run the tests, and begun poking around with the sample code and the docume... no. The nice thing about llvmlite is that it's a really thin layer around the LLVM library, so the LLVM documentation is going to be my reference.
I proceeded to hack one of the test scripts to output LLVM code to match one of my earlier tests.
and, as advertised, it prints 7:
I've installed the prerequisites, run the tests, and begun poking around with the sample code and the docume... no. The nice thing about llvmlite is that it's a really thin layer around the LLVM library, so the LLVM documentation is going to be my reference.
I proceeded to hack one of the test scripts to output LLVM code to match one of my earlier tests.
and, as advertised, it prints 7:
Wednesday, March 4, 2015
Or, maybe I could do it that way.
So, my language recognizes struct declarations. And allocations. And I've been working on assigning to instances of these structs.
I had false starts that I've mentioned here before, specifically about creating anonymous structs, which are probably useful for a language feature I want to do later, but not really relevant right now.
And so, I've been looking for examples of how it's supposed to be done. I found vague references to "getelementptr" in LLVM, which in LLVMPY is exposed as "gep", because hey, it's hard to type stuff.
Interesting side trip: my language is pretty close to C, so I wrote a little bit of C code demonstrating the behavior I want in my language, and I used "clang" to compile it to LLVM.
clang -S -emit-llvm teststruct.c
Easy peasy. Dropped a bunch of LLVM asm into teststruct.s, which used that getelementptr thing that I was trying to use. So, maybe I'm going in the right direction.
More searching, more experimentation (they call it Computer Science because we use the scientific method) and I start exploring the LLVMPY issue tracker, and I discover a feature request to support structures. Ok, good, it's a priority... ooooh. No. That's been outstanding for going on two years now.
And I start poking around the LLVMPY users' mailing list, and I discover a post suggesting that LLVMPY users switch over to LLVMlite.
So, I guess that's the next thing on my TODO list - migrate over to LLVMlite. And maybe, you know, read the documentation.
I had false starts that I've mentioned here before, specifically about creating anonymous structs, which are probably useful for a language feature I want to do later, but not really relevant right now.
And so, I've been looking for examples of how it's supposed to be done. I found vague references to "getelementptr" in LLVM, which in LLVMPY is exposed as "gep", because hey, it's hard to type stuff.
Interesting side trip: my language is pretty close to C, so I wrote a little bit of C code demonstrating the behavior I want in my language, and I used "clang" to compile it to LLVM.
clang -S -emit-llvm teststruct.c
Easy peasy. Dropped a bunch of LLVM asm into teststruct.s, which used that getelementptr thing that I was trying to use. So, maybe I'm going in the right direction.
More searching, more experimentation (they call it Computer Science because we use the scientific method) and I start exploring the LLVMPY issue tracker, and I discover a feature request to support structures. Ok, good, it's a priority... ooooh. No. That's been outstanding for going on two years now.
And I start poking around the LLVMPY users' mailing list, and I discover a post suggesting that LLVMPY users switch over to LLVMlite.
So, I guess that's the next thing on my TODO list - migrate over to LLVMlite. And maybe, you know, read the documentation.
Tuesday, March 3, 2015
On my way to implementing classes
I've been working on adding classes to my language, and I've got very little to show for it so far. Previous modifications have been relatively small, adding a little bit to the grammar, handling one new production, a small test case, and post here.
The test case I'm working on is this:
I added class definitions to my language, and that seems to be working OK, and I seem to be able to declare the instances of my classes just fine, but assigning to the members (e.g. "scrpos.x = 160;") has been giving me a hard time up to this point.
The place where I was stuck was in handling the "MemberRef" production (what happens when the parser processes the "." member operator). I have an object that I know is a struct, and I know it's got two int elements, and I know I want to refer to the "x" element, but I was banging my head, trying to figure out how to get the "Vec2i" name for that object.
Turns out, when I was constructing my struct object originally, I had ignored an optional parameter that is the struct's name. Now I think I have what I need.
On the left, in black-on-white, you'll see llvm.core.Type.struct(types, self.classname). This is where I'm passing in "Vec2i" in as the name of the struct I'm constructing. On the right, in white-on-black, you'll see my debugging spew, where it is reporting that the "base type name", the name of the type of scrpos in my sample at the top, that's "Vec2i", where I need to be able to figure out what "x" means.
A couple lessons here:
1) (boring) llvm.core.Type.struct can take more than one parameter, just because the code runs doesn't mean that it's doing what you want.
2) LLVMPY's documentation can be sparse. However, in this case, I could have discovered my problem by rereading the documentation on the struct constructor. By passing in no name, earlier, I was implicitly asking for an "opaque" struct.
3) source control is your friend. I don't think I need to tell you that, but I've ripped out a lot of stuff, rebuilding stuff, and the only way I can feel confident to do this is to know that I've got a checked in "working" version to revert to if things go wrong. Indeed, I've added a bunch of stuff that's now unnecessary, and I can filter out the unuseful stuff before I commit it.
Thursday, February 26, 2015
Better (non-int) return values
On my list of annoyances in the code that I've been writing so far has been declaring all my functions as returning ints, because that was what worked. I just dug in and figured out what was keeping void functions from working correctly - turns out, all I needed to do was to do a ret_void() at the right places, and everything worked great.
I also thought that returning floats would require extra work, since I didn't have float support in the language until recently, and I figured there'd be something that'd have to pay attention to whether I was returning an int or a float, but the above code worked just great on the first time. Well, the second time, if you count the compiler catching a legitimate bug in the convert_rad_to_deg source code. But even that's a success.
In the "note to self" category, or maybe for people who are following these breadcrumbs, learning about the LLVM experience, when I was implementing const variable declarations, I banged my head against parsing errors - I was having a hard time parsing "const int x = 8;", for reasons that weren't making sense to me. I was thinking that there was some precedence issue, and the LALR parser was maybe not sufficient to parse the ambiguous grammar that I had built.
I ended up (after many other stabs in the dark) turning on the bit that prints out the output of the tokenizer, and I noticed that "const" was being interpreted as an identifier, not as the keyword. I moved "const" up in my list of tokens, nope, same thing. I added 'const' into my list of reserved words, and then things worked right.
Another note: as I was implementing "const", I began putting in support for function static variables - variables whose visibility is restricted to a function, but whose lifetime is for the length of the application. I haven't finished that, and it's not a priority, but I did notice that LLVM variables have a flag, so maybe it's as simple as detecting that qualifier on the declaration and setting the flag.
Next up: structures / classes / objects. This will make a significant improvement to the architecture of my Pong game, in a few ways.
- data modularity - I can have a 2d vector representing velocity, and refer to the velocity as a higher-level object, moving it around as a unit, rather than having to remember to move pairs of vx,vy.
- API cleanliness - the events coming back from SDL for keyboard events or mouse actions are complex things. They have an event type (e.g. keyboard event) and usually one or more pieces of data (e.g. what button is pressed, maybe modifier keys that are also pressed, like 'shift').
Being able to clean up the structure of the Pong game will make it substantially less gross for me to look at, which is really the point of that game - show me what I can't stand, and I fix it.
Wednesday, February 25, 2015
Dear Constants
If you read that title and start humming "Dear Prudence" by the Beatles, you can skip the rest of this post.
I've added a few things, like being able to initialize variables on the same line as their declaration, and declaring that variables are constants. Except, if they're "constant" they don't vary. I'm still going to call them "variables".
I've also begun tidying stuff up to make a public repository. Things are rather messier than I'd like for real distribution, but getting stuff pushed to a public repository is going to force my hand some to make things cleaner as a habit.
As I was implementing constants, I was surprised to find that global variables in LLVM have a flag for being constants, but I didn't see the same thing on locals. So, I just made a separate dictionary for them in my frame objects. Nothing too fancy.
I also made parsing rules initially for constant declarations without initializers. Then, I stuck some code into my code generation to flag that as an error, and then I realized that I could catch it at parse time. As early as possible seems the best for that sort of thing.
No screenshots, as constants and variable declarations with initializers aren't pretty to look at.
I've added a few things, like being able to initialize variables on the same line as their declaration, and declaring that variables are constants. Except, if they're "constant" they don't vary. I'm still going to call them "variables".
I've also begun tidying stuff up to make a public repository. Things are rather messier than I'd like for real distribution, but getting stuff pushed to a public repository is going to force my hand some to make things cleaner as a habit.
As I was implementing constants, I was surprised to find that global variables in LLVM have a flag for being constants, but I didn't see the same thing on locals. So, I just made a separate dictionary for them in my frame objects. Nothing too fancy.
I also made parsing rules initially for constant declarations without initializers. Then, I stuck some code into my code generation to flag that as an error, and then I realized that I could catch it at parse time. As early as possible seems the best for that sort of thing.
No screenshots, as constants and variable declarations with initializers aren't pretty to look at.
Tuesday, February 24, 2015
Local Variable Progress
Evidence of new local variable behavior:
I added a stack of frames for local variables, and am looking up variables on that stack. Now, variables are local to the curly-brace block where they were defined (which has a name, which may aid debugging), they're looked up by most local to least local, and then in the global namespace (which is separate).
One way that this test had been failing was that the two different 'a' variables were getting confused, and the assignment "a=7" was stepping on the toplevel a, which should still have been 1. Now, everything's working correctly, and the assertions are happy.
Next up: being able to initialize variables at the time of declaration like "int x = 8;"
Monday, February 23, 2015
I've been working on my Compiler
I've been working on my compiler, a little bit, in the half year since I've written here. This isn't going to be a complete summary of everything that's been going on, but a few notes, as much for myself as anyone. I'll get chattier in later posts.
There are a few goals of my language project. These aren't in any priority order, just in me jotting down ideas as they come to me:
WIP, hopefully still around by the time you read this: http://bigdicegames.com/BDGRT/Pong/pong.html
One nice thing about the Pong test is that it's showing me what features I can live without, and what features are painful in their absence. I only had integer variables, and they were all global. This worked, but it was gross, so now I've been adding local variables and float support to the language.
As of this morning, I've got a little sample program that does a Fahrenheit to Celsius conversion using locally allocated float variables. There's a few pieces that I expect from local variables:
Also working is reading and writing local variables. I didn't list those, above. Not really variables if you can't assign to and read from them.
The rest of that remains to be done, and I'll post here as I get it working.
There are a few goals of my language project. These aren't in any priority order, just in me jotting down ideas as they come to me:
- lightweight language, with a feel like C, with only a little bit of C++'s object-oriented features, without C++'s bulk.
- targeting platforms ranging from desktops (Linux, Mac, Windows) to mobile (Android, iOS) to the web (HTML5 / JS) to maybe crazy unreasonable stuff like the Apple ][
- primarily for writing games (interactive, graphical), but maybe command-line stuff is OK (server support?)
- should be fun to program in
- should be performant, but this isn't the highest priority. If I can achieve performance through tools, rather than by burdening the developer with decisions, that's nice.
- David Beazley's PLY (Python lex/yacc) http://www.dabeaz.com/ply/
- LLVMPY, a python wrapper around LLVM: http://www.llvmpy.org/
- SDL, a media layer for games: https://www.libsdl.org/
- Emscripten, a tool for taking C (and LLVM bitcode) and turning it into Javascript: https://kripken.github.io/emscripten-site
- gnu make (for now)
- python (because I like python, I do. I was going to say I write everything in python. Except this is about writing a language that isn't python.)
WIP, hopefully still around by the time you read this: http://bigdicegames.com/BDGRT/Pong/pong.html
One nice thing about the Pong test is that it's showing me what features I can live without, and what features are painful in their absence. I only had integer variables, and they were all global. This worked, but it was gross, so now I've been adding local variables and float support to the language.
As of this morning, I've got a little sample program that does a Fahrenheit to Celsius conversion using locally allocated float variables. There's a few pieces that I expect from local variables:
- declared in the function where they're used
- or, declared in a block (a chunk of code delimited by curly braces, contrasted with a basic block, which is something a little different inside the guts of the compiler)
- has a well-defined lifetime (it goes away when the allocating block is exited)
- initialized to something well-defined, by default
- can be initialized as part of declaration (float x=8.0; #remember, Mallory, x=8)
- scopes can nest, so a more-specific scope will have priority over a less-specific scope.
Also working is reading and writing local variables. I didn't list those, above. Not really variables if you can't assign to and read from them.
The rest of that remains to be done, and I'll post here as I get it working.
Monday, August 25, 2014
PLY lexing and parsing
Having not posted for a while, it's overdue to report some progress here. There's not much in the way of visuals, but the above "screenshot" shows what I've been up to - I've been working on writing my own language, which has a very C-like feel to it. There are functions and curly braces, so nothing too exciting there.
I think that I want to add multiple return values, something like this:
(boolean, uint) find (string x, char c);
This would declare a function called "find" which takes a string and a character and returns a boolean and an unsigned integer - the boolean would indicate if the character was found, and the integer would say where.
I know that there are ways to accomplish this through magic numbers, like allowing the function to return a signed value and then returning -1, but that's a little clumsy for this example, and there are cases where it's even worse.
The bit that I'm showing off here is that, using PLY, a Python Lex/Yacc implementation, I've got code that recognizes a factorial function, shown in the screenshot. (As opposed to issuing errors on language use that it doesn't recognize, ie 'syntax errors'.) It doesn't build an Abstract Syntax Tree (AST) yet, but the productions are in place, and it's relatively easy (or so the PLY code tells me) to build up a tree from what I have.
One thing I'm discovering is that it's tricky to debug errors in language grammar specifications.
Subscribe to:
Posts (Atom)
















