Writing a parser

Edit: Information on this page is out-dated. Please go here to find out more about the new JSON game maker library.

As you have seen in a previous post, I designed a JSON encoder that encodes data into a JSON string for Game Maker. That was super easy to write. But, I also needed a JSON parser or decoder which is a lot harder to write.

Well, I’ve had the pleasure of working with Game Maker. It has a scripting language, but unfortunately, no built in JSON-parser or any kind of regular expression engine. So, obviously, I have to write one.

Some of you may be asking, “but dayum, why not just get an existing JSON parser and turn it into a dll for Game Maker to use?”

The answer is:

  • I suck at programming, therefore:
  • I can’t understand how to use many of the JSON parsers. Either I’m extremely stupid, there isn’t enough documentation or, most likely, both.
  • It isn’t very efficient because it cannot directly access Game Maker’s data structures. Well, using GML is technically, also very inefficient (being a scripted language) but at least you can directly access resources.
  • I also have to deal with resource management (which I hate).
  • I’m bored and I want to write a parser from scratch.

Basically troubles of putting together a library for Game Maker to use is right out of the picture for me. I’m aware of programs like YACC but I don’t think they work for Game Maker.

So, on with my parser-writing adventure. I scrapped four designs before finally getting one that actually works. It took me 4 design-scrapping to realize you can’t simply use regular expressions or string finding functions to parse JSON. Yeah, four designs.

There was also the problem of dealing with GML’s data structure limitations. My parser would have been 10 times easier had I written it in PHP. PHP, allows maps (associative arrays) of basically unlimited levels while Game Maker only allows 2 dimensional arrays. You can see from the code snippet of what I mean:

$array[‘foo’][‘bar’] = "hello world";
$array[‘key’] = array(‘hello’=> array(‘foo’=>’bar’));

Compared with Game Maker

array[0,0] = “hello world”;

This is a pain in the ass because that means if we want to do a associative array, we’ll have to use the ds_map functions. Even that has it’s limitations since it can only go one dimension and is much more awkward to access data than arrays.

The problem with one dimensional maps is that say I get a string of json like this:

{ "foo": { "hello": "world }, "foobar": "helloworld" }

I can’t simply put the whole thing into one map, unlike PHP.

In the end, I went with parsing one dimension and just storing the rest into a string for further processing when required. So the above JSON string would be stored like so:

Key Value
foo { “hello”: “world }
foobar helloworld

The next thing that really got me was ignoring whitespace. I have to iterate through strings and I decided to break it up into parts rather than do the whole thing at once.

So, I’d keep iterating until I got a number, bracket or quotation mark then select the appropriate function to deal with the rest. That function automatically trims out parsed data and passes the string back to the calling function which does another iteration until there are no more to parse.

The string parser, iterates through every character and stores it’s state (whether the next character needs escaping, etc) until it reaches a quotation mark. Then it stores the acquired data and returns to the caller.

So, first parser I’m writing. I hope it turns out successful. I’ll post snippets of code tomorrow of what I have. If anyone reading this has experience in writing a parser or knows a better way of doing things, please leave a comment. Because, quite frankly, I’m not doing so well.

That’s all, good night.

3 thoughts on “Writing a parser

  1. I was lucky enough to google a GM JSON interpreter on google today, and found this post. I need one desperately for my IRC Bot, and would be willing to help you.
    Email me if you are interested.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s