Tree data structures provide the foundation for most interactive systems. In most cases one can take an existing app, class, or collection of objects and represent it with a tree structure. HTML, XML, JSON, DOM, Flash displayList, file systems, and programming languages are all hierarchical in nature.
I have been interested in serializing & deserializing tree based systems for a long time and feel like there is a gap in terms of file formats. Tag based formats (XML/HTML) tend to be fragile given closing tags and are generally hard to read and edit accurately without tools. Minimalistic formats like JSON and YAML lack type metadata and can be confusing to read and edit as well. Also parsers for these formats tend to lean towards generating a complete object hierarchy as a result vs emitting stream of events allowing a single pass of object generation.
Long story short, I haven’t found somethings that fits well and decided to build one. I settled on calling the format “Meta”. Here is the base definition:
Meta – A simple readable ASCII based format for typed tree serialization & deserialization.
//comment <NAME>:<TYPE>=<VALUE> <NAME>:<TYPE> <NAME>:<TYPE>=<VALUE> <NAME>:<TYPE> <NAME>:<TYPE>=<VALUE>
- <NAME>,<TYPE>,<VALUE> are strings using any non-whitespace char, must quote if “:” or “=” is used.
- Indention denotes object heirarchy
- Types follows the colon “:” similar to type declarations in ActionScript 3
- Values follows the “=” and vary in syntax to the type used.
s1:String = "Hello" n1:Number = 1234 b1:Boolean = true //comment innerobj:Object src = logo.png x : Number = 0 y : Number = 0 isFullResolution : Boolean = true rotation : Number = -30 alpha : Number = 0.5 c1 : Color = #FF00FF55 an1 : Array = [1,2,3,4,5,6,7] obj0:Object 1:Number = 1 2 = 2 3 = 0 4: Boolean = true obj1:Object src = "logo.png" //comment x : Number = 0 y : Number = 0 isFullResolution : Boolean = true rotation : Number = -30 innerObj:Object src = "logo.png" x : Number = 0 y : Number = 0 isFullResolution : Boolean = true rotation : Number = -30 alpha : Number = 0.5
One thing I really like about the format is that it is streamable in that parsing of the file can start as soon as downloading begins, all you need is a complete line regardless of hierarchy. This makes it very useful for data exchange as sometimes it is handy to stream data(large array collections or ui) to a client. Within the game usage the format sets up large object trees of box2b enabled displayobjects (think of a level in Angry Birds…). Combined with a class that manages object creation/destruction/recycling and it gets really scalable. I especially like the readability aspect, just glancing at the file you can see the object tree and all you need is a text editor. Within the game engine I implemented Library and Reference types to allow for reuse of generated objects within downstream objects and properties. This sort of mapping made wiring up function handlers and material types far easier.
Obviously Meta is far from complete but I have found it very useful thus far. As soon as I apply a bit more polish to the codebase I will post the Lua and ActionScript classes for parsing and serialization.
Thoughts and comments always welcome.