Saturday, February 18, 2012

Native Qson Formats

There are four forms of Qson, sqlite table, in memory graph, flat format for files and sending, and Json. The code comes with a run time switch, you can go from any to any. Any file to/from any table to/from any network to/from any console to/from any memory. What about the flat file for sending? here is a complete example (I did by hand):

POST\r\nContent-Type:text/qson\n\rContent-Length:00000029+00200041234,10003123

 So everything up through the colon is magic fixed size header.  Then eight bytes of content length (the Endianess thingy can be adjusted). Then, for each triple; three bytes of row count, and key_count and key_bytes.  This is all subject to change, of course. The more likely is that link gets four bytes, and pointer gets four and key count gets four, and data is a minimum of four.  So the graph of one node is 16 bytes, the minimum atomic unit. Then make the total count an eight byte value, but it counts in big units of eight bytes at a time. The minimum additional key data increment becomes eight bytes, and  a twelve byte string (zero terminator not needed) for a name become perfect.

MyWebAddress,is a twelve bytes  index into a table maintained by your service provider. The provider can add: CheapSeats$MyWebAdress , indicating you are a dead beat CutnPaster, like me. CheapSeats point to a secondary look up table.  The syntax here is still under discussion, it will be what works the fastest and most efficient in the engine itself. Keep short names is not a requirement, but a foreknowledge that clients learn.

The in memory format is an array of triples struct  {int;int,int*;}  and the key pointer points to buffs containing key_count,key_data.

The sqlite format is a three column: link,pointer,blob  Sqlite wants the blob move to the last column, and it also means the engine can caste the triple format onto a buffer and match the variable names. So all formats keep key_count,key_bytes last.

For Json its all about replacing the key_count with brackets and colons appropriately.  Going the other way is Lazy J.

That is the plan, subject to change for native Qson.

No comments: