Wednesday, February 15, 2012

Bson has one effiiciency that Qson doesn't

Bson names are generally shorter than the data, so Bson makes then c strings, their total storage is much less when we can assume short strings. From the point of view of a query syntax with joins, the grammar of joins, it goes easier when the name structure looks mostly like any other structure. This is a trade off, structured and jin ready, ro compact and requiring more byte counting. For a short name with eight letters, Qson will waste about 30 bytes, total I imagine, including B tree inefficiency. When the key is always a blob, sqlite3 has the character count, always handy from beginning to end. That recovers a lot of the inefficiency in the B Tree. But native Qson sends over the wire will be bloated compared to Bson, the more naming the more bloated. But Bson still has the forward count or pointer, so fundamentally they still make Bson convolution work.
The native bundle format for Qson is:

|KeyLen|KeyData|Link|RowCount|

Qson maintains the row count, the size of the enclosed object, and the key length keep with key data. So, between machines, Qson just shuffles in an out. Qson count in rows and byte counts. But look at the waste of space when a key is a short name. In local memory, the key data is stored with malloc, and the output system can stream inefficient malloced buffers to the socket.

But that is a small trade compared to the power of being able to name data points arbitrarily along its length, andthen have an efficiet indexing on the name.

No comments: