Wednesday, February 22, 2012

Making and loading square tables with Lazy J, best approach

I always say, the fastest path is best. The machine has the IO_struct thing, it knows about square. I have already define machine_get_row and machine_get_colinfo. Sqlite3 will make a square for us, but how to we specify in Lazy J? Hmm...

We have machine index space, lets make a SystemSquare.
MySquare@SystemSquare,"name text who int where blob"
What did I do with the quotes? Something interesting, I said they enclosed a semi ordered set of words. Isn't that more interesting than "Don't look here" or "Some bytes" Or I suppose:
MySquare@SystemSquare,name:text.who:int.where:blob,xx
I like it, easy. Lazy J naturally parses that into binary object for @ interpreter. It look up SystemCall, System call gets the argument structure and does a fetch until the schema is done. Then courtesy of Sqlite3, it does the create.

So, in you use Lazy J as CSV files on steroids, identify a proprietary table format, up front. Then keep loading million of records, using named:value pairs, or just grouping values together. I have tested this machine with long list of variably groups key words, no problemo.

Let's make square tables, tables with many columns. Lazy J, the ninth wonder of the world.

No comments: