Friday, November 4, 2011

Speaking of distributed file systems

I currently ponder that problem with the ontology engine. When we introduce a convolution operation, we may may require access to thousand or even millions of ontology maps, stored as object stores in our embedded sql micro machine. One ontology block descending into another, hopefully near by.  I imagine that we should identify a particular object store by some URL or RDF style consruction, then hand it off to a distributed file system.

But the ontology engine can manage block store directly on its own, probably much better than Google's BigTable.  How do we find any particular block embedded in some database? How about this:

convolve('some rdf pointer',BlockStoreLookup) BlokStoreLookup is simply an entry point into the web ontology, like QueryExpander. Behind the scenes we have much large graphs, the Optimizing Graph. They traverse the ontology, including block store links, and constantly optimize access steps to minimize entropy. It places less accessed storage deeper into the archives as it is less frequently required by clients. And keep frequently accessed locks close by in small chunks. That is the old -iLog(i), where the i are a set of discrete access ranks, the step number to arrive at a conclusion. Log(i) generally should be the block size. The open source ontology engine has background optimizing graphs in continual convolution making this so. I suggest we allow this process to run is own locator for object store. Looking at the typical operations of convolve(gG1,G2) the execution unit has a choice, send G1 to the next block of G2, or bring the next piece of G2 to G1.

No comments: