Our bots ca have private look up tables, anywhere there is a machine and a cooperating linux box. The bots can even contract with other bot companies to have their index tables updated daily. You see, our bots can still move and collect using ontological word navigation.
How does the system bot update my remote look update table?
MatsTable:@{NewIndexTree,*}
NewIndexTree being a Bson arrival to the usual indexing site. The system prepares the mew index updates, basically just replaces my current table, ad sends them out each day. So key search Bot can just hop along its own private network, looking coclecting my daily news.
This whole scheme of private look up tables implies security, but that is handled well I hear.
That leaves global indexing. Even on these, regular global indices are collected for me, tuned to my needs, so the Huffman encoding is already done, asynchronously by a service. The service knows my needs, and uses local tables, hence never a need to look up anything far away. Remember the underlying assumption is that hops are not costly due to network time, the greatest amount of information being stored in huge warehouses where network distance is a few feet.
That leaves global indexing for new or wildcard sites. I dunno, a lot of these problem goes away wit intelligent collecting bots, local, personal indexing tables, all that free stuff you get with an open source Regex for key words.
Maybe I will let this one simmer, this idea that we need distributed indexing right away. It just seems to me the work should still focus on local key index tables, easily made and destroyed.
No comments:
Post a Comment