If they know a group of pages are related, or even not, the bot will cross reference the one with the other, many tomes over many pages and collect the common matches, pairwise, The common matches are word lists. With a bit of work the bot can find segmentation of web pages into groups, by adjusting wordlist to magnify compactness of page sets. A lot of NSA style mass processing of data, and the learning process is like neural nets with word lists at the nodes. Matching new words change lists; over time, the pit boss runs through the directed graph of lists, and prunes words off while adding new. Like a high speed, automated Webster.
Anybody can have a bot reader, all the mechanics make sense, and I am sure the mathematicians are on the job here to prove convergence.
No comments:
Post a Comment