web internet sites in the sense that a file is downloaded to the user’s browser when he or she surfs to these addresses. But that’s where the similarity ends. These web pages are front-ends, gates to underlying databases. The databases contain records relating to the plots, themes, characters and other features of, respectively, motion pictures and books. Each and every user-query generates a distinctive web page whose contents are determined by the query parameters. Hidden wiki of singular pages as a result capable of being generated is thoughts boggling. Search engines operate on the same principle – vary the search parameters slightly and entirely new pages are generated. It is a dynamic, user-responsive and chimerical sort of web.
These are very good examples of what http://www.brightplanet.com contact the “Deep Net” (previously inaccurately described as the “Unknown or Invisible World-wide-web”). They think that the Deep Internet is 500 instances the size of the “Surface World-wide-web” (a portion of which is spidered by standard search engines). This translates to c. 7500 TERAbytes of information (versus 19 terabytes in the entire recognized internet, excluding the databases of the search engines themselves) – or 550 billion documents organized in one hundred,000 deep web internet sites. By comparison, Google, the most comprehensive search engine ever, stores 1.4 billion documents in its immense caches at http://www.google.com. The all-natural inclination to dismiss these pages of information as mere re-arrangements of the same info is incorrect. Truly, this underground ocean of covert intelligence is generally more precious than the info freely out there or very easily accessible on the surface. Therefore the capacity of c. 5% of these databases to charge their users subscription and membership fees. The average deep web web page receives 50% far more targeted traffic than a typical surface site and is substantially additional linked to by other web sites. But it is transparent to classic search engines and little known to the surfing public.
It was only a query of time before a person came up with a search technology to tap these depths (www.completeplanet.com).
LexiBot, in the words of its inventors, is…
“…the initial and only search technologies capable of identifying, retrieving, qualifying, classifying and organizing “deep” and “surface” content material from the World Wide Net. The LexiBot enables searchers to dive deep and explore hidden data from various sources simultaneously making use of directed queries. Corporations, researchers and customers now have access to the most worthwhile and challenging-to-discover details on the Web and can retrieve it with pinpoint accuracy.”
It places dozens of queries, in dozens of threads simultaneously and spiders the benefits (rather as a “1st generation” search engine would do). This could prove really beneficial with huge databases such as the human genome, weather patterns, simulations of nuclear explosions, thematic, multi-featured databases, intelligent agents (e.g., buying bots) and third generation search engines. It could also have implications on the wireless net (for instance, in analysing and producing place-specific marketing) and on e-commerce (which amounts to the dynamic serving of net documents).
This transition from the static to the dynamic, from the given to the generated, from the one particular-dimensionally linked to the multi-dimensionally hyperlinked, from the deterministic content material to the contingent, heuristically-made and uncertain content material – is the real revolution and the future of the net. Search engines have lost their efficacy as gateways. Portals have taken more than but most folks now use internal hyperlinks (inside the exact same web web-site) to get from one place to another. This is where the deep net comes in. Databases are about internal links. Hitherto they existed in splendid isolation, universes closed but to the most persistent and knowledgeable. This may well be about to adjust. The flood of top quality relevant data this will unleash will significantly dwarf anything that preceded it.