Leading enterprise search vendors have started to release “indexes” of the internet, in a move they claim signals a new phase for web search technology.

UK-based Autonomy and US specialist Convera have announced web indexes, which will be available to providers of web services to improve their content delivery.

“The launch of indexes is, in many ways, stage two of web search development,” said Mike Lynch, CEO of Autonomy, which recently acquired major rival Verity ( see full story ). Autonomy has released its Conceptual Index of the World Wide Web. The index is said to contain one billion items, including web pages, video, audio and image files. Convera has created an index of 500 million images, 20 million audio files and five million videos as part of its Excalibur web search technology.

“Google and Yahoo search engines have a keyword index, but with this we are launching an index of concepts, which will enable natural language
search and implicit query,”said Lynch. This technology has become increasingly important in the past 12 months as reputation management applications come on to the market from the likes of aggregators Factiva and LexisNexis .

“The conceptual abilities of the index allows users to monitor the web without being discovered,” he said. Indexes may also overcome other problems associated with existing search providers. Patrick Condo, president and chief executive of Convera, said: “Research shows end users tend to only look at the first page of search results.”

Lynch said major media organisations will increasingly bring their content online because of the web indexes and implicit search technologies, as users will be able to search for particular scenes and parts of shows. Autonomy claims to have several major organisations interested in the index.