Metacrawler

Description of a search engine that utilizes a bot to crawl through web sites and index the page by the data contained in the meta tags or the first portions of text on the page. Because of abuse, major search engines have abandoned this technique as the only way to index web pages and now rely mainly on the content contained on the page.

Also see: Bot, Crawl, Index, Internet definitions, Meta tags, Search engine