Elements of a search engine

There are basically 3 parts:

1) The "spider", "crawler" or "bot" collects data from the web.

2) The "indexer" analyzes the content, decides on appropriate keywords and makes some preliminary determination about the "importance" of a page by some algorithm (Google uses PageRank, for instance).

3) The "engine" is the component that services the users request and matches the keywords to specific pages or other content.

- extracted from a reply by Paul Williams in LinkedIn Answers: Web Development

Comments