Google Search - Then & Now

Speaking at WSDM 2009, Google Fellow Jeff Dean in a keynote talk shared how Google has grown from 1999 to 2009. Various search engine enthusiasts have blogged about the facts presented in the talk from their own notes.

Excerpts:

• Google handles 1000 times more queries now with 1000 times more processing power (# machines * speed of the machines).
• It uses 1000 machines to handle a single query compared to just 12 previously.
• Query latency has improved from under 1000ms to normally under 200ms now.
• Crawler updates now take minutes compared to months in 1999.
• For many pages, search results now change within minutes of the page changing.
• Rolled out seven major rearchitecture efforts in ten years.
• In-house design from the ground up: rack design, pc class motherboards, linux, and in-house software (GFS, BigTable, etc...)
• Google's machine translation models use a million lookups in a multi-terabyte model just to translate one sentence.

Popularity comes at a price. Google is now accused of harming the environment with it's energy consumption (a typical search uses "half the energy as boiling a kettle of water" and produces 7 grams of CO2) and for not practicing what it preaches.

Also see: How LARGE websites manage performance & scalability