Manual Review

All major search engines combine a manual review process with their automated relevancy algorithms to help catch search spam and train their relevancy algorithms. Abnormal usage data or link growth patterns may also flag sites for manual review.

See also:

Inktomi Spam Database Left Open to Public – article about Inktomi’s spam database from 2001
Search Bistro – links to Google’s General Guidelines on Random Query Evaluation [PDF] and Google’s Spam Guide for Raters
Google Image Labeler – example of how humans can be used to review content

Categories

Manual Review