Blog » Search Engines
Search 101 - how search engines work - part 2
In Part 1 we covered how a search engine crawler visits web pages. In this part we're going to investigate how words on web pages are indexed. You'll recall the three phases of search engines: Crawling (or spidering) the web, finding pages people want to search Indexing words on web pages Searching the index (i.e. the bit that happens when you type a search into Google) A search engine index works very like the way the index in book works: in a book each word in the index lists page numbers the word appears on; in a search index each word has a list of pages the word appears on.
Search 101 - how search engines work - part 1
Having some background on how search engines work is very useful when you're trying to optimize your site. We hope we've a bit of perspective on this, having spent the best part of a decade implementing search engines and web crawlers. How do words on a web page end up searchable? This happens in three phases: Crawling (or spidering) the web, finding pages people want to search Indexing words on web pages Searching the index (i.