Search Engine Technology

By: Chelsea Canulo

BS PSYCHOLOGY 1A
What are search engine?

 A program that searches for

and identifies items ina
database that correspond to
keywords or characters
specified by the user, used
especially for finding particular
sites on the World Wide Web.
 A search engine is a web-
based tool that enables users
to locate information on the
WWW.
 The information gathered by
the spiders is used to create a
searchable index of the web.
Modern web search engines are complex software
systems using the technology that has evolved over the
years.
Google and Yahoo - are the largest search engines that
utilize tens or hundreds of thousands of of computer to
process billions of web pages and return results from
thousands of searches per second.
Modern Search Engines have the following components:
Crawl
What is Crawl Method?
Crawling is the process used by
search engine web crawlers to
visit and download a page and
extracts its links in order to
discover additional pages.
Most modern search engines
now utilize a continuous crawl
method rather than discovery
based on a seed list. The
continuous crawl method is
just an extension of discovery
method but there is no seed
list because the crawl never
stops. The current list of pages
is visited on regular intervals
and new pages are found
when links are added or
deleted from those pages.
Link Map
Pages discovered by crawlers are fed into service that creates a link
map of the pages.
Link Map-is a graph structure in pages are represented as nodes
connected by the links among those pages.

 this data is stored data structures that allows fast access to the
data by certain algorithms which compute the popularity score of
pages of the web, essentially based on how many links to a web
page and a quality of those links.
Index
Indexing is the process of extracting
text from web pages, tokening it
and the creating an index structure
that can be used to quickly find
which pages contain a particular
word.
The index is where your
discovered pages are
stored. After a crawler finds
a page, the search engine
renders it just like a
browser would. In the
process of doing so, the
search engine analyzes
that pages's contents. All of
that information is stored in
its index.
End