Search Engines

I. How search engines work

Search engines are based on computer programs that explore the Net in search of webpages. They do this by using so-called "spiders," "crawlers," or "robots".
Like any human surfer, the robots are able to follow the links present on webpages. When finding a new webpage, they copy all or parts of the text present on the page into their search engine database.
Other spiders revisit these pages on a regular interval in order to register changes or dead links. The interval varies greatly from search engine to search engine and from site to site.

II. Ranking algorithm

The search engine will have billions of webpages in its database. In order to deliver relevant results, the search engine companies have developed special ranking algorithms that calculate the order of search query results.
By a search query we mean the keyword or keyword phrase the visitor enters into the search form of the search engine.
A special program analyzes the search query, and then searches the database for pages that contain this keyword or keyword phrase.
Next it analyzes every single one of the relevant pages in order to determine how important that phrase is on that page. The pages the search engine find most relevant will be listed first.
For instance, if you are searching for "Albert Einstein," and the search engine finds a main headline on that page containing the phrase "Albert Einstein," it will guess that this page is fairly relevant to your needs. If, on the other hand, it finds the phrase only once, buried in a long text, it will reckon it is less relevant.

III. Getting search engine points

The search engine gives each and every page points based on how often and where it finds the keyword phrase (the keyword location). Some locations give more points than others, the exact factor varying from search engine to search engine.
Search engines also take other factors into consideration, including link popularity. We will come back to that.
One very important part of search engine marketing is to know where to put the relevant keyword phrases in your webpages.

IV. Designing webpages for search engines

The search engine does not "see" the webpage the way we see it; it reads only the text-based code behind it. This is will help you to have an elementary insight into HTML, the coding language used to produce webpages. If you do not know any HTML, you could for instance read through one of the beginner's guides to HTML.
If you do not have the patience for this, by all means read on -- we will help you understand. Keep in mind, though, that an HTML file is a plain text file that contains the text you can read on a web page, as well as information on how the web browser is to format it. We are going to look at the elements of this text file.

V. On search directories

This guide is about search engines only. There are also so-called "search directories" produced by human beings that visit and consider websites for inclusion. The most important ones are the Yahoo! Directory and The Open Directory. In order to get listed in these, you have to use entirely different techniques.
See our article on "Preparing your site for the search directories") for more information.

More