Search Engines
What do people do when they want to find some information online? Yes, they indeed go to a search engine like Google and type their query to get relevant search result that provides them with the requisite information. “Simply Google it” is the new mantra to find answers to all your queries on the internet. How do you define a search engine? It is a kind of software system devised to search for information / data on the WWW (World Wide Web) and produce relevant search results. The search result pages produced as output for your queries are also known as SERPs (Search Engine Results Pages). Search engines help to find out information in the form of web pages, images, videos, listings, etc. they consider the query and try to find the most suitable answers. Search engines are so powerful that they can even produce search results with real-time data. They are able to do so with the help of search engine algorithms that are executed on a web crawler. Some of the popular search engines of today are from Google, Yahoo, Microsoft, etc.
History of search engines
Before the advent of search engines, the internet world made use of some efficient search tools to find information. Some of the earliest search tools were:
- Archie (the first search tool)
- Veronica
- Jughead
In the 1993, the world witnessed the first search engine in the form of “W3 Catalog”. It was a primitive search engine that just about managed to show results for information queries. With the advancement of internet, several new search engines emerged such as Yahoo search, MSN search, etc. However, it was emergence of Google around 1998 that made the world to sit up and take notice of search engines and their utility. With a simple search bar, Google changed the dynamics of search engine globally.
Overview of Google Search Engine
As of today, Google is the most popular search engine that receives more search enquiries than any other search engine on a daily basis. Secret behind Google’s massive growth- its simplicity and increased focus on making search more relevant. If you have a relevant website (designed and presented on the web to reflect the business products and services), it will appear in Google search results and the traffic gathered for your website will be genuine. However, the main task is getting your website found by Google search algorithms. Google focuses mainly on its search business including natural search and sponsored listings.
How does a web search engine function?
Insert your information query, click on “Search” or “submit”, check the database and display the information. It may sound pretty simple and straight-forward. But the process is not easy. There are plenty of activities that a search engines needs to perform in the background to produce the most relevant search results. Some of the activities performed by a search engine are:
- Crawling
- Indexing
- Processing
- Relevancy calculation
- Retrieving
Crawling
The basic thing that almost every search engine does is to crawl all over the web to check out what exists on different websites and web pages. A software known as Spider or Crawler or GoogleBot (for Google search engine) is used for the purpose of web crawling. The so-called crawlers (or Spiders) visit web pages and crawl up different links from one page to another. For a website, it is important to be crawled by such spiders so that they appear in the concerned search engine results. Crawlers are unable to view images, password-protected pages, javascript, etc. and so it is important that all website owners (or webmasters) check the stuff not crawled on their website using spider simulator. This crawling activity is extremely important because if plenty of stuff on your site is not visible to the spider, then, all that content will not be crawled, indexed and processed. In simple words, such content will cease to exist for search engine.
Indexing
Once the spider is able to crawl a web page, it is necessary to get the page content indexed. Indexing a web page is a process of identifying the words or expressions that describe the page perfectly following by assigning that page to specific keywords. Once a web page is indexed, it is stored in a database and can be easily accessed (or retrieved) later. For successful indexing, it is important that every page on your website or blog conveys it’s meaning clearly. Only then will the search engines will understand the meaning correctly and classify your web pages using the appropriate keywords. For this purpose, it may sometimes be a better idea to optimize some of the content on your web pages.
Processing
The earlier activities like crawling and indexing happen on the content (web pages) that are already present on the World Wide Web. The processing activity takes place during the search operation i.e. when someone places a search query in the search bar of a search engine. The processing activity carried out by a search engine involves comparing the search string in the search query (request) with the indexed web pages available in database. Now, there are millions of web pages that get crawled as well as indexed and later stored in the database. So, comparing the search string of the search request becomes a tedious task as there will be several similar pages containing that search string. To overcome this problem, search engines need to perform the activity of Relevancy calculation.
Relevancy Calculation
It becomes necessary for search engines to calculate the relevance of the indexed web pages with the search string of the search request. To calculate relevancy, there are different algorithms used by different search engines. This leads to different weightage for factors like links, Meta tags, keywords, etc. So, it is a common occurrence that even for the same search string different search results pages get displayed. Search engines often try and tweak the algorithms used for relevancy calculations. For this reason, it is necessary to implement SEO (Search Engine Optimisation) so that pages of your website are not affected by these algorithm changes as far as appearing in search results is concerned.
Retrieving
Once the relevancy calculation is done by a search engine, the final activity performed is retrieval of the search result and displaying them for access by the end user (the one who raised the search request). In this retrieval process, the search engine display tons of pages of search results for the relevant search string with descending order of relevancy i.e. the most relevant search results appear in the top pages and the less relevant search result appear in the last pages. Thus, the search engine does a lot more than simple tasks like querying the database and displaying the results. All these activities need to be performed consistently by search engine to successfully satisfy the requirements of end users.
I’m a freelance Web Designer and SEO Expert I have been working as a web designer and Search Engine Optimiser for over 8 years, and have done lots of work over that time. My skills cover many aspects of graphic design, web design, illustration, search engine optimisation (SEO), Social media optimization (SMO) & Search engine marketing (SEM) and other technical knowledge.
Web Presence and SEO 
Leave a comment