SEO – Webpage/Robots/Spiders

What are the importance steps we have to do in order to optimize a single page in a website? Do we have to optimize only the homepage or all the pages of the website? How can we focus on particular keyword for the website? Wow that’s lot of question to be answered. There are so many questions that will arise when you really start doing SEO for a website.

The important steps that has be considered serious in optimizing a webpage are

  • Proper title for the page in the header section
  • Page title again in the body of the page with header tags h1,h2 etc
  • The body of the page should have rich content based on the title of the page.

I think so the above points are the basic stuff that has to be followed while doing any decent website. It is always better to have all the pages in the website optimized. No SEOs has ever optimized all the keywords he wants to target in homepage of the website. In-fact it not a good practice to do that and it is not possible. Distribute your keywords to be focused in different pages and concentrate it better.

I know you think I am really mad!!! Now what is search engine spider bots? What does it do? Who sends them? Why they send them? What is benefit of spider bots visiting a website? How often do these bots visit a website? What are the parameters that control the frequency of the visit? Will more visit by these bots benefit the website or vice versa?

Search Engine Robots

Search engine spiders are software designed to get the webpage and save them to the search engine database. These are sent by the search engines like Google, MSN, and Yahoo etc. They send these bots to retrieve the information present in your website and save the same in their database for them search and give results. The benefit of bots visiting your website will be your site will be included while search engine performing a search. And chances are there for your website to appear in the search. One cannot really predict how often these bots comes to the website. For some it comes every few minutes for some it may takes months. The parameter that controls the frequency of the bots visiting your site will be based on the fresh content, site popularity, etc. It is good if the bots comes more often to the website. It means your website is given more importance and search engine will always keeps your site index up to date.

All spiders/robots obey one thing called robots.txt. Please read this if you wish to know more about how to write your own robots.txt file. Sorry I can really feel that I am jumping like monkey here and there. I am not good at writing anyway..!! More to come… 😉

Robots.txt Ultimate Guide

Categorized as SEO, Web

By Imthiaz

Programmer, SAAS, CMS & CRM framework designer, Love Linux & Apple products, Currently addicted to mobile development & working @bluebeetle


  1. Thanks for this Nice post, Really usefull all of us. just bookmarked this post in my digg profile, hope you will update more post soon.
    I really liked your blog!


Comments are closed.