Why Robots.txt File Is Important For SEO?

Today, Q-vantage is going to start new SEO topic and topic name is “Importance of Robots.txt”. On this blog we have published basic information of robots.txt file and why do we use robots.txt file for SEO?. Wihout wasting our precious time, lets start:

What is Robots.txt File?: When any search engine spider or crawler come to visit your website then it will find out one most important file on your website and that special file name is robots.txt. Your robots.txt file instructs to all search engine spiders and web robots,  which webpages of website should be crawled or indexed and which web pages of website should be ignored or not indexed in search engine.

Robots.txt file is simple txt file with no html coding and it must be uploaded in your root directory.

Site With Robots.txt File

For example:

http://www.yourwebsitename.com/robots.txt

Importance of Robots.txt file:

  • If we want search engines to ignore all duplicate pages on our website
  • If we don’t want search engines to index our internal search results pages
  • If we don’t want search engines to index certain areas of our  website or a whole website
  • If we don’t want search engines to index certain files of our website (images, PDFs, etc.)
  • If we want to tell search engines where our sitemap is placed.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s