Hey Bloggers, Have you heard the name robots.txt or have you added your own custom robots.txt file in your Blogger blog! Today we will discuss about Robots.txt File benefits in SEO and How To Add Custom Robots.txt file in blogger blogspot you can customize this file according to your needs.
Robots.txt is a simple text file which is used by webmasters and bloggers to instruct search engines Google, Bing, Yahoo and others about how to crawl your blog or website i.e. which pages or directory of your blog should be crawled and which should not crawled..
That means you can restrict any web page on your blog/website from web crawlers so that it can’t get indexed in search engines like your blog post, blog labels page, your demo page or any other page that is not important to get indexed. Use Custom Robots.txt Generator Tool to create robots.txt file for your Blogger blog.
https://www.blogger4ever.net
https://www.blogger4ever.net change these lines with your blog address. How To Create Blogger XML Sitemap And Submit It To Google Search Console
2. Navigate to Settings > Scroll Down at Crawlers and Indexing first Enable Custom Robots.txt
3. Now open Custom Robots.txt box.
4. Now copy above given code and paste in the box.
5. Click on Save button.That's it your done!
For Example:
https://www.yourblog.com/robots.txt
https://www.yourblog.blogspot.com/robots.txt
There are a number of third-party tools available that can help you to view and test robots.txt file. Simply, enter the URL of any blog/website you want to test and select the User-Agent to check if any file or directory is blocked for that web crawler. Technicalseo Robots.txt Validator and Testing Tool
Conclusion: Hope this article clearly explained you about Robots.txt file and How To Add Custom Robots.Txt File In Blogger Blogspot. Please share it and if you have any problem on blogger robots.txt file then feel free to ask in comment below.
You might also like: Custom Robots.txt Generator Tool For Blogger
How To Fix 404 Error Page (Not Found) In Blogger Blog
Create Blogger XML Sitemap And Submit It To Google Search Console
Robots.txt is a simple text file which is used by webmasters and bloggers to instruct search engines Google, Bing, Yahoo and others about how to crawl your blog or website i.e. which pages or directory of your blog should be crawled and which should not crawled..
That means you can restrict any web page on your blog/website from web crawlers so that it can’t get indexed in search engines like your blog post, blog labels page, your demo page or any other page that is not important to get indexed. Use Custom Robots.txt Generator Tool to create robots.txt file for your Blogger blog.
Bloggers Blog default robots.txt file:
Code
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: https://www.blogger4ever.net/sitemap.xml Sitemap: https://www.blogger4ever.net/sitemap-pages.xml
https://www.blogger4ever.net change these lines with your blog address. How To Create Blogger XML Sitemap And Submit It To Google Search Console
How To Add Custom Robots.txt File To Blogger Blogspot
1. Go to your blogger Dashboard2. Navigate to Settings > Scroll Down at Crawlers and Indexing first Enable Custom Robots.txt
3. Now open Custom Robots.txt box.
4. Now copy above given code and paste in the box.
5. Click on Save button.That's it your done!
How To Check Your Robots.txt File in Blogger Blogspot
You can check this file on your blog by adding /robots.txt at the end of your blog URL in the web browser. Once you visit the robots.txt file URL you will see the entire code which you are using in your Custom Robot.txt file in your blogger.For Example:
https://www.yourblog.com/robots.txt
https://www.yourblog.blogspot.com/robots.txt
How To View & Test Your Robots.txt File
You can view and test the robots.txt file of your Blogger blogspot using Google Robots.txt Tester Tool If there are any syntax warnings and logic errors, Google will display them. Enter your Blogger URL and test if it is blocked from any Google web crawler.There are a number of third-party tools available that can help you to view and test robots.txt file. Simply, enter the URL of any blog/website you want to test and select the User-Agent to check if any file or directory is blocked for that web crawler. Technicalseo Robots.txt Validator and Testing Tool
Conclusion: Hope this article clearly explained you about Robots.txt file and How To Add Custom Robots.Txt File In Blogger Blogspot. Please share it and if you have any problem on blogger robots.txt file then feel free to ask in comment below.
You might also like: Custom Robots.txt Generator Tool For Blogger
How To Fix 404 Error Page (Not Found) In Blogger Blog
Create Blogger XML Sitemap And Submit It To Google Search Console
