Robots.txt Generator

Default - All Robots are:

Crawl-Delay:

Sitemap: (leave blank if you don't have)

Search Robots:	Google
	Google Image
	Google Mobile
	MSN Search
	Yahoo
	Yahoo MM
	Yahoo Blogs
	Ask/Teoma
	GigaBlast
	DMOZ Checker
	Nutch
	Alexa/Wayback
	Baidu
	Naver
	MSN PicSearch

Restricted Directories:	The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

What is robots.txt?

One of the most popular tools for bloggers and website owners is the Robots.txt Generator. The robots.txt file is one of the most important files for any website that wants to give search engines instructions on how to index their content. You can tell the search engine bots what to crawl and index and what not to crawl and index using this file. This prevents duplicate, sensitive, and unwanted content from being indexed by search engines.

Our Robots.txt Generator tool provides you with a variety of options for creating the exact robots.txt file you desire. It contains a list of the most widely used search engine bots from around the world.