Posts Tagged ‘robots.txt’

What's robots.txt? Learn some more

Well, we all have been implementing robots.txt in our website to block search engines to crawl our private pages. But sometimes search engines crawl that page and index them. So, what can be the cause. Here, in this video Matt Cutts is describing how Google handles robots.txt.

Powered by WordPress | Get Cheap or Free T-Mobile Cell Phones Online. | Thanks to Best Savings Accounts, Credit Cards and Reverse lookup