The Robot Exclusion Standard also known as the Robots Exclusion Protocol or robots.txt protocol. By default search engine frequently fetching the content from your site and show in search result. To avoid your site or some pages from the search engine, you can use robots.txt file. Why robots.txt file This robots.txt text file contains the instructions in a specific […]
