The pages indexed by search engines ensure stable traffic to the resource. But if at the moment your site is under development, then the appearance of search robots on the pages of the resource is highly undesirable, since information about unfilled sections that has got into the search can deprive your resource of target visitors for a long time. To temporarily protect the site from crawling by robots, you need to prohibit it from indexing. To do this, it is enough to make a few simple changes in the resource code.
It is necessary
- - have at least basic knowledge of HTML
- - know how to open the root folder of your site's file directory
Instructions
Step 1
First, check if your site has a robots.txt file that is responsible for correctly indexing the resource. To do this, go to http: ⁄ ⁄ www site ru ⁄ robots.txt ⁄, replacing http: ⁄ ⁄ www site ru with your website address.
Step 2
If, when clicking on this link, a record of the form "User-agent: * Disallow …" opens, this means that the required file is present on your site. In this case, go to the root folder where all your site files are stored and find the robots.txt file.
Step 3
If your site's management system allows you to edit this file directly from the root folder, then open robots.txt through an auxiliary service. If it is impossible to make any changes to the file through the system interface, save the document to your computer, and then open it through the Notepad program.
Step 4
Change the first two lines of the document to the following:
User-agent: *
Disallow: /.
The inscription "User-agent: *" indicates that the following rules apply to all search robots, and "Disallow: /" means that the entire site is not indexed. After correcting the content, save the file.
Step 5
If your site does not have a robots.txt file, create a new document in Notepad. Enter the same two entries in it, placing each of them on a new line, and save the file using the "File-Save As …" command under the name robots.txt.
Step 6
Upload the created document to the root folder of your site and check its functioning by again following the link http: ⁄ ⁄ www · site · ru ⁄ robots.txt ⁄, where instead of "http: ⁄ ⁄ www · site · ru" enter the address of your resource.
Step 7
Another way to prohibit indexing is to insert special meta tags into the HTML code of site pages. In order to use this method, find in the code of one of the pages of the site the inscriptions " and place it immediately after the " line ".
Step 8
If your site is written in HTML, then this code must be inserted on every page. For a PHP resource, it is enough to place such an entry in the header.php file.
