Sometimes a webmaster needs to remove one of the indexed pages from the search engine. Most often, this operation is resorted to after an erroneous entry of the page address into the general list of the sitemap. At one time, specialists from the Megafon company encountered this error (sms messages became available to any user of the Yandex search engine).
It is necessary
Personal site
Instructions
Step 1
One of the easiest ways to remove your own web page from the archives of search engines is to physically delete it, change its location, and falsely delete it (you need to set the attribute of the deleted page). After transforming this page, the search robot will see the following line instead of the content: HTTP / 1.1 404 Not Found. However, do not forget that search robots can visit the site every 3 hours, and maybe once every 2-3 days. Therefore, you need to wait a while to get the result.
Step 2
The next way is to edit the robots.txt file, which determines the path of the crawler as soon as it comes to your site. This text document always has one location - the root of the site. In the first paragraph, the indexing parameters for the Yandex robot are usually indicated (it differs markedly from other robots), in the second paragraph for all other search engines.
Step 3
At the beginning of the paragraph, you must specify the title of the agent "User-Agent: *" and the addresses of the pages to be hidden - "Disallow: /wp-content/foto/fotojaba.html". In the same way, you must specify the addresses of pages or sections that you want to close from indexing. Please note that this method does not provide quick results. If your site has low activity and the news is not broadcast on social networks, the processing of new data can reach a period of several days. In addition, you will need to delete versions of these pages from the archive of the search service.
Step 4
An alternative method for setting links in the robots.txt file is to use the meta robots tag of the same name. The syntax for this tag is as follows: it must be placed between the paired [head] and [/head] tags. The robots value should be placed in the meta name tag. An example would look like this:.
