The robots.txt file for your WORDPRESS site: Create with the All in One SEO Pack plugin.
We go into the settings of All in One SEO Pack and activate the file editor module:
You can also download the robots.txt file via an FTP client to the root directory of your site.
Rules for robots.txt:
User-agent: Agent Type Specification (Search Engine Spiderman Crawling All Sites)
Disallow: indexing bar for admin panels, tags, search phrases, etc.
Anything that is not forbidden will be indexed.