The robots
https://simontgyl770.tearosediner.net/wordpress-web-development-resources-every-developer-needs
The robots.txt dossier guides seek engine crawlers on which pages to index or now not, making sure that merely important content material receives prioritized in seek outcome.