I think robots.txt isn't suitable for this for the same reason it's not suitable for keeping other bots from crawling your site: adhering to what robots.txt ...
The blog post you link has a bunch of alternatives, but I agree they're not great. If there are a lot of webmasters who want to be able to noindex through ...
Old Hard to Find TV Series on DVD
... Google to explore alternatives to robots.txt. Seems like it's intended for content stealing from every place that doesn't immediately implement Google's New ...
Google to explore alternatives to robots.txt ... Welcome to r/patient_hackernews! Remember that in this subreddit, commenting requires a special ...
Google is exploring alternatives or supplemental ways of controlling crawling and indexing beyond the 30 year standard of the robots.txt ...
curl www.google.com/humans.txt Google is built by a large team of engineers, designers, researchers, robots, and others in many different ...
You are stating that Google has never acted in bad faith and that robots.txt is the only thing that Google looks at when crawling/scraping the web.
Robots.txt is used to manage crawler traffic. Explore this robots.txt introduction guide to learn what robot.txt files are and how to use them.
Yes, Google is currently developing its own version of ai.txt, which will be similar to robot.txt. This new file will provide instructions for ...
Google to explore alternatives to robot.txt. Calls for public discussion on new web standards and protocols in the wake of new AI and tech.