AI51Bot
AI51Bot is the reader behind the “Try it on your website” demo on opsdom.com. If you run a website and found AI51Bot in your server logs, this page explains what it did.
Why it visited
Someone typed your website address into the demo on opsdom.com. AI51Bot read a few of your public pages so that AI51, our AI agent, could answer that person’s questions as if it worked for you. It does not visit sites on its own and it does not crawl the web.
What it does
- It asks for your
robots.txtfirst and follows it. If the file cannot be read, it reads nothing. - It reads at most five public pages of your site, one request each, and only text pages.
- It stays on your site. It does not follow a link to another website.
- It sends no cookies, signs in to nothing and submits no forms.
- It identifies itself as
AI51Botin the user-agent line of every request, together with the address of this page. - It keeps nothing. The text it reads is used for that one demo and is not stored by Opsdom.
How to keep it out
Add this to your robots.txt file. AI51Bot then reads nothing from your site:
User-agent: AI51Bot Disallow: /
You can also block only some folders, for example Disallow: /members/.
Questions
If AI51Bot did something you did not expect, tell us through the early-access form and say so in the message. We will reply to the email address you used.