About FrankBot
FrankBot is the crawler behind FR^NK, an SEO and AI visibility tool operated by Frank Online Ltd. If you found FrankBot/1.0 in your logs, this page explains what it was doing there. Its full user agent is:
FrankBot/1.0 (+https://frank.online/crawler)
What it reads
FrankBot fetches pages that are already accessible to the public. It presents no credentials, defeats no paywalls and submits no forms. It reads a site when one of our clients analyses their own site, their market or their competitors, and it stores findings about pages rather than mirroring sites.
The AI-visibility check
FrankBot does not send requests under other crawlers' names. To tell whether AI engines can read a site, FR^NK reads the site's robots.txt and asks the engines themselves, which reach the site from their own networks.
Opting out
The robots.txt token is FrankBot. The FR^NK free audit does not run on a site whose robots.txt carries this group, and does not open it as a competitor:
User-agent: FrankBot Disallow: /
Load
Requests are bounded per audit and retried with backoff. A typical audit fetches a few dozen pages once, not continuously.
Questions or blocking
If you want your site excluded from anything else, or have any question about a request you saw, write to us at [email protected] or through frank.online/talk, and include your domain. We add it to our exclusion list. Blocking the literal user agent string FrankBot at your edge also works.