Skip to content
RivalBrief

For website owners

About RivalBriefBot

If you saw RivalBriefBot/1.0 in your logs, it was RivalBrief checking a public page on behalf of one of our customers.

What it does

  • Reads a small number of public pages per site (at most ten), usually once a week or once a day.
  • Reads robots.txt first and obeys it, including Crawl-delay.
  • Waits several seconds between requests to the same site.
  • Never logs in, fills in forms, or gets past paywalls. Pages that ask for a password are skipped.
  • Doesn’t collect personal data or download images and files.

How to block it

Add this to your robots.txt. RivalBrief will stop checking your site at its next visit.

User-agent: RivalBriefBot
Disallow: /