The BacklinksBot crawler

If you found this address in your server logs, our crawler visited your site. It identifies itself with this user agent:

Mozilla/5.0 (compatible; BacklinksBot/1.0; +https://backlinksbot.com/bot)

What it does

• Reads public pages and sitemaps to build a link index: which pages link to which sites.
• Opens a page a customer is pitching to read its topic and find a published contact page.
• Checks that a link agreed with a site owner is live on the page, and later that it is still there.

It reads only public HTML. It does not fill in forms, sign in, post comments or download files beyond the page itself.

Rules it follows

• It honours robots.txt rules for * and for BacklinksBot.
• It reads at most two pages of a site at a time, pauses between batches and stops when your server answers "too many requests" or "service unavailable".
• It does not crawl private networks or addresses that resolve to them.

How to block or limit it

Add this to your robots.txt to block it from the whole site:

User-agent: BacklinksBot
Disallow: /

Or block only part of the site by listing the paths you want excluded. Changes are picked up the next time the crawler reads your robots.txt.

Questions

Write to support@backlinksbot.com with your domain and we will answer within one business day.