AtlasCatalogBot, a product catalogue crawler
Did you see AtlasCatalogBot in your server logs? This page explains who operates it, what it reads, how it behaves and how to have crawling stopped or content removed.
How to recognise it
The crawler always identifies itself with this user agent. It never pretends to be a browser.
AtlasCatalogBot/0.1 (+https://ecommasterhq.com/robot; [email protected])
The version number may change; the name AtlasCatalogBot, the address of this page and the contact address stay the same.
Who operates it
AtlasCatalogBot is operated by the team that runs a group of European online shops, whose internal management tool is hosted on ecommasterhq.com.
Contact: [email protected].
Why
The crawler reads product catalogues published online, for market analysis and to prepare the catalogues of our shops. The information is kept internally, together with its source.
- Nothing is published automatically: a person reviews every product page before it goes online.
- Customer reviews are never republished; they are only used for internal analysis, without their authors' names.
What is collected
- Public product and category pages, sitemaps and the
robots.txtfile. - Public images and documents attached to product pages (photographs, data sheets).
Never requested: carts, customer accounts, login and payment pages, or any page that requires authentication.
How it behaves
robots.txtis obeyed. The file is read before any other request; a page it disallows is not requested.- The request rate adapts to your site. The crawler starts at one request per second and only speeds up while the site responds normally: at most 4 pages per second per site in the usual setting, never more than 8, and at most 8 images per second.
- It slows down by itself. A 429, 403 or 503 response, or a response time that doubles, halves the rate and triggers a pause. If refusals keep coming, crawling of the site is suspended.
Retry-Afteris obeyed: the crawler waits for the requested delay before any new request.Crawl-delayis obeyed whenrobots.txtdeclares it: that delay then replaces the rate described above.- No login to any account, no form submitted, no cookie kept.
- No circumvention: when it meets an access refusal, a verification page or a CAPTCHA, the crawler stops. It does not try another way.
Limiting or blocking the crawler
The crawler follows the robots.txt rules that name it. To block it from your whole site:
User-agent: AtlasCatalogBot
Disallow: /
To slow it down, set a delay in seconds between two requests:
User-agent: AtlasCatalogBot
Crawl-delay: 10
Asking for crawling to stop or for removal
Write to [email protected] and tell us:
- the site concerned;
- your request: stop crawling, or remove content;
- for a removal, the address of the pages or images concerned.
Every request is read and handled by a person. An image whose removal is requested is removed and is no longer collected.
Cookies and local storage
This page sets no cookie and uses no audience measurement or advertising tool.
If you close the information message shown at the bottom of the page, your browser remembers that for six months in its local storage (key atlas-info-cookies, which only holds the date). Nothing is sent to a server.
Last updated: 4 October 2026.