TechSussdBot, our crawler
TechSussdBot reads public pages so the register can say what a maker says about its watches, with a link and a date. It is polite, slow and easy to turn away.
How to recognise it
It sends this user agent on every request:
TechSussdBot/1.0 (+https://techsussd.com/bot)
When it uses a research service's official API, it adds our contact address, as those services ask.
What it fetches
- Makers' product and compatibility pages, and phone software release notes, for individual facts about specific watches.
- Research databases (PubMed, Crossref, OpenAlex) through their official APIs, for study details.
- App store details only through the store's own API. It does not read Google services at all.
- Reviewers' pages only with the reviewer's written permission.
How it behaves
- It reads robots.txt first and follows it. If robots.txt cannot be reached, it fetches nothing from that site until it can.
- It honours Crawl-delay. A very long delay is taken as a request not to crawl.
- It makes one request at a time to a site, at the rate set for that source and by default no more than 6 a minute.
- It asks whether a page has changed before downloading it again, uses HTTPS only, and stops at a size cap.
- It keeps the facts it needs, a short quotation and a fingerprint of the page. It keeps a full copy only where the page's licence allows.
How to stop it
Add this to your robots.txt:
User-agent: TechSussdBot Disallow: /
It re-reads robots.txt at least every 24 hours, so the change takes effect within that time. Or email hello@techsussd.com and we will stop by hand.