AutoApplyBot

Last updated: August 17, 2026

AutoApply is a desktop app that helps people find and apply to jobs. Part of it is a crawler, and this page is where we say plainly what that crawler does, how to recognise it, and how to tell it to go away.

Run by Paul Colombo. Questions, complaints, or "please stop fetching this": [email protected]. A real person reads that inbox.

1. How to recognise it

Every request the crawler makes sends this User-Agent:

The version number changes with each release. The AutoApplyBot product token does not, and it never will, because that is the token you write rules for.

That covers everything AutoApply fetches on its own schedule: reading job-board APIs, checking careers pages, and re-checking whether a posting we already know about is still open. It is not what you will see when a person is actually applying through the app. That runs in a real browser, sends that browser's own User-Agent, and is one person on one page, so it looks like a person because it is one. Section 4 explains why we keep those two apart.

2. What it fetches, and why

AutoApplyBot builds an index of open roles so that people looking for work can search one list instead of forty career sites. It fetches two kinds of thing:

• Public job-board APIs. The JSON endpoints applicant tracking systems publish for exactly this purpose (Greenhouse, Lever, Ashby, Workday and friends).
• Employers' public careers pages. Usually to find which job board a company uses, and to check whether a posting we already know about is still open.

It reads pages. It does not fill anything in, click anything, sign in, or submit anything. It stays on the public web: no logged-in pages, no paywalled pages, no pages behind a form.

It is also small and slow on purpose. Requests are rate limited and spread out, one company's board is checked on a rotation rather than all at once, and the whole thing runs on a schedule measured in hours. If AutoApplyBot ever shows up in your logs as a spike, that is a bug and we want to hear about it.

3. robots.txt

AutoApplyBot reads your robots.txt before it fetches your pages, and it honours what it finds. Rules addressed to AutoApplyBot win; if you have not written any, it follows your * rules like any other well-behaved crawler.

To keep it off your site entirely:

That takes effect on the next fetch, and we cache your robots.txt for a day at most. If we cannot reach it because your server is erroring or timing out, we treat that as "do not fetch" for that pass and try again later. A missing robots.txt (a plain 404) means what it has always meant: no rules, go ahead.

We do not honour Crawl-delay yet. Our own rate limiting is well under what anyone sets it to, but if that is not true for your site, email us.

4. The two commitments

There are two completely different kinds of traffic here, and lumping them together is how this conversation usually goes wrong. So, separately:

One. The crawler behaves like a crawler should. It identifies itself honestly, it reads and obeys robots.txt, it is rate limited, and there is a human address on this page you can reach. If you tell it to stop, it stops.

Two. A person applying for a job is not crawling. When someone uses AutoApply to apply, the app opens that one posting, in a real browser, because that specific person asked for that specific job right now. A human reads what got filled in and a human clicks Submit. The app never submits an application on its own, and it never will. That traffic is a person's own browser doing what they told it to do, so it is not governed by robots.txt, which is a rule for crawlers, not a rule about who is allowed to apply to your job. Pointing robots.txt at it would let a file meant to manage indexing decide who gets to apply for work, which is not what anybody wants it to mean.

5. The API exemption

The robots.txt rules above cover the crawling we do against arbitrary sites: careers pages, employer domains, anything we discovered by following a link. Requests to an applicant tracking system's own public integration API are handled differently, because that API exists to be called, and how it may be called is set by the vendor's terms rather than by a file at the root of a marketing domain.

The control there is a per-vendor policy review, and it has teeth. If a vendor's terms or robots policy says no, we switch that vendor off in the product. SmartRecruiters is currently off. The crawler no longer sources listings from SmartRecruiters boards, does not probe them, does not call their API, and does not fetch their pages, including any SmartRecruiters form embedded in someone else's careers page. People can still apply to SmartRecruiters jobs themselves, which is not something we would ever try to stop.

If you run an ATS and you would rather we used your API differently, or not at all, email us and we will do it.

6. Contact

[email protected]. Crawler questions, takedown requests, rate complaints, partnership questions, anything. Include a hostname and a rough timestamp and it makes the answer a lot faster.