Direct Answer
To scrape AutoScout24 car-market data cleanly, request only public listing pages at a slow, human-like cadence, honour the site's robots.txt, and route through one dedicated German mobile IP with sticky sessions. Aggressive bursts trip the site's bot management and rate limits within a few requests; a polite client and a well-reputed IP reduce blocks far more reliably than any evasion attempt.
What "Car-Market Data" Actually Means Here
German used-car marketplaces like AutoScout24 and mobile.de hold the richest public dataset on European vehicle pricing: make and model, year, mileage, fuel type, transmission, asking price, and the seller's region. For a data engineer, that is a live signal on how residual values move by segment and location — useful for pricing models, market research, and dealer-side analytics.
The data you want is the public listing data anyone can see without logging in. That distinction matters: this article is about collecting publicly visible listing facts at a respectful rate, not about accessing anything behind an account or storing the personal details of private sellers. Public prices and specs, yes; personal data, no. That boundary is both good engineering and good GDPR practice.
Why AutoScout24 Blocks Aggressive Scrapers
These sites sit behind enterprise-grade bot management (the same class of protection as Akamai and Cloudflare). It does not look for a single "bot" flag — it scores every request on several signals at once and blocks when the combined score crosses a threshold. Three signals do most of the work.
1. IP reputation
The first thing the edge sees is your IP, and it already has an opinion about it. Datacenter ranges from the big cloud providers are known, published, and heavily associated with automated traffic — so they start with a poor score before you send a single header. This is why datacenter proxies get blocked on protected sites almost on sight. A real German mobile IP, by contrast, is an address a genuine shopper could be using, so it carries the IP reputation of an ordinary person, not a server farm.
2. Request rate
Bot management watches how fast requests arrive from one source. A human browsing cars opens a page, reads it, scrolls, and clicks a few times a minute. A naive scraper fires dozens of requests per second. That burst is the clearest tell there is, and it triggers rate limiting — the 429 Too Many Requests response — long before any fingerprinting comes into play. Slow down and this signal disappears.
3. Consistency across the session
The edge also checks whether your requests hang together: does the IP stay stable within a browsing session, do headers stay consistent, does the traffic pattern resemble a person moving through the catalogue. Jumping IP on every request, or mismatched headers, reads as automation. This is where sticky sessions matter — holding one IP for the length of a logical browsing session looks exactly like one shopper working through results.
The honest framing: you are not defeating this system, and you should not try to. You are lowering the two or three signals that make you look like an attack — reputation and rate above all — so that your polite, public-data collection isn't mistaken for something it isn't.
The Clean Approach, Step by Step
Step 1 — Read robots.txt first
Before writing any request code, fetch autoscout24.de/robots.txt (and mobile.de/robots.txt) and read which paths are disallowed. Treat it as binding. If a section is off-limits, stay out of it — collect only from paths that are open to crawlers. This single habit keeps you on the defensible side of the line and out of the areas most aggressively defended.
Step 2 — Set a slow, polite cadence
Pace your requests like a person, not a pipeline. A few pages per minute per session, with randomised pauses between them, is the right order of magnitude. Add exponential backoff: if you receive a 429 or a challenge, stop, wait longer, and resume slower — never retry harder. The goal is to stay comfortably under whatever the current limit is rather than to find it.
Step 3 — Use one clean dedicated German mobile IP
Route the collection through a single dedicated German mobile connection rather than a datacenter IP or a shared pool. Because AutoScout24 is a German site, a German mobile IP is the region-true, low-suspicion address a real local buyer would have. A dedicated endpoint means the reputation is yours alone — you are not inheriting the behaviour of strangers sharing a pool, which is a common reason a dedicated proxy beats a shared one for a sustained job. Fewer suspicious signals also means fewer challenges: clean mobile IPs reduce CAPTCHAs rather than fighting through them.
Step 4 — Hold sticky sessions per crawl unit
Keep the same IP for the duration of one logical browsing session — say, paging through one search result set — using a sticky session, then rotate on demand when you move to the next unit of work. This mirrors how a real shopper behaves: they don't teleport to a new network between page two and page three. Rotation is a tool for separating jobs, not a per-request reflex.
Step 5 — Extract only what you need, store it responsibly
Pull the structured facts — model, year, mileage, price, region — and leave the rest. Don't scoop up seller names, phone numbers, or anything that identifies a private individual. Collecting minimal, public, non-personal fields keeps the dataset clean, smaller, and firmly within GDPR expectations.
A Quick Comparison of IP Choices
| IP type | Reputation on a German site | Blocks under load | Fit for this job |
|---|---|---|---|
| Datacenter | Known server range, poor | Fast, often on sight | Weak |
| Shared residential/mobile pool | Mixed — inherits pool behaviour | Variable, unpredictable | Fair |
| Dedicated German mobile | Region-true, clean, yours alone | Lowest at a polite rate | Strong |
The pattern is consistent: the closer your IP is to a real German consumer connection, and the more it belongs to you alone, the less the site's defences have to react to. Pair that with a slow cadence and most blocking pressure simply never builds.
What Not to Do
- Don't hammer the site to find its rate limit. Probing for the ceiling is the fastest way to burn an IP's reputation. Stay well under it.
- Don't rotate IP on every request. It looks less human, not more, and breaks the session consistency the edge expects.
- Don't touch data behind a login or personal seller details. Public listing facts only.
- Don't ignore a challenge or a 429. Back off and slow down; treating a soft block as noise turns it into a hard one.
None of this is about hiding what you're doing. It's about being a well-behaved client on a public site — which is both the compliant path and, in practice, the one that keeps working.
FAQ
Is it legal to scrape AutoScout24 listings?
Collecting publicly visible listing data — model, price, mileage, region — is generally defensible when you respect the site's robots.txt and terms, request at a reasonable rate, and avoid storing personal data of private sellers. Legality depends on jurisdiction and how you use the data, so treat this as engineering guidance, not legal advice. See our overview of whether web scraping is legal in the EU.
Why does AutoScout24 block my scraper so fast?
German car marketplaces run enterprise-grade bot management that scores each request on IP reputation, request rate and header consistency. A datacenter IP firing dozens of requests per second is flagged almost immediately. Slowing down and routing through a clean German mobile IP removes the two signals — poor reputation and burst rate — that trip blocks first.
How many requests per minute is safe on AutoScout24?
There is no published number, and it shifts with site load and detection tuning. The safe approach is to stay well below what a fast human could do — a few pages per minute per session, with pauses — rather than probing for the exact limit. Polite pacing keeps your IP reputation intact and avoids 429 responses.
Related guides
- Scrape ImmobilienScout24 Real-Estate Market Data Responsibly
- Scrape OTTO.de and Zalando Data with a German Mobile Proxy
- Price-Monitoring Scraper with a German Mobile Proxy
- The Web-Scraping Use Case: Real German 5G Mobile IPs
CTA
Building a German car-market dataset the clean way? Run it through one dedicated German mobile IP with sticky sessions from a transparent EU-based provider — browse the German mobile proxy plans.
Get a German mobile proxy: Real 4G/5G IPs from Germany, fair-use 20 GB/day — see German 4G/5G mobile proxy options or try a shared 5G German proxy.