10% off ALL new Proxies – CODE: 10pro

Price-Monitoring Scraper with a German Mobile Proxy

Price-Monitoring Scraper with a German Mobile Proxy

Direct Answer

To monitor public German e-commerce prices reliably, drive a headless browser (Playwright) through a German mobile proxy so pages render the region-true prices a local shopper sees, then extract and store the values on a schedule. The mobile IP keeps runs from stalling where datacenter ranges get blocked, and a sticky session holds one IP through multi-step pages.


Why Prices Need a German Mobile IP

Prices, currency, availability and shipping on German e-commerce pages depend on where the request appears to come from. A datacenter IP often gets a distorted view — or no view at all, once it's restricted. A real German mobile IP (shared across many real subscribers via CGNAT) returns the same region-true page a German shopper sees, so your dataset is decision-ready rather than misleading.

This walkthrough collects public, non-personal price data for legitimate market research. It is not a way around access controls — see the Scrape Responsibly section below.


What You'll Need

  • Python with Playwright installed (pip install playwright then playwright install chromium).
  • A German mobile proxy endpoint: host, port, username, password from your GermanProxy.io account.
  • A target page that's publicly accessible (the example uses https://www.example.de/product).

Step 1 — Route Playwright Through the Mobile Proxy

Pass the endpoint to Playwright's proxy option so every page render exits through the German mobile IP:

from playwright.sync_api import sync_playwright

PROXY = {
    "server": "http://PROXY_HOST:PORT",
    "username": "USER",
    "password": "PASS",
}

with sync_playwright() as p:
    browser = p.chromium.launch(proxy=PROXY, headless=True)
    page = browser.new_page()
    page.goto("https://www.example.de/product", wait_until="domcontentloaded")
    price = page.locator(".price").first.inner_text()
    print("Price:", price)
    browser.close()

The full proxy reference is in our Python (requests & Playwright) guide.


Step 2 — Hold One IP for Multi-Step Pages

If a price sits behind a region or cookie prompt, you'll click through several steps. Use a sticky session so the IP doesn't change mid-flow — a changed IP can reset the page state. Keep one browser context for the whole sequence, and rotate only between products. The concepts are in sticky sessions and IP rotation.


Step 3 — Extract and Store

Pull the fields you actually need — price, currency, availability — and write them with a timestamp:

import csv, datetime, os

row = {
    "url": "https://www.example.de/product",
    "price": price,
    "checked_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
}
write_header = not os.path.exists("prices.csv")
with open("prices.csv", "a", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=row)
    if write_header:
        writer.writeheader()
    writer.writerow(row)

Run it on a schedule (cron, a task runner, or a workflow tool like n8n) to build a price history over time.


Step 4 — Verify the German IP

Before a long run, confirm the proxy is in the path: load an IP-echo page in the same context and check it returns a German mobile IP. If it shows your own address, re-check the proxy block. See test your mobile proxy setup.


Scrape Responsibly

This is for legitimate, public-data collection — not for circumventing access controls or platform rules:

  • Respect robots.txt and each site's terms of service.
  • Throttle with delays; don't hammer a server.
  • Collect only public, non-personal data, in line with the GDPR.
  • Identify your client honestly in the User-Agent.

Scraping public data can be lawful, but rules vary by site and country — see is web scraping legal? EU & GDPR.


Takeaway

The code is the easy part; the IP decides whether the run completes and whether the prices are region-true. A German mobile proxy gives Playwright a genuine German vantage point, with predictable fixed-price cost instead of per-GB billing.


Get a German mobile proxy: Real 4G/5G IPs from Germany, fair-use 20 GB/day — compare German mobile proxy plans or start with a dedicated 5G proxy.

Telegram Support
Nach oben scrollen