If your scraper keeps hitting 429 errors or getting soft-banned after a few hundred requests, the problem probably isn't your code. Proxy pool size scraping strategy is almost always the culprit. A pool that's too small means the same IPs rotate back too quickly, and even the most forgiving target site will flag you. In this guide, you will learn exactly what proxy pool size means, how to calculate the right number of IPs for your workload, why mobile proxies give you a structural advantage, and how to stop wasting money on proxies that don't rotate fast enough to matter. Specifically, we'll cover:
- What proxy pool size actually means in practice
- How to calculate the minimum pool size your project needs
- Why mobile 4G IPs outperform datacenter pools of the same size
- Configuration tips to squeeze more performance out of every IP
What Is Proxy Pool Size in Scraping?
Proxy pool size refers to the total number of unique IP addresses available to your scraper at any given time. Think of it as the deck of cards you shuffle through when sending requests. If your deck only has five cards and you're playing a game that penalizes repeated cards, you'll run out of safe plays very quickly.
But pool size isn't just a headcount. It's the relationship between three variables: the number of IPs you have, the rate at which you send requests, and how long each IP stays "warm" before a site starts treating it with suspicion. A pool of 100 IPs sending 10 requests per second is effectively a much smaller pool than 100 IPs sending 1 request per minute.
Static Pools vs. Dynamic Pools
A static pool assigns you a fixed list of IPs that don't change unless you manually swap them. A dynamic pool rotates IPs automatically, pulling from a larger shared or dedicated reservoir. For most scraping workloads, dynamic rotation wins. But here's the catch: if that dynamic pool is still pulling from only 50 real IPs, the "rotation" is mostly theater.
- Static pool: predictable, easier to allowlist, but burns out fast under heavy load
- Dynamic pool: better for high-volume scraping, but only valuable if the underlying IP inventory is genuinely large
- Dedicated port with API rotation: the middle path, one port but fresh IP on demand via a single API call
Key takeaway: Pool size is meaningless without context. What matters is the ratio of unique IPs to request volume over time.
How Pool Size Directly Affects Scraping Success Rate
Every target website, whether it's an e-commerce store, a search engine, or a social platform, tracks request patterns per IP. When the same IP appears too often in a short window, it triggers rate limiting, CAPTCHAs, or outright blocks. This is why proxy pool size scraping decisions have a direct, measurable impact on your success rate.
In our testing across e-commerce targets like Allegro and price comparison sites, scrapers using pools smaller than 20 IPs for 500+ requests per hour saw block rates above 60%. The same request volume distributed across 200+ IPs dropped block rates to under 5%. That's not a marginal improvement. It's the difference between a working scraper and a broken one.
The Cool-Down Problem
Even if you rotate IPs, you need to wait long enough before reusing each one. If your pool has 10 IPs and you rotate every 10 seconds, each IP gets reused every 100 seconds. For most sites, that's too frequent. A pool of 100 IPs rotating every 10 seconds means each IP waits 1,000 seconds before reuse. Much safer.
- Google Search: recommends at least 200 unique IPs per 10,000 queries
- Instagram: extremely sensitive, needs IP-level session consistency plus pool depth
- Amazon product pages: tolerant of moderate volume but bans aggressively on CAPTCHA failure
- Nike SNKRS: burst-sensitive, benefits from pools with instant IP refresh
So the formula isn't just "more IPs = better." It's about matching pool depth to your request cadence and the target's detection sensitivity.
How to Calculate the Right Pool Size for Your Project
You don't need to guess. There's a simple formula scraping engineers use to size their proxy pools. Start with these inputs:
- Total requests per hour (R): How many HTTP requests does your scraper send per hour?
- Safe requests per IP per hour (S): How many requests can one IP safely make before risk of detection? This depends on the target. For Google, it might be 20. For a generic e-commerce site, it might be 100.
- Minimum pool size = R / S
Example: you scrape 5,000 product pages per hour from a mid-sensitivity target where 50 requests per IP per hour is safe. You need at least 100 IPs in your pool. Add a 30% buffer for failures and retries, and you're looking at 130 IPs as your working minimum.
Buffer for Failures and Retries
Real scrapers don't run clean. You'll get timeouts, partial responses, and occasional bans that pull IPs out of rotation. Always add at least 20-30% to your theoretical minimum. And track your actual block rate. If it climbs above 5%, your pool is probably too small for your current request rate.
Key takeaway: Calculate pool size from your request volume and per-IP request tolerance, then add a safety buffer. Don't just buy "a proxy package" without doing this math first.
Datacenter vs. Mobile Proxies: Pool Size Is Not Equal
Here's something most guides skip over. A pool of 100 datacenter IPs and a pool of 100 mobile IPs are not equivalent, even if they cost the same. Mobile IPs, especially real 4G LTE addresses from residential carrier networks, carry far more trust with target websites than datacenter IPs.
Why? Because websites use IP reputation scoring. Datacenter IP ranges are well-known and heavily flagged. Many sites block entire ASN ranges from AWS, DigitalOcean, or Hetzner on sight. Mobile carrier IPs, on the other hand, are the same addresses used by millions of real smartphone users. Blocking them means blocking real customers. Websites are much more conservative about rate-limiting mobile IPs.
The CGNAT Advantage
Mobile networks use CGNAT (Carrier-Grade NAT), which means thousands of real users share the same public IP address. When a website sees a request from a mobile carrier IP, it can't easily distinguish you from the crowd. This structural anonymity means your effective "safe request threshold" per IP is dramatically higher on mobile than on datacenter.
In practice, you can often send 3-5x more requests per mobile IP before hitting detection thresholds compared to a datacenter IP. This means a pool of 20 real mobile 4G IPs can outperform a pool of 100 datacenter IPs for many scraping targets. That changes the economics completely.
- Datacenter IPs: high volume, low trust, need very large pools to compensate
- Residential IPs: better trust, but often sourced from botnet-style networks with ethical concerns
- Mobile 4G IPs: highest trust, CGNAT coverage, ideal for sensitive targets with moderate volume needs
You can verify how target sites see your IP using our IP detection tool to confirm whether your proxy appears as a mobile carrier address.
Rotation Strategy: Making Every IP in Your Pool Count
Pool size matters less if your rotation strategy is poor. Even a large pool will burn out if you rotate IPs incorrectly. The goal is to space out requests per IP, maintain session continuity where needed, and trigger rotation at the right moment.
When to Rotate
There are two main rotation triggers you should configure:
- Time-based rotation: change IP every N minutes regardless of request count. Good for passive monitoring or crawls where request frequency is low.
- Request-based rotation: change IP after every N requests. Better for high-volume scraping where you need precise control over per-IP exposure.
For scraping sensitive platforms like Instagram or Google, session-based rotation is often necessary: keep the same IP for an entire user session (login, browse, extract), then rotate between sessions. Rotating mid-session is a red flag for most detection systems.
API-Triggered Rotation on Mobile Proxies
With Proxy Poland's infrastructure, you can trigger a new IP on your dedicated 4G modem port with a single API call. The IP changes in under 2 seconds. This means you can implement precise request-based rotation without managing a large pool of separate proxy endpoints. One port, one API call, fresh IP. Your scraper stays connected to the same host and port while the underlying IP changes on the carrier network.
This approach simplifies your scraper architecture significantly. Instead of maintaining a list of 200 proxy endpoints and load-balancing across them, you maintain one connection and rotate on demand. You can also check for DNS leaks during setup using our DNS leak test to make sure your real IP never slips through.
Key takeaway: Rotation strategy multiplies the effective value of your pool. A smaller pool with smart rotation often beats a large pool with naive round-robin cycling.
Proxy Pool Size for Specific Use Cases
Different scraping projects have different pool size requirements. Here's a practical breakdown based on real workloads.
E-commerce Price Monitoring
You're hitting 10-20 product pages per SKU, across thousands of SKUs, probably a few times per day. Request volume is moderate but spread across time. A pool of 10-30 mobile IPs with hourly rotation is usually sufficient. The goal is consistency and low latency, not raw IP count.
SERP Scraping (Google, Bing)
Search engines are among the most aggressive about IP bans. For Google, you need a minimum of 1 unique IP per 15-20 queries, which means a pool of at least 50-100 IPs for any meaningful volume. Mobile IPs dramatically reduce ban rates here. Tools like Semrush and Ahrefs scrapers work best on pools of 100+ mobile IPs with sub-30-second rotation.
Social Media Automation
Instagram and similar platforms tie accounts to IPs. One IP per account is the safest configuration. Pool size here equals the number of accounts you manage. Losing IP-account consistency mid-session triggers immediate flags. Use session-locked rotation and confirm your proxy speed with our proxy speed test to avoid slow connections that trigger timeout-based detection.
Sneaker Bots
Drop events are burst-heavy. You'll send hundreds of requests in a narrow time window. Here, pool size and rotation speed matter more than almost anything else. Mobile IPs with 2-second rotation give you a fresh IP for each checkout attempt without needing hundreds of separate proxy subscriptions.
Conclusion
Getting proxy pool size scraping right isn't about buying the biggest package you can find. It's about understanding the relationship between your request volume, your target's detection sensitivity, and the trust level of the IPs you're using. A small pool of high-trust mobile 4G IPs with smart rotation will outperform a large datacenter pool on almost every meaningful metric. Calculate your minimum pool size from your request rate, add a buffer, and match rotation strategy to your use case. Whether you're scraping product prices from Allegro, tracking SERPs, or managing social accounts, the math is the same: more trust per IP means you need fewer IPs to do the job safely. Proxy Poland runs real LTE 4G/5G modems on Polish carrier networks, giving you CGNAT-backed mobile IPs with 2-second rotation and unlimited bandwidth, starting from a free 1-hour trial.
Ready to stop burning IPs and start scraping reliably? See Proxy Poland plans and start your free trial today.