Rotating proxies and managing sessions are essential techniques for avoiding detection, reducing blocks, and maintaining stable access during web scraping or automation. By controlling how and when IPs are reused, users can mimic natural behavior, maintain continuity across requests, and increase overall success rates—especially on sites with strict anti-bot protections.
## **Proxy types overview** Squid Proxies supports two proxy types: | Proxy Type | Format | Rotation Style | Authentication | Use Cases | |--------------------|--------------------------------|----------------------------|------------------------------|-------------------------------------------| | Datacenter Proxies | `IP:PORT` | Manual (user-controlled) | IP whitelisting or user/pass | High-frequency scraping, SEO audits | | Residential Proxies| `HOST:PORT` (e.g., `p102...`) | Automatic (~10 minutes) | IP whitelisting or user/pass | Localized scraping, account automation |
## **What is proxy rotation?** **Proxy rotation** is the practice of changing your IP address on a regular basis or per request to avoid: - IP-based rate limiting - CAPTCHA triggers - Session drops - Blacklist flagging Rotation can be: - **Per request** – maximizes anonymity - **Per session** – preserves state while avoiding overuse - **Per time interval or task** – balances anonymity and stability
## **Using Squid Proxies for rotation** ### **Datacenter proxies** - Each assigned proxy is static - Rotation is manual: switch IPs per request, thread, or domain - Use your full pool to distribute traffic **Sample list:** ```python proxies = [ "http://username:password@123.45.67.89:3128", "http://username:password@98.76.54.32:3128" ] ``` Use round-robin or random assignment based on your concurrency model. ### **Residential proxies** - Use a rotating hostname and port (e.g., `p102.squidproxies.com:8907`) - Rotation occurs approximately every 10 minutes - You don’t control individual IPs, but can maintain stickiness per session by reusing the port or connection **Example configuration:** ```python proxies = { "http": "http://p102.squidproxies.com:8907", "https": "http://p102.squidproxies.com:8907" } ```
## **Maintaining proxy session state** Many workflows require IP consistency during a session: - Logging in - Filling multi-step forms - Navigating dashboards - Adding items to a cart If the IP changes mid-session, servers may terminate the session or flag it as suspicious. ### **How to maintain sessions** - **Datacenter proxies**: Use the same IP for the full session - **Residential proxies**: Stick to the same connection or port for under 10 minutes **Session map pattern:** ```python session_map = {} def get_session(user_id): if user_id not in session_map: proxy = select_proxy() session = requests.Session() session.headers.update({"User-Agent": get_random_user_agent()}) session.proxies.update({"http": proxy, "https": proxy}) session_map[user_id] = session return session_map[user_id] ```
## **Proxy rotation logic patterns** | Strategy | When to Use | Notes | |----------------|----------------------------------|-------------------------------| | Round-Robin | Equal distribution of load | Simple, avoids overuse | | Random | Avoid predictable IP sequences | Good for stealth | | Time-Based | Rotate every X seconds/minutes | Mimics human behavior | | Sticky Sessions| Maintain IP for session length | Ideal for logins or cart flows|
## **Scaling proxy rotation across threads** When running concurrent threads: - Assign a dedicated proxy (or proxy + user-agent) per thread - Never share the same IP across threads at the same time - Use thread-safe proxy pools or task queues **Example with a queue:** ```python from queue import Queue proxy_queue = Queue() for proxy in proxy_list: proxy_queue.put(proxy) def worker(): proxy = proxy_queue.get() try: # perform request pass finally: proxy_queue.put(proxy) ```
## **Proxy authentication methods** Squid Proxies supports: - **IP whitelisting**: Recommended for stable network environments. You’ll authorize your device’s IP in the dashboard. Works with both datacenter and residential proxies. - **Username/password**: Ideal when using dynamic IPs, cloud servers, CI/CD pipelines, or tools that require embedded login credentials. **Auth examples:** - **Datacenter with IP whitelist** ```python proxies = {"http": "http://123.45.67.89:3128"} ``` - **Datacenter with username/password** ```python proxies = {"http": "http://user:pass@123.45.67.89:3128"} ``` - **Residential with IP whitelist (hostname format)** ```python proxies = {"http": "http://p102.squidproxies.com:8907"} ``` - **Residential with username/password** ```python proxies = {"http": "http://user:pass@p102.squidproxies.com:8907"} ```
## **Troubleshooting proxy rotation issues** | Issue | Possible Cause | Suggested Fix | |------------------------|-------------------------------|------------------------------------------------| | IP does not rotate | Using static datacenter proxy | Rotate proxy from your assigned pool | | Session drops mid-task| Rotation is too aggressive | Use sticky proxy or keepalive | | Requests blocked | Same IP used too often | Increase rotation frequency | | DNS leaks | Not routing DNS through proxy | Use DNS-over-HTTPS or proxy-aware tools | | Residential IP changed early| Long session duration | Keep tasks under 10 minutes |
## **Testing proxy rotation setup** Use these tools to verify IP rotation and session behavior: - https://httpbin.org/ip – confirms IP address - https://ip-api.com/json – checks geolocation - https://httpbin.org/headers – inspects request headers - Developer tools – monitor cookies and session continuity ```bash curl -x http://proxy:port https://httpbin.org/ip ```
## **Final recommendations for proxy rotation & session management** | Scenario | Best Practice | |----------------------------|------------------------------------------| | Login flows, carts, dashboards | Use sticky residential proxy sessions | | Large-scale scraping | Use datacenter proxies with rotation logic | | Country-specific testing | Use geo-targeted residential pools | | Complex sessions | Bind proxy + UA + cookies per user/task | | High concurrency | Isolate proxies per thread or domain |
---
**More Guides:** - [Test Your Proxies Before Integrating](https://www.squidproxies.com/developer-docs/dev-getting-started/test-your-proxies-before-integrating) - [Use Proxies in Your Code](https://www.squidproxies.com/developer-docs/dev-getting-started/use-proxies-in-your-code)