How to Rotate Proxies with Python requests

If you send every scraping request through one IP, you eventually hit rate limits, CAPTCHAs, or outright bans. In Python, the fastest fix is to route traffic through a proxy layer that can rotate exit IPs without forcing you to rewrite the rest of your requests client.

This guide uses the current NinjaProxy proxy format documented in the product docs and OpenAPI spec. The important detail is that the endpoint itself stays fixed while rotation behavior is controlled through the username and the endpoint you copy from your account.

Why proxy choice matters

Rotating proxy setup is not only about hiding one IP. It changes how your client behaves under load.

• Assigned/static endpoints are useful when you need a specific IP from your account and want to cycle across a known list.
• Rotating gateways are better when you want fresh routes, sticky sessions, or country-level controls without maintaining your own pool.
• requests works with both models because it only needs a standard proxy URL in the form http://:@.

For the query "python rotating proxy", the rotating gateway is the more relevant setup, so the example below uses the current username-control format from /docs/rotating.

Step-by-step NinjaProxy setup

1. Open Portal → Rotating Gateway IPs and copy your rotating HTTP endpoint.
2. Open the credentials section in the portal and copy your portal username and per-user apiKey.
3. Keep the endpoint unchanged and add routing controls to the username only when you need them.
4. Use http://:@ for default rotation.
5. Add controls like --session-, --duration-, --provider-res, and --geo-country-us when you need sticky or geo-specific routes.

The current docs also expose a customer API endpoint at /api/v1/myProxies?apiKey=... for listing proxy rows, but you do not need that API call for the basic rotating-gateway flow shown here.

Full working Python example

The script below rotates by changing the session token between requests. If you remove the session controls, NinjaProxy can choose a new route on each request through the same rotating endpoint.

from __future__ import annotations

import itertools
from dataclasses import dataclass

import requests

ROTATING_HTTP_ENDPOINT = "<ROTATING_HTTP_ENDPOINT>"
USERNAME = "<USERNAME>"
API_KEY = "<API_KEY>"


@dataclass(frozen=True)
class RouteConfig:
    session_id: str
    country: str = "us"
    duration_seconds: int = 90
    provider: str = "res"


def build_proxy_url(config: RouteConfig) -> str:
    routed_username = (
        f"{USERNAME}"
        f"--session-{config.session_id}"
        f"--duration-{config.duration_seconds}"
        f"--provider-{config.provider}"
        f"--geo-country-{config.country}"
    )
    return f"http://{routed_username}:{API_KEY}@{ROTATING_HTTP_ENDPOINT}"


def fetch(url: str, config: RouteConfig) -> dict:
    proxy = build_proxy_url(config)
    response = requests.get(
        url,
        proxies={"http": proxy, "https": proxy},
        timeout=20,
    )
    response.raise_for_status()
    return response.json()


if __name__ == "__main__":
    targets = [
        "https://httpbin.org/ip",
        "https://httpbin.org/headers",
        "https://httpbin.org/user-agent",
    ]

    routes = itertools.cycle(
        [
            RouteConfig(session_id="python-rotate-1"),
            RouteConfig(session_id="python-rotate-2"),
            RouteConfig(session_id="python-rotate-3"),
        ]
    )

    for url in targets:
        route = next(routes)
        payload = fetch(url, route)
        print(route.session_id, payload)

Common errors and fixes

• 407 Proxy Authentication Required usually means the username, API key, or username-control syntax is wrong. Re-copy the portal credentials and verify the controls are appended to the username, not the host.
• Connection timeouts usually mean the endpoint was typed manually instead of copied from the portal. Use the exact rotating HTTP endpoint shown in your account.
• Unexpectedly sticky IPs usually mean you reused the same --session-... token. Change or remove the session token if you want a different route.
• 401 or invalid API key errors from /api/v1/myProxies mean the portal API key was regenerated. Update every script before rotating credentials again.

Relevant docs

• Python integration
• Rotating proxies
• Authentication
• API reference