How to Set Up Selenium Proxy: Complete Guide
Rohit Rajpal
Learn how to configure unauthenticated and authenticated proxies in Selenium WebDriver with examples, use cases, and best practices.
A proxy server acts as a middle layer between Selenium-driven browsers and the websites they access. It decides how requests leave your system and how responses come back. In testing and scraping, proxies are used to access geo-restricted sites, rotate IP addresses to avoid blocking, or control network traffic for debugging.
Selenium allows proxy integration, but the setup is not universal. Chrome, Firefox, and other browsers require different configuration methods. The approach also changes when working with proxies that need authentication compared to those that do not.
This guide explains how Selenium proxy works, when to use it, and how to configure it correctly with practical steps and best practices.
Understanding Selenium Proxy: Definition and Use Cases
In Selenium, a proxy server is an external service that sits between the WebDriver browser and the destination website. Instead of the browser connecting directly, all traffic passes through the proxy. This gives testers and developers more control over how requests are routed and how responses are received.
Here are the main scenarios where Selenium proxy is used:
- Geo-specific testing: Run tests from a particular country or city to verify region-based features, pricing, or content availability.
- Web scraping at scale: Rotate IP addresses through a proxy pool to reduce the risk of being blocked by target sites.
- Traffic inspection: Capture and analyze request–response data to debug headers, cookies, or authentication issues.
- Access behind firewalls: Reach internal or restricted systems by routing traffic through a corporate proxy.
- Load simulation: Distribute test traffic across multiple IPs to mimic real-world user activity patterns.
Benefits of Using Selenium Proxy for Web Scraping
Web scraping often involves sending a large number of automated requests to the same site. Many websites track request volume, IP reputation, and geographic origin to detect bots. A proxy helps overcome these limits by controlling how requests appear to the target server.
Here are the key benefits of using a proxy with Selenium for scraping:
- Avoiding IP bans: Sending repeated requests from a single IP can trigger blocking. Rotating through a proxy pool keeps requests distributed and lowers the chance of detection.
- Accessing region-restricted content: Some sites serve different data depending on location. A proxy allows Selenium to appear as if it is browsing from the required region.
- Stabilizing long-running scrapers: Proxies help manage connection reliability. If one route becomes unstable, traffic can switch to another without breaking the scraping session.
- Managing request headers and cookies: Many proxy providers allow control over headers, cookies, and user-agent strings. This adds flexibility for scraping complex sites that require session handling.
- Scaling operations: Large-scale scraping setups depend on proxy management. With Selenium, proxies ensure scripts can run for extended periods without interruption.
Selenium Proxy vs Regular Proxy: Key Differences
A regular proxy manages traffic at the system or browser level. A Selenium proxy works inside automated sessions, which introduces extra steps for configuration, authentication, and scaling.
The table below compares Selenium Proxy and Regular Proxy in detail:
Aspect | Regular Proxy | Selenium Proxy |
Setup | Configured once in the browser or OS, applies to all traffic | Passed as WebDriver capabilities or options, applies per automated session |
Session Control | Static, stays active until changed manually | Dynamic, can be assigned or switched during test execution |
Authentication | Credentials often stored in system or browser settings | Handled in code using username/password or browser extensions |
Integration with Workflows | Independent of automation, used for manual browsing or system-wide traffic | Tied directly to Selenium scripts, part of test or scraping workflows |
Debugging and Logging | Requires external tools like Fiddler or Wireshark | Can capture request–response data within automation for detailed debugging |
Flexibility | One proxy applied across all browser activity | Different proxies can be set for different test cases or parallel sessions |
Performance Impact | Minimal for casual browsing | Can affect execution speed if switching proxies frequently in large test runs |
Scalability | Not designed for handling multiple automated sessions | Supports proxy pools and rotation for large-scale scraping or distributed tests |
Error Handling | Errors usually show up as failed connections in browser | Requires explicit exception handling in code to retry with another proxy |
How to Implement Proxy Authentication in Selenium WebDriver
Selenium does not provide a built-in way to supply proxy credentials. The approach depends on whether the proxy requires authentication. If no credentials are needed, you only configure the proxy address in the browser options.
If authentication is required, you must use additional methods such as browser extensions, local intercepting proxies, or libraries that can inject headers.
Configuring Unauthenticated Proxy Servers in Selenium
When a proxy does not need credentials, setup is simple. You just point the browser to the proxy through WebDriver options or a browser profile.
Chrome (Python example):
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("--proxy-server=http://proxy.example.com:3128")
driver = webdriver.Chrome(options=opts)
driver.get("https://api.ipify.org/?format=json")
print(driver.page_source)
driver.quit()
Firefox (Java example):
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.http", "proxy.example.com");
profile.setPreference("network.proxy.http_port", 3128);
FirefoxOptions options = new FirefoxOptions();
options.setProfile(profile);
WebDriver driver = new FirefoxDriver(options);
driver.get("https://api.ipify.org/?format=json");
driver.quit();
These setups work for basic routing. Always verify by loading an IP-checking site to confirm traffic is routed through the proxy.
Configuring Authenticated Proxy Setup for Selenium
When a proxy requires a username and password, WebDriver cannot handle the login prompt directly. You need a workaround that passes credentials before the request reaches the proxy.
Two practical methods are shown below.
1. Chrome with Extension (Python example):
Use a small Chrome extension that injects the Proxy-Authorization header. The extension is loaded when the browser starts, and the proxy server is specified in the options.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_extension('/path/to/proxy_auth_extension.crx')
opts.add_argument("--proxy-server=http://proxy.example.com:3128")
driver = webdriver.Chrome(options=opts)
driver.get("https://api.ipify.org/?format=json")
driver.quit()
2. Firefox with Extension (Java example):
Firefox can also load extensions at startup. The extension adds the required auth header, while the profile handles the proxy server configuration.
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("network.proxy.type", 1);
profile.setPreference("network.proxy.http", "proxy.example.com");
profile.setPreference("network.proxy.http_port", 3128);
FirefoxOptions options = new FirefoxOptions();
options.setProfile(profile);
options.addExtensions(new File("/path/to/proxy_auth_extension.xpi"));
WebDriver driver = new FirefoxDriver(options);
driver.get("https://api.ipify.org/?format=json");
driver.quit();
These setups bypass the authentication pop-up by injecting credentials before requests leave the browser. They work reliably in headed mode, but you should test them in headless mode since extension APIs can behave differently.
Best Practices for Selenium Proxy Configuration
A proxy in Selenium is not only about routing traffic. If configured poorly, it can slow tests, leak credentials, or cause sessions to fail unpredictably. To avoid these issues and keep tests reliable, apply the following practices.
- Use isolated proxies per test session: Assign a dedicated proxy to each parallel browser to prevent race conditions, connection reuse, or provider rate limits from affecting multiple tests.
- Validate proxy routing at startup: Load an IP verification endpoint after the driver launches to confirm the browser is using the expected proxy and not bypassing it for some requests.
- Rely on provider authentication features: Prefer tokens, API keys, or IP allowlisting offered by proxy providers instead of embedding credentials in code or extensions.
- Protect sensitive information: Store proxy credentials in a secrets manager or CI vault and inject them at runtime rather than hardcoding them in test scripts.
- Manage HTTPS interception properly: If using an intercepting proxy, install its root certificate in the browser trust store to avoid TLS handshake errors and test instability.
- Implement resilience in automation: Add retry logic with exponential backoff for proxy failures, and rotate to a backup proxy after repeated connection drops or 407 responses.
- Monitor and measure proxy performance: Track latency, request failures, and throughput per proxy endpoint, then feed metrics into your monitoring system to detect degradation early.
Conclusion
Using a proxy with Selenium lets you route requests through specific servers, test region-based content, and avoid network restrictions. The setup process varies between unauthenticated and authenticated proxies, and each requires the right configuration to work reliably.
To check that these setups behave correctly in real user environments, you should run them on actual browsers and devices. BrowserStack provides a cloud testing platform with over 3,500+ real browser and device combinations, which makes it possible to confirm that proxy configurations hold up across environments and match real-world conditions.
Written by
Rohit Rajpal
Rohit Rajpal is a B2B Content Strategist at BrowserStack, helping SaaS brands build AI-first strategies that drive authority and revenue. He writes about content, strategy, and the future of search in the zero-click era.
Contents
- Understanding Selenium Proxy: Definition and Use Cases
- Benefits of Using Selenium Proxy for Web Scraping
- Selenium Proxy vs Regular Proxy: Key Differences
- How to Implement Proxy Authentication in Selenium WebDriver
- Configuring Unauthenticated Proxy Servers in Selenium
- Configuring Authenticated Proxy Setup for Selenium
- Best Practices for Selenium Proxy Configuration
- Conclusion
Subscribe for latest updates
Share this article
Related posts
Endpoint ft. Mahdi Taghizadeh: A CTO’s guide to product thinking, observability, and growth
Engineering leader Mahdi Taghizadeh shares insights on agile thinking, AI’s impact on development, and the underrated skills that define great engineers
Dinesh Thakur
October 7, 2025
A Beginner’s Guide to WCAG Testing Checklist
Discover the essential WCAG testing checklist for ensuring web accessibility, covering key areas like contrast, keyboard navigation, and mobile responsiveness.
Azma Banu
October 6, 2025
How to Use Maestro for Flutter Testing
Learn how to use Maestro for Flutter testing including setup steps, test strategies, component validation, performance checks, and accessibility compliance.
Rohit Rajpal
October 6, 2025