Proxy settings are the configuration options that tell a device, browser, or app whether to send its traffic through a proxy server and how to reach it, such as the proxy's address, port, and login details. Without them, traffic goes directly to websites.

What Proxy Settings Include

  • Address and port: the proxy's IP address or hostname and its port number.
  • Protocol: macOS, for example, has separate entries for a web (HTTP) proxy, a secure web (HTTPS) proxy, and a SOCKS proxy.
  • Login details: a user name and password when the proxy requires them. See proxy authentication.
  • Bypass list: hosts and domains that should skip the proxy. Windows also has an option to skip the proxy for local (intranet) addresses.
  • Automatic setup: instead of typing an address, a device can detect the network's proxy settings automatically or load a setup script, also called a PAC file. WPAD is the protocol designed for automatic discovery.

Where Proxy Settings Are Configured

  • Operating system: Windows and macOS store system-wide settings that many apps follow. On Windows, a VPN connection has its own proxy settings that you set separately.
  • Browsers: Chromium-based browsers use the system network settings, which the --proxy-server command-line flag can override. Firefox can use the system settings or its own: no proxy, auto-detect, manual configuration, or a PAC file URL.
  • Command-line tools: curl reads environment variables. http_proxy works only in lowercase, while HTTPS_PROXY, ALL_PROXY (all protocols), and NO_PROXY (hosts to skip) can also be uppercase.
  • Code: scrapers and other scripts can set the proxy in code for each request or session.

Settings to Check When a Proxy Doesn't Work

  • Which settings the app reads: a command-line tool that reads environment variables ignores the system settings, and a browser with its own configuration may ignore them too.
  • Local addresses: Microsoft recommends keeping the option to skip the proxy for local addresses turned on unless your organization says otherwise.
  • DNS with SOCKS: if the app resolves hostnames itself instead of through the proxy, lookups can bypass it and cause a DNS leak. Firefox's setting to send DNS through a SOCKS5 proxy has been on by default since Firefox 128.

What It Means for Web Scraping

System-wide settings send every app through one proxy. A scraper that sets the proxy per request or session can send each session through a different address instead, which is the basis of IP rotation.