TLS and network control

Send requests through a proxy

Configure an HTTP or SOCKS proxy and identify the additional trust boundary it introduces.

A proxy is a server that forwards your requests for you. Instead of connecting to the destination, curl connects to the proxy and asks it to reach the destination on its behalf.

You meet proxies in corporate networks. In debugging tools like mitmproxy that run on your own machine. In scrapers that route traffic through specific exit addresses.

A proxy sits in the middle of the connection. That has consequences for the protocol, and consequences for trust. Let’s look at both.

Point curl at a proxy

Send a request through a local proxy:

curl --proxy http://127.0.0.1:8080 --verbose https://example.org/ -o /dev/null

For HTTPS through an HTTP proxy, curl first sends a CONNECT request, then negotiates TLS with the destination. You can watch it in the verbose output:

* Connected to 127.0.0.1 port 8080
> CONNECT example.org:443 HTTP/1.1
< HTTP/1.1 200 Connection established
* SSL connection using TLSv1.3

Read that sequence, because it separates two relationships.

First, a plain connection to the proxy. Then CONNECT, which asks the proxy to open a raw tunnel to example.org:443. The proxy answers 200 Connection established. Only then does curl negotiate TLS, with the destination, through the tunnel. The proxy shuttles encrypted bytes it can’t read.

That split is your debugging map. A failure before CONNECT is a proxy problem. A failure after it is between you and the destination. If you have access to the proxy logs, read both sides.

If nothing is listening on port 8080, you get curl: (7) Failed to connect to 127.0.0.1 port 8080. Exit code 7. curl never got past the first step.

SOCKS proxies

For a SOCKS proxy, like the one ssh -D 1080 gives you, change the scheme:

curl --proxy socks5h://127.0.0.1:1080 https://example.org/

The h in socks5h matters. It makes the proxy resolve the hostname, so no DNS query leaves your machine. Plain socks5 resolves locally first, and that DNS query is visible to anyone watching your network.

The proxy you didn’t ask for

curl also honors environment variables like https_proxy and HTTP_PROXY. Remember this when curl uses a proxy you never configured. Check with env | grep -i proxy.

--noproxy '*' disables that for one command.

The trust boundary

A proxy can see every destination you visit and when. Without end-to-end TLS, it can read the content too. If a proxy asks you to install its own CA certificate to “inspect HTTPS”, it can then read everything.

Use only a proxy you trust and understand. You’re adding a party to every conversation.

Try it: run ssh -D 1080 -N to a server you own, then send a request through socks5h://127.0.0.1:1080 to https://httpbin.org/ip. The address in the response is the server’s, not yours.

Lesson completed

Take this course offline

Get every free book, course edition, and software download.

Get the download library →