TLS and network control

Set timeouts and safe retries

Bound connection and total time, then retry only operations whose repetition is acceptable.

By default, curl waits a long time for a server that never answers. In a terminal that costs you patience. In automation it costs you a hung script, a stuck cron job, or a CI pipeline blocked for minutes on a dead host.

Networks hang and fail temporarily. So automation needs three things: time bounds, clear exit codes, and a retry policy that matches what the operation does.

Bound the time

Add a connection timeout, a total timeout, and a limited retry:

curl --connect-timeout 3 --max-time 10 --retry 2 https://example.org/

The two timeouts answer different questions.

--connect-timeout 3 bounds how long curl waits to establish the connection. That’s DNS, plus TCP, plus the TLS handshake.

--max-time 10 bounds the entire transfer, download included. A server that accepts connections instantly but dribbles out one byte per second sails past the first limit and hits the second.

I put both on every curl command that runs unattended. No exceptions.

Test it once against an address that never answers:

time curl --connect-timeout 3 --max-time 10 https://10.255.255.1/

You get this, after about three seconds:

curl: (28) Failed to connect to 10.255.255.1 port 443 after 3002 ms: Timeout was reached

echo $? prints 28. That’s curl’s timeout exit code. The three-second duration proves the connection timeout did the work. Without these options, the same command can hang for minutes.

Retry what deserves retrying

--retry 2 makes curl try up to two more times. But only for failures curl considers transient: timeouts, and HTTP responses like 408, 429, 500, 502, 503, and 504.

A 404 doesn’t get retried. That’s correct. The resource will still be missing on attempt three.

curl waits between attempts and backs off progressively. You can cap the whole budget with --retry-max-time 30, so the retries never push you past thirty seconds total.

The safety rule

Retries repeat side effects. Retrying a GET is harmless. Reading twice changes nothing.

Retrying a POST that creates an order can create two orders. “The response timed out” does not mean “the server did nothing”. The request may have succeeded right as the connection dropped.

So retry freely only for operations that are idempotent, meaning running them twice gives the same result as running them once. Or for operations protected by an idempotency key, where the server recognizes a repeated request and refuses to apply it twice.

For everything else, let the failure surface to something that can decide. A person, or a queue with proper deduplication. Don’t silently fire the same mutation again.

Try this on a script you already have: add --connect-timeout and --max-time to every curl call in it. Then find any --retry next to a POST and ask whether that endpoint is safe to hit twice.

Lesson completed

Take this course offline

Get every free book, course edition, and software download.

Get the download library →