Security and troubleshooting
Complete the service review
Review one production-style unit from identity and dependencies through recovery, logging, limits, and deployment.
A good unit file is a small operational contract. Another person should be able to read it and understand how the service starts, stops, fails, and recovers, without asking you. This closing lesson turns everything from the course into one repeatable review.
Collect the evidence
Go through the service’s account, paths, configuration, dependencies, restart policy, timeouts, resources, sandboxing, logs, timer relationships, health check, and rollback procedure. That sounds like a lot, but each item maps to a command you already know:
systemctl cat demo-api.service
systemctl show demo-api.service -p User,WorkingDirectory,Wants,After
systemctl show demo-api.service -p Restart,RestartSec,TimeoutStopSec,NRestarts
systemctl show demo-api.service -p MemoryMax,TasksMax,Result
systemd-analyze security demo-api.service
systemctl list-timers --all
journalctl -u demo-api.service -p warning -b
systemctl cat shows the contract as written, drop-ins included. The show queries reveal the effective values, including defaults nobody set on purpose. A MemoryMax=infinity or a TimeoutStopSec=1min 30s in that output means nobody decided. systemd did.
The journal query surfaces warnings the service has been logging while everyone looked the other way. I am always surprised by what shows up there.
Two questions have no command and matter most. Where is the health check? What is the rollback procedure? If the answer to either is “ask Flavio”, the contract is incomplete. The person who knows will be on holiday when it matters.
Write it down, small
A review that lives in your head expires in a week. Keep the written artifact tiny:
Service: demo-api.service Reviewed: 2026-08-03
Strength: restart policy verified. Killed the process,
recovered in 2s, NRestarts incremented as expected
Risk: DATABASE_URL passed via Environment=,
visible in `systemctl show` to any local user
Improvement (tested): moved the password to
LoadCredential=, restarted, login path verified
Three lines. One verified strength, one concrete risk, one reversible improvement you tested.
Each claim needs evidence. The strength was exercised, not assumed: you actually killed the process and watched it come back. The risk names how you would trigger or exploit it. The improvement was applied, verified, and can be rolled back by deleting a single drop-in.
Observations, not opinions
The trap in service reviews is producing opinions instead of observations. “Restart policy looks fine” is an opinion. “I ran kill -9 1423 and the service was serving again two seconds later” is a review.
The difference matters when something breaks later. An opinion gives you nothing to compare against. An observation tells you what changed.
Here is how I turn a vague worry into an observation. Instead of “memory might be tight”, I run systemctl show demo-api.service -p MemoryCurrent at peak traffic and write down the number. Now the review says “peaks at 380M against a 512M limit”. Someone can act on that.
Try this against one real service you operate. Run the commands above, fill in the three lines, and keep the file next to the unit. It usually takes twenty minutes and almost always finds something.
Lesson completed