Observe and back up
Design actionable alerts
Alert on meaningful access, privilege, availability, and integrity signals with an owner and an immediate investigation path.
8 minute lesson
An alert nobody investigates is noise. Start with events that could cause real harm.
Examples include a new sudo user, an unexpected listening port, repeated SSH failures followed by success, a disabled firewall, full disk, or changed deployment files. Set thresholds from normal traffic and test notification delivery.
Alerting on every failed SSH login creates noise. Alerting when failures are followed by success highlights a plausible account attack but can still catch a forgetful administrator.
Send alerts through a path independent from the host when possible. A full disk or compromised server may break the same local mail command that was supposed to report it.
Define signal, threshold, owner, evidence link, and first action for three alerts. Trigger one real alert and one near-threshold case, then prove notification delivery and suppression both behave as documented.
Lesson completed