← Selected engineering

Case study / Infrastructure / Portfolio project

Less alert noise.
A clearer next step.

Ops Triage turns sample operational alerts into prioritized incident briefs, keeping the evidence and suggested checks together.

ProjectOps Triage
FocusIncident grouping & handoff
StackPython · Standard library · JSON
Get the code ↗

01 / The problem

Repeated alerts hide the operational picture.

A service can report the same failure several times as it gets worse. Operators need the highest observed severity, the original evidence, and a defined incident window, rather than a list of disconnected notifications.

This small local tool groups matching service and alert fingerprints into fixed windows, preserves unique messages, and suggests checks by incident category.

02 / Sample result

8 alerts. 4 incident briefs.

The included fixture covers capacity, connectivity, access, and a completed backup. These counts come from the sample run; they describe the fixture, rather than a production performance result.

P1

file-server

3 alerts · capacity

  • Data volume disk utilization is above 85%.
  • Data volume disk utilization is above 95%.
Suggested checks
  • Check disk and volume utilization.
  • Identify growth and retention policy.
  • Review capacity or cleanup changes with the service owner.
P2

vpn-gateway

2 alerts · connectivity

  • VPN connection timeout from the branch office.
Suggested checks
  • Check service health and recent changes.
  • Inspect DNS, routing, VPN, and firewall paths.
  • Record the failing hop before proposing a change.
P2

identity-service

1 alerts · access

  • Repeated authentication denied events need identity review.
Suggested checks
  • Confirm the affected identity and scope.
  • Review authentication logs and recent policy changes.
  • Escalate unexpected access patterns for review.
P4

backup-worker

2 alerts · general

  • Scheduled backup job completed.
Suggested checks
  • Confirm user impact and affected service.
  • Compare the alert timeline with recent changes.
  • Collect evidence and assign an incident owner.

03 / Design decisions

Small enough to inspect. Clear enough to test.

Fixed windows

Grouping is anchored to the first alert, preventing a continuous stream from stretching one incident indefinitely.

Explicit priorities

Critical, error, warning, and informational severities map to P1–P4. Each brief keeps its highest observed severity.

Preserved evidence

Counts include repeated events. Distinct messages and first/last timestamps remain in the brief.

Human review

Suggested checks guide investigation. The tool makes no connections to infrastructure and runs no remediation commands.

04 / Run it yourself

No services or credentials to configure.

python triage.py fixtures/alerts.json
python -m unittest -v

Seven tests cover severity escalation, grouping windows, service isolation, deterministic output, malformed data, timezone handling, and empty input. The project runs on Python 3.10 or later.

It is a portfolio demonstration, with rule-based categorization and sample data. Production work would add source adapters, service ownership, persistence, redaction, and site-specific runbooks.