How do you conduct an incident post-mortem and Root Cause Analysis (5 Whys framework)?

In Simple Words: The 5 Whys framework drills past superficial human error to uncover organizational, architectural, and procedural root causes.

Example 5 Whys in Practice:

• Problem: Our user database crashed and went offline for 32 minutes.
• Why 1?: Because disk storage on the primary database instance hit 100% capacity.
• Why 2?: Because un-truncated audit log tables grew by 180GB in 4 hours.
• Why 3?: Because a developer enabled verbose debug logging in production.
• Why 4?: Because there was no automated configuration linting preventing debug logging in production environments.
• Why 5?: Because our CI pipeline lacked environment-specific config validation.
• Systemic Fix: Add automated CI check blocking LOG_LEVEL=debug in production, add auto-expanding EBS storage with alert at 80% disk capacity.

Practical Script & Summary Template

### 5 Whys Root Cause Analysis Matrix:
| Level | Question | Finding | Systemic Action Item |
| :--- | :--- | :--- | :--- |
| **Why 1** | Why did the service fail? | Disk hit 100% full | Auto-expand EBS storage volume |
| **Why 2** | Why did disk fill up? | Verbose debug log explosion | Set log rotation to 24 hours |
| **Why 3** | Why was debug on? | Developer enabled for testing | Prevent manual env changes via GitOps |
| **Why 4** | Why did CI allow it? | Missing config schema validation | Add Zod schema validation in CI |