DevOps & Kubernetes (50 Qs)•Part 1: Candid Experience, Container Internals & Docker (1 - 10)
How do you systematically debug an unhealthy or crashing container in production (Exit Code 137, 139, 1)?
Systematic Diagnostic Workflow
When a container terminates unexpectedly or fails health checks, analyze the container state and exit codes systematically before restarting it.
sql
+-----------------------+ +------------------------+ +-----------------------+
| Check Container State | --> | Interpret Exit Code | --> | Inspect Events & Logs |
| docker ps -a | | 137 = OOM, 1 = App Err | | docker logs / events |
+-----------------------+ +------------------------+ +-----------------------+
Common Container Exit Code Matrix
| Exit Code | Reason | Meaning & Root Cause | Immediate Action |
|---|---|---|---|
| 137 | 128 + 9 (SIGKILL) | OOMKilled or forcefully killed by host kernel / docker stop timeout. | Run docker inspect -f '{{.State.OOMKilled}}' <id>. If true, raise memory limits or fix memory leaks. |
| 139 | 128 + 11 (SIGSEGV) | Segmentation Fault. Native C/Rust addon or corrupted binary memory access. | Inspect core dumps; check native library compatibility across Alpine/musl and Debian/glibc. |
| 143 | 128 + 15 (SIGTERM) | Graceful shutdown requested by host or orchestrator, completed properly. | Normal behavior during rolling update or manual container termination. |
| 1 | Generic Error | Application crashed (uncaught exception, missing environment variable, syntax error). | Run docker logs --tail 100 <id> to inspect application stack traces. |
Diagnostic Command Toolkit
bash
# 1. Inspect exit code, OOM flag, and start/finish timestamps
docker inspect <container-id> --format='{{json .State}}' | jq
# 2. Check kernel out-of-memory logs on the host
dmesg -T | grep -i oom
# 3. View resource utilization live (CPU, Memory, Network I/O)
docker stats --no-stream
# 4. Debug an already crashing container by overriding entrypoint to interactive shell
docker run --rm -it --entrypoint /bin/sh my-broken-image:v1