DevOps & Kubernetes (50 Qs)•Part 3: Production K8s Scaling, Reliability & SRE (26 - 35)
Custom Resource Definitions (CRDs) & The Operator Pattern: The Reconcile Loop in Go/controller-runtime.
Extending the Kubernetes API
A Custom Resource Definition (CRD) teaches the Kubernetes API Server a new object schema (e.g. DatabaseCluster). The Operator Pattern combines a CRD with a custom Controller that runs an ongoing Reconcile Loop to automate human operational domain knowledge (backups, failover, upgrades).
sql
User applies CRD:
[ kubectl apply -f postgres-cluster.yaml ]
|
v
[ kube-apiserver ] ---> Writes CRD to etcd
|
v (Informer watches event)
+-------------------------------------------------------+
| Custom Operator Controller Loop |
| |
| 1. Observe: Read desired state from CRD |
| 2. Compare: Query actual state (Running Pods/PVCs) |
| 3. Reconcile: If mismatch: |
| - Provision new Replica Pod |
| - Execute WAL replication command |
| - Update CRD status: "Healthy" |
+-------------------------------------------------------+
The Reconcile Function in Go (controller-runtime)
go
func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
// 1. Fetch the Database instance
var db customv1.Database
if err := r.Get(ctx, req.NamespacedName, &db); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
// 2. Check if underlying StatefulSet exists
var sts appsv1.StatefulSet
err := r.Get(ctx, req.NamespacedName, &sts)
if errors.IsNotFound(err) {
// 3. Desired state does not exist: Create it
newSts := r.buildStatefulSet(&db)
if err := r.Create(ctx, newSts); err != nil {
return ctrl.Result{}, err
}
return ctrl.Result{Requeue: true}, nil
}
// 4. Update status subresource
db.Status.ReadyReplicas = sts.Status.ReadyReplicas
_ = r.Status().Update(ctx, &db)
return ctrl.Result{RequeueAfter: time.Minute * 5}, nil
}