Chapter 11: Advanced Patterns

Authored by syscook.dev

What are Advanced Patterns in Kubernetes?

Advanced patterns in Kubernetes involve sophisticated deployment strategies, architectural patterns, and operational techniques that go beyond basic deployments. These patterns help solve complex problems like stateful applications, multi-tenancy, and high availability.

Key Concepts:

• StatefulSets: Managing stateful applications
• DaemonSets: Running pods on every node
• Jobs and CronJobs: Batch processing and scheduled tasks
• Operators: Custom resource management
• Service Mesh: Advanced networking and observability
• Multi-tenancy: Isolating workloads and resources

Why Use Advanced Patterns?

1. Stateful Applications

Manage applications that require persistent storage and stable network identities.

# Example: StatefulSet for database
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-statefulset
  namespace: production
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB
          value: "myapp"
        - name: POSTGRES_USER
          value: "user"
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

2. DaemonSets for System Services

Run system-level services on every node.

# Example: DaemonSet for log collection
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:latest
        ports:
        - containerPort: 24224
        env:
        - name: FLUENTD_CONF
          value: "fluent.conf"
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: fluentd-config
          mountPath: /fluentd/etc
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: fluentd-config
        configMap:
          name: fluentd-config
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule

3. Jobs and CronJobs

Handle batch processing and scheduled tasks.

# Example: CronJob for backup
apiVersion: batch/v1
kind: CronJob
metadata:
  name: database-backup
  namespace: production
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:13
            command:
            - /bin/bash
            - -c
            - |
              pg_dump -h postgres-service -U user -d myapp > /backup/backup-$(date +%Y%m%d).sql
              aws s3 cp /backup/backup-$(date +%Y%m%d).sql s3://my-backup-bucket/
            env:
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: password
            volumeMounts:
            - name: backup-storage
              mountPath: /backup
          volumes:
          - name: backup-storage
            emptyDir: {}
          restartPolicy: OnFailure

How to Use Advanced Patterns?

1. StatefulSet Configuration

Database StatefulSet

# Example: Complete database StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-statefulset
  namespace: production
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB
          value: "myapp"
        - name: POSTGRES_USER
          value: "user"
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        - name: POSTGRES_REPLICATION_USER
          value: "replicator"
        - name: POSTGRES_REPLICATION_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: replication-password
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
        - name: postgres-config
          mountPath: /etc/postgresql
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "1Gi"
            cpu: "1000m"
      volumes:
      - name: postgres-config
        configMap:
          name: postgres-config
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: postgres-headless
  namespace: production
spec:
  clusterIP: None
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432

2. DaemonSet Configuration

Monitoring DaemonSet

# Example: Node monitoring DaemonSet
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
      - name: node-exporter
        image: prom/node-exporter:latest
        ports:
        - containerPort: 9100
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"
        volumeMounts:
        - name: proc
          mountPath: /host/proc
          readOnly: true
        - name: sys
          mountPath: /host/sys
          readOnly: true
        - name: root
          mountPath: /rootfs
          readOnly: true
      volumes:
      - name: proc
        hostPath:
          path: /proc
      - name: sys
        hostPath:
          path: /sys
      - name: root
        hostPath:
          path: /
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule

3. Job and CronJob Configuration

Batch Processing Job

# Example: Batch processing job
apiVersion: batch/v1
kind: Job
metadata:
  name: data-processing-job
  namespace: production
spec:
  parallelism: 3
  completions: 10
  template:
    spec:
      containers:
      - name: processor
        image: data-processor:latest
        env:
        - name: BATCH_SIZE
          value: "1000"
        - name: INPUT_SOURCE
          value: "s3://input-bucket/"
        - name: OUTPUT_DEST
          value: "s3://output-bucket/"
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
      restartPolicy: Never

Scheduled Maintenance CronJob

# Example: Maintenance CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: maintenance-cronjob
  namespace: production
spec:
  schedule: "0 3 * * 0"  # Weekly on Sunday at 3 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: maintenance
            image: maintenance-tool:latest
            command:
            - /bin/bash
            - -c
            - |
              echo "Starting maintenance tasks..."
              # Clean up old logs
              find /var/log -name "*.log" -mtime +30 -delete
              # Update system packages
              apt-get update && apt-get upgrade -y
              # Clean up Docker images
              docker system prune -f
            resources:
              requests:
                memory: "128Mi"
                cpu: "100m"
              limits:
                memory: "256Mi"
                cpu: "200m"
          restartPolicy: OnFailure

Practical Examples

1. Complete Stateful Application

Step 1: Create StatefulSet

#!/bin/bash
# create-statefulset.sh

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-statefulset
  namespace: production
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB
          value: "myapp"
        - name: POSTGRES_USER
          value: "user"
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: postgres-headless
  namespace: production
spec:
  clusterIP: None
  selector:
    app: postgres
  ports:
  - port: 5432
    targetPort: 5432
EOF

echo "StatefulSet created successfully!"

Step 2: Create DaemonSet

#!/bin/bash
# create-daemonset.sh

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: fluentd
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:latest
        ports:
        - containerPort: 24224
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
EOF

echo "DaemonSet created successfully!"

Step 3: Create CronJob

#!/bin/bash
# create-cronjob.sh

kubectl apply -f - <<EOF
apiVersion: batch/v1
kind: CronJob
metadata:
  name: database-backup
  namespace: production
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: postgres:13
            command:
            - /bin/bash
            - -c
            - |
              pg_dump -h postgres-headless -U user -d myapp > /backup/backup-$(date +%Y%m%d).sql
              echo "Backup completed: backup-$(date +%Y%m%d).sql"
            env:
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secret
                  key: password
            volumeMounts:
            - name: backup-storage
              mountPath: /backup
          volumes:
          - name: backup-storage
            emptyDir: {}
          restartPolicy: OnFailure
EOF

echo "CronJob created successfully!"

Best Practices

1. StatefulSet Configuration

# Good: Proper StatefulSet configuration
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres-statefulset
spec:
  serviceName: postgres-headless
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:13
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

2. DaemonSet Configuration

# Good: Proper DaemonSet configuration
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
      - name: node-exporter
        image: prom/node-exporter:latest
        volumeMounts:
        - name: proc
          mountPath: /host/proc
          readOnly: true
      volumes:
      - name: proc
        hostPath:
          path: /proc
      tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule

3. CronJob Configuration

# Good: Proper CronJob configuration
apiVersion: batch/v1
kind: CronJob
metadata:
  name: maintenance-cronjob
spec:
  schedule: "0 3 * * 0"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: maintenance
            image: maintenance-tool:latest
            resources:
              requests:
                memory: "128Mi"
                cpu: "100m"
              limits:
                memory: "256Mi"
                cpu: "200m"
          restartPolicy: OnFailure

Common Pitfalls and Solutions

1. StatefulSet Scaling Issues

# ❌ Problem: StatefulSet not scaling properly
# Error: pod not ready

# ✅ Solution: Check pod readiness and storage
kubectl get pods -l app=postgres
kubectl describe pod postgres-statefulset-0
kubectl get pvc

2. DaemonSet Node Affinity Issues

# ❌ Problem: DaemonSet not running on all nodes
# Error: pod not scheduled

# ✅ Solution: Check tolerations and node affinity
kubectl get nodes
kubectl describe pod fluentd-xxx
kubectl get daemonset fluentd -o yaml

3. CronJob Schedule Issues

# ❌ Problem: CronJob not running
# Error: schedule not valid

# ✅ Solution: Check schedule format and timezone
kubectl get cronjob
kubectl describe cronjob maintenance-cronjob
kubectl get jobs

Conclusion

Advanced patterns in Kubernetes provide powerful tools for managing complex applications and workloads. By understanding:

• What advanced patterns are and their use cases
• Why they're important for complex applications
• How to implement them effectively

You can create sophisticated, scalable applications that handle stateful workloads, system services, and batch processing. Proper use of advanced patterns ensures that your applications are robust, maintainable, and performant.

Next Steps

• Practice with different advanced patterns
• Learn about production best practices
• Move on to Chapter 12: Production Best Practices

This tutorial is part of the Kubernetes Mastery series by syscook.dev