Kubernetes Storage Best Practices

by Bala

Best practices for managing persistent storage in Kubernetes

Kubernetes Storage Best Practices

Managing storage effectively in Kubernetes is crucial for stateful applications. This guide covers essential storage best practices and patterns.

Prerequisites

  • Basic understanding of Kubernetes
  • Access to a Kubernetes cluster
  • kubectl CLI tool installed
  • Familiarity with storage concepts

Project Structure

.
├── storage/
│   ├── storage-classes/    # StorageClass definitions
│   ├── pvcs/              # PersistentVolumeClaim configs
│   ├── statefulsets/      # StatefulSet configurations
│   └── backups/           # Backup configurations
└── monitoring/
    ├── storage-metrics/   # Storage monitoring configs
    └── alerts/           # Storage alert definitions

Storage Classes

1. Basic StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp3
  fsType: ext4
reclaimPolicy: Retain
allowVolumeExpansion: true

2. High-Performance StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/aws-ebs
parameters:
  type: io2
  iopsPerGB: "50"
  fsType: ext4
reclaimPolicy: Retain
allowVolumeExpansion: true

Persistent Volume Claims

1. Basic PVC

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: standard
  resources:
    requests:
      storage: 10Gi

2. StatefulSet PVC

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
spec:
  serviceName: database
  replicas: 3
  template:
    spec:
      containers:
      - name: database
        image: postgres:14
        volumeMounts:
        - name: data
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: fast
      resources:
        requests:
          storage: 100Gi

Backup Strategies

1. Velero Backup

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: daily-backup
spec:
  includedNamespaces:
  - default
  includedResources:
  - persistentvolumeclaims
  - persistentvolumes
  ttl: 720h
  hooks:
    resources:
      - name: backup-hook
        includedNamespaces:
        - default
        labelSelector:
          matchLabels:
            app: database

2. Volume Snapshots

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: data-snapshot
spec:
  volumeSnapshotClassName: csi-hostpath-snapclass
  source:
    persistentVolumeClaimName: data-pvc

Storage Monitoring

1. Prometheus Storage Metrics

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: storage-monitor
spec:
  selector:
    matchLabels:
      app: storage-metrics
  endpoints:
  - port: metrics

2. Storage Alerts

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: storage-alerts
spec:
  groups:
  - name: storage
    rules:
    - alert: HighStorageUsage
      expr: kubelet_volume_stats_used_bytes / kubelet_volume_stats_capacity_bytes > 0.85
      for: 5m
      labels:
        severity: warning

Storage Encryption

1. Encrypted StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: encrypted-storage
provisioner: kubernetes.io/aws-ebs
parameters:
  encrypted: "true"
  kmsKeyId: arn:aws:kms:region:account:key/key-id

2. Pod with Encrypted Volume

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  containers:
  - name: app
    image: nginx
    volumeMounts:
    - name: secure-storage
      mountPath: /data
  volumes:
  - name: secure-storage
    persistentVolumeClaim:
      claimName: encrypted-pvc

Best Practices Checklist

  1. ✅ Use appropriate StorageClasses
  2. ✅ Implement backup strategies
  3. ✅ Configure volume snapshots
  4. ✅ Monitor storage usage
  5. ✅ Implement encryption
  6. ✅ Use StatefulSets for stateful apps
  7. ✅ Configure storage alerts
  8. ✅ Plan capacity
  9. ✅ Regular backup testing
  10. ✅ Storage performance monitoring

Common Storage Pitfalls

  1. ❌ Using wrong storage class
  2. ❌ Missing backups
  3. ❌ No encryption
  4. ❌ Poor monitoring
  5. ❌ Inadequate capacity planning

Storage Types for Different Workloads

Databases

  • Use high-performance storage
  • Enable encryption
  • Regular backups
  • Volume snapshots

File Storage

  • Use appropriate access modes
  • Configure quotas
  • Monitor usage
  • Regular cleanup

Cache Storage

  • Use fast storage
  • Configure proper retention
  • Monitor performance
  • Plan for scalability

Conclusion

Following these storage best practices ensures reliable and efficient data management in your Kubernetes clusters. Regular monitoring and maintenance of storage resources is crucial for application reliability.

Additional Resources