Setting Up a K3s Kubernetes Cluster

K3s is a lightweight, certified Kubernetes distribution designed for IoT & Edge computing. This guide covers installation, configuration, and best practices for running K3s.

Video Tutorial

Learn more about setting up K3s Kubernetes clusters in this comprehensive video tutorial:

Prerequisites

• Linux machine (Ubuntu 20.04+ recommended)
• 512MB RAM (minimum)
• 1 CPU core (minimum)
• Sudo access

Installation

1. Single Node Setup

# Install K3s
curl -sfL https://get.k3s.io | sh -

# Verify installation
sudo k3s kubectl get nodes

# Get kubeconfig
sudo cat /etc/rancher/k3s/k3s.yaml

2. Multi-Node Setup

Server Node

# Install K3s server
curl -sfL https://get.k3s.io | sh -

# Get token for agent nodes
sudo cat /var/lib/rancher/k3s/server/node-token

Agent Nodes

# Install K3s agent
curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

Configuration

1. Basic Configuration

# /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: "0644"
tls-san:
  - "my-kubernetes-domain.com"
node-label:
  - "environment=production"

2. Advanced Configuration

# /etc/rancher/k3s/config.yaml
cluster-init: true
cluster-cidr: "10.42.0.0/16"
service-cidr: "10.43.0.0/16"
cluster-dns: "10.43.0.10"
cluster-domain: "cluster.local"
flannel-backend: "vxlan"
token: "your-secure-token"

Storage Configuration

1. Local Path Provisioner

# local-path-storage.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

2. Longhorn Storage (Optional)

# Install Longhorn
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml

# Set as default StorageClass
kubectl patch storageclass longhorn -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Network Configuration

1. Basic Network Policy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
spec:
  podSelector: {}
  policyTypes:
  - Ingress

2. Load Balancer (Optional)

# metallb-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.1.240-192.168.1.250

Security Configuration

1. Role-Based Access Control (RBAC)

# rbac-config.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]

2. Pod Security Policy

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  seLinux:
    rule: RunAsAny
  runAsUser:
    rule: MustRunAsNonRoot
  fsGroup:
    rule: RunAsAny

Monitoring Setup

1. Metrics Server

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

2. Prometheus & Grafana (Optional)

# Add Helm repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install Prometheus Stack
helm install monitoring prometheus-community/kube-prometheus-stack

Maintenance

1. Upgrading K3s

# Server node
curl -sfL https://get.k3s.io | sh -

# Agent nodes
curl -sfL https://get.k3s.io | K3S_URL=https://myserver:6443 K3S_TOKEN=mynodetoken sh -

2. Backup and Restore

Backup

# Stop K3s
systemctl stop k3s

# Backup data directory
tar czf k3s-backup-$(date +%Y%m%d).tar.gz /var/lib/rancher/k3s

# Restart K3s
systemctl start k3s

Restore

# Stop K3s
systemctl stop k3s

# Restore data directory
tar xzf k3s-backup-20250120.tar.gz -C /

# Restart K3s
systemctl start k3s

Troubleshooting

Common Issues

1. Node Not Ready

# Check node status
kubectl get nodes
kubectl describe node <node-name>

# Check system logs
journalctl -u k3s

2. Pod Networking Issues

# Check CNI configuration
kubectl -n kube-system get pods
kubectl -n kube-system logs -l k8s-app=flannel

# Verify network policy
kubectl get networkpolicies

3. Storage Issues

# Check PVC status
kubectl get pvc
kubectl describe pvc <pvc-name>

# Check storage provisioner
kubectl -n kube-system get pods | grep local-path

Best Practices

1. Resource Management

   • Set resource requests and limits
   • Use namespace quotas
   • Monitor resource usage

2. High Availability

   • Deploy multiple server nodes
   • Use external database (etcd or MySQL)
   • Configure proper backup strategy

3. Security

   • Enable RBAC
   • Use network policies
   • Regularly update K3s
   • Implement pod security policies

4. Monitoring

   • Deploy metrics server
   • Set up proper logging
   • Configure alerts

Additional Resources

• Official K3s Documentation
• K3s GitHub Repository
• Rancher Support Forums
• K3s Slack Channel