Free to read · CKA · Cluster architecture and kubectl fluency

Node components and the CRI, CNI and CSI interfaces

How the control plane and the nodes of a kubeadm cluster work together, how the API is organised, and the kubectl and exam habits that make every later module faster.

Lesson · 12 min. This module is free on this site. The rest of the course, the quizzes and the labs open with a free account.

Why this matters

The kubelet does not run containers, set up networking or attach disks by itself. It delegates each job to a plugin through a standard interface. The curriculum names these interfaces explicitly, and node troubleshooting is mostly a matter of finding which plugin in the chain has failed.

InterfaceJobOn our lab clusters
CRI (Container Runtime Interface)Run pod sandboxes and containers, pull imagescontainerd, socket unix:///run/containerd/containerd.sock
CNI (Container Network Interface)Give each pod an IP and connect it to the pod networkCalico, config in /etc/cni/net.d, binaries in /opt/cni/bin
CSI (Container Storage Interface)Provision, attach and mount volumesWhatever drivers are installed; listed as CSIDriver objects

The kubelet

The kubelet is a systemd service on every node, the control plane nodes included. It is the only Kubernetes component that is not a pod on a kubeadm cluster, because something has to start the pods.

Its configuration is spread over a few files. Learn them; they come up in node troubleshooting tasks again and again.

FilePurpose
/var/lib/kubelet/config.yamlThe KubeletConfiguration: static pod path, cgroup driver, cluster DNS, runtime endpoint
/var/lib/kubelet/kubeadm-flags.envExtra command-line flags kubeadm chose for this node
/etc/kubernetes/kubelet.confKubeconfig the kubelet uses to reach the API server
/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.confsystemd drop-in that wires the files above into the command line

Find the real values instead of trusting memory:

systemctl status kubelet
ps -o args= -C kubelet | tr ' ' '\n' | grep -- '--'
grep -E 'staticPodPath|cgroupDriver|containerRuntimeEndpoint|clusterDNS' -A1 /var/lib/kubelet/config.yaml
journalctl -u kubelet -n 50 --no-pager

When a node is NotReady, check in this order: is the kubelet running (systemctl status kubelet), is it enabled so it survives a reboot (systemctl is-enabled kubelet), what does its log say, and can it reach the runtime and the API server?

CRI: the container runtime

The kubelet talks gRPC to the runtime over a Unix socket. containerd is the common choice on kubeadm clusters. Docker Engine is no longer built into the kubelet; if you meet Docker it is through the separate cri-dockerd adapter.

crictl is a CRI client. It shows what the runtime sees, which is handy when the API server is down or when you need low-level IDs.

crictl info | head -20          # runtime status and config
crictl version                  # runtime name and version
crictl pods                     # pod sandboxes on this node
crictl ps -a                    # containers, including exited ones
crictl logs <container-id>      # container logs straight from the runtime
crictl images                   # images present on this node

crictl reads its endpoint from /etc/crictl.yaml. If that points at a dead socket, every command fails with a connection error even though containerd is fine.

runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10

RuntimeClass

A node can offer more than one runtime handler (for example plain runc and a sandboxed runtime). A RuntimeClass gives a handler a name pods can ask for. It can also add a fixed overhead that the scheduler counts on top of the pod's requests, and a scheduling section that steers the pods to nodes that have the handler.

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: hardened
handler: runc
overhead:
  podFixed:
    cpu: 100m
    memory: 64Mi
scheduling:
  nodeSelector:
    sandbox: enabled

A pod opts in with spec.runtimeClassName: hardened. If the handler does not exist on the node, the pod fails at sandbox creation and the event says so.

CNI: the pod network

When the runtime creates a pod sandbox it calls the CNI plugin. The plugin creates the pod's network interface, assigns an IP from the node's pod range and sets up routes.

  • Network configs live in /etc/cni/net.d. The runtime loads the file whose name sorts first and ignores the others. A stray 00-something.conf can hijack networking.
  • Plugin binaries live in /opt/cni/bin. The config's type field names the binary to run.
  • The CNI's agent usually runs as a DaemonSet (on our clusters, calico-node in kube-system).
ls /etc/cni/net.d /opt/cni/bin
kubectl -n kube-system get ds
kubectl get node node01 -o jsonpath='{.spec.podCIDR}{"\n"}'

Without a working CNI, nodes report NotReady with a message that the network plugin is not ready, and new pods hang in ContainerCreating.

kube-proxy

kube-proxy runs as a DaemonSet and turns Services into packet-forwarding rules on each node. On Linux it uses the iptables mode by default; the newer nftables mode is also available. Its settings are in the kube-proxy ConfigMap in kube-system. If kube-proxy is not running on a node, pods there cannot reach Service IPs even though pod-to-pod traffic works. Some CNI plugins can replace kube-proxy entirely; you will see that described as "kube-proxy replacement".

CSI: storage drivers

A CSI driver has a controller part (create and attach volumes) and a node part, usually a DaemonSet that mounts volumes into pods. Each installed driver registers a cluster-scoped CSIDriver object, and nodes list the drivers they run in CSINode objects.

kubectl get csidrivers
kubectl get csinodes
kubectl get storageclass

A StorageClass's provisioner field names the driver that creates its volumes. If no CSI driver is installed, the cluster can still use static PersistentVolumes such as hostPath and local.

Common mistakes

  • Starting a stopped kubelet with systemctl start but forgetting systemctl enable. It dies again on the next reboot.
  • Running crictl against the wrong socket and concluding containerd is down.
  • Reporting a CNI name from a file that is not the first one in /etc/cni/net.d.
  • Expecting docker ps on a containerd node. Use crictl.
  • Confusing CSIDriver objects (drivers) with StorageClasses (policies that use a driver).

Exam angle

Report-style tasks ask you to find facts such as the runtime endpoint, the CNI in use or the kubelet config path and write them to a file. Read the facts from the live system (process arguments, config files, API objects), not from defaults you remember. The live system is what the grader checks.

Practise it

Investigate the runtime, network and storage plugins of a real cluster here:

Practise this on a real cluster in your account: C12, C28, C30, C29, T2.

Official documentation