A Beginner's Guide to Container Runtimes (Part 1): Core Mechanics
“Container runtime” is one of the most overloaded terms in cloud native. Some use it to describe Docker, others mean containerd, and some specifically mean runc. Depending on context, none of them are wrong. That ambiguity exists because running a container requires coordination across several architectural tiers—and virtually every tier has been called a runtime at some point.
As the first installment of a two-part series, this article establishes three foundational mental models:
- Containers are not virtual machines; they are ordinary processes restricted by the Linux kernel: Isolation is enforced by namespaces, resource limits by cgroups, and privileges are constrained by capabilities and seccomp.
- Runtimes are split into two distinct tiers: high-level and low-level: High-level runtimes (like containerd and CRI-O) handle image pulls and lifecycle management; low-level runtimes (like runc, crun, gVisor, and Kata) follow the OCI specification to invoke kernel primitives and spawn processes.
- Kubernetes communicates with high-level runtimes via the CRI: The kubelet—the node-level agent in Kubernetes—operates containers through a standard gRPC interface, entirely agnostic to whether the underlying engine is containerd or CRI-O.
Mastering these underlying mechanisms makes diagnosing practical issues—such as cgroup driver mismatches, OOMKilled events, and crictl troubleshooting—much easier.
The Nature of Containers: Kernel-Constrained Processes
Debunking the “Lightweight Virtual Machine” Myth
Many people first encounter containers through the analogy that they are “lightweight virtual machines.” While helpful as an initial visual hook, this analogy becomes actively misleading when debugging system architecture.
A virtual machine (VM) runs its own dedicated guest operating system and kernel, with a hypervisor emulating a complete hardware environment. A container, by contrast, has no separate kernel; it shares the host’s Linux kernel alongside every other process running on that machine.
In other words, a container is fundamentally an ordinary Linux process wrapped in four layers of kernel-enforced defenses:
| Mechanism | Core Responsibility | Plain Analogy |
|---|---|---|
| namespaces | Dictates what the process can see | Filtered lenses that restrict visibility to its own processes, interfaces, and mount points |
| cgroups | Dictates how much the process can consume | A quota card strictly capping CPU, memory, and process count |
| capabilities | Dictates which privileged operations it can perform | Carves root’s all-or-nothing authority into dozens of granular capabilities, granting only what is required |
| seccomp | Dictates which system calls it can invoke | A whitelist filter at the kernel boundary that blocks any syscall not explicitly permitted |
Namespaces: Defining Visibility Boundaries
The Linux kernel provides several namespaces (see namespaces(7)). The five most critical are:
| Namespace | Isolated Resource | Effect Inside the Container |
|---|---|---|
| PID | Process ID | The primary container process becomes PID 1, blind to processes on the host and other containers |
| Mount | File mount points | The container has its own root filesystem, unable to see host directories like /home |
| Network | Network devices, routing, ports | The container gets a dedicated eth0 and IP, allowing independent binding to port 80 |
| User | UID and GID | The root user inside the container can map to an unprivileged user on the host |
| UTS | Hostname and domain name | The container can set an independent hostname |
Other namespaces—such as IPC (inter-process communication), Cgroup (restricting visibility to the process’s own cgroup hierarchy), and Time (independent boot and monotonic clocks, primarily used for live container migration)—provide supplementary isolation.
Namespaces alter only the process’s field of view; they do not create a separate kernel. Running ps -ef on the host reveals every process running inside every container. Consequently, a kernel vulnerability can compromise all containers running on that host.
Cgroups: Enforcing Resource Limits
Namespaces do not restrict resource consumption. Without explicit constraints, a single container can exhaust all host memory. Control Groups (cgroups) organize processes into hierarchical groups and allocate limits for CPU, memory, I/O, and process counts (see cgroups(7)).
The interface for cgroups is a virtual filesystem, typically mounted at /sys/fs/cgroup. Adjusting a quota is as simple as writing a value to a file.
In cgroup v2, for instance, writing 268435456 to memory.max sets that group’s memory ceiling to 256 MiB. If usage exceeds this limit, the kernel triggers the OOM (Out Of Memory) Killer within that cgroup, terminating processes to reclaim memory (see the Linux Kernel cgroup v2 documentation).
The resources.limits defined in Kubernetes manifests are ultimately translated by the runtime into values written to these filesystem paths.
Capabilities and Seccomp: Constraining Execution Privileges
Traditional Linux security makes a binary distinction between root and non-root users. Linux capabilities divide traditional superuser privileges into dozens of distinct units (such as CAP_NET_BIND_SERVICE and CAP_SYS_ADMIN; see capabilities(7)). By default, the root user inside a container receives only a minimal subset of capabilities, preventing operations like loading arbitrary kernel modules.
Seccomp (Secure Computing Mode) enforces boundaries at the system call layer. Docker’s default seccomp profile blocks approximately 44 of the more than 300 available Linux syscalls—allowing the vast majority through while filtering out hazardous calls such as reboot and mount (see the Docker seccomp documentation).
Kubernetes does not enable strict seccomp restrictions by default. Unless seccompDefault is enabled in the kubelet configuration or RuntimeDefault is explicitly specified in a Pod’s securityContext, Pods run in Unconfined mode (see the Kubernetes seccomp tutorial).
Hands-On: Assembling a Minimal Container Without a Runtime
The following demonstration uses only the standard unshare utility (which runs a program with namespaces unshared from the parent; see unshare(1)) and the cgroup v2 filesystem. Run this on a disposable Linux sandbox with root privileges:
# 1. Prepare an Alpine root filesystem
mkdir rootfs
docker export "$(docker create alpine)" | tar -C rootfs -xf -
# 2. Unshare namespaces and switch root using chroot
sudo unshare --uts --ipc --net --pid --mount --fork \
chroot rootfs /bin/sh -c 'mount -t proc proc /proc && hostname demo && exec /bin/sh'
Inside the resulting container shell, verify isolation:
ps: Shows only two or three lines;/bin/shis PID 1 (PID namespace in effect).hostname: Displaysdemo; the host hostname remains unchanged (UTS namespace in effect).ip link: Only shows the loopback interfacelo, which is in aDOWNstate (Network namespace in effect).
Open a second terminal window on the host to attach this process to a custom cgroup:
# Locate the actual PID of the shell process on the host
ps -ef --forest | rg -A3 unshare
# Create a cgroup limiting CPU to 0.5 cores (50,000 max per 100,000 us) and memory to 64 MiB
sudo mkdir /sys/fs/cgroup/demo
echo "50000 100000" | sudo tee /sys/fs/cgroup/demo/cpu.max
echo $((64*1024*1024)) | sudo tee /sys/fs/cgroup/demo/memory.max
echo <HOST_PID> | sudo tee /sys/fs/cgroup/demo/cgroup.procs
# Inspect the process's namespaces and cgroup assignment
sudo ls -l /proc/<HOST_PID>/ns
cat /proc/<HOST_PID>/cgroup
When finished, type exit inside the container shell and clean up with sudo rmdir /sys/fs/cgroup/demo.
For simplicity, this manual demonstration uses standard chroot. Production low-level runtimes like runc switch the root filesystem using pivot_root to prevent directory traversal escapes, automating the entire workflow with standardized security profiles.
High-Level vs. Low-Level Runtimes: Division of Labor
To prevent container tooling from fracturing around single-vendor lock-in, the industry established the OCI (Open Container Initiative), which formalized two key specifications:
- OCI Image Spec: Defines the container image layout and layered archive structure.
- OCI Runtime Spec: Defines how a low-level runtime unpacks configuration (
config.json) and a rootfs directory (collectively known as an OCI Bundle) to spawn a container.
These specifications split the container runtime landscape into two clear tiers:
- High-Level Runtimes (The Front-of-House Manager): Responsible for taking orders, preparing ingredients (pulling and unpacking OCI images), assigning tables (coordinating networking and storage), and tracking customer state throughout the meal (lifecycle management). Notable implementations: containerd and CRI-O.
- Low-Level Runtimes (The Chef): Focused strictly on following a standardized recipe (the OCI
config.json) to prepare the dish quickly (invoking kernel primitives to create the process), and hands off control as soon as the container is running. Notable implementations: runc, crun, gVisor, and Kata Containers.
| Dimension | High-Level Runtime | Low-Level Runtime |
|---|---|---|
| Representative Projects | containerd, CRI-O | runc, crun, gVisor, Kata |
| Daemon State | Yes (long-running background daemon) | No (exits once the container is launched) |
| Upstream Interface | gRPC API, Kubernetes CRI | OCI Runtime Spec (CLI driven by config.json) |
| Image Resolution | Full support (pull, unpack, cache) | None (operates strictly on unpacked bundles) |
| Network Management | Coordinates CNI plugins for Pod networks | Only joins specified Network namespaces |
| Direct User Interaction | Occasional (via crictl, nerdctl) | Rare (almost never called directly) |
Thanks to the standardized boundary of the OCI Runtime Spec, high-level and low-level runtimes can be mixed and matched, allowing the underlying execution engine to be swapped transparently.
NOTE
Further reading: OCI 101: The Common Language of the Container World
Container Execution Flow: From docker run to Process
The complete call chain from command invocation to process execution proceeds as follows:
- docker CLI
- REST API (/var/run/docker.sock)dockerd
- gRPCcontainerdHigh-level Runtime
- ttRPCcontainerd-shim-runc-v2One per Container or Pod
- CLI invocation (runc create/start)runcLow-level Runtime
- clone(), pivot_root, cgroup, seccompLinux Kernel
- Spawns PID 1Your Containershim is the parent process
Breakdown of tiered responsibilities:
- docker CLI: A pure API client that serializes user input into HTTP requests directed to the Unix socket
/var/run/docker.sock. - dockerd: The Docker Engine daemon responsible for image builds, volumes, and network orchestration. Since Docker 1.11, container execution has been fully decoupled and delegated to containerd (see the Docker official blog).
- containerd: The high-level runtime. Pulls images, unpacks layers, generates the OCI
config.jsonspecification, and launches the shim. It does not spawn the container process directly; it orchestrates the higher-level workflow (see the containerd runtime-v2 documentation). - containerd-shim-runc-v2: A persistent daemon process (one per container in Docker; optimized to one per Pod in Kubernetes).
- Adopts the Container as Parent: Because runc exits immediately after launching the container, the shim assumes parentage of the container process. It collects exit codes when the container terminates and keeps standard I/O (stdio) pipes open. This ensures that restarting containerd does not kill running containers.
- Decouples Runtime Implementations: Swapping the low-level runtime requires swapping only the shim (e.g., using
containerd-shim-kata-v2for Kata Containers) without modifying containerd itself.
- runc: The low-level runtime. Reads the OCI Bundle, configures namespaces and cgroups, performs
pivot_root, applies seccomp profiles, executes the target binary, and exits. - Target Process: Executes as a standard host process constrained and isolated by kernel primitives.
On a Kubernetes node, this call chain is streamlined: the kubelet communicates directly with containerd via the CRI, removing the docker CLI and dockerd layers entirely.
Observing the Process Tree on a Live Node
On such a node, you can inspect the process hierarchy directly:
# View process hierarchy
ps -ef --forest | rg -A3 containerd-shim
# Inspect tasks managed by containerd
sudo ctr -n k8s.io tasks ls
Typical output resembles the following:
PID PPID CMD
1234 1 /usr/bin/containerd-shim-runc-v2
-namespace k8s.io -id 9f3c...
1250 1234 \_ /pause
1302 1234 \_ nginx: master process nginx -g daemon off;
1330 1302 \_ nginx: worker process
Key architectural takeaways visible in this hierarchy:
- The shim’s PPID is 1 (adopted by systemd/init), not containerd.
- The
/pausecontainer lives under the shim. Its sole role is to hold open the shared Network and IPC namespaces that workload containers (likenginx) join. - runc has already exited and is absent from the process tree.
CRI: How the Kubelet Controls Runtimes
Why the CRI Exists
Early Kubernetes releases embedded Docker support directly into kubelet source code through a hardcoded translation layer called dockershim. As alternative runtimes emerged, supporting each new engine required adding specialized code directly to the kubelet codebase.
In 2016, Kubernetes 1.5 introduced the CRI (Container Runtime Interface), consolidating node-level container operations into a standardized gRPC interface (see the Kubernetes official blog post). This paved the way for the eventual deprecation and removal of embedded dockershim in favor of a clean, runtime-agnostic architecture.
NOTE
Further reading: Why Kubernetes Removed Docker
CRI Architecture and Pod Startup Lifecycle
Defined in api.proto under kubernetes/cri-api, the CRI consists of two primary gRPC services:
| CRI Service | Core RPC Calls | Primary Function |
|---|---|---|
RuntimeService | RunPodSandbox, CreateContainer, StartContainer, ContainerStatus, RuntimeConfig | Manages Pod Sandbox and container lifecycles |
ImageService | PullImage, ListImages, RemoveImage | Manages container image pulls and local cache |
When a Pod is scheduled to a node, the kubelet executes the following sequence:
kubelet High-level Runtime
| |
|-- 1. RunPodSandbox -->|
| | Create Network Namespace /
| | pause container / CNI IP
| |
|-- 2. PullImage ------>|
| | Download application image
| |
|-- 3. CreateContainer >|
| | Create container per OCI Spec
| |
|-- 4. StartContainer ->|
| | Launch application process
v v
Among the calls above, RuntimeConfig plays a distinct role: it lets the kubelet query runtime configuration (such as the active cgroup driver), enabling automated alignment between node components during startup. This alignment has been enabled by default since Kubernetes 1.31 (the KubeletCgroupDriverFromCRI feature, GA in 1.34) and requires runtime support for the call (containerd v2.0+, CRI-O v1.28+); otherwise the kubelet falls back to its own cgroup driver setting.
The Container Runtime Landscape at a Glance
In the modern cloud native ecosystem, prevalent runtimes divide responsibilities based on architectural layer and isolation requirements:
| Component | Layer | Maintainer & Governance | Typical Use Case |
|---|---|---|---|
| containerd | High-level | CNCF Graduated | Default runtime for managed Kubernetes (EKS, GKE, AKS); core of Docker Engine |
| CRI-O | High-level | CNCF Graduated | Red Hat OpenShift; lightweight runtime built specifically for Kubernetes |
| runc | Low-level | OCI Reference Implementation | Default low-level execution engine for containerd (CRI-O now defaults to crun) |
| crun | Low-level | containers organization (GitHub) | Written in C for extreme startup speed and low memory footprint; default in Podman and CRI-O |
| gVisor | Low-level (Sandbox) | Untrusted multi-tenant workloads; backing engine for GKE Sandbox | |
| Kata Containers | Low-level (MicroVM) | OpenInfra Foundation | Hardware-isolated virtualization for strong multi-tenant boundaries |
Low-level runtimes fall into three primary isolation paradigms:
- Shared Host Kernel (runc / crun): Relies entirely on native Linux isolation primitives. Offers peak performance but carries kernel escape risks.
- User-Space Kernel (gVisor): The Sentry engine intercepts and services syscalls entirely in user space, trading some performance and compatibility for substantial security hardening.
- Lightweight MicroVMs (Kata Containers): Spawns an isolated microVM for each Pod. Delivers hypervisor-grade isolation at the cost of requiring hardware virtualization support.
Common Misconceptions Clarified
- “Docker images stopped working when Kubernetes removed dockershim”: False. Kubernetes removed only the internal dockershim translation layer from the kubelet. Images built by Docker strictly adhere to the OCI Image Spec and run natively on containerd and CRI-O.
- “You cannot debug a node without the
dockerCLI”: False. Thecrictlutility, designed specifically for CRI runtimes, provides direct pod and container inspection and debugging capabilities. - “Root inside a container is equivalent to root on the host”: Not entirely. Inside a container, root’s capabilities are severely restricted. However, if User Namespaces are not enabled, any container breakout (where a process breaches kernel isolation) results in UID 0 access on the host.
- “Running Docker on macOS executes containers directly on the macOS kernel”: False. macOS lacks native Linux namespaces and cgroups; Docker on macOS runs containers inside an underlying lightweight Linux VM.
Conclusion and What’s Next
This article laid out the nature of container runtimes and their tiered division of labor. “Part 2: Operational Practice” will dive into production troubleshooting and node configuration:
- The practical differences between cgroup v1 and v2, and ensuring cgroup driver parity between kubelet and containerd.
- How Kubernetes resource constraints translate into kernel parameters, dissecting the trigger conditions for OOMKilled and CPU throttling.
- Essential node maintenance tooling: practical
crictltechniques and tuning/etc/containerd/config.toml. - Hardening multi-tenant clusters using RuntimeClass with gVisor and Kata Containers.
NOTE
Further reading: A Beginner’s Guide to Container Runtimes (Part 2): Operational Practice