How kube-proxy Works (Part 1): A Rule Synchronizer, Not a Proxy Server

The name “kube-proxy” is misleading. Many engineers picture a daemon sitting on each node like Nginx or Envoy, accepting packets in user space and forwarding them to backends.

In reality, kube-proxy itself never forwards application packets.

At its core, it is a node-local controller for Service forwarding rules. Running in the background, it watches the Kubernetes API and translates declarative Services and EndpointSlices into Linux kernel forwarding rules (iptables, nftables, or IPVS). The actual per-packet work—filtering, load-balancing selection, and destination network address translation (DNAT)—is mostly performed by the Linux kernel’s network stack.


Core Role: Synchronizing Layer 4 Forwarding Rules

To define kube-proxy in a single sentence:

kube-proxy turns the stable virtual entry point declared by a Kubernetes Service into Layer 4 forwarding rules enforced on every node.

This role comes down to four properties:


Why Do We Need Services and Virtual IPs?

In Kubernetes, Pods are ephemeral. Rolling updates, node failovers, and horizontal autoscaling constantly replace Pods, and every replacement gets a new Pod IP.

If client Pods cached backend Pod IPs directly, every client would have to implement health checks, cache invalidation, load-balancing algorithms, and race-condition handling whenever endpoints changed. The Service mechanism separates these two responsibilities:

  1. Client Pod
  2. Send to Service ClusterIP: stable virtual IPService ClusterIP
  3. Kernel DNATSelected Backend PodDynamic Pod IP
  1. Clients talk to a stable virtual entry point: They communicate through a fixed Service DNS name and ClusterIP.
  2. The control plane dynamically maintains the backend set: The EndpointSlice controller in the control plane (covered next) continuously tracks which Pods are healthy and ready.

Note: When you need to bypass virtual IP proxying—for example, with a Headless Service (clusterIP: None)—CoreDNS returns all backend Pod IPs directly, leaving load balancing to the client.


How the Five Networking Components Work Together

Internal service access in Kubernetes relies on a tight division of labor across control plane controllers, a node daemon, and the Linux kernel:

  1. ServiceVIP definition
  2. EndpointSlice Controller reconciles selectorEndpointSliceReady endpoint IPs & ports
  3. Watch APIkube-proxyNode Daemon
  4. Write rulesLinux Kernel Datapathiptables / IPVS / nftables
  1. Client Pod
  2. Send to VIP:PortLinux Kernel Datapath
  3. Kernel DNAT to Pod IP & route via CNIBackend Pod

1. Service: Declaring the Entry Point and Intent

When a developer creates a Service, the Kubernetes API server allocates a ClusterIP (for example, 10.96.0.50). This object simply records an intent in etcd: “Connections sent to 10.96.0.50:80 should route to backend containers carrying the label app: api.”

2. EndpointSlice: Tracking Available Backends

The EndpointSlice controller in the control plane watches Pods matching the Service selector and compiles their IPs, ports, node placement, and readiness conditions (Ready, Serving, Terminating) into EndpointSlice objects (solving the performance and update-broadcast bottlenecks that the legacy monolithic Endpoints object caused in large clusters).

EndpointSlices serve as kube-proxy’s source of truth when choosing forwarding targets. If a Pod has not passed its Readiness Probe, its address is excluded from the active forwarding endpoints.

3. kube-proxy: The Reconciling Rule Controller

On every node, kube-proxy watches the API server for changes to Services and EndpointSlices. Internally, it runs a control loop: whenever it detects updates, it batches the changes and triggers a local sync, updating the kernel forwarding rules to match the latest desired state.

4. Linux Kernel Data Plane: Per-Packet Forwarding

Once kube-proxy writes the rules, packets entering the Linux kernel’s network stack are intercepted on the fly. Following those rules, the kernel picks a suitable backend endpoint and performs DNAT to rewrite the destination address from the ClusterIP to the Pod IP. The entire process happens in kernel space without switching back to a user-space application.

5. CNI and the Pod Network: Underlying Packet Delivery

After the kernel completes DNAT, the packet’s destination IP is now an actual Pod IP. At this point, the underlying network topology built by the CNI (Container Network Interface) plugin takes over, delivering the packet to the target Pod on the same node or across nodes.


Why Decoupling the Control and Data Planes Matters

Separating the control plane (declaring state) from the data plane (forwarding packets) brings three main advantages:


Common Misconceptions Clarified

When learning or troubleshooting Kubernetes service networking, three misconceptions come up repeatedly:

Misconception 1: Does restarting kube-proxy immediately drop connections to Services?

No. Kernel forwarding rules live in the Linux kernel’s memory, independent of the kube-proxy process lifecycle.

When kube-proxy restarts or goes temporarily offline, the configured forwarding rules normally remain in effect, and connections to existing Services continue to flow. The main impact during that window is that the rules stop tracking current state: if Pods are added or deleted, or their readiness state changes, the node cannot immediately sync the latest endpoint list, and Services created during the outage do not get forwarding rules yet.

Misconception 2: Are HTTP requests distributed evenly across backend Pods?

Not necessarily. kube-proxy is a Layer 4 (transport layer) mechanism. The Linux kernel’s conntrack (connection tracking) module selects a backend endpoint and applies DNAT only when it receives the initial TCP SYN packet. Once the connection state is established, every subsequent packet reuses that exact same NAT mapping.

As a result, when applications use HTTP/2, gRPC, keep-alive connections, or connection pools, a large volume of HTTP requests shares a single long-lived TCP connection, which lands all of those requests on the same Pod.

If you need per-request load balancing based on HTTP headers or paths, use a Layer 7 Ingress controller or the Gateway API.

Misconception 3: Does a cluster need kube-proxy in order to use Service virtual IPs?

Not necessarily. kube-proxy is Kubernetes’ default component for synchronizing Service forwarding rules, but it is not the only option.

If a cluster runs a modern eBPF-capable CNI—such as Cilium’s kube-proxy-replacement—the CNI attaches eBPF programs directly in the Linux kernel: a cgroup hook performs ClusterIP address translation at the socket layer, the TC (traffic control) layer handles DNAT, and XDP serves as an optional accelerator. In this architecture, Services and EndpointSlices normally continue to work even with kube-proxy completely disabled.


In Part 2, we will step into the Linux kernel data plane, trace concrete IP topologies across iptables, nftables, and IPVS, and follow the SNAT and cross-node packet flows that NodePort and LoadBalancer introduce.