← All field notes

Why One Kubernetes Node Hit 96% Utilization While Another Sat at 48%

Diagnose uneven node utilization by reviewing pod placement, resource requests, topology spread constraints and autoscaling behavior.

Kubernetes worker nodes showing uneven utilization of 96 percent, 48 percent and 51 percent.

A cluster was set to autoscale between 2 and 10 nodes. On paper, it had room. In practice, one worker sat at 96% CPU, another at 78%, and a third at 48%.

The instinct is to add nodes. But more nodes don't help if the scheduler keeps stacking the busiest pods on the same machine. Before adding infrastructure, we asked one question: why was one worker at 96% while another sat at 48%?

1. Start with pod placement

Node metrics show how busy each worker is, not which workloads cause it. First, see where the pods actually landed:

bash
kubectl get pods -n <namespace> -o wide

Then compare with node usage:

bash
kubectl top nodes

A common assumption is that three replicas on a three-node cluster land one per node. Kubernetes does not guarantee that. Without spread rules, two or even three replicas can end up on the same node.

2. Requests vs actual usage

The scheduler places pods using their resource requests, not their real usage. If a pod requests 100m CPU but uses 900m, the scheduler thinks the node has plenty of room and keeps adding pods to it.

See what each node has been promised:

bash
kubectl describe node <node-name>

Look at the "Allocated resources" section near the bottom. Then compare with real usage per pod:

bash
kubectl top pods -n <namespace>

3. What usually causes it

When one node runs hot and others sit idle, we usually find a combination of three things:

1 of 2 sections shown

Anees Khan

Co-Founder & CTO at Turf Dev, focused on Kubernetes, platform engineering, cloud infrastructure, and production reliability. I write about real-world infrastructure problems, how we investigate them, and the engineering approaches used to solve them.

Meet Turf Dev ↗

Discussion

Have a question or something to add? Share it below.

Comments appear immediately. Please keep the discussion respectful.

No comments yet. Be the first to join the discussion.

Keep exploring

More from the field.

All articles ↗

Platform Mandate

Request Architecture Review

Review platform posture, reliability constraints, and cost governance priorities with an infrastructure engineering team.

Request Architecture Review