Post cover image

May 23, 2026

Part 5: The Compute Layer — Karpenter for Nodes, HPA for Pods

Tested against EKS 1.31 / 1.32 and Karpenter 1.11.x. If you’re on an older Karpenter (v1beta1), some of the field names below will not…

By M Bilal

12 min read

Tested against EKS 1.31 / 1.32 and Karpenter 1.11.x. If you're on an older Karpenter (v1beta1), some of the field names below will not apply — check your CRDs before copying anything.

Where we are in the series

Parts 1 through 4 covered the parts of an EKS cluster that don't move much once you build them: the VPC, the control plane settings, the IAM story, and Pod Identity. They're the bones. Part 5 is about the part that has to move — the compute layer — and doing it in a way that the finance team doesn't show up at your desk at the end of the month.

The default temptation when you first run EKS in production is to over-provision a couple of large managed node groups and call it a day. It works. It also wastes a meaningful fraction of your AWS bill on idle CPU. The other failure mode is the opposite — node groups sized so tight that the first traffic spike pages somebody at 2 a.m. Neither is acceptable on a serious platform.

This part wires up two things that, together, get you out of that trap:

  • HPA scales pods in response to CPU, memory, or custom metrics.
  • Karpenter scales nodes in response to pods that HPA (or anything else) has marked unschedulable.

Used together they form a feedback loop: load goes up → HPA adds replicas → some replicas can't be scheduled → Karpenter launches the right-shaped EC2 instance → pods land → load drops → Karpenter consolidates the cluster back down. That loop is the whole game.

Why I'm not recommending Cluster Autoscaler here

Cluster Autoscaler is still a fine tool, and if you're running across AWS, Azure, and GCP and need one mental model, it's the pragmatic choice. The upstream kubernetes-sigs/karpenter project is generalizing toward multi-cloud and Azure already has a working provider, but in mid-2026 CA is still the safer bet outside AWS.

Inside AWS, though, the tradeoffs have shifted. CA is bound to Auto Scaling Groups, which means you spend your time managing node groups instead of describing workloads. If a pod needs r7g.xlarge and your ASG is m5.large, CA can't help you — you build another ASG. Production clusters I've inherited often have a dozen or more node groups for exactly this reason, and the operational cost of keeping them all patched and right-sized adds up.

Karpenter inverts the model. You describe what your workloads need; Karpenter picks an instance from the entire EC2 catalog that satisfies the constraints at the lowest price. It calls CreateFleet directly with a wide override list (the API allows up to 60 overrides per request, which is usually plenty), so you tend to actually get Spot capacity even when a single instance family is exhausted.

The other thing Karpenter does that CA doesn't is consolidation — actively repacking pods onto fewer or cheaper nodes when it can. In real workloads I've seen this drive 40–70% savings versus a static On-Demand baseline, depending on how Spot-tolerant the workload is. The 80%+ numbers you'll see in marketing decks are real but assume aggressive Spot adoption and forgiving SLAs.

What's actually happening when Karpenter scales

Worth understanding before you debug it at 3 a.m.:

  1. Batching. Karpenter watches for unschedulable pods and groups them so a single node launch can serve several pods at once. It's bin-packing, not one-pod-one-node.
  2. NodeClaim. This is the object Karpenter creates to represent its intent to bring up a node. kubectl describe nodeclaim is the first place to look when scale-out stalls — it tells you whether the failure is on the AWS side (ICE errors, capacity, quota) or the Kubernetes side (registration timeout).
  3. EC2 Fleet call. Karpenter calls CreateFleet with the override list it derived from your NodePool requirements.
  4. Registration and initialization. Instance boots, joins the cluster, Karpenter applies labels/taints, the node moves to Ready, taints come off, scheduler places pods.

The whole loop is usually 30–60 seconds end-to-end. CA on the same workload tends to land around 2–3 minutes once you count the ASG polling interval plus the EC2 launch. Both numbers depend on AMI size and bootstrap script, so measure your own.

Installing the controller

The pattern I use, and the one this article assumes, is to run the Karpenter controller itself on the Managed Node Group from Part 2. Karpenter cannot bootstrap the nodes it runs on — that's a circular dependency — so something has to host it. A small, statically-sized MNG is the simplest answer.

IAM: two identities, kept separate

There are two roles in play and confusing them is the most common installation bug:

  • Controller role. Attached to the Karpenter pod via Pod Identity (Part 4). This is what calls ec2:CreateFleet, ec2:TerminateInstances, etc.
  • Node role. Assumed by the EC2 instances that Karpenter launches. This is what lets the kubelet pull from ECR, send logs to CloudWatch, and join the cluster.

Reusing the Part 2 node-group role for Karpenter-launched nodes works and keeps things simple, but it does broaden the blast radius — if that role gets over-permissioned, everything inherits it. On larger or more security-sensitive clusters I create a dedicated KarpenterNodeRole-<cluster> so the two populations of nodes have separate identities. I'll show that pattern below, and you can pick the trade-off you want.

Terraform

module "karpenter" {
  source       = "terraform-aws-modules/eks/aws//modules/karpenter"
  cluster_name = module.eks.cluster_name

  # Pod Identity, not IRSA. This is the current best practice.
  enable_pod_identity             = true
  create_pod_identity_association = true

  # Create a dedicated node role for Karpenter-launched instances.
  # Set create_node_iam_role = false and pass node_iam_role_arn
  # if you want to reuse the Part 2 MNG role instead.
  create_node_iam_role = true
  node_iam_role_name   = "KarpenterNodeRole-${module.eks.cluster_name}"

  tags = {
    Environment = "prod"
    Owner       = "platform"
  }
}

Helm Release

resource "helm_release" "karpenter" {
  namespace        = "karpenter"
  create_namespace = true

  name       = "karpenter"
  repository = "oci://public.ecr.aws/karpenter"
  chart      = "karpenter"
  # Pin to a current 1.x patch you've tested. Don't track latest in prod.
  version    = "1.11.1"

  values = [
    <<-EOT
    replicas: 2
    settings:
      clusterName:       ${module.eks.cluster_name}
      clusterEndpoint:   ${module.eks.cluster_endpoint}
      interruptionQueue: ${module.karpenter.queue_name}
    controller:
      resources:
        requests:
          cpu: 500m
          memory: 512Mi
        limits:
          cpu: 1
          memory: 1Gi
    # Datadog Agent autodiscovery — picks up the /metrics endpoint
    # and ships everything into Datadog as karpenter.* metrics.
    podAnnotations:
      ad.datadoghq.com/controller.checks: |
        {
          "openmetrics": {
            "instances": [
              {
                "openmetrics_endpoint": "http://%%host%%:8000/metrics",
                "namespace": "karpenter",
                "metrics": [".*"],
                "max_returned_metrics": 2000
              }
            ]
          }
        }
    nodeSelector:
      intent: system
    tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
    topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app.kubernetes.io/name: karpenter
    EOT
  ]
}

A few things worth pointing out:

  • Two replicas, spread across AZs. Karpenter is a leader-elected controller; one pod is active, the other is a hot standby. Single-replica installs are fine for dev, not for prod.
  • clusterEndpoint is set explicitly. Some installer paths discover it automatically, but being explicit avoids a class of "works on my cluster" bugs.
  • Resource requests on the controller itself. Without these you'll eventually see the Karpenter pod get evicted under memory pressure on a busy cluster — embarrassing, given what it does for a living.
  • interruptionQueue connects to the SQS queue the Terraform module created. This is what gives Spot the two-minute grace period to drain before AWS reclaims the instance. Don't run Spot in production without it.

Describing what you want: NodePools and EC2NodeClasses

The mental model: EC2NodeClass describes the AWS-side infrastructure (subnets, security groups, AMI, role). NodePool describes the policy (what instance types are allowed, Spot vs On-Demand, how aggressively to consolidate, lifetime).

You almost always want more than one NodePool. A common layout:

  • A system NodePool, On-Demand only, weighted high, for workloads you don't want disrupted (ingress controllers, observability agents).
  • A default NodePool, Spot-preferred, for stateless application workloads.
  • Optionally a batch or gpu NodePool with its own taints for specialized work.

EC2NodeClass

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  # In v1 you must declare amiSelectorTerms. The `alias` form is the
  # idiomatic way to pick an AL2023 image. Pin a version in prod rather
  # than tracking @latest so node rotations are deterministic.
  amiSelectorTerms:
    - alias: al2023@latest

  role: "KarpenterNodeRole-${cluster_name}"   # must match Terraform above

  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${cluster_name}"

  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "${cluster_name}"

  tags:
    karpenter.sh/discovery: "${cluster_name}"
    Environment: prod
    Owner: platform
    intent: apps

Two things people miss here. First, in v1 the schema requires amiSelectorTerms — amiFamily alone is no longer sufficient and the cluster will reject the manifest. Second, @latest is convenient but it means every node rotation might pick up a new AMI; for production I pin to a specific dated alias and bump it deliberately.

NodePools

A system pool first, for things that must not be evicted onto Spot:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: system
spec:
  weight: 100
  template:
    metadata:
      labels:
        intent: system
    spec:
      taints:
        - key: CriticalAddonsOnly
          effect: NoSchedule
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      expireAfter: 720h        # rotate nodes every 30 days
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 1m
    budgets:
      - nodes: "1"             # never disrupt more than one system node at a time
  limits:
    cpu: 200
    memory: 200Gi

Then a default pool, the one most of your workloads will land on:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  weight: 10
  template:
    metadata:
      labels:
        intent: apps
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["5"]
        - key: karpenter.k8s.aws/instance-size
          operator: NotIn
          values: ["nano", "micro", "small"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["us-east-1a", "us-east-1b", "us-east-1c"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      expireAfter: 336h        # 14 days
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 1m
    budgets:
      - nodes: "10%"           # cap disruptions at 10% of the pool
      - nodes: "0"             # ...and never during business hours
        schedule: "0 9 * * mon-fri"
        duration: 8h
  limits:
    cpu: 1000
    memory: 1000Gi

A handful of details worth dwelling on, because they're the ones that bite people:

  • WhenEmptyOrUnderutilized, not WhenUnderutilized. That older value was a v1beta1 thing. On a v1 NodePool the schema will reject it. The two valid values in v1 are WhenEmpty (only delete empty nodes) and WhenEmptyOrUnderutilized (also do replacement consolidation).
  • Disruption budgets are not optional in production. Without them, Karpenter can in theory consolidate a meaningful fraction of your fleet at once. The example above caps churn at 10% of the pool and blocks all consolidation during the weekday business window — pick numbers that match your real risk tolerance.
  • expireAfter forces rotation. Even if nothing else changes, you want your nodes cycled regularly so AMI patches, kernel updates, and config drift all get reset. 720h (30 days) for system, 336h (14 days) for workloads is a reasonable starting point.
  • limits is a guardrail against bill shock, not a target. It's an aggregate cap across the whole NodePool — if you hit it, Karpenter stops launching new nodes, period. Set it well above your real expected peak and treat it as a circuit breaker.
  • Instance generation and size filters stop Karpenter from picking up ancient or pathologically small instance types. Without them you can end up with t2.nano nodes serving production traffic, which is a story you don't want to tell.

A note on consolidation

Two kinds:

  • Empty-node consolidation. A node has zero non-DaemonSet pods; Karpenter terminates it. Cheap and uncontroversial.
  • Replacement consolidation. Karpenter notices that the pods on two m5.large nodes could fit on one m5.xlarge, and the xlarge costs less than two larges. It cordons the originals, brings up the replacement, drains, and terminates. This is where the real money comes from, and also where you most need disruption budgets and PDBs.

Speaking of PDBs: if you have a workload that genuinely cannot be disrupted (a stateful database, a long-running batch job), set a PodDisruptionBudget on it, or annotate the pod with karpenter.sh/do-not-disrupt: "true". Karpenter respects both.

The other half: HPA

Karpenter alone gives you nodes that match your pods. You still need something to scale the pods in the first place. That's the Horizontal Pod Autoscaler.

The wiring is straightforward. You need:

  1. The Kubernetes Metrics Server installed (most EKS clusters already have it, but verify with kubectl top nodes).
  2. Resource requests set correctly on the deployment — HPA can't compute utilization without them.
  3. An HPA object pointing at the deployment.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image: my-registry/api:1.4.2
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1
              memory: 512Mi
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300   # don't yo-yo
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30

The behavior block is the part most teams skip and later regret. The defaults are conservative on scale-up and aggressive on scale-down, which tends to produce visible latency during traffic ramps. The settings above scale up fast, scale down slowly, and use a five-minute stabilization window to stop the autoscaler from oscillating during spiky traffic.

For anything driven by metrics other than CPU and memory — queue depth, request rate, custom Prometheus metrics — you'll want KEDA in front of HPA. That's beyond the scope of this part, but flag it for Part 7 if the series gets one.

How HPA and Karpenter compose

Worth being explicit about this because new operators get confused: HPA and Karpenter don't talk to each other. They don't need to.

The chain is:

  1. HPA sees CPU climb past 70%, asks the deployment for more replicas.
  2. The scheduler tries to place the new pods. Some don't fit on existing nodes.
  3. Karpenter sees pending pods, launches a node sized to fit them.
  4. New nodes register, pods land, CPU comes back down.
  5. Later, HPA scales replicas back, leaving empty space on nodes.
  6. Karpenter's consolidation loop removes or replaces underutilized nodes.

Each loop is independent and the system as a whole is robust because of it. The thing you have to get right is making sure resource requests are honest — if your pods request 100m CPU but actually use 800m, HPA will scale far too aggressively and Karpenter will dutifully launch nodes you don't need.

Observability with Datadog

You cannot operate Karpenter blind. Throughout this series we've been using Datadog as the observability layer (set up in an earlier part), so the question for Part 5 is: how do we make sure the Karpenter controller and the nodes it launches show up there cleanly?

The controller exposes Prometheus-format metrics on :8000/metrics. The Helm values above already include a Datadog Agent autodiscovery annotation, so as long as your Agent is running in cluster mode with autodiscovery enabled, it'll pick the endpoint up automatically and ship the metrics into Datadog under the karpenter.* namespace. No ServiceMonitor, no Prometheus, no extra scrape config.

The Karpenter-specific metrics I keep on a dashboard:

  • karpenter.nodepools.usage and karpenter.nodepools.limit — how close each NodePool is to its limits block. If usage climbs toward limit during a normal week, your guardrails are too tight.
  • karpenter.nodeclaims.launched, .registered, .disrupted — the lifecycle counts. A widening gap between launched and registered almost always means AWS-side capacity errors (ICE), subnet misconfiguration, or bootstrap problems on the new node.
  • karpenter.pods.state (tagged by phase) — pods Karpenter is currently treating as unschedulable.
  • karpenter.disruption.evaluation_duration_seconds — how long the consolidation loop is taking. A sudden jump here usually means you've grown the cluster faster than the controller can reason about it; check controller CPU.
  • karpenter.cloudprovider.instance_type_price_estimate (tagged with instance_type, capacity_type, zone) — useful when you want to confirm Karpenter is actually preferring cheaper instances.

For the nodes themselves, the standard Datadog Kubernetes integration already covers them — the Cluster Agent picks them up the moment they register, and DaemonSet-deployed node Agents start emitting kubernetes.cpu.* and kubernetes.memory.* immediately. The thing worth doing is making sure the tags Karpenter applies to instances (intent, karpenter.sh/nodepool, karpenter.sh/capacity-type) flow through to Datadog. Set the Datadog Agent's DD_KUBERNETES_NODE_LABELS_AS_TAGS to include those keys — without it, you can't slice a cost or saturation dashboard by NodePool, which is the whole point of running multiple pools.

A few monitors worth creating before you turn Spot loose on real workloads:

  • karpenter.nodeclaims.launched - karpenter.nodeclaims.registered rising for more than a few minutes — early signal for ICE storms or AMI bootstrap failures.
  • Karpenter controller pod restarts (kubernetes.containers.restarts{kube_deployment:karpenter} > 0 over 10m).
  • karpenter.nodepools.usage{nodepool:default} / karpenter.nodepools.limit{nodepool:default} > 0.8 — you're approaching your guardrail and likely about to stop scaling.
  • Spot interruption rate (aws.ec2.spot.interruption from the AWS integration, sliced by your cluster's eks:cluster-name tag) above whatever baseline your workloads can absorb.

Datadog also ships an out-of-the-box Karpenter integration dashboard that wires most of the above up if you'd rather not build one from scratch. It's a fine starting point, but I'd treat it as the floor of what you monitor, not the ceiling.

Verifying the install

The smoke test I run on every new install:

# 1. The controller is up, two replicas, leader elected.
kubectl -n karpenter get pods
kubectl -n karpenter logs -l app.kubernetes.io/name=karpenter | grep -i leader

# 2. NodePools and EC2NodeClass are healthy.
kubectl get nodepool,ec2nodeclass

# 3. Force a scale-out and watch what happens.
kubectl create deployment burst --image=public.ecr.aws/eks-distro/kubernetes/pause:3.9 --replicas=50
kubectl get nodeclaims -w
kubectl get nodes -l intent=apps -w

# 4. Scale back to zero and watch consolidation.
kubectl scale deployment burst --replicas=0
kubectl get nodes -l intent=apps -w

If step 3 stalls, kubectl describe nodeclaim <name> is where the truth lives — it'll show you whether AWS returned an ICE error, whether the subnet/security-group selectors matched anything, whether the registration timed out, etc.

What this part actually buys you

Concretely, from clusters I've moved from CA to a Karpenter-plus-HPA setup like this one:

  • Scale-out time from pending pod to scheduled pod dropped from roughly 2–3 minutes to roughly 30–60 seconds.
  • Steady-state node count dropped meaningfully — usually 20–40% — because consolidation packs things tighter than ASGs ever did.
  • Compute cost dropped further on top of that once Spot was turned on with the interruption queue, typically into the 40–70% range against an On-Demand baseline. The exact number depends entirely on how Spot-tolerant your workloads are.
  • Number of node groups we had to manage went from "a list" to "two NodePools plus a small system MNG."

None of that comes for free — you have to actually configure disruption budgets, PDBs, and pod priorities, and you have to watch the metrics. But the ceiling is high.

What's in Part 6

Part 6 closes out the series with secrets management — wiring AWS Secrets Manager into the cluster via the Secrets Store CSI Driver, so application credentials never sit in a Kubernetes Secret object in plain etcd. The compute layer is now elastic; Part 6 makes sure the things it runs aren't leaking their credentials.