Karpenter + Consolidation enabled == CostSaving + AutoScaler

Karpenter:

Karpenter is an open-source cluster autoscaler that automatically provisions new nodes in response to unschedulable pods. Karpenter evaluates the aggregate resource requirements of the pending pods and chooses the optimal instance type to run them. It will automatically scale-in or terminate instances that don’t have any non-daemonset pods to reduce waste. It also supports a consolidation feature which will actively move pods around and either delete or replace nodes with cheaper versions to reduce cluster cost.

How Karpenter Works:

Karpenter works in tandem with the Kubernetes scheduler by observing incoming pods over the lifetime of the cluster. It launches or terminates nodes to maximize application availability and cluster utilization. When there is enough capacity in the cluster, the Kubernetes scheduler will place incoming pods as usual. When pods are launched that cannot be scheduled using the existing capacity of the cluster, Karpenter bypasses the Kubernetes scheduler and works directly with your provider’s compute service, (for example, Amazon EC2), to launch the minimal compute resources needed to fit those pods and binds the pods to the nodes provisioned. As pods are removed or rescheduled to other nodes, Karpenter looks for opportunities to terminate under-utilized nodes. Running fewer, larger nodes in your cluster reduces overhead from daemonsets and Kubernetes system components and provides more opportunities for efficient bin-packing.

Benefits of Kapeneter:

1. Cost -saving/management:
2. Easy Auto-scaler:
3. k-native:
4. Zero cost:

Consolidation

Karpenter has two mechanisms for cluster consolidation:

• Deletion — A node is eligible for deletion if all of its pods can run on free capacity of other nodes in the cluster.
• Replace — A node can be replaced if all of its pods can run on a combination of free capacity of other nodes in the cluster and a single cheaper replacement node.

Consolidation has three mechanisms that are performed in order to attempt to identify a consolidation action:

1. Empty Node Consolidation — Delete any entirely empty nodes in parallel
2. Multi-Node Consolidation — Try to delete two or more nodes in parallel, possibly launching a single replacement that is cheaper than the price of all nodes being removed
3. Single-Node Consolidation — Try to delete any single node, possibly launching a single replacement that is cheaper than the price of that node

It’s impractical to examine all possible consolidation options for multi-node consolidation, so Karpenter uses a heuristic to identify a likely set of nodes that can be consolidated. For single-node consolidation we consider each node in the cluster individually.

When there are multiple nodes that could be potentially deleted or replaced, Karpenter chooses to consolidate the node that overall disrupts your workloads the least by preferring to terminate:

• nodes running fewer pods
• nodes that will expire soon
• nodes with lower priority pods

Note: For spot nodes, Karpenter only uses the deletion consolidation mechanism. It will not replace a spot node with a cheaper spot node. Spot instance types are selected with the price-capacity-optimized strategy and often the cheapest spot instance type is not launched due to the likelihood of interruption. Consolidation would then replace the spot instance with a cheaper instance negating the price-capacity-optimized strategy entirely and increasing interruption rate.

Code Sample:

---
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:

  #ttlSecondsAfterEmpty: 60 # scale down nodes after 60 seconds without workloads (excluding daemons)
  #ttlSecondsUntilExpired: 604800 # expire nodes after 7 days (in seconds) = 7 * 60 * 60 * 24
  limits:
    resources:
      cpu: 100 # limit to 100 CPU cores
  consolidation:
    enabled: true
  requirements:
    # Exclude general purpose instance families
    - key: karpenter.k8s.aws/instance-family
      operator: NotIn
      values: [c1, cc1, cc2, cg1, cg2, cr1, cs1, g1, g2, hi1, hs1, m1, m2, m3,t1]
  providerRef:
    name: my-provider
---
apiVersion: karpenter.k8s.aws/v1alpha1
kind: AWSNodeTemplate
metadata:
  name: my-provider
spec:
  subnetSelector:
    kubernetes.io/cluster/demo: owned
  securityGroupSelector:
    kubernetes.io/cluster/demo: owned

Before & After using Consolidation:

Before 7/03 Consolidation was turned off and we were using ttlSecondsAfterEmpty/ttlSecondsUntilExpired So you can spot the difference in cost of ec2 resources before and after it.

Note: Can use either ttlSecondsAfterEmpty/ttlSecondsUntilExpired or Consolidation, can’t use both at a time.

After enabling it all resources are used efficiently , like there is no wastage of resources.

NOTE: If upgrading Karpenter , update CRD’s also , see below commands:-

kubectl replace -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.sh_provisioners.yaml

kubectl replace -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.k8s.aws_awsnodetemplates.yaml

Next blog:- Full Installation with terraform on eks.