Autoscaling with the HPA

How the HorizontalPodAutoscaler turns CPU or memory use into a replica count, what it needs to work, how to slow it down with behavior, and where the other autoscalers fit.

Your Django app is quiet at night and busy at lunch. You do not want to pick its replica count by hand. A HorizontalPodAutoscaler (HPA) watches how much CPU or memory the Pods use, and adds or removes Pods to match. Horizontal means more or fewer Pods, not bigger ones.

On the exam, that means an HPA with exact numbers. Sometimes it also needs a behavior block, which kubectl autoscale cannot write. The grader reads the HPA object; it does not create load and wait for scaling. It can also check whether your Pods request the resource the HPA measures. That is the step most people miss.

How the HPA decides

1. 1.The HPA reads each Pod's usage from metrics-server, which this lab has installed.
2. 2.It divides the usage by the Pod's request. That gives a percentage, called utilization.
3. 3.It averages that over all the Pods and compares it with your target.
4. 4.It sets a new replica count, kept between minReplicas and maxReplicas.

The formula is desired = ceil(current replicas × current value / target value), where ceil means round up. Example: 4 replicas average 90% CPU, and the target is 60%. Then ceil(4 × 90 / 60) = 6 replicas.

• Utilization is measured against requests. So if a container has no CPU request, the HPA cannot compute anything, and it reports missing request for cpu.
• Once an HPA manages a Deployment, leave spec.replicas out of manifests you apply again. Otherwise every apply fights the autoscaler.

So before you create an HPA, check the two things it needs: working metrics, and requests on the containers.

kubectl get apiservice v1beta1.metrics.k8s.io           # Available True means metrics-server answers
kubectl -n wl-hpa top pods                                 # usage per Pod
kubectl -n wl-hpa get deploy api -o jsonpath='{.spec.template.spec.containers[*].resources}'; echo
kubectl -n wl-hpa set resources deploy/api -c nginx --requests=cpu=100m   # add the missing request (starts a rollout)

Create one

# Current kubectl (the exam's v1.37 included)
kubectl -n wl-hpa autoscale deployment api --min=2 --max=8 --cpu=60%
# Older kubectl only knows this form; newer releases still accept it with a deprecation warning
kubectl -n wl-hpa autoscale deployment api --min=2 --max=8 --cpu-percent=60

kubectl -n wl-hpa autoscale deployment api --min=2 --max=8 --cpu=60% --dry-run=client -o yaml > hpa.yaml

Recent kubectl uses --cpu and --memory instead of the older --cpu-percent. Anything beyond min, max and one CPU or memory target needs YAML. A target comes in two types, and they are easy to mix up:

Here is a full HPA with two metrics and a behavior block:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker
  namespace: wl-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 1
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization          # percent of the request
          averageUtilization: 75
    - type: Resource
      resource:
        name: memory
        target:
          type: AverageValue         # absolute usage per Pod
          averageValue: 100Mi
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 60 # act on the highest recommendation of the last 60s
      policies:
        - type: Pods
          value: 1                   # remove at most 1 Pod ...
          periodSeconds: 30          # ... per 30 seconds

With several metrics, the HPA works out a replica count for each one and uses the largest. Other metric types exist for custom metrics, but they need an extra metrics adapter.

When one direction has several policies, selectPolicy decides which one wins:

• Max (the default): the policy that allows the biggest change.
• Min: the policy that allows the smallest change.
• Disabled: no scaling in that direction at all.

If you leave behavior out, these defaults apply. Note the 300-second window on scale-down: an HPA waits 5 minutes before it removes Pods.

Reading an HPA

kubectl -n wl-hpa get hpa
# NAME   REFERENCE        TARGETS              MINPODS   MAXPODS   REPLICAS
# api    Deployment/api   cpu: <unknown>/60%   2         8         2        <- no metrics or no requests
kubectl -n wl-hpa describe hpa api | sed -n '/Conditions/,$p'
#   ScalingActive  False  FailedGetResourceMetric  ... missing request for cpu in container nginx ...

A new HPA shows <unknown> for up to a minute, while metrics-server collects its first numbers. If it stays that way, read the condition message.

Other autoscalers

Two other autoscalers change different things. Keep the three apart:

Neither of the last two runs on kind, and the exam treats both as concepts.