Pod admission, Pod Security and priority

Where admission controllers sit in the API request path and how to see which are enabled, Pod Security Admission with namespace labels and a restricted-compliant Pod, and PriorityClasses with preemption.

This is the admission half of Configure Pod admission and scheduling (limits, node affinity, etc.). Admission is where a Pod gets defaulted, mutated or rejected after you are authorized but before it is stored. LimitRange and ResourceQuota from the earlier lesson are admission plugins; this lesson covers how plugins are switched on, Pod Security Admission, and the Priority plugin that feeds preemption.

Where admission sits

Every write to the API server passes authentication, then authorization (RBAC), then mutating admission, schema validation, validating admission, and only then reaches etcd. Any validating step can reject the request with a message that kubectl prints. Built-in plugins are compiled into kube-apiserver and switched on with --enable-admission-plugins, which adds to a default set, and --disable-admission-plugins. Webhooks and CEL-based admission policies add custom rules without touching the flags.

# Explicitly enabled plugins (exam: ssh to the control plane node, then sudo grep)
docker exec cka-control-plane grep -- --enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml
#    - --enable-admission-plugins=NodeRestriction

# The same flag from the API, without a node shell
kubectl -n kube-system describe pod -l component=kube-apiserver | grep admission

# The default set for this exact version
kubectl -n kube-system exec kube-apiserver-cka-control-plane -- kube-apiserver -h | grep -A2 -- '--enable-admission-plugins strings'

Changing the flag means editing /etc/kubernetes/manifests/kube-apiserver.yaml on the control plane. The kubelet notices the file change and recreates the static Pod; the API is unavailable for up to a minute, and a typo keeps it down. Back the file up outside the manifests directory first, and wait for kubectl get nodes to answer again before the next question.

Pod Security Admission

A namespace opts in with labels of the form pod-security.kubernetes.io/<mode>: <level>. enforce rejects violating Pods, warn returns a warning to the client, audit annotates the audit log. Each mode can be pinned with <mode>-version (for example latest). Warn and audit also check the Pod templates of Deployments and other workload objects; enforce applies to Pods only, so a Deployment whose Pods are rejected looks fine until you read its ReplicaSet's events.

# Dry run on the server: lists existing Pods that would violate, changes nothing
kubectl label --dry-run=server --overwrite ns wl-psa pod-security.kubernetes.io/enforce=restricted

kubectl label --overwrite ns wl-psa pod-security.kubernetes.io/enforce=restricted pod-security.kubernetes.io/warn=restricted
kubectl get ns wl-psa --show-labels

# A rejected Pod names every missing field:
#   violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (...), unrestricted capabilities (...),
#   runAsNonRoot != true (...), seccompProfile (...)

apiVersion: v1
kind: Pod
metadata:
  name: tidy
  namespace: wl-psa
spec:
  securityContext:              # Pod level: applies to every container
    runAsNonRoot: true
    runAsUser: 1000             # needed when the image would run as root
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: busybox:1.36
      command: ["sleep", "3600"]
      securityContext:          # container level: these two only exist here
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]

runAsNonRoot: true alone is not enough for images that default to root, such as busybox and nginx: the API admits the Pod, then the kubelet refuses to start the container with container has runAsNonRoot and image will run as root. Set a numeric runAsUser, or use an image built to run unprivileged. Labeling a namespace never evicts Pods that are already running, which is why the dry run above is the honest check.

PriorityClass and preemption

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: business-critical
value: 1000000                   # higher wins; user classes go up to 1,000,000,000
globalDefault: false             # true makes it the default for Pods without a class
preemptionPolicy: PreemptLowerPriority   # or Never
description: "Customer-facing services"
---
apiVersion: v1
kind: Pod
metadata:
  name: checkout
  namespace: wl-prio
spec:
  priorityClassName: business-critical
  containers:
    - name: nginx
      image: nginx:1.27
# Imperative: kubectl create priorityclass business-critical --value=1000000 --description="Customer-facing services"

The Priority admission plugin copies the class's value into spec.priority at creation; a Pod naming a class that does not exist is rejected, and a Pod without one gets the global default or 0. The scheduler queues higher-priority Pods first. If one cannot fit anywhere, it looks for a node where evicting lower-priority Pods would make room, gracefully terminates them, and records the node in the Pod's status.nominatedNodeName; PodDisruptionBudgets are respected only on a best-effort basis. preemptionPolicy: Never keeps the queue position but never evicts anyone. The built-in system-cluster-critical and system-node-critical classes are reserved for cluster components. A class's value cannot be edited; delete and re-create it, and remember that existing Pods keep the priority they were admitted with.