Pod admission, Pod Security and priority

Where admission sits in the API server and which plugins are on, how Pod Security labels block unsafe Pods, and how PriorityClasses decide which Pods go first.

Before the API server saves a new Pod, it sends the request through admission. Admission is a set of plugins that can fill in values, change the request, or reject it. You have already met two of them. LimitRange and ResourceQuota from the requests and limits lesson are admission plugins.

Example: your team's namespace must never run Pods as root. A Pod Security label on the namespace makes the API server refuse such Pods before they ever start. This lesson covers how plugins are switched on, Pod Security Admission, and the Priority plugin, which decides which Pods the scheduler serves first.

Where admission sits

Every change you send to the API server goes through the same steps, in order:

• Admission works in two passes. First, mutating plugins may change the object, for example to fill in defaults. Then validating plugins may only accept or reject it.
• Built-in plugins are part of kube-apiserver. The flag --enable-admission-plugins switches more of them on; it adds to a default set.
• The flag --disable-admission-plugins turns off plugins from the default set.

# Explicitly enabled plugins (exam: ssh to the control plane node, then sudo grep)
docker exec cka-control-plane grep -- --enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml
#    - --enable-admission-plugins=NodeRestriction

# The same flag from the API, without a node shell
kubectl -n kube-system describe pod -l component=kube-apiserver | grep admission

These are the plugins worth knowing for this exam:

Other default plugins do jobs you meet in other lessons:

• The default ServiceAccount for every Pod, in RBAC.
• The default StorageClass for new claims, in StorageClasses.
• The 5-minute tolerations every Pod gets, in Worker node failures.

To change the flag, you edit the API server's static Pod file. How that works, and how to do it safely, is in Control plane and node components.

Pod Security Admission

Some Pod settings are dangerous: running as root, using the node's network, or mounting the node's disk. Kubernetes defines three Pod Security Standards, from loose to strict:

A namespace picks a level with a label of the form pod-security.kubernetes.io/<mode>: <level>. There are three modes:

One trap: warn and audit also check the Pod templates inside Deployments and other workloads, but enforce checks only Pods. So a Deployment whose Pods are rejected looks fine. You only see the problem in its ReplicaSet's events.

# Dry run on the server: lists existing Pods that would violate, changes nothing
kubectl label --dry-run=server --overwrite ns wl-psa pod-security.kubernetes.io/enforce=restricted

kubectl label --overwrite ns wl-psa pod-security.kubernetes.io/enforce=restricted pod-security.kubernetes.io/warn=restricted
kubectl get ns wl-psa --show-labels

# A rejected Pod names every missing field:
#   violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (...), unrestricted capabilities (...),
#   runAsNonRoot != true (...), seccompProfile (...)

Here is a Pod with every setting the restricted level asks for:

apiVersion: v1
kind: Pod
metadata:
  name: tidy
  namespace: wl-psa
spec:
  securityContext:              # Pod level: applies to every container
    runAsNonRoot: true
    runAsUser: 1000             # needed when the image would run as root
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: busybox:1.36
      command: ["sleep", "3600"]
      securityContext:          # container level: these two only exist here
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["ALL"]

• runAsNonRoot: true alone is not enough for images that run as root by default, such as busybox and nginx. The API admits the Pod, but then the kubelet refuses to start the container: container has runAsNonRoot and image will run as root. Set a numeric runAsUser, or use an image built to run as a normal user.
• Labeling a namespace never removes Pods that are already running. That is why the server-side dry run above is the honest check.

PriorityClass and preemption

When the cluster is full, which Pods go first? A PriorityClass gives Pods a number: the higher the number, the more important the Pod. Example: your checkout Pods get a higher priority than a nightly report, so checkout always gets room first.

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: business-critical
value: 1000000                   # higher wins; user classes go up to 1,000,000,000
globalDefault: false             # true makes it the default for Pods without a class
preemptionPolicy: PreemptLowerPriority   # or Never
description: "Customer-facing services"
---
apiVersion: v1
kind: Pod
metadata:
  name: checkout
  namespace: wl-prio
spec:
  priorityClassName: business-critical
  containers:
    - name: nginx
      image: nginx:1.27
# Imperative: kubectl create priorityclass business-critical --value=1000000 --description="Customer-facing services"

How a Pod gets its priority

• The Pod names a class in priorityClassName.
• When the Pod is created, the Priority plugin copies the class's value into spec.priority.
• A Pod that names a class that does not exist is rejected. A Pod with no class gets the global default class, or 0.
• A class's value cannot be edited: delete the class and create it again. Pods that already exist keep the priority they were given.

Preemption

• The scheduler serves higher-priority Pods first.
• If such a Pod fits on no node, the scheduler looks for a node where removing lower-priority Pods would make room. It then stops those Pods gracefully. This is called preemption.
• preemptionPolicy: Never keeps the Pod's place at the front of the queue, but it never removes anyone.