CRDs and operators

How a CustomResourceDefinition adds a new API, how to read the ones a cluster already has, and how operators pair CRDs with a controller you install and configure.

Architecture · 25%35 min2 tasks

After this lesson

01You can write a CustomResourceDefinition with a validated schema, short names and printer columns, then create and query custom resources.
02You can find which CRD and API group serve a given kind, its scope, and its served and storage versions.
03You can install an operator from manifests or a chart and list exactly which CRDs it brought in.

The competency is Understand CRDs, install and configure operators. Much of what you install on a real cluster arrives this way: the Gateway API, cert-manager, Prometheus Operator, Calico's own configuration. Exam tasks ask you to create a small CRD from a specification, to inspect installed ones (which group, which scope, which version is stored), or to install an operator and configure it through its custom resources. Everything here is plain kubectl; no extra tooling is needed.

Anatomy of a CustomResourceDefinition

A namespaced CRD with validation, a default, printer columns and a status subresourceyaml

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: cacheclusters.data.example.com     # must be <plural>.<group>
spec:
  group: data.example.com
  scope: Namespaced                        # or Cluster
  names:
    kind: CacheCluster                     # used in manifests
    plural: cacheclusters                  # used in URLs and kubectl get
    singular: cachecluster
    shortNames: ["cc"]
    categories: ["all"]                    # kubectl get all now includes it
  versions:
    - name: v1
      served: true                         # the API answers on /apis/data.example.com/v1
      storage: true                        # exactly one version is written to etcd
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: ["engine"]
              properties:
                engine:
                  type: string
                  enum: ["redis", "memcached"]
                replicas:
                  type: integer
                  minimum: 1
                  maximum: 9
                  default: 1
                version:
                  type: string
            status:
              type: object
              x-kubernetes-preserve-unknown-fields: true
      subresources:
        status: {}                         # status is written via /status, spec edits bump generation
      additionalPrinterColumns:
        - name: Engine
          type: string
          jsonPath: .spec.engine
        - name: Replicas
          type: integer
          jsonPath: .spec.replicas
        - name: Age                        # custom columns replace the default AGE; add it back
          type: date
          jsonPath: .metadata.creationTimestamp

Structural schema. apiextensions.k8s.io/v1 requires a schema for every version, with a type on every field. Unknown fields are pruned on write unless you mark a subtree x-kubernetes-preserve-unknown-fields: true. The schema is also what kubectl explain reads, so a CRD without descriptions explains as <empty>.
Validation comes from the schema keywords (required, enum, minimum, pattern, maxLength) and from CEL rules in x-kubernetes-validations. Invalid objects are rejected by the API server with a message naming the field.
Versions. Several versions may be served; exactly one has storage: true. status.storedVersions lists every version that may still exist in etcd. If schemas differ between versions, the CRD needs a conversion webhook; with conversion.strategy: None only apiVersion is rewritten.
Establishment. A new CRD reports NamesAccepted and then Established; only after that can custom resources be created. kubectl wait --for=condition=Established crd/<name> removes the race in scripts.
Deletion cascades. Deleting a CRD deletes every custom resource of that kind in every namespace. Treat kubectl delete crd like deleting a namespace.

Discover and inspect what a cluster already has

From a kind to its CRD, group, scope and versionsbash

kubectl get crd                                        # every CRD and when it was created
kubectl api-resources --api-group=gateway.networking.k8s.io   # NAME, SHORTNAMES, APIVERSION, NAMESPACED, KIND
kubectl api-resources | grep -i referencegrant          # find the resource for a kind you were given
kubectl explain referencegrants                         # served version, fields, descriptions
kubectl explain httproutes.spec.parentRefs --api-version=gateway.networking.k8s.io/v1
kubectl explain cacheclusters.spec --recursive          # your own CRD, whole tree

crd=referencegrants.gateway.networking.k8s.io
kubectl get crd $crd -o jsonpath='{.spec.group}{" "}{.spec.scope}{" "}{.spec.names.kind}{"\n"}'
kubectl get crd $crd -o jsonpath='{range .spec.versions[*]}{.name}{" served="}{.served}{" storage="}{.storage}{"\n"}{end}'
kubectl get crd $crd -o jsonpath='{.status.storedVersions}{"\n"}'
kubectl get crd $crd -o jsonpath='{range .status.conditions[*]}{.type}={.status}{" "}{end}{"\n"}'

# All CRDs of one group, and every object in a category
kubectl get crd -o custom-columns=NAME:.metadata.name,GROUP:.spec.group,SCOPE:.spec.scope | grep projectcalico
kubectl get gateway-api -A

Operators: CRDs plus a controller

A CRD on its own only stores data. An operator adds a controller, usually a Deployment with a ServiceAccount and ClusterRole, that watches its custom resources and reconciles the cluster toward them: you create a Certificate and cert-manager produces a Secret; you create a Postgres cluster resource and the operator builds StatefulSets, Services and backups. Installing one means applying its CRDs, its RBAC and its controller, from a release manifest or a Helm chart. Configuring it means creating its custom resources, plus any flags or ConfigMap the project documents. Big CRDs can exceed the 256 KiB annotation that client-side kubectl apply writes, which is why many projects tell you to use kubectl apply --server-side or kubectl create.

Install an operator from its release manifest and see what it broughtbash

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.2/cert-manager.yaml
kubectl -n cert-manager rollout status deploy/cert-manager-webhook --timeout=180s

kubectl get crd -l app.kubernetes.io/name=cert-manager    # if the project labels its CRDs
kubectl get crd | grep cert-manager.io                    # otherwise filter by group
kubectl api-resources --api-group=cert-manager.io
kubectl -n cert-manager get deploy,sa                     # the controller side
kubectl get clusterroles | grep cert-manager

# Configure it through a custom resource
kubectl apply -f - <<'YAML'
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}
YAML
kubectl get clusterissuers

# Remove it: custom resources first, then the manifest (this deletes the CRDs and any remaining CRs)
kubectl delete clusterissuer selfsigned
kubectl delete -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.2/cert-manager.yaml

CRDs installed by	Upgrades	Removal
`kubectl apply -f` of a release manifest	Apply the newer manifest	`kubectl delete -f` removes the CRDs and every custom resource
A Helm chart's `crds/` directory	Installed on first install only; Helm never upgrades them	Never deleted by `helm uninstall`; delete them yourself. `--skip-crds` skips them at install
Helm templates behind a value (often `crds.enabled` or `installCRDs`)	Upgraded with the release	Deleted with the release unless the chart marks them `helm.sh/resource-policy: keep`

In the docs · the tab you get on exam day

Extend the Kubernetes API with CustomResourceDefinitions ↗search: “customresourcedefinition”
Custom Resources ↗search: “custom resources”
Versions in CustomResourceDefinitions ↗search: “crd versioning”
Operator pattern ↗search: “operator pattern”

Taskar-crd-1

Architecture7%~9 min

context · kubectl config use-context kind-cka

Create a CRD with validation, then a custom resource

Create a CustomResourceDefinition for the kind BackupPolicy in API group ops.example.com: version v1 (served and used for storage), namespaced, plural backuppolicies, singular backuppolicy, short name bp. Its schema must require spec.schedule (a string) and spec.retentionDays (an integer of at least 1). kubectl get bp must show the columns Schedule and Retention. Then create a BackupPolicy named nightly in namespace arx-crd with schedule 0 2 * * * and retentionDays 14.

01
Set up the scenario
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/setup | bash
02
Solve it in your terminal
Work the way the exam makes you: kubectl, the docs, nothing else. Put any answer files under /tmp/cka/.
03
Check your work
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/verify | bash
Counts toward your readiness score.
04
Clean up
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/cleanup | bash

Taskar-crd-2

Architecture4%~5 min

context · kubectl config use-context kind-cka

Trace a kind back to its CRD

Objects of kind ReferenceGrant can be created in this cluster. Find the CustomResourceDefinition that defines this kind and write four lines to /tmp/cka/crd/referencegrant.txt, in this order: the CRD's name, its API group, its scope (as the API reports it), and the version its objects are stored in. Then write the names of all CRDs that belong to the same API group, one per line, to /tmp/cka/crd/group-crds.txt.

01
Set up the scenario
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/setup | bash
02
Solve it in your terminal
Work the way the exam makes you: kubectl, the docs, nothing else. Put any answer files under /tmp/cka/.
03
Check your work
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/verify | bash
Counts toward your readiness score.
04
Clean up
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/cleanup | bash

Finished reading?

0 of 2 tasks passed. Readiness counts tasks, so run them before you move on.

Mark complete, next lesson →