CRDs and operators

A CustomResourceDefinition adds a new kind of object to Kubernetes; an operator is a controller that acts on those objects.

35 min 2 hands-on tasksArchitecture · 25% of the exam

Key ideas in this lesson · tap one for a simple explanation

What you will be able to do

Write a CRD with required fields, a short name and extra columns, then create objects of the new kind.
Find which CRD and API group serve a kind, its scope, and which version is stored.
Install an operator and list the CRDs it brought in.

Kubernetes knows kinds such as , and . A (CRD) teaches it a new kind. Example: you install cert-manager, and suddenly kubectl get certificates works, because cert-manager added a Certificate CRD. Objects of a new kind are called custom resources.

A CRD only stores data. An operator is a program that watches those objects and does the real work. On the exam you create a small CRD from a description, inspect the CRDs a cluster already has, or install an operator. It is all plain kubectl.

How a CRD and an operator work together

CRD

adds a new kind to the API

certificates.cert-manager.io

Custom resource

one object of that kind, made by you

kind: Certificate

Operator

a controller that watches those objects

cert-manager

The result

a Secret that holds the certificate

The CRD adds the kind. You create objects of that kind. The operator watches them and does the work.

Inside a CRD

Here is a small CRD for a kind called CacheCluster. The parts you must get right on the exam are the names, the version, the schema (the allowed fields) and the extra columns.

A namespaced CRD with a schema, a short name and two extra columns· YAML

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: cacheclusters.data.example.com     # must be <plural>.<group>
spec:
  group: data.example.com
  scope: Namespaced                        # or Cluster
  names:
    kind: CacheCluster                     # used in manifests
    plural: cacheclusters                  # used in URLs and kubectl get
    singular: cachecluster
    shortNames: ["cc"]
  versions:
    - name: v1
      served: true                         # the API answers on /apis/data.example.com/v1
      storage: true                        # exactly one version is written to etcd
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: ["engine"]
              properties:
                engine:
                  type: string
                replicas:
                  type: integer
                  minimum: 1
      additionalPrinterColumns:
        - name: Engine
          type: string
          jsonPath: .spec.engine
        - name: Replicas
          type: integer
          jsonPath: .spec.replicas

One CRD gives a kind five names that look alike. Each is used in a different place:

Name	Form	Example
The CRD's own name	`<plural>.<group>`	`cacheclusters.data.example.com`
`apiVersion` of its objects	`<group>/<version>`	`data.example.com/v1`
`kind`	Capitalized, used in	`CacheCluster`
`plural`	Lowercase, used by `kubectl get` and rules	`cacheclusters`
Short name	Your shortcut for `kubectl get`	`cc`

Rules worth knowing

Schema. Every version needs a schema with a type on every field. The rejects objects that break it, with a message naming the field. kubectl explain reads the same schema.
Versions. A CRD can serve several versions, but exactly one has storage: true. That is the version its objects are saved as in . status.storedVersions lists every version that may still be in etcd.
Established. A new CRD needs a moment before you can create objects of its kind. Wait for it with kubectl wait --for=condition=Established crd/<name>.
Deleting a CRD deletes every object of its kind, in every . Treat kubectl delete crd like deleting a namespace.

Inspect the CRDs a cluster already has

Clusters often come with CRDs already installed; this lab has the and ones. To trace a kind back to its CRD, first find its plural and its group:

From a kind to its CRD· Terminal

kubectl get crd                                        # every CRD and when it was created
kubectl api-resources --api-group=gateway.networking.k8s.io   # NAME, SHORTNAMES, APIVERSION, NAMESPACED, KIND
kubectl api-resources | grep -i referencegrant          # find the resource for a kind you were given
kubectl explain referencegrants                         # served version, fields, descriptions
kubectl explain cacheclusters.spec --recursive          # your own CRD, once you have applied it

The CRD's name is the plural and the group joined by a dot, here referencegrants.gateway.networking.k8s.io. Then read the details from the CRD itself:

Read a CRD's group, scope and versions· Terminal

crd=referencegrants.gateway.networking.k8s.io
kubectl get crd $crd -o jsonpath='{.spec.group}{" "}{.spec.scope}{" "}{.spec.names.kind}{"\n"}'
kubectl get crd $crd -o jsonpath='{range .spec.versions[*]}{.name}{" served="}{.served}{" storage="}{.storage}{"\n"}{end}'
kubectl get crd $crd -o jsonpath='{.status.storedVersions}{"\n"}'
kubectl get crd $crd -o jsonpath='{range .status.conditions[*]}{.type}={.status}{" "}{end}{"\n"}'

# All CRDs of one group
kubectl get crd -o custom-columns=NAME:.metadata.name,GROUP:.spec.group,SCOPE:.spec.scope | grep projectcalico

Careful: kubectl api-resources and kubectl explain show the preferred version, which is not always the stored one. Only storage: true in the CRD tells you which version is stored.

Operators: CRDs plus a controller

An operator adds a controller to its CRDs. The controller is a program, usually a Deployment, that watches the custom resources and changes the cluster to match them. Two examples:

You create a Certificate, and cert-manager makes a that holds the certificate.
You create a Postgres object, and a database operator builds the , Services and backups for it.

To install an operator, you apply its CRDs, its RBAC and its controller, from a release or a . To configure it, you create its custom resources.

Install an operator from its release manifest and see what it brought· Terminal

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.2/cert-manager.yaml
kubectl -n cert-manager rollout status deploy/cert-manager-webhook --timeout=180s

kubectl get crd -l app.kubernetes.io/name=cert-manager    # if the project labels its CRDs
kubectl get crd | grep cert-manager.io                    # otherwise filter by group
kubectl api-resources --api-group=cert-manager.io
kubectl -n cert-manager get deploy,sa                     # the controller side
kubectl get clusterroles | grep cert-manager

# Configure it through a custom resource
kubectl apply -f - <<'YAML'
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
spec:
  selfSigned: {}
YAML
kubectl get clusterissuers

# Remove it: custom resources first, then the manifest (this deletes the CRDs and any remaining CRs)
kubectl delete clusterissuer selfsigned
kubectl delete -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.2/cert-manager.yaml

One Helm detail: Helm installs the CRDs in a chart's crds/ folder once, and never upgrades or deletes them. helm uninstall leaves them behind.

In the official docs· open on exam day too

Extend the Kubernetes API with CustomResourceDefinitionssearch: customresourcedefinition
Custom Resourcessearch: custom resources
Versions in CustomResourceDefinitionssearch: crd versioning
Operator patternsearch: operator pattern

Hands-on task 1 of 2ar-crd-1

~9 min7% weightArchitecture

Create a CRD with validation, then a custom resource

Create a CustomResourceDefinition for the kind BackupPolicy in API group ops.example.com: version v1 (served and used for storage), namespaced, plural backuppolicies, singular backuppolicy, short name bp. Its schema must require spec.schedule (a string) and spec.retentionDays (an integer of at least 1). kubectl get bp must show the columns Schedule and Retention. Then create a BackupPolicy named nightly in namespace arx-crd with schedule 0 2 * * * and retentionDays 14.

Cluster:kubectl config use-context kind-cka

1
Set up the scenario
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/setup | bash
2
Solve it in your terminal
Use kubectl and the Kubernetes docs, like on exam day. Save any answer files under /tmp/cka/.
3
Check your work
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/verify | bash
Got FAIL? Read which check failed, fix it, run it again.
4
Clean up (optional)
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-1/cleanup | bash

Hands-on task 2 of 2ar-crd-2

~5 min4% weightArchitecture

Trace a kind back to its CRD

Objects of kind ReferenceGrant can be created in this cluster. Find the CustomResourceDefinition that defines this kind and write four lines to /tmp/cka/crd/referencegrant.txt, in this order: the CRD's name, its API group, its scope (as the API reports it), and the version its objects are stored in. Then write the names of all CRDs that belong to the same API group, one per line, to /tmp/cka/crd/group-crds.txt.

Cluster:kubectl config use-context kind-cka

1
Set up the scenario
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/setup | bash
2
Solve it in your terminal
Use kubectl and the Kubernetes docs, like on exam day. Save any answer files under /tmp/cka/.
3
Check your work
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/verify | bash
Got FAIL? Read which check failed, fix it, run it again.
4
Clean up (optional)
curl -fsSL https://codewithmuh.com/labs/cka/t/ar-crd-2/cleanup | bash

Finished this lesson?

0 of 2 tasks passed. Your readiness counts tasks, so run them before you move on.

Complete and continue