NetworkPolicies

How NetworkPolicies control which Pods may talk, how peers and ports combine, why egress breaks DNS, and how to test a policy.

By default, every Pod can connect to every other Pod. A NetworkPolicy is a firewall rule for Pods: it says who may connect to them, or where they may connect to. For example: only the frontend Pods may reach the backend Pods, and only on port 8080. The exam topic is Define and enforce Network Policies.

It is a common task, and easy to get almost right. The API accepts any well-formed policy. Whether it does what the question meant depends on one dash in the YAML, on remembering DNS, and on testing it. Also, a policy only works if the CNI plugin enforces it. Calico in this lab does; Flannel on its own ignores policies without any warning.

The rules of isolation

Five rules explain how every NetworkPolicy behaves:

• A policy picks Pods in its own namespace. Its podSelector chooses them, and podSelector: {} means every Pod in the namespace.
• No policy means open. A Pod that no policy selects allows all traffic. Once a policy selects it for Ingress (or Egress), the Pod is isolated in that direction. Then only traffic that some policy allows gets through.
• Policies only add. There are no deny rules. A Pod allows everything that any of its policies allows, added together.
• Set policyTypes yourself. It says which directions the policy controls. If you leave it out, Ingress is always included, but Egress only when the policy has an egress section.
• Replies are always allowed. A policy only decides who may open a connection. For Pod A to reach Pod B, A's egress and B's ingress must both allow it.

Most tasks start from a default deny: a policy that selects every Pod and allows nothing. Here are the common starting points:

# Deny all incoming traffic to every Pod in the namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: nw-netpol
spec:
  podSelector: {}
  policyTypes: ["Ingress"]
---
# Deny all traffic in both directions (remember DNS, see below)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: nw-netpol
spec:
  podSelector: {}
  policyTypes: ["Ingress", "Egress"]
---
# Allow all incoming traffic (overrides an isolation for the Pods it selects)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all-ingress
  namespace: nw-netpol
spec:
  podSelector: {}
  policyTypes: ["Ingress"]
  ingress:
    - {}

Peers: who is allowed

A peer is an allowed source (under from) or destination (under to). You can pick peers in four ways:

The classic trap is the third row. The two rules below differ by one dash, and they mean different things:

ingress:
  - from:
      - namespaceSelector:          # ONE peer: Pods labeled role=monitor
          matchLabels:              # AND in namespace "observability"
            kubernetes.io/metadata.name: observability
        podSelector:
          matchLabels:
            role: monitor
---
ingress:
  - from:
      - namespaceSelector:          # TWO peers: every Pod in "observability"
          matchLabels:
            kubernetes.io/metadata.name: observability
      - podSelector:                # OR Pods labeled role=monitor in this namespace
          matchLabels:
            role: monitor

Every namespace gets the label kubernetes.io/metadata.name=<name> automatically, and it cannot be changed. Use it to pick a namespace by name, without labeling anything. If the question names its own label, such as team=payments, check it with kubectl get ns --show-labels. Add it with kubectl label ns only if the question says to.

Ports

ports sits next to from (or to) in the same rule. Four things to know:

• It matches the Pod's port. kube-proxy has already turned the Service address into a Pod address. So a Service on port 80 with targetPort: 8080 needs a policy that allows 8080.
• protocol is TCP when you leave it out.
• port can also be the name of a container port, such as http.
• A rule with from but no ports allows every port. A rule with ports but no from allows every source on those ports.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend
  namespace: nw-netpol
spec:
  podSelector:
    matchLabels:
      app: backend               # the policy protects the backend Pods
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

Egress and the DNS exception

Once a Pod is isolated for egress, it cannot reach CoreDNS either. Every connection by name then fails with bad address before it even starts. So almost every egress policy needs a DNS rule: port 53, over UDP and TCP. Allow it to the CoreDNS Pods, as below. Allowing port 53 to any destination is simpler, and also accepted unless the question says otherwise.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: client-egress
  namespace: nw-egress
spec:
  podSelector:
    matchLabels:
      role: client
  policyTypes: ["Egress"]
  egress:
    - to:                                   # DNS, to the CoreDNS Pods only
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    - to:                                   # the api Pods, on their container port
        - podSelector:
            matchLabels:
              app: api
      ports:
        - protocol: TCP
          port: 8080

Prove it works

Never trust a policy you have not tested. Test from one Pod that should get in, and from one that should not. Use wget -T 3, so a blocked connection fails after 3 seconds instead of hanging.

kubectl -n nw-netpol get netpol
kubectl -n nw-netpol describe netpol allow-frontend      # reads the selectors back in plain English

# -T 3 makes a blocked connection fail in 3 seconds instead of hanging
kubectl -n nw-netpol exec frontend -- wget -qO- -T 3 http://backend:8080/hostname   # expect a name
kubectl -n nw-netpol exec intruder -- wget -qO- -T 3 http://backend:8080/hostname   # expect "download timed out"

# Blocked looks like a timeout; "connection refused" means the policy allowed it and nothing listens
kubectl -n nw-egress exec client -- nslookup api.nw-egress.svc.cluster.local         # DNS still works?