Helm & Kustomize

The two dominant ways to manage Kubernetes YAML for non-trivial systems. Helm templates them. Kustomize patches them. They solve overlapping problems differently, and the right answer is usually "both, in their lanes."

Analogy

Helm is shipping a piece of furniture flatpack — every part is parameterised, you give the buyer an instruction sheet (values.yaml), they decide the colour, the shelf count, the leg height. Kustomize is owning your own house — you have one set of base furniture, and each room (environment) gets specific tweaks: a different rug here, a different lamp there. You don't write a generic "couch template" for your own rooms. You don't ship raw house tweaks to an Ikea customer.

What's the actual problem?

You have manifests like:

apiVersion: apps/v1
kind: Deployment
metadata: { name: web }
spec:
  replicas: 3
  template:
    spec:
      containers:
      - { name: web, image: my-app:v1.4.2 }

For one cluster, fine. Now you need:

• 3 environments (dev / staging / prod) with different replicas, image tags, resource limits, env vars, ingress hosts.
• Easy upgrades without copy-pasting YAML.
• Reusability — when you ship my-redis-operator, every user wants their own namespace, name, password, replica count.

Two tools, two philosophies.

Helm — templated charts

A chart is a directory of templated YAML + a values.yaml of defaults:

mychart/
├── Chart.yaml
├── values.yaml
└── templates/
    ├── deployment.yaml
    ├── service.yaml
    └── ingress.yaml

templates/deployment.yaml uses Go templates:

apiVersion: apps/v1
kind: Deployment
metadata: { name: {{ .Release.Name }}-web }
spec:
  replicas: {{ .Values.replicaCount }}
  template:
    spec:
      containers:
      - name: web
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        resources:
          {{- toYaml .Values.resources | nindent 10 }}

Install:

helm install my-app ./mychart -f prod-values.yaml --set replicaCount=12

What you get:

• Reproducible install: chart version + values = manifest set.
• Built-in upgrade: helm upgrade my-app ./mychart -f prod-values.yaml.
• Atomic rollback: helm rollback my-app 3 returns to release 3.
• Distribution: charts publish to OCI registries (Docker Hub, ECR, GHCR).

The downside: Go templates over YAML are painful. Quoting, indentation, type coercion — all infinite footguns. A chart of any size grows a _helpers.tpl file with macros to keep the templates readable.

Kustomize — strategic patches

Kustomize takes the opposite approach: no templating language. Just plain YAML in a base/ directory, with overlays that apply patches.

deploy/
├── base/
│   ├── kustomization.yaml
│   ├── deployment.yaml
│   └── service.yaml
└── overlays/
    ├── dev/
    │   ├── kustomization.yaml
    │   └── replica-patch.yaml
    └── prod/
        ├── kustomization.yaml
        ├── replica-patch.yaml
        └── resources-patch.yaml

base/deployment.yaml is plain YAML — no {{ }} markers anywhere. Real, readable, kubeval-passing manifests.

overlays/prod/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod
namePrefix: prod-
resources:
- ../../base
patches:
- path: replica-patch.yaml
- path: resources-patch.yaml
images:
- name: my-app
  newTag: v1.4.2
configMapGenerator:
- name: web-config
  literals:
  - LOG_LEVEL=info

Apply with:

kubectl apply -k ./overlays/prod

Kustomize is built into kubectl since 1.14 — no extra binary required. The patch model is strategic merge (Kubernetes-aware: it knows how to merge a list of containers by name, etc.) plus JSON 6902 patches when you need surgical access.

When to pick which

A simple rubric:

Situation	Tool
Distributing software to other teams / users	Helm
Installing third-party software (Postgres, etc.)	Helm
Managing your own services across environments	Kustomize
You want kubectl-only, no extra binaries	Kustomize
You need versioned, atomic rollback	Helm
You hate Go templates	Kustomize
You're inheriting upstream Helm charts	Both — Helm to template, Kustomize to patch

The "Helm for distribution, Kustomize for ops" split is the most common pattern in mature teams.

The combination trick

You're using a third-party Helm chart (say, the official Postgres chart), and you need to add a custom annotation on one of its services that the chart doesn't expose as a value. You can:

helm template postgres bitnami/postgresql -f values.yaml | kustomize build /dev/stdin

Or — better — use Argo CD's Helm-with-Kustomize mode, which renders the chart and applies overlays in one pass. The ergonomics let you treat upstream charts as immutable input and apply your own overlays on top.

Helm chart anatomy

mychart/
├── Chart.yaml          # name, version, description, dependencies
├── values.yaml         # defaults
├── values.schema.json  # optional — validates user-provided values
├── templates/
│   ├── _helpers.tpl    # named templates (macros)
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── NOTES.txt       # printed after install
│   └── tests/          # `helm test` connection probes
├── charts/             # vendored sub-chart dependencies
└── crds/               # CustomResourceDefinitions installed before templates

A few high-leverage features:

• Hooks (helm.sh/hook: pre-install) — run a Job before install / upgrade.
• required template function — fail-fast if a required value is missing.
• tpl — render a string AS a template. Useful for letting users template values themselves.
• Sub-charts — depend on other charts. Versioned in Chart.yaml.dependencies.

Kustomize features beyond the basics

• configMapGenerator / secretGenerator — generates a ConfigMap / Secret with a hash suffix. When the data changes, the name changes, forcing a rolling update on consumers.
• replicas field — override replica counts without writing a patch.
• patches with target selectors — apply the same patch to multiple resources matching a selector (e.g. all Deployments).
• components — reusable patch sets composed across overlays.
• SOPS integration — encrypted secrets in git, decrypted by the controller.

Common bugs

Helm: indentation hell. {{- toYaml .Values.resources | nindent 10 }} is the canonical fix; the leading dash strips whitespace, nindent adds the correct indent. Get it wrong and your YAML silently parses wrong.

Helm: forgetting --atomic. Without it, a partial install leaves orphaned resources. Always helm install --atomic --timeout 5m in CI.

Kustomize: patches not applying. Patch targets must match by apiVersion + kind + name. The error message points at the offending mismatch.

Kustomize: name collisions. Two overlays both inheriting from base can produce duplicate resources at install time. Use namespace: and namePrefix: consistently.

Both: forgetting to bump app.kubernetes.io/version in metadata. Doesn't break anything, but breaks ArgoCD's "what version is deployed" view.

A reasonable production setup

• Third-party operators (cert-manager, ingress-nginx, prometheus, vault): Helm, pinned to specific chart versions.
• In-house services (my-api, my-worker): Kustomize, with base/ shared and overlays per env.
• Bridge: Argo CD or Flux as the GitOps controller, supporting both natively.

That layout will scale from one team to fifty without rewriting.