Terraform variable reference#

This is the supported way to reach chart settings the module does not model — including scheduling (node selectors, tolerations) and Grafana ingress, neither of which the module surfaces yet. See the README.

Each element is one YAML document. .tfvars is HCL, so an indented heredoc is the readable way to write one — <<-YAML strips the common leading indent, and ${...} escapes a literal $ ahead of a brace. The marker is arbitrary; YAML rather than the usual EOT only because this description is itself a heredoc:

additional_values = [
  <<-YAML
    grafana:
      ingress:
        enabled: true
        hosts: ["grafana.example.com"]
  YAML
]

• enabled — install the bundled rules at all. The chart’s default is true.
• capabilities — what the deployment contains beyond what the chart derives, such as crdb-dedicated or cilium. A rule installs only where every capability it requires is present.
• selected — alert names, rule-group names, or "*", to install beyond the default set.
• disabled — alert names never to install.
• overrides — per alert, a for_duration (the rule’s for) and labels to add or replace. An override never changes an expression. The usual reason for one is a cluster whose normal hydration takes longer than cluster-hydration-stuck’s hour.
• environment_namespaces — where Materialize environments run. Null uses materialize_instance_namespace.
• excluded_namespaces — namespaces no rule alerts on.
• infra_workloads — the workloads each infrastructure tier holds (core, important, nonessential, daemonset). A tier given here replaces the chart’s list for that tier.

An unknown capability, alert or group name fails the Helm render, as does an override whose for_duration is not a duration.

• preset — which severity-to-class mapping routes alerts: critical-infrastructure, important (the chart’s default), evaluation, or a key of presets.
• presets — severity-to-class mappings to add, or cells to change, keyed by preset name.
• unknown_severity — the severity an alert with none, or an unknown one, is routed as.
• receivers — notification receivers keyed by name, each with class (the class or list of classes it serves), config (an Alertmanager receiver, verbatim, without name) and optional route options. Empty means no alert reaches anybody.
• routes — root for the top-level route’s grouping and timing, and extra for routes ahead of the preset’s severity routes, in Alertmanager’s own format. Every bundled alert carries an audience label, platform, workload or development, which is the usual thing an extra route matches.
• inhibit_rules, time_intervals, global — Alertmanager’s own blocks, verbatim.
• templates — notification templates keyed by file name, each ending in .tmpl.
• default_templates — make the chart’s notification templates every receiver’s default (the chart’s default is on): a readable subject and Slack body, and links into Grafana.
• grafana_url — the address people open Grafana at, for those links. Null derives it from Grafana’s root_url, which is unset unless additional_values exposes Grafana.

The Alertmanager-native parts are typed any and passed through as written, so every integration Alertmanager documents works without a module release.

Credentials never belong here. Reference them with a receiver field’s _file variant — api_url_file, routing_key_file, credentials_file — at /etc/alertmanager/secrets/alertmanager-receivers/<key>, and supply <key> through alerting_receiver_secrets. The chart fails the render on an inline credential.

Set it to alertmanager under the chart’s split-namespace profile, which moves Alertmanager there. Not inferred: the module cannot see where an additional_values override put it. The namespace has to exist before apply.

Keep certificate_renew_before well under this — a third or less. cert-manager renews at duration - renewBefore, so a value close to duration renews continuously; on a small cluster that has been observed to livelock the controller, after which it stops renewing and reports certificates as healthy while they expire.

Requires cert-manager to already be installed, with its CRDs present. This module does not install it — the same shared-responsibility split as buckets and workload identity — so with the flag on and cert-manager absent the apply fails on an unknown cert-manager.io/v1 kind.

Off by default rather than on. The design calls for the Terraform path to be secure by default, and that becomes safe once a wrapper that installs cert-manager owns the default; flipping it here today would break every existing consumer’s next apply.

Issuing certificates does not turn TLS on anywhere — var.internal_tls is what moves the hops off plaintext, and it needs this. The two are separate because a component only leaves plaintext once its renewal behaviour is proven, so the material exists before every hop is ready to use it.

Leave null, which is the supported path: the module reads the version out of the chart’s own Chart.yaml in this repository, so a module ref always installs the chart it shipped with and the two cannot drift. Set it only to pin a chart version different from the module’s.

Set it whenever more than one cluster writes to the same log or metrics store, or notifies the same channel or incident tool — the label is the only thing that tells them apart. Null leaves the chart’s default, default.

Required when the Grafana resource lives somewhere other than namespace, which is what the chart’s split-namespace profile does. Not inferred: the module cannot see where an additional_values override put the instance.

Note that the CRDs forbid turning this back off in place; the resource has to be recreated.

Null derives the label the main chart puts on its own Grafana resource, which is correct unless that release narrowed the selector through connections.grafana.labels in additional_values — the dashboards charts cannot read that, so set the same map here.

Null leaves the charts’ own default, which is every dashboard. Narrow it to hold one back — env-upgrade depends on operator instrumentation that not every Materialize version has. A dashboards chart the patterns match nothing in is not installed.

Delivered as a Secret this module creates (mzmon-alloy-gateway-env), never through the Helm values — anything in values is readable with helm get values by anyone who can read the release Secret. The gateway reads it from the environment at startup.

An app key is not needed and is not accepted here; the metrics intake authenticates with the API key alone.

site is your Datadog site — datadoghq.com, datadoghq.eu, us3.datadoghq.com, and so on. Getting it wrong is a 403 from the intake, not a routing error.

min_importance picks a metric tier — essential, recommended, extended, diagnostic, or all — and each tier includes the ones below it. This is a cost control, and a sharper one than for most backends: Datadog bills per custom metric, so all is rarely what you want.

metric_endpoint and logs_endpoint override the intake URLs the exporter derives from site. Leave them null unless you are routing through a proxy or PrivateLink — a hand-written endpoint that disagrees with site fails at the intake rather than at plan time.

The dashboards are Helm releases of their own, one per Grafana folder, because Helm stores a release in a Kubernetes Secret and a Secret may not exceed 1 MiB, which the rendered set had grown past. The module installs them against the folders and Grafana instance the main release creates, so they stay in step without either side having to discover the other.

Set false when dashboards are managed elsewhere — a platform team’s own Grafana provisioning, or a GitOps pipeline that owns them.

Set false when the cluster already has them from elsewhere — kube-prometheus-stack, or a platform team that owns CRDs centrally — since Terraform would otherwise fail trying to create objects it does not own.

Every CRD carries helm.sh/resource-policy: keep, so destroying this release, or turning this off later, leaves the CRDs and every custom resource in the cluster in place. Removing the CRDs is a deliberate step of its own; see the “Uninstalling” page in the docs.

It watches every postgres_exporter_targets entry, and Grafana’s own database whenever this module configures one with a password, as a target named grafana. Grafana’s connects to the postgres database rather than grafana_database_name, because Grafana’s own tables would add thousands of per-table series that no panel reads.

Each target’s credentials go into a Secret this module creates, postgres-exporter, never into the Helm values. The role each target connects as needs pg_monitor to see other roles' sessions; see the PostgreSQL exporter page.

On by default. The chart currently works with an empty password.

Required by a sigv4 remote-write destination, which has no other source of credentials. Merged with any annotations object_storage contributes, so both can be present.

min_importance picks a metric tier — essential, recommended, extended, diagnostic, or all — and each tier includes the ones below it. This is a cost control: the series land as prometheus.googleapis.com/ metrics, billed per sample ingested, and all sends the entire surface.

The project needs the telemetry.googleapis.com API enabled. Authentication is ADC only. Bind the gateway ServiceAccount to a Google service account holding roles/monitoring.metricWriter through object_storage.gateway_service_account_annotations; failing that it falls back to the node’s service account, which works only if that account has the role.

prefix is deprecated and ignored. The Telemetry API names every metric prometheus.googleapis.com/<name>/<kind>, so there is no prefix to choose.

Null infers it from grafana_database_host, which is right whenever that host is a literal. Set it explicitly when the host is computed from a resource created in the same apply.

Null infers it from grafana_database_password. Set it explicitly when that password is generated in the same apply.

False is not a way to supply the password by another route: it means no Secret and no $__file{} reference at all, so a connection that needs one has to get both from additional_values. That is also the shape for a genuinely passwordless connection — a Cloud SQL Auth Proxy sidecar, or peer authentication.

Never inlined into grafana.ini, which renders into a ConfigMap.

Null when the connection needs no password — a Cloud SQL Auth Proxy sidecar with --auto-iam-authn, or a trust/peer-authenticated database. Note that IAM database authentication without a proxy does not work: Grafana reads its password once at startup and has no refresh hook, so the first reconnect after the token expires fails.

require encrypts but does not authenticate the server. verify-full also authenticates it and is the better choice — but it needs a CA bundle on disk, which this module does not mount: supply grafana.ini.database.ca_cert_path and the matching grafana.extraSecretMounts through additional_values when you use it.

Only needed behind an L4 load balancer, which passes TCP through and leaves TLS to terminate at the pod, so the material has to exist in the cluster. An L7 load balancer terminating with a cloud-managed certificate (ACM, Google Certificate Manager, Azure Key Vault) attaches it by ARN or resource ID and the key never enters the cluster — for that shape leave this empty and pass the annotation through grafana.service.annotations in additional_values.

Setting this with no var.issuer_ref issues nothing, and the render says so.

Leave false when the Materialize operator module installs it (the default topology), and set it true when that module has install_metrics_server = false — otherwise nothing provides the metrics API and the Materialize Console silently loses cluster metrics.

On by default: node-level metrics are part of the stack’s baseline, and nothing else in it collects them. Set false when the cluster already runs its own node-exporter DaemonSet — a second one wastes a per-node slot and produces the same series twice under two job labels, which double-counts in any sum() over them.

This writes the chart’s node-exporter.enabled circuit breaker rather than a tag. Tags are OR’d, so tags.node-exporter = false would not turn it off while tags.default is true.

The stages are the chart’s own profiles/mtls*.values.yaml, composed in order, and they exist because Kubernetes does not order a server’s rollout against its clients’. On a fresh install there is nothing to strand, so go straight to authenticate. On a running stack step through present first, or on any hop where the server pod happens to roll before the writers, ingestion stops until they catch up — which on a busy pipeline is data loss rather than latency.

authenticate is authentication, not authorization: none of these components can express “this identity may write and that one may not”, so the size of the trust domain is the security property. That is the argument for leaving issuer_ref null and letting the chart bootstrap a root scoped to this release.

Two ports stop short of authenticate by nature rather than by choice, and the chart refuses to configure them otherwise: Loki’s HTTP port and Alertmanager’s API port are probed by the kubelet, and a httpGet probe has no field for a client certificate, so their terminal state is present. The Loki gateway’s listener stops at present for the same reason. Grafana’s datasources verify the backend, and from present they present a certificate to Alertmanager and to the Loki gateway; the Thanos datasource presents none.

Leave null with certificates_enabled on and the chart bootstraps a self-signed root of its own, scoped to this release — which is the recommended shape rather than a fallback. None of the receiving components here implement per-client authorization, so “signed by the CA we trust” is the entire authorization decision; an issuer shared with every workload in the cluster reduces that to “has any certificate”.

An ownership label for cost-center allocation is the expected use. Each arrives as label_<key>, with every character outside [a-zA-Z0-9_] mapped to _, and joins to any kube_pod_* family on namespace and pod.

Appended to the chart’s pods allowlist entry rather than replacing it. Setting kube-state-metrics.metricLabelsAllowlist through additional_values instead replaces the whole list, node labels included.

Name each label. * copies every label on every pod, and is refused.

flavor is rds (the DB instance identifier), cloudsql (the instance name, without the project) or azure-postgres (the flexible server name). A postgres-exporter target is declared with its own metadata_database flag instead.

Set this when that assumption does not hold, because the chart’s constraints fail closed rather than degrading:

• 0 — no node carries a topology.kubernetes.io/zone label. Common on kind and on many on-premises distributions. The hard constraints are dropped; the soft host spread stays.
• 1 — a single zone. minDomains becomes 1, which is satisfiable now and turns into real protection the day a second zone appears.
• 2 or more — minDomains is set to the real count, which is stricter than the chart’s floor of 2 whenever you have more zones than that.

Leaving this null on a cluster with fewer than two zones leaves those pods Pending forever rather than unbalanced: below minDomains Kubernetes treats the global minimum as 0, so one zone holding every replica computes a skew equal to the replica count. One zone is exactly as broken as none. Alertmanager’s rule sets no minDomains, so it survives one zone and breaks only at zero.

Not applied to the Alloy agent: it is a DaemonSet that must reach every node to collect logs and node metrics, so constraining it to a workload pool would silently stop collection everywhere else.

cloud selects the objstore dialect. The *_service_account_annotations maps carry the workload-identity annotation for each component’s ServiceAccount — the chart validates that the annotation’s cloud matches the objstore backend, so a mismatched pair fails at render time rather than at pod start.

azure_storage_account is required when cloud is azure, and only then: both Loki and Thanos name the account separately from the container. Azure needs nothing else — the annotation plus the pod label the module applies are all the Entra webhook requires.

endpoint applies to aws only, and both backends need one — their shared objstore client rejects an empty endpoint rather than resolving the AWS SDK’s regional default. It is optional because the module derives s3.<region>.amazonaws.com from region, falling back to the global host when neither is given; name it to point at a VPC endpoint or an S3-compatible store.

Prefer workload identity wherever it exists: it rotates, and the object_storage.*_service_account_annotations maps are how it is configured. These two variables are the fallback, not the default path.

Set both or neither. aws only — GCS takes a service-account key and Azure a storage-account key, neither of which is an access-key pair; configure those through additional_values.

Reaches the backends as a Secret in both cases, never a ConfigMap: Thanos already renders its objstore config into one, and the module switches Loki’s configStorageType to Secret when these are set — the chart’s default puts the rendered config in a ConfigMap, which would publish this value to anyone with read access to the namespace.

Delivered as a Secret this module creates (mzmon-alloy-gateway-env) rather than through the Helm values. Cannot be combined with otlp_auth_header_secrets or otlp_metrics.auth_headers: the chart has one auth slot per OTLP destination.

Each value is delivered as a Secret this module creates (mzmon-alloy-gateway-env) rather than through the Helm values, and the gateway reads it from the environment at startup. The module derives the variable name from the header (x-honeycomb-team becomes GATEWAY_OTEL_DEST_HEADER_X_HONEYCOMB_TEAM); nothing else depends on it.

Non-secret headers belong in otlp_metrics.auth_headers, which renders them inline. The two compose into one header set. Cannot be combined with otlp_auth_bearer_token: the chart has one auth slot per OTLP destination.

url is a host[:port] with no scheme; https:// in it fails at gateway start. protocol selects the exporter: grpc for OTLP/gRPC, http for OTLP/HTTP.

min_importance picks a metric tier — essential, recommended, extended, diagnostic, or all — and each tier includes the ones below it. Metered backends are the reason it defaults below all here.

auth_headers sets non-secret request headers, such as Honeycomb’s x-honeycomb-dataset. They render into the gateway’s pipeline ConfigMap as literals, so put credentials in otlp_auth_header_secrets or otlp_auth_bearer_token instead — those reach the gateway through a Secret.

name becomes the target’s target label and keys its password in postgres_exporter_passwords. database is the one the connection opens, and per-table statistics come from it alone; everything instance-wide covers every database on the instance. Point it at the metadata database’s own database (materialize in the shipped examples), and leave it at postgres for anything whose tables you do not need.

metadata_database = true declares the target a metadata database, which is what the postgres recording-rule adapter reads.

The key names the destination and becomes its Alloy component label, so it must match [a-zA-Z_][a-zA-Z0-9_]* and appears in the gateway’s own metrics. Two keys are special only by convention: thanos is the chart’s built-in destination, so setting it here retunes that one rather than adding a second — which is how you drop the bundled backend to a cheaper tier, or turn it off with enabled = false while keeping another destination.

Each destination gets its own remote-write component and its own upstream tier filter, so min_importance — essential, recommended, extended, diagnostic, or all, each including the ones below it — is genuinely per destination: a metered backend on essential never buffers the metrics it would only discard. Amazon Managed Prometheus bills per sample ingested and per active series, which is what that is for.

auth_type is none, sigv4, basicAuth, or bearer. sigv4 needs no credentials at all — the gateway signs with the IRSA identity from gateway_service_account_annotations. The other two take their credentials from prometheus_remote_write_credentials, which delivers them through a Secret rather than the Helm values.

Delivered as a Secret this module creates (mzmon-alloy-gateway-env) rather than through the Helm values, which are readable with helm get values and land in Terraform state. The module derives the variable names from the destination name — amp becomes GATEWAY_PROMETHEUS_DEST_AMP_USERNAME and friends — and writes those same names into the chart’s values, so the two cannot disagree.

sigv4 destinations need no entry here: they sign with the gateway pod’s IRSA identity.

Profiles are read from the chart directory in this repository at the same commit as the pinned chart version, so they cannot drift from it. A profile that does not exist yet is skipped, which is how Thanos sizing will start applying once those profiles land.

Four are PVC-backed by default: Alertmanager, the Loki ruler, and the Thanos Store Gateway and Compactor. Loki’s ingesters and Thanos Receive use node-local emptyDir by design — durability is the replication factor there, and a volume would pin them to one availability zone. The Thanos ruler runs stateless and holds only a remote-write WAL. The class is still fanned out to Receive and to the ruler so that re-enabling their persistence picks the class up rather than silently missing it.

Required where the default class cannot serve the nodes: GCP’s C4 and N4 families take only Hyperdisk, and every Persistent Disk class fails to attach with pd-balanced disk type cannot be used by <machine-type>.

Changing it on an existing install does not move the volumes. volumeClaimTemplates are immutable, so the old PVCs must be deleted first — discarding their contents.