Queries as Code#

Monitoring queries are authored once as YAML under packages/queries/, validated against a project-owned JSONSchema, and rendered per backend query language. A query is written for PromQL first; the other engines are translations of the same definition, carried in the same entry.

The audience for this section is repo contributors — the people adding a panel, an alert, or a metric to the registry. The audience for the rendered queries is the operator reading a dashboard, so the descriptions attached to each query are end-user voice; see Dashboard Style Guidelines for that.

In this section#

• Authoring Alerts — how an alerts: entry becomes an installed Prometheus rule: the alerting context, capabilities, the default set, and what gen-rules rejects.
• Authoring Recording Rules — how a rules: entry becomes a recorded series: the naming convention, the normalized ext:* contract, and the install-time facts its adapters need.
• Datadog Translations — the PromQL→Datadog mapping, the OTLP naming assumptions it rests on, and the gaps where Datadog’s language cannot express what the PromQL does.

Where things live#

The loader is deliberately lenient; structural strictness is the schema validator’s job. The check-queries pre-commit hook runs it on every changed query file:

bin/mz-monitoring-check check-queries packages/queries/materialize-compute.yaml

What a registry file holds#

Each file carries a description, a metricImportanceHint, and at least one of four content branches — queries, rules (recording rules), alerts, or metricOverrides. Alerts and recording rules reference a query either by queryId or by defining one inline, which is then promoted to a top-level registry entry at load time.

Every query carries a stability level, and that level is a contract: changes to canonical and best-effort queries are breaking changes, and both must be deprecated before removal. experimental and playground queries can be removed outright.

The engines#

A query’s expression fields are the per-engine translations. promQL is the source of truth; the rest are rendered from the same registry entry.

promQL and datadogQuery accept either one expression or a list of them, where a list means several distinct series on one panel.

Templating#

Expressions are templates, not literals. %%{param} placeholders are filled by a TemplateContext, which supplies both the parameter values and the template functions (orZero, mzClusterName, mzObjectName) — so the same registry entry renders differently per engine. The parameter names are shared across engines; the values are not. A PromQL context supplies label matchers (mzEnvironmentFilter → materialize_cloud_organization_name=~"…"), a Datadog context supplies tag matchers (materialize_cloud_organization_name:…).

The permitted parameter names are an enum in the schema (knownParameter), and check-queries checks every %%{…} against it, so a typo fails validation rather than rendering an empty string. Every one of them must be implemented by the dashboard and extraction contexts, which a test reads from the schema to enforce. The alerting context implements a subset on purpose; see Authoring Alerts.

Consumers#

Adding or changing a query has effects beyond the query itself:

• Grafana dashboards take both their PromQL and their panel descriptions from the registry, through mz_dashboards::grafana::queries — see Dashboards → SDKs and Schemas. A panel names a query id and gets the pair; nothing about a query is restated in the dashboard, so a change here reaches the rendered dashboards on the next make dashboards.

• Metric extraction (mz-monitoring-build extract-metrics) parses the PromQL to derive the metric set, which lands in docs/assets/metrics/metrics.yaml and backs Reference Metrics.

• Metric tiers (mz-monitoring-build gen-metric-tiers) roll each query’s stability and importance up into the per-destination allowlists in charts/materialize-monitoring/pre-rendered/metrics/metric-tiers.yaml. This is why a new query can change what a deployment ships to a metered backend.

  A query is the only way a metric reaches a tier. The tiers are built by walking the registry’s queries, so metricOverrides re-weights a metric some query already references and cannot introduce one — however many patterns match it. A family worth keeping therefore has to be named by a query even when no dashboard draws it. infra-loki.yaml once carried six such queries for the Alloy-side loki_* components; they moved to infra-alloy.yaml when a dashboard came to draw them.

  The converse also holds, because tiers select by metric name. A query naming a generic family such as go_goroutines admits every target’s copy of it, not only the copy the query was written for, which is why infra-alloy.yaml holds its Go runtime families at extended.

• Alerting and recording rules (mz-monitoring-build gen-rules) render every alerts: and rules: entry into charts/materialize-monitoring/pre-rendered/rules/, which the chart installs as PrometheusRule resources. A rule’s query renders through its own context and is held to stricter checks than a panel’s; see Authoring Alerts and Authoring Recording Rules.

• The docs read packages/queries/ directly: Hugo mounts it at assets/queries/, and the list-queries shortcode renders Common Queries from it. There is no generated intermediate to refresh.

Both generated outputs declare packages/queries/*.yaml as a prerequisite, so make metrics rebuilds them after a query change. Note that make all covers only the metric tiers — the metrics.yaml docs asset is not in the synced chain, so run make metrics when you have changed which metrics a query names.