WIP (2026Q2)
This is a new repository built around a first-class observability initiative for Materialize.
As of Materialize 27.0 (2026Q2), this repository is still under active development and not supported.

Generating and Pushing Dashboards#

How dashboard code is structured, how we keep generation deterministic, and how dashboards flow from source code into a live Grafana instance.

Code structure#

Dashboards live in their respective packages within packages/. The current Python implementation lives in packages/grafana-dashboards as a uv workspace. Python helpers live in packages/py-mzmon-lib.

Within packages/grafana-dashboards, the top-level packages represent the family of concerns (e.g., mz_environment, infra). Within a family, each dashboard has its own sub-package (such as overview) with the main dashboard entrypoint suffixed with _dashboard.py.

The full path to a given dashboard looks like:

packages/grafana-dashboards/dashboards/<family>/<dashboard_name>/<dashboard_name>_dashboard.py

The main dashboard class derives from py_mzmon_lib.dashboard.MzDashboard.

Other modules alongside a dashboard are typically tabs (when the dashboard has multiple tabs) or particularly intricate rows.

Sharing panels, rows, or even tabs between dashboards is acceptable, but prefer to have the code live within the most appropriate package and have other modules import it directly.

Code quality#

We use the following tools for Python code quality:

  • ruff for linting and formatting (we use aggressive rules).
  • pyright for type checking.
  • pytest for testing.
    • Unit tests are recommended to be placed next to their code with the _test.py suffix.

Linter configuration lives in the root pyproject.toml. Familiarize yourself with the configured rules before adding new files — the rules will surface ambiguous-character issues (e.g., RUF001/RUF002 for Unicode → or em-dash in identifiers/titles) and other issues that aren’t obvious without reading the config.

Determinism in dashboards#

We try to maximize deterministic and idempotent behavior of dashboards. It is acceptable for a dashboard to be “upgraded” on import into Grafana, but we want to target a minimal diff.

UID selection and behavior#

UIDs should be selected consistently based on the name of the dashboard. UIDs are not required to be random, but must be unique. Upgraded dashboards should continue using the same UIDs unless they break workflows.

Even though we have different Grafana targets, we should not encode the Grafana version in the UID (since dashboards may be upgraded across versions).

UIDs must follow the strict UID format introduced in Grafana 11.2: Latin alphanumeric with dashes and underscores, 40 characters max. We use the mz-mon- prefix for all UIDs.

Dashboard v2 caveat: in v2 the UID is not part of the dashboard spec — it lives in the surrounding Kubernetes-style metadata.name on the dashboard.grafana.app/v2 resource. The MzDashboard.UID value (with the mz-mon- prefix) is what we want as the canonical resource name, but Grafana will happily auto-generate a UID at first upload if one isn’t supplied. Once a dashboard exists, its UID becomes immutable; the way to “fix” a mismatched UID is to delete the existing dashboard and re-upload.

Element key stability#

In a v2 dashboard, panels are referenced by string keys in spec.elements{} and in spec.layout.…ElementReference.name. The Python source uses human-readable keys (e.g. "pod-cpu-percent"); Grafana may rewrite them to "panel-<id>" form on some save paths and leave them alone on others. Both forms are valid and the round-trip is non-destructive — do not rely on a specific naming convention when reading dashboards back.

Generating dashboards#

Dashboards can generally be generated by running the relevant dashboard module. Each dashboard module should include a __main__ entrypoint that emits the dashboard JSON:

if __name__ == "__main__":
    from grafana_foundation_sdk.cog.encoder import JSONEncoder

    print(JSONEncoder(indent=2).encode(MyDashboard()))  # noqa: T201

Include the # noqa: T201 lint suppression — print is otherwise disallowed in committed code.

Pushing dashboards to Grafana#

The canonical production path is gcx dashboards update, which handles the wrapping and the API call. The notes below cover the ad-hoc / verification path when iterating from a Claude Code session against the Grafana MCP.

Use the v2 API directly#

mcp-grafana’s built-in get_dashboard_by_uid and update_dashboard tools convert dashboards to the v1 representation on the way out, which strips queries from v2-only panel/layout features. For anything that must round-trip a v2 dashboard, hit the v2 resource API via grafana_api_request:

GET /apis/dashboard.grafana.app/v2/namespaces/default/dashboards/<uid>
PUT /apis/dashboard.grafana.app/v2/namespaces/default/dashboards/<uid>

PATCH is generally unavailable in our deployments (service accounts only receive the update verb, not patch); use the full PUT.

PUT body shape#

PUTs must wrap the dashboard spec in the Kubernetes-style envelope:

{
  "apiVersion": "dashboard.grafana.app/v2",
  "kind": "Dashboard",
  "metadata": {
    "name": "<uid>",
    "namespace": "default",
    "resourceVersion": "<rv from current GET>",
    "annotations": {
      "grafana.app/folder": "<folder uid from current GET>",
      "grafana.app/message": "<one-line summary of this change>"
    }
  },
  "spec": { /* JSONEncoder output of MyDashboard() */ }
}

Gotchas:

  • Folder annotation is required on update. Without metadata.annotations["grafana.app/folder"], Grafana treats the PUT as a move-to-root and returns 403 "not allowed to create resource in the destination folder". Always fetch the current resource first and carry the folder annotation forward.
  • Always set grafana.app/message. This is the dashboard’s version history entry — populate it with a one-line summary describing the change in this revision (same role as a git commit message).
  • resourceVersion enables optimistic concurrency. Fetch + PUT, not fire-and-forget; otherwise concurrent saves can clobber each other.

Service account permissions#

Reads work with a Viewer-scoped token, but PUT requires Edit on the destination folder. The clearest error tells you which:

  • "not allowed to update resource in the source folder" = no edit on the existing folder.
  • "not allowed to create resource in the destination folder" = missing folder annotation or no edit on the target folder.
Backward Style Guidelines Testing Forward