Documentation Grafana Alloy Reference Components prometheus prometheus.exporter.databricks

prometheus.exporter.databricks

The prometheus.exporter.databricks component embeds the databricks_exporter for collecting billing, jobs, pipelines, and SQL warehouse metrics from Databricks System Tables via HTTP for Prometheus consumption.

Usage

Alloy Copy

prometheus.exporter.databricks "LABEL" {
    server_hostname     = "<DATABRICKS_SERVER_HOSTNAME>"
    warehouse_http_path = "<DATABRICKS_WAREHOUSE_HTTP_PATH>"
    client_id           = "<DATABRICKS_CLIENT_ID>"
    client_secret       = "<DATABRICKS_CLIENT_SECRET>"
}

Arguments

You can use the following arguments with prometheus.exporter.databricks:

Blocks

The prometheus.exporter.databricks component doesn’t support any blocks. You can configure this component with arguments.

Exported fields

Component health

prometheus.exporter.databricks is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

prometheus.exporter.databricks doesn’t expose any component-specific debug information.

Debug metrics

prometheus.exporter.databricks doesn’t expose any component-specific debug metrics.

Prerequisites

Before using this component, you need:

1. Databricks Workspace with Unity Catalog and System Tables enabled
2. Service Principal with OAuth2 M2M authentication configured
3. SQL Warehouse for querying System Tables (serverless recommended for cost efficiency)

See the Databricks documentation for detailed OAuth2 M2M setup instructions.

Example

The following example uses a prometheus.scrape component to collect metrics from prometheus.exporter.databricks:

Alloy Copy

prometheus.exporter.databricks "example" {
  server_hostname     = "dbc-abc123-def456.cloud.databricks.com"
  warehouse_http_path = "/sql/1.0/warehouses/xyz789"
  client_id           = "my-service-principal-id"
  client_secret       = "my-service-principal-secret"
}

// Configure a prometheus.scrape component to collect databricks metrics.
prometheus.scrape "demo" {
  targets         = prometheus.exporter.databricks.example.targets
  forward_to      = [prometheus.remote_write.demo.receiver]
  scrape_interval = "5m"
  scrape_timeout  = "4m"
}

prometheus.remote_write "demo" {
  endpoint {
    url = "<PROMETHEUS_REMOTE_WRITE_URL>"

    basic_auth {
      username = "<USERNAME>"
      password = "<PASSWORD>"
    }
  }
}

Replace the following:

• <PROMETHEUS_REMOTE_WRITE_URL>: The URL of the Prometheus remote_write compatible server to send metrics to.
• <USERNAME>: The username to use for authentication to the remote_write API.
• <PASSWORD>: The password to use for authentication to the remote_write API.

Tuning recommendations

• scrape_interval: Default is 5 minutes. The exporter queries Databricks System Tables which can be slow. Increase to reduce SQL Warehouse costs.
• scrape_timeout: Default is 4 minutes. The exporter typically takes 90-120 seconds per scrape depending on data volume.

High cardinality warning

The collect_task_retries flag adds task-level retry metrics which can significantly increase cardinality for workspaces with many jobs. Only enable if needed.

Compatible components

prometheus.exporter.databricks has exports that can be consumed by the following components:

• Components that consume Targets