Prometheus metrics

기준일: 2026-07-26
공식 기준: Prometheus metrics

Prometheus metrics 문서는 OpenClaw 공식 문서(gateway/prometheus)를 한국어로 정리한 가이드입니다. Expose OpenClaw diagnostics as Prometheus text metrics through the diagnostics-prometheus plugin 명령·설정 키·코드 예시는 공식 문서를 그대로 보존하며, 해석과 절차 안내는 한국어로 제공합니다. 최종 동작은 설치된 CLI 버전과 공식 원문을 확인하세요.

핵심 요약

Expose OpenClaw diagnostics as Prometheus text metrics through the diagnostics-prometheus plugin

한국어 가이드 범위: gateway/prometheus 경로의 설정·명령·제약·예시를 학습용으로 재구성합니다.

문서 구성

공식 문서의 주요 섹션은 다음과 같습니다.

• 빠른 시작
• Metrics exported
• Label policy
• PromQL recipes
• Tokens per minute, split by provider
• Spend (USD) over the last hour, by model
• 95th percentile model run duration
• Queue wait time SLO (95p under 2s)
• Skill usage, split by bounded source
• Dropped Prometheus series (cardinality alarm)
• Choosing between Prometheus and OpenTelemetry export
• 트러블슈팅
• 관련 문서

상세 내용

본문

OpenClaw can expose diagnostics metrics through the official diagnostics-prometheus plugin. It listens to trusted diagnostics plus internally tagged, dispatcher-owned diagnostic events (queue, memory, and session-recovery signals), and renders a Prometheus text endpoint at:

위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 --help를 확인하세요.

GET /api/diagnostics/prometheus

빠른 시작

The HTTP route is registered at plugin startup, so reload after enabling.

위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 --help를 확인하세요.

주요 항목:

• job_name: openclaw
• targets: ["openclaw-gateway:18789"]

openclaw plugins install clawhub:@openclaw/diagnostics-prometheus

{
          plugins: {
            allow: ["diagnostics-prometheus"],
            entries: {
              "diagnostics-prometheus": { enabled: true },
            },
          },
          diagnostics: {
            enabled: true,
          },
        }

openclaw plugins enable diagnostics-prometheus

curl -H "Authorization: Bearer $OPENCLAW_GATEWAY_TOKEN" \
      http://127.0.0.1:18789/api/diagnostics/prometheus

Metrics exported

For model-call metrics, observation_unit="request" measures one observable provider request. observation_unit="turn" measures a synthetic Claude Code or Codex CLI agent turn that can contain multiple hidden provider requests. Keep those series separate when comparing latency.

위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 --help를 확인하세요.

Metric	Type	Labels
openclaw_run_completed_total	counter	channel, model, outcome, provider, trigger
openclaw_run_duration_seconds	histogram	channel, model, outcome, provider, trigger
openclaw_model_call_total	counter	api, error_category, model, observation_unit, outcome, provider, transport
openclaw_model_call_duration_seconds	histogram	api, error_category, model, observation_unit, outcome, provider, transport
openclaw_model_failover_total	counter	from_model, from_provider, lane, reason, suspended, to_model, to_provider
openclaw_model_tokens_total	counter	agent, channel, model, provider, token_type
openclaw_gen_ai_client_token_usage	histogram	model, provider, token_type
openclaw_model_cost_usd_total	counter	agent, channel, model, provider
openclaw_model_usage_duration_seconds	histogram	agent, channel, model, provider
openclaw_skill_used_total	counter	activation, agent, skill, source
openclaw_tool_execution_total	counter	error_category, outcome, params_kind, tool, tool_owner, tool_source
openclaw_tool_execution_duration_seconds	histogram	error_category, outcome, params_kind, tool, tool_owner, tool_source
openclaw_tool_execution_blocked_total	counter	denied_reason, params_kind, tool, tool_owner, tool_source
openclaw_harness_run_total	counter	channel, error_category, harness, model, outcome, phase, plugin, provider
openclaw_harness_run_duration_seconds	histogram	channel, error_category, harness, model, outcome, phase, plugin, provider
openclaw_webhook_received_total	counter	channel, webhook
openclaw_webhook_error_total	counter	channel, webhook
openclaw_webhook_duration_seconds	histogram	channel, webhook
openclaw_message_received_total	counter	channel, source
openclaw_message_dispatch_started_total	counter	channel, source
openclaw_message_dispatch_completed_total	counter	channel, outcome, reason, source
openclaw_message_dispatch_duration_seconds	histogram	channel, outcome, reason, source
openclaw_message_processed_total	counter	channel, outcome, reason
openclaw_message_processed_duration_seconds	histogram	channel, outcome, reason
openclaw_message_delivery_started_total	counter	channel, delivery_kind
openclaw_message_delivery_total	counter	channel, delivery_kind, error_category, outcome
openclaw_message_delivery_duration_seconds	histogram	channel, delivery_kind, error_category, outcome
openclaw_talk_event_total	counter	brain, event_type, mode, provider, transport
openclaw_talk_event_duration_seconds	histogram	brain, event_type, mode, provider, transport
openclaw_talk_audio_bytes	histogram	brain, event_type, mode, provider, transport
openclaw_queue_lane_size	gauge	lane
openclaw_queue_lane_wait_seconds	histogram	lane
openclaw_session_state_total	counter	reason, state
openclaw_session_queue_depth	gauge	state
openclaw_session_turn_created_total	counter	agent, channel, trigger
openclaw_session_stuck_total	counter	reason, state
openclaw_session_stuck_age_seconds	histogram	reason, state
openclaw_session_recovery_total	counter	action, active_work_kind, state, status
openclaw_session_recovery_age_seconds	histogram	action, active_work_kind, state, status
openclaw_liveness_warning_total	counter	reason
openclaw_liveness_sessions	gauge	state
openclaw_liveness_event_loop_delay_p99_seconds	histogram	reason
openclaw_liveness_event_loop_delay_max_seconds	histogram	reason
openclaw_liveness_event_loop_utilization_ratio	histogram	reason
openclaw_liveness_cpu_core_ratio	histogram	reason
openclaw_payload_large_total	counter	action, channel, plugin, reason, surface
openclaw_payload_large_bytes	histogram	action, channel, plugin, reason, surface
openclaw_memory_bytes	gauge	kind
openclaw_memory_rss_bytes	histogram	none
openclaw_memory_pressure_total	counter	level, reason
openclaw_telemetry_exporter_total	counter	exporter, reason, signal, status
openclaw_prometheus_series_dropped_total	counter	none
openclaw_diagnostic_async_queue_dropped_total	counter	drop_class
openclaw_diagnostic_async_queue_length	gauge	none

Label policy

Prometheus labels stay bounded and low-cardinality. The exporter does not emit raw diagnostic identifiers such as runId, sessionKey, sessionId, callId, toolCallId, message IDs, chat IDs, or provider request IDs.

위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 --help를 확인하세요.

주요 항목:

• prompt text, response text, tool inputs, tool outputs, system prompts
• Talk transcripts, audio payloads, call ids, room ids, handoff tokens, turn ids, and raw session ids
• raw provider request IDs (only bounded hashes, where applicable, on spans — never on metrics)
• session keys and session IDs
• hostnames, file paths, secret values

PromQL recipes

### Tokens per minute, split by provider

sum by (provider) (rate(openclaw_model_tokens_total[1m]))

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### Spend (USD) over the last hour, by model

sum by (model) (increase(openclaw_model_cost_usd_total[1h]))

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### 95th percentile model run duration

histogram_quantile( 0.95, sum by (le, provider, model) (rate(openclaw_run_duration_seconds_bucket[5m])) )

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### Queue wait time SLO (95p under 2s)

histogram_quantile( 0.95, sum by (le, lane) (rate(openclaw_queue_lane_wait_seconds_bucket[5m])) ) < 2

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### Skill usage, split by bounded source

sum by (skill, source) (increase(openclaw_skill_used_total[24h]))

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### Dropped Prometheus series (cardinality alarm)

increase(openclaw_prometheus_series_dropped_total[15m]) > 0 ```

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

### Choosing between Prometheus and OpenTelemetry export

OpenClaw supports both surfaces independently. You can run either, both, or neither.

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

주요 항목:

- **Pull** model: Prometheus scrapes `/api/diagnostics/prometheus`.
- No external collector required.
- Authenticated through normal Gateway auth.
- Surface is metrics only (no traces or logs).
- Best for stacks already standardized on Prometheus + Grafana.
- **Push** model: OpenClaw sends OTLP/HTTP to a collector or OTLP-compatible backend.
- Surface includes metrics, traces, and logs.
- Bridges to Prometheus through an OpenTelemetry Collector (`prometheus` or `prometheusremotewrite` exporter) when you need both.
- See OpenTelemetry export for the full catalog.

### 트러블슈팅

The endpoint requires the Gateway operator scope (`auth: "gateway"` with `gatewayRuntimeScopeSurface: "trusted-operator"`). Use the same token or password Prometheus uses for any other Gateway operator route. There is no public unauthenticated mode.

> 위 내용은 공식 문서의 해당 섹션 요지입니다. 세부 플래그·기본값은 원문과 `--help`를 확인하세요.

주요 항목:

- Check that `diagnostics.enabled` is not set to `false` in config (it defaults to `true`).
- Confirm the plugin is enabled and loaded with `openclaw plugins list --enabled`.
- Generate some traffic; counters and histograms only emit lines after at least one event.

### 관련 문서

주요 항목:

- Diagnostics export — local diagnostics zip for support bundles
- Health and readiness — `/healthz` and `/readyz` probes
- Logging — file-based logging
- OpenTelemetry export — OTLP push for traces, metrics, and logs

## 실습 체크리스트

1. 공식 문서와 로컬 버전을 대조합니다: `https://docs.openclaw.ai/gateway/prometheus`
2. 관련 CLI는 `openclaw --help` 및 하위 명령 `--help`로 옵션을 확인합니다.
3. 설정 변경 시 `openclaw config` / `openclaw doctor`로 유효성을 검사합니다.
4. Gateway·채널·플러그인 변경 후에는 필요 시 Gateway를 재시작합니다.

## 자주 쓰는 명령·설정 예시

```text
GET /api/diagnostics/prometheus

openclaw plugins install clawhub:@openclaw/diagnostics-prometheus

{
          plugins: {
            allow: ["diagnostics-prometheus"],
            entries: {
              "diagnostics-prometheus": { enabled: true },
            },
          },
          diagnostics: {
            enabled: true,
          },
        }

openclaw plugins enable diagnostics-prometheus

curl -H "Authorization: Bearer $OPENCLAW_GATEWAY_TOKEN" \
      http://127.0.0.1:18789/api/diagnostics/prometheus

# prometheus.yml
    scrape_configs:
      - job_name: openclaw
        scrape_interval: 30s
        metrics_path: /api/diagnostics/prometheus
        authorization:
          credentials_file: /etc/prometheus/openclaw-gateway-token
        static_configs:
          - targets: ["openclaw-gateway:18789"]

관련 링크

• 공식 원문: gateway/prometheus
• OpenClaw 문서 홈

이 가이드는 공식 문서를 한국어 학습용으로 재구성한 것입니다. 옵션 기본값·플래그 이름은 설치 버전에 따라 달라질 수 있습니다.