Testing: live suites

Testing: live suites 문서는 OpenClaw 공식 문서(help/testing-live)를 한국어로 정리한 가이드입니다. Live (network-touching) tests: model matrix, CLI backends, ACP, media providers, credentials 명령·설정 키·코드 예시는 공식 문서를 그대로 보존하며, 해석과 절차 안내는 한국어로 제공합니다. 최종 동작은 설치된 CLI 버전과 공식 원문을 확인하세요.

핵심 요약

Live (network-touching) tests: model matrix, CLI backends, ACP, media providers, credentials

한국어 가이드 범위: help/testing-live 경로의 설정·명령·제약·예시를 학습용으로 재구성합니다.

문서 구성

공식 문서의 주요 섹션은 다음과 같습니다.

Live tests vs your real gateway
Live: local smoke commands
Live: Android node capability sweep
Live: model smoke (profile keys)
Layer 1: Direct model completion (no gateway)
Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)
Live: CLI backend smoke (Claude, Gemini, or other local CLIs)
Live: APNs HTTP/2 proxy reachability
Live: ACP bind smoke (/acp spawn ... --bind here)
Live: Codex app-server harness smoke
Live: OpenAI repeated compaction
Recommended live recipes
Live: model matrix (what we cover)
Aggregators / alternate gateways
Credentials (never commit)
Deepgram live (audio transcription)
BytePlus coding plan live
ComfyUI workflow media live
Image generation live
Music generation live
Video generation live
Media live harness
관련 문서

상세 내용

본문

For quick start, QA runners, unit/integration suites, and Docker flows, see Testing. This page covers live (network-touching) tests: model matrix, CLI backends, ACP, media providers, and credential handling.

Live tests vs your real gateway

Live suites and ad hoc smokes must never disturb a gateway that is already serving real traffic (yours or another operator's):

주요 항목:

Bring your own gateway: use the in-process gateway (Layer 2 below) or start a
Do not openclaw gateway stop/restart (or launchctl/systemctl/tmux
Need realistic data? Copy the live state/DB into your dev state dir and test

Live: local smoke commands

Export the needed provider key in the process environment before ad hoc live checks.

pnpm openclaw infer tts convert --local --json \
  --text "OpenClaw live smoke." \
  --output /tmp/openclaw-live-smoke.mp3

pnpm openclaw voicecall setup --json
pnpm openclaw voicecall smoke --to "+15555550123"

Live: Android node capability sweep

주요 항목:

Test: src/gateway/android-node.capabilities.live.test.ts
Script: pnpm android:test:integration
Goal: invoke every command currently advertised by a connected Android node and assert command contract behavior.
Scope:
Preconditioned/manual setup (the suite does not install/run/pair the app).
Command-by-command gateway node.invoke validation for the selected Android node.
Required pre-setup:
Android app already connected + paired to the gateway.
App kept in foreground.
Permissions/capture consent granted for capabilities you expect to pass.
Optional target overrides:
OPENCLAW_ANDROID_NODE_ID or OPENCLAW_ANDROID_NODE_NAME.
OPENCLAW_ANDROID_GATEWAY_URL / OPENCLAW_ANDROID_GATEWAY_TOKEN / OPENCLAW_ANDROID_GATEWAY_PASSWORD.
Full Android setup details: Android App

Live: model smoke (profile keys)

Live model tests are split into two layers so failures are isolated:

주요 항목:

"Direct model" tells you whether the provider/model can answer at all with the given key.
"Gateway smoke" tells you whether the full gateway+agent pipeline works for that model (sessions, history, tools, sandbox policy, etc.).

Layer 1: Direct model completion (no gateway)

주요 항목:

Test: src/agents/models.profiles.live.test.ts
Goal:
Enumerate discovered models
Use getApiKeyForModel to select models you have creds for
Run a small completion per model (and targeted regressions where needed)
How to enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
Set OPENCLAW_LIVE_MODELS=modern, small, or all (alias for modern) to actually run this suite; otherwise it skips, so pnpm test:live on its own stays focused on gateway smoke.
How to select models:
OPENCLAW_LIVE_MODELS=modern runs the curated high-signal priority list (see Live: model matrix)
OPENCLAW_LIVE_MODELS=small runs the curated small-model priority list
OPENCLAW_LIVE_MODELS=all is an alias for modern
or OPENCLAW_LIVE_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,..." (comma allowlist)
Local Ollama small-model runs default to http://127.0.0.1:11434; set OPENCLAW_LIVE_OLLAMA_BASE_URL only for LAN, custom, or Ollama Cloud endpoints.
Modern/all and small sweeps default to their curated-list length as a cap; set OPENCLAW_LIVE_MAX_MODELS=0 for an exhaustive selected-profile sweep or a positive number for a smaller cap.
Exhaustive sweeps use OPENCLAW_LIVE_TEST_TIMEOUT_MS for the whole direct-model test timeout. 기본값: 60 minutes.
Direct-model probes run with 20-way parallelism by default; set OPENCLAW_LIVE_MODEL_CONCURRENCY to override.
How to select providers:
OPENCLAW_LIVE_PROVIDERS="google,google-antigravity,google-gemini-cli" (comma allowlist)
Where keys come from:
By default: profile store and env fallbacks
Set OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1 to enforce profile store only
Why this exists:
Separates "provider API is broken / key is invalid" from "gateway agent pipeline is broken"
Contains small, isolated regressions (example: OpenAI Responses/Codex Responses reasoning replay + tool-call flows)

Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)

To see what you can test on your machine (and the exact provider/model ids), run:

주요 항목:

Test: src/gateway/gateway-models.profiles.live.test.ts
Goal:
Spin up an in-process gateway
Create/patch an agent:dev:* session (model override per run)
Iterate models-with-keys and assert:
"meaningful" response (no tools)
a real tool invocation works (read probe)
optional extra tool probes (exec+read probe)
OpenAI regression paths (tool-call-only -> follow-up) keep working
Probe details (so you can explain failures quickly):
read probe: the test writes a nonce file in the workspace and asks the agent to read it and echo the nonce back.
exec+read probe: the test asks the agent to exec-write a nonce into a temp file, then read it back.
image probe: the test attaches a generated PNG (cat + randomized code) and expects the model to return cat <CODE>.
Implementation reference: src/gateway/gateway-models.profiles.live.test.ts and test/helpers/live-image-probe.ts.
How to enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
How to select models:
기본값: the curated high-signal (modern) priority list
OPENCLAW_LIVE_GATEWAY_MODELS=small runs the curated small-model list through the full gateway+agent pipeline
OPENCLAW_LIVE_GATEWAY_MODELS=all is an alias for modern
Or set OPENCLAW_LIVE_GATEWAY_MODELS="provider/model" (or comma list) to narrow
Modern/all and small gateway sweeps default to their curated-list length as a cap; set OPENCLAW_LIVE_GATEWAY_MAX_MODELS=0 for an exhaustive selected sweep or a positive number for a smaller cap.
How to select providers (avoid "OpenRouter everything"):
OPENCLAW_LIVE_GATEWAY_PROVIDERS="google,google-antigravity,google-gemini-cli,openai,anthropic,zai,minimax" (comma allowlist)
Tool + image probes are always on in this live test:

openclaw models list
openclaw models list --json

Live: CLI backend smoke (Claude, Gemini, or other local CLIs)

This does not ask Gemini to generate a response. It writes the same system settings OpenClaw gives Gemini, then runs gemini --debug mcp list to prove a saved transport: "streamable-http" server is normalized to Gemini's HTTP MCP shape and can connect to a local streamable-HTTP MCP server.

주요 항목:

Test: src/gateway/gateway-cli-backend.live.test.ts
Goal: validate the Gateway + agent pipeline using a local CLI backend, without touching your default config.
Backend-specific smoke defaults live with the owning plugin's cli-backend.ts definition.
Enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
OPENCLAW_LIVE_CLI_BACKEND=1
Defaults:
Default provider/model: claude-cli/claude-sonnet-4-6
Command/args/image behavior come from the owning CLI backend plugin metadata.
Overrides (optional):
OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6"
OPENCLAW_LIVE_CLI_BACKEND_COMMAND="/full/path/to/claude"
OPENCLAW_LIVE_CLI_BACKEND_ARGS='["-p","--output-format","json"]'
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_PROBE=1 to send a real image attachment (paths are injected into the prompt). Off by default in Docker recipes.
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_ARG="--image" to pass image file paths as CLI args instead of prompt injection.
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_MODE="repeat" (or "list") to control how image args are passed when IMAGE_ARG is set.
OPENCLAW_LIVE_CLI_BACKEND_RESUME_PROBE=1 to send a second turn and validate resume flow.
OPENCLAW_LIVE_CLI_BACKEND_MODEL_SWITCH_PROBE=1 to opt into the Claude Sonnet -> Opus same-session continuity probe when the selected model supports a switch target. Off by default, including in Docker recipes.
OPENCLAW_LIVE_CLI_BACKEND_MCP_PROBE=1 to opt into the MCP/tool loopback probe. Off by default in Docker recipes.
The Docker runner lives at scripts/test-live-cli-backend-docker.sh.
It runs the live CLI-backend smoke inside the repo Docker image as the non-root node user.
It resolves CLI smoke metadata from the owning plugin, then installs the matching Linux CLI package (@anthropic-ai/claude-code or @google/gemini-cli) into a cached writable prefix at OPENCLAW_DOCKER_CLI_TOOLS_DIR (default: ~/.cache/openclaw/docker-cli-tools).
codex-cli is no longer a bundled CLI backend; use openai/* with the Codex app-server runtime instead (see Live: Codex app-server harness smoke).
pnpm test:docker:live-cli-backend:claude-subscription requires portable Claude Code subscription OAuth through either ~/.claude/.credentials.json with claudeAiOauth.subscriptionType or CLAUDE_CODE_OAUTH_TOKEN from claude setup-token. It first proves direct claude -p in Docker, then runs two Gateway CLI-backend turns without preserving Anthropic API-key env vars. This subscription lane disables the Claude MCP/tool and image probes by default because it consumes the signed-in subscription's usage limits and Anthropic can change Claude Agent SDK / claude -p billing and rate-limit behavior without an OpenClaw release.
Claude and Gemini support the same probe set (text turn, image classification, MCP cron tool call, model-switch continuity) through the flags above, but none of those probes run by default - opt in per flag as needed.

OPENCLAW_LIVE_CLI_BACKEND=1 \
  OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6" \
  pnpm test:live src/gateway/gateway-cli-backend.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  pnpm test:live src/agents/cli-runner/bundle-mcp.gemini.live.test.ts

pnpm test:docker:live-cli-backend

pnpm test:docker:live-cli-backend:claude
pnpm test:docker:live-cli-backend:claude-subscription
pnpm test:docker:live-cli-backend:gemini

Live: APNs HTTP/2 proxy reachability

주요 항목:

Test: src/infra/push-apns-http2.live.test.ts
Goal: tunnel through a local HTTP CONNECT proxy to Apple's sandbox APNs endpoint, send the APNs HTTP/2 validation request, and assert Apple's real 403 InvalidProviderToken response comes back through the proxy path.
Enable:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_APNS_REACHABILITY=1 pnpm test:live src/infra/push-apns-http2.live.test.ts
Optional timeout:
OPENCLAW_LIVE_APNS_TIMEOUT_MS=30000

Live: ACP bind smoke (/acp spawn ... --bind here)

주요 항목:

Test: src/gateway/gateway-acp-bind.live.test.ts
Goal: validate the real ACP conversation-bind flow with a live ACP agent:
send /acp spawn <agent> --bind here
bind a synthetic message-channel conversation in place
send a normal follow-up on that same conversation
verify the follow-up lands in the bound ACP session transcript
Enable:
pnpm test:live src/gateway/gateway-acp-bind.live.test.ts
OPENCLAW_LIVE_ACP_BIND=1
Defaults:
ACP agents in Docker: claude,codex,gemini
ACP agent for direct pnpm test:live ...: claude
Synthetic channel: Slack DM-style conversation context
ACP backend: acpx
Overrides:
OPENCLAW_LIVE_ACP_BIND_AGENT=claude
OPENCLAW_LIVE_ACP_BIND_AGENT=codex
OPENCLAW_LIVE_ACP_BIND_AGENT=droid
OPENCLAW_LIVE_ACP_BIND_AGENT=gemini
OPENCLAW_LIVE_ACP_BIND_AGENT=opencode
OPENCLAW_LIVE_ACP_BIND_AGENTS=claude,codex,gemini
OPENCLAW_LIVE_ACP_BIND_AGENT_COMMAND='npx -y @agentclientprotocol/claude-agent-acp@<version>'
OPENCLAW_LIVE_ACP_BIND_CODEX_MODEL=gpt-5.6-luna
OPENCLAW_LIVE_ACP_BIND_OPENCODE_MODEL=opencode/kimi-k2.6
OPENCLAW_LIVE_ACP_BIND_IMAGE_PROBE=1 (or on/true/yes) to force the image probe on; any other value forces it off. Runs by default for every agent except opencode.

OPENCLAW_LIVE_ACP_BIND=1 \
  OPENCLAW_LIVE_ACP_BIND_AGENT=claude \
  pnpm test:live src/gateway/gateway-acp-bind.live.test.ts

pnpm test:docker:live-acp-bind

pnpm test:docker:live-acp-bind:claude
pnpm test:docker:live-acp-bind:codex
pnpm test:docker:live-acp-bind:droid
pnpm test:docker:live-acp-bind:gemini
pnpm test:docker:live-acp-bind:opencode

Live: Codex app-server harness smoke

command that should be approved and one fake-secret upload that should be denied so the agent asks back

주요 항목:

Goal: validate the plugin-owned Codex harness through the normal gateway
load the bundled codex plugin
select an OpenAI model through /model <ref> --runtime codex
send a first gateway agent turn with the requested thinking level
send a second turn to the same OpenClaw session and verify the app-server
run /codex status and /codex models through the same gateway command
optionally run two Guardian-reviewed escalated shell probes: one benign
Test: src/gateway/gateway-codex-harness.live.test.ts
Enable: OPENCLAW_LIVE_CODEX_HARNESS=1
Harness baseline model: openai/gpt-5.6-luna
Fresh OpenAI API-key selection default: openai/gpt-5.6
Default thinking: low
Model override: OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/<model>
Thinking override: OPENCLAW_LIVE_CODEX_HARNESS_THINKING=<level>
Non-default model effort assertion:
Matrix override: OPENCLAW_LIVE_CODEX_HARNESS_TARGETS=<model>=<thinking>,...
Auth mode: OPENCLAW_LIVE_CODEX_HARNESS_AUTH=codex-auth (default) uses the
Optional image probe: OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1
Optional MCP/tool probe: OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1
Optional Guardian probe: OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1
Optional resume stress: OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 adds
Optional fan-out stress: set OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_PROBE=1
Optional compaction stress: OPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS=1
Full direct-API context: OPENCLAW_LIVE_CODEX_HARNESS_FULL_CONTEXT=1 applies
Optional loop-relay opt-out probe:

OPENCLAW_LIVE_CODEX_HARNESS=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/gpt-5.6-luna \
  pnpm test:live -- src/gateway/gateway-codex-harness.live.test.ts

pnpm test:docker:live-codex-harness

OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 \
pnpm test:docker:live-codex-harness

OPENCLAW_LIVE_CODEX_HARNESS_AUTH=api-key \
  OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_COUNT=8 \
  OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS=1 \
  pnpm test:docker:live-codex-harness

Live: OpenAI repeated compaction

least two real automatic compactions, then verify a durable marker survives.

주요 항목:

Goal: exercise the embedded OpenClaw openai-responses agent loop through at
Test: src/agents/sessions/agent-session.openai-compaction.live.test.ts
Enable: OPENCLAW_LIVE_OPENAI_COMPACTION=1
Default model: gpt-5.6-luna
Model override: OPENCLAW_LIVE_OPENAI_COMPACTION_MODEL=<model>
The normal stress mode uses a reduced client context budget to reach the same
Full-context mode sets the client budget to 922000 and compaction reserve to
The Docker runner lives at scripts/test-live-codex-harness-docker.sh.
It passes OPENAI_API_KEY, copies Codex CLI auth files when present, installs
Docker enables the image, MCP/tool, and Guardian probes by default. Set
Docker uses the same explicit Codex runtime config, so legacy aliases or OpenClaw
Matrix targets run sequentially in one container. The Docker script scales its

OPENCLAW_LIVE_TEST=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION=1 \
  pnpm test:live -- src/agents/sessions/agent-session.openai-compaction.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION_FULL=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION_MODEL=gpt-5.6-terra \
  pnpm test:live -- src/agents/sessions/agent-session.openai-compaction.live.test.ts

OPENCLAW_LIVE_GATEWAY_OPENAI_API_DEFAULT=1 \
  OPENCLAW_LIVE_GATEWAY_PROVIDERS=openai \
  OPENCLAW_LIVE_GATEWAY_THINKING=off \
  pnpm test:live -- src/gateway/gateway-models.profiles.live.test.ts

OPENCLAW_LIVE_GATEWAY_THINKING=ultra \
  OPENCLAW_LIVE_GATEWAY_PROVIDERS=openai \
  OPENCLAW_LIVE_GATEWAY_MODELS='openai/gpt-5.6-sol,openai/gpt-5.6-terra,openai/gpt-5.6-luna' \
  pnpm test:live -- src/gateway/gateway-models.profiles.live.test.ts

Recommended live recipes

Narrow, explicit allowlists are fastest and least flaky:

주요 항목:

Single model, direct (no gateway):
OPENCLAW_LIVE_MODELS="openai/gpt-5.6-luna" pnpm test:live src/agents/models.profiles.live.test.ts
Small-model direct profile:
OPENCLAW_LIVE_MODELS=small pnpm test:live src/agents/models.profiles.live.test.ts
Small-model gateway profile:
OPENCLAW_LIVE_GATEWAY_MODELS=small pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Ollama Cloud API smoke:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_OLLAMA=1 OPENCLAW_LIVE_OLLAMA_BASE_URL=https://ollama.com OPENCLAW_LIVE_OLLAMA_MODEL=glm-5.1:cloud OPENCLAW_LIVE_OLLAMA_WEB_SEARCH=0 pnpm test:live -- extensions/ollama/ollama.live.test.ts
Single model, gateway smoke:
OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Tool calling across several providers:
OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,google/gemini-3.5-flash,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Z.AI Coding Plan GLM-5.2 direct smoke:
ZAI_CODING_LIVE_TEST=1 pnpm test:live src/agents/zai.live.test.ts
Google focus (Gemini API key + Antigravity):
Gemini (API key): OPENCLAW_LIVE_GATEWAY_MODELS="google/gemini-3.5-flash" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Antigravity (OAuth): OPENCLAW_LIVE_GATEWAY_MODELS="google-antigravity/claude-opus-4-thinking,google-antigravity/gemini-3-pro-high" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Google adaptive thinking smoke (qa manual from the private QA CLI - requires OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 and a source checkout; see QA overview):
Gemini 3 dynamic default: OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 pnpm openclaw qa manual --provider-mode live-frontier --model google/gemini-3.1-pro-preview --alt-model google/gemini-3.1-pro-preview --message '/think adaptive Reply exactly: GEMINI_ADAPTIVE_OK' --timeout-ms 180000
Gemini 2.5 dynamic budget: OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 pnpm openclaw qa manual --provider-mode live-frontier --model google/gemini-2.5-flash --alt-model google/gemini-2.5-flash --message '/think adaptive Reply exactly: GEMINI25_ADAPTIVE_OK' --timeout-ms 180000
google/... uses the Gemini API (API key).
google-antigravity/... uses the Antigravity OAuth bridge (Cloud Code Assist-style agent endpoint).
google-gemini-cli/... uses the local Gemini CLI on your machine (separate auth + tooling quirks).
Gemini API vs Gemini CLI:
API: OpenClaw calls Google's hosted Gemini API over HTTP (API key / profile auth); this is what most users mean by "Gemini".

Live: model matrix (what we cover)

Live is opt-in, so there is no fixed "CI model list." OPENCLAW_LIVE_MODELS=modern / OPENCLAW_LIVE_GATEWAY_MODELS=modern (and their all alias) run the curated priority list from HIGH_SIGNAL_LIVE_MODEL_PRIORITY in src/agents/live-model-filter.ts, in this priority order:

주요 항목:

codex and codex-cli providers are excluded from the default modern sweep (they cover CLI-backend/ACP behavior, tested separately above). openai/gpt-5.5 itself routes through the Codex app-server harness by default; see Live: Codex app-server harness smoke.
fireworks, google, openrouter, and xai only run their explicitly curated model ids in the modern sweep (no automatic "every model from this provider" expansion).
Include at least one image-capable model (Claude/Gemini/OpenAI-family vision variants, etc.) in OPENCLAW_LIVE_GATEWAY_MODELS to exercise the image probe.
Mistral: mistral/...
Cerebras: cerebras/... (if you have access)
LM Studio: lmstudio/... (local; tool calling depends on API mode)

Provider/model	Notes
`anthropic/claude-opus-5`
`anthropic/claude-opus-4-8`
`anthropic/claude-sonnet-5`
`anthropic/claude-sonnet-4-6`
`anthropic/claude-opus-4-7`
`google/gemini-3.1-pro-preview`	Gemini API
`google/gemini-3.5-flash`	Gemini API
`cohere/command-a-plus-05-2026`
`moonshot/kimi-k3`
`anthropic/claude-opus-4`
`deepseek/deepseek-v4-flash`
`deepseek/deepseek-v4-pro`
`minimax/MiniMax-M3`
`openai/gpt-5.5`
`openrouter/openai/gpt-5.2-chat`
`openrouter/minimax/minimax-m2.7`
`opencode-go/glm-5`
`openrouter/ai21/jamba-large-1.7`
`xai/grok-4.5`
`xai/grok-4.20-0309-reasoning`
`zai/glm-5.1`
`fireworks/accounts/fireworks/models/glm-5p1`
`minimax-portal/minimax-m3`

Provider/model
`lmstudio/qwen/qwen3.5-9b`
`vllm/qwen/qwen3-8b`
`sglang/qwen/qwen3-8b`
`ollama/gemma3:4b`
`openrouter/qwen/qwen3.5-9b`
`openrouter/z-ai/glm-5.1`
`openrouter/z-ai/glm-5`
`zai/glm-5.1`

OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,google/gemini-3.1-pro-preview,google/gemini-3.5-flash,google-antigravity/claude-opus-4-thinking,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts

Aggregators / alternate gateways

If you have keys enabled, you can also test via:

주요 항목:

OpenRouter: openrouter/... (hundreds of models; use openclaw models scan to find tool+image capable candidates)
OpenCode: opencode/... for Zen and opencode-go/... for Go (auth via OPENCODE_API_KEY / OPENCODE_ZEN_API_KEY)
Built-in: anthropic, cerebras, github-copilot, google, google-antigravity, google-gemini-cli, google-vertex, groq, mistral, openai, openrouter, opencode, opencode-go, xai, zai
Via models.providers (custom endpoints): minimax (cloud/API), plus any OpenAI/Anthropic-compatible proxy (LM Studio, vLLM, LiteLLM, etc.)

Credentials (never commit)

Live tests discover credentials the same way the CLI does. Practical implications:

주요 항목:

If the CLI works, live tests should find the same keys.
If a live test says "no creds", debug the same way you'd debug openclaw models list / model selection.
Per-agent auth profiles: ~/.openclaw/agents/<agentId>/agent/auth-profiles.json (this is what "profile keys" means in the live tests)
Config: ~/.openclaw/openclaw.json (or OPENCLAW_CONFIG_PATH)
Legacy OAuth dir: ~/.openclaw/credentials/ (copied into the staged live home when present, but not the main profile-key store)
Local live runs copy the active config (with agents.*.workspace / agentDir overrides stripped) and each agent's auth-profiles.json - not the rest of that agent's directory, so workspace/ and sandboxes/ data never reaches the staged home - plus the legacy credentials/ dir and supported external CLI auth files/dirs (.claude.json, .claude/.credentials.json, .claude/settings*.json, .claude/backups, .codex/auth.json, .codex/config.toml, .gemini, .minimax) into a temp test home.

Deepgram live (audio transcription)

주요 항목:

Test: extensions/deepgram/audio.live.test.ts
Enable: DEEPGRAM_API_KEY=... DEEPGRAM_LIVE_TEST=1 pnpm test:live extensions/deepgram/audio.live.test.ts

BytePlus coding plan live

주요 항목:

Test: extensions/byteplus/live.test.ts
Enable: BYTEPLUS_API_KEY=... BYTEPLUS_LIVE_TEST=1 pnpm test:live extensions/byteplus/live.test.ts
Optional model override: BYTEPLUS_CODING_MODEL=ark-code-latest

ComfyUI workflow media live

주요 항목:

Test: extensions/comfy/comfy.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts
Scope:
Exercises the bundled comfy image, video, and music_generate paths
Skips each capability unless plugins.entries.comfy.config.<capability> is configured
Useful after changing comfy workflow submission, polling, downloads, or plugin registration

Image generation live

For the shipped CLI path, add an infer smoke after the provider/runtime live test passes:

주요 항목:

Test: test/image-generation.runtime.live.test.ts
Command: pnpm test:live test/image-generation.runtime.live.test.ts
Harness: pnpm test:live:media image
Scope:
Enumerates every registered image-generation provider plugin
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs each configured provider through the shared image-generation runtime:
<provider>:generate
<provider>:edit when the provider declares edit support
Current bundled providers covered:
deepinfra
fal
google
minimax
openai
openrouter
vydra
xai
Optional narrowing:
OPENCLAW_LIVE_IMAGE_GENERATION_PROVIDERS="openai,google,openrouter,xai"
OPENCLAW_LIVE_IMAGE_GENERATION_PROVIDERS="deepinfra"
OPENCLAW_LIVE_IMAGE_GENERATION_MODELS="openai/gpt-image-2,google/gemini-3.1-flash-image,openrouter/google/gemini-3.1-flash-image-preview,xai/grok-imagine-image"
OPENCLAW_LIVE_IMAGE_GENERATION_CASES="google:flash-generate,google:pro-edit,openrouter:generate,xai:default-generate,xai:default-edit"

OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_INFER_CLI_TEST=1 pnpm test:live -- test/image-generation.infer-cli.live.test.ts
openclaw infer image providers --json
openclaw infer image generate \
  --model google/gemini-3.1-flash-image \
  --prompt "Minimal flat test image: one blue square on a white background, no text." \
  --output ./openclaw-infer-image-smoke.png \
  --json

Music generation live

주요 항목:

Test: extensions/music-generation-providers.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts
Harness: pnpm test:live:media music
Scope:
Exercises the shared bundled music-generation provider path
Currently covers fal, google, minimax, and openrouter
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs both declared runtime modes when available:
generate with prompt-only input
edit when the provider declares capabilities.edit.enabled
comfy has its own separate live file, not this shared sweep
Optional narrowing:
OPENCLAW_LIVE_MUSIC_GENERATION_PROVIDERS="google,minimax"
OPENCLAW_LIVE_MUSIC_GENERATION_MODELS="google/lyria-3-clip-preview,minimax/music-2.6"
Optional auth behavior:
OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1 to force profile-store auth and ignore env-only overrides

Video generation live

주요 항목:

Test: extensions/video-generation-providers.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/video-generation-providers.live.test.ts
Harness: pnpm test:live:media video
Scope:
Exercises the shared bundled video-generation provider path across alibaba, byteplus, deepinfra, fal, google, minimax, openai, openrouter, pixverse, qwen, runway, together, vydra, xai
Defaults to the release-safe smoke path: one text-to-video request per provider, one-second lobster prompt, and a per-provider operation cap from OPENCLAW_LIVE_VIDEO_GENERATION_TIMEOUT_MS (180000 by default)
Skips FAL by default because provider-side queue latency can dominate release time; pass OPENCLAW_LIVE_VIDEO_GENERATION_PROVIDERS="fal" (or clear the skip list) to run it explicitly
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs only generate by default
Set OPENCLAW_LIVE_VIDEO_GENERATION_FULL_MODES=1 to also run declared transform modes when available:
imageToVideo when the provider declares capabilities.imageToVideo.enabled and the selected provider/model accepts buffer-backed local image input in the shared sweep
videoToVideo when the provider declares capabilities.videoToVideo.enabled and the selected provider/model accepts buffer-backed local video input in the shared sweep
Current declared-but-skipped imageToVideo provider in the shared sweep:
vydra (buffer-backed local image input is not supported in this lane)
Provider-specific Vydra coverage:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_VYDRA_VIDEO=1 pnpm test:live -- extensions/vydra/vydra.live.test.ts
That file runs veo3 text-to-video plus a kling image-to-video lane that uses a remote image URL fixture by default (OPENCLAW_LIVE_VYDRA_KLING_IMAGE_URL to override).
Provider-specific xAI coverage:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_XAI_VIDEO=1 pnpm test:live -- extensions/xai/xai.live.test.ts -t "classic Grok Imagine"
The classic case generates a square local PNG first frame, omits geometry, requests a one-second image-to-video clip, polls to completion, and verifies the downloaded buffer.
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_XAI_VIDEO=1 pnpm test:live -- extensions/xai/xai.live.test.ts -t "Grok Imagine Video 1.5"
The 1.5 case generates a local PNG first frame, requests a one-second 1080P image-to-video clip, polls to completion, and verifies the downloaded buffer.
Current videoToVideo live coverage:

Media live harness

주요 항목:

Command: pnpm test:live:media
Entrypoint: test/e2e/qa-lab/media/hosted-media-provider-live.ts, which runs pnpm test:live -- <suite-test-file> per selected suite, so heartbeat and quiet-mode behavior stay consistent with other pnpm test:live runs.
Purpose:
Runs the shared image, music, and video live suites through one repo-native entrypoint
Auto-loads missing provider env vars from ~/.profile
Auto-narrows each suite to providers that currently have usable auth by default
Flags:
--providers <csv> global provider filter; --image-providers / --music-providers / --video-providers scope a filter to one suite
--all-providers skips the auth-based auto-filter
--allow-empty exits 0 when filtering leaves no runnable providers
--quiet / --no-quiet passed through to test:live
Examples:
pnpm test:live:media
pnpm test:live:media image video --providers openai,google,minimax
pnpm test:live:media video --video-providers openai,runway --all-providers
pnpm test:live:media music --quiet

실습 체크리스트

공식 문서와 로컬 버전을 대조합니다: https://docs.openclaw.ai/help/testing-live
관련 CLI는 openclaw --help 및 하위 명령 --help로 옵션을 확인합니다.
설정 변경 시 openclaw config / openclaw doctor로 유효성을 검사합니다.
Gateway·채널·플러그인 변경 후에는 필요 시 Gateway를 재시작합니다.

자주 쓰는 명령·설정 예시

pnpm openclaw infer tts convert --local --json \
  --text "OpenClaw live smoke." \
  --output /tmp/openclaw-live-smoke.mp3

pnpm openclaw voicecall setup --json
pnpm openclaw voicecall smoke --to "+15555550123"

openclaw models list
openclaw models list --json

OPENCLAW_LIVE_CLI_BACKEND=1 \
  OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6" \
  pnpm test:live src/gateway/gateway-cli-backend.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  pnpm test:live src/agents/cli-runner/bundle-mcp.gemini.live.test.ts

pnpm test:docker:live-cli-backend

Testing: live suites

핵심 요약

Live (network-touching) tests: model matrix, CLI backends, ACP, media providers, credentials

한국어 가이드 범위: help/testing-live 경로의 설정·명령·제약·예시를 학습용으로 재구성합니다.

문서 구성

공식 문서의 주요 섹션은 다음과 같습니다.

Live tests vs your real gateway
Live: local smoke commands
Live: Android node capability sweep
Live: model smoke (profile keys)
Layer 1: Direct model completion (no gateway)
Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)
Live: CLI backend smoke (Claude, Gemini, or other local CLIs)
Live: APNs HTTP/2 proxy reachability
Live: ACP bind smoke (/acp spawn ... --bind here)
Live: Codex app-server harness smoke
Live: OpenAI repeated compaction
Recommended live recipes
Live: model matrix (what we cover)
Aggregators / alternate gateways
Credentials (never commit)
Deepgram live (audio transcription)
BytePlus coding plan live
ComfyUI workflow media live
Image generation live
Music generation live
Video generation live
Media live harness
관련 문서

상세 내용

본문

Live tests vs your real gateway

Live suites and ad hoc smokes must never disturb a gateway that is already serving real traffic (yours or another operator's):

주요 항목:

Bring your own gateway: use the in-process gateway (Layer 2 below) or start a
Do not openclaw gateway stop/restart (or launchctl/systemctl/tmux
Need realistic data? Copy the live state/DB into your dev state dir and test

Live: local smoke commands

Export the needed provider key in the process environment before ad hoc live checks.

pnpm openclaw infer tts convert --local --json \
  --text "OpenClaw live smoke." \
  --output /tmp/openclaw-live-smoke.mp3

pnpm openclaw voicecall setup --json
pnpm openclaw voicecall smoke --to "+15555550123"

Live: Android node capability sweep

주요 항목:

Test: src/gateway/android-node.capabilities.live.test.ts
Script: pnpm android:test:integration
Goal: invoke every command currently advertised by a connected Android node and assert command contract behavior.
Scope:
Preconditioned/manual setup (the suite does not install/run/pair the app).
Command-by-command gateway node.invoke validation for the selected Android node.
Required pre-setup:
Android app already connected + paired to the gateway.
App kept in foreground.
Permissions/capture consent granted for capabilities you expect to pass.
Optional target overrides:
OPENCLAW_ANDROID_NODE_ID or OPENCLAW_ANDROID_NODE_NAME.
OPENCLAW_ANDROID_GATEWAY_URL / OPENCLAW_ANDROID_GATEWAY_TOKEN / OPENCLAW_ANDROID_GATEWAY_PASSWORD.
Full Android setup details: Android App

Live: model smoke (profile keys)

Live model tests are split into two layers so failures are isolated:

주요 항목:

"Direct model" tells you whether the provider/model can answer at all with the given key.
"Gateway smoke" tells you whether the full gateway+agent pipeline works for that model (sessions, history, tools, sandbox policy, etc.).

Layer 1: Direct model completion (no gateway)

주요 항목:

Test: src/agents/models.profiles.live.test.ts
Goal:
Enumerate discovered models
Use getApiKeyForModel to select models you have creds for
Run a small completion per model (and targeted regressions where needed)
How to enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
Set OPENCLAW_LIVE_MODELS=modern, small, or all (alias for modern) to actually run this suite; otherwise it skips, so pnpm test:live on its own stays focused on gateway smoke.
How to select models:
OPENCLAW_LIVE_MODELS=modern runs the curated high-signal priority list (see Live: model matrix)
OPENCLAW_LIVE_MODELS=small runs the curated small-model priority list
OPENCLAW_LIVE_MODELS=all is an alias for modern
or OPENCLAW_LIVE_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,..." (comma allowlist)
Local Ollama small-model runs default to http://127.0.0.1:11434; set OPENCLAW_LIVE_OLLAMA_BASE_URL only for LAN, custom, or Ollama Cloud endpoints.
Modern/all and small sweeps default to their curated-list length as a cap; set OPENCLAW_LIVE_MAX_MODELS=0 for an exhaustive selected-profile sweep or a positive number for a smaller cap.
Exhaustive sweeps use OPENCLAW_LIVE_TEST_TIMEOUT_MS for the whole direct-model test timeout. 기본값: 60 minutes.
Direct-model probes run with 20-way parallelism by default; set OPENCLAW_LIVE_MODEL_CONCURRENCY to override.
How to select providers:
OPENCLAW_LIVE_PROVIDERS="google,google-antigravity,google-gemini-cli" (comma allowlist)
Where keys come from:
By default: profile store and env fallbacks
Set OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1 to enforce profile store only
Why this exists:
Separates "provider API is broken / key is invalid" from "gateway agent pipeline is broken"
Contains small, isolated regressions (example: OpenAI Responses/Codex Responses reasoning replay + tool-call flows)

Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)

To see what you can test on your machine (and the exact provider/model ids), run:

주요 항목:

Test: src/gateway/gateway-models.profiles.live.test.ts
Goal:
Spin up an in-process gateway
Create/patch an agent:dev:* session (model override per run)
Iterate models-with-keys and assert:
"meaningful" response (no tools)
a real tool invocation works (read probe)
optional extra tool probes (exec+read probe)
OpenAI regression paths (tool-call-only -> follow-up) keep working
Probe details (so you can explain failures quickly):
read probe: the test writes a nonce file in the workspace and asks the agent to read it and echo the nonce back.
exec+read probe: the test asks the agent to exec-write a nonce into a temp file, then read it back.
image probe: the test attaches a generated PNG (cat + randomized code) and expects the model to return cat <CODE>.
Implementation reference: src/gateway/gateway-models.profiles.live.test.ts and test/helpers/live-image-probe.ts.
How to enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
How to select models:
기본값: the curated high-signal (modern) priority list
OPENCLAW_LIVE_GATEWAY_MODELS=small runs the curated small-model list through the full gateway+agent pipeline
OPENCLAW_LIVE_GATEWAY_MODELS=all is an alias for modern
Or set OPENCLAW_LIVE_GATEWAY_MODELS="provider/model" (or comma list) to narrow
Modern/all and small gateway sweeps default to their curated-list length as a cap; set OPENCLAW_LIVE_GATEWAY_MAX_MODELS=0 for an exhaustive selected sweep or a positive number for a smaller cap.
How to select providers (avoid "OpenRouter everything"):
OPENCLAW_LIVE_GATEWAY_PROVIDERS="google,google-antigravity,google-gemini-cli,openai,anthropic,zai,minimax" (comma allowlist)
Tool + image probes are always on in this live test:

openclaw models list
openclaw models list --json

Live: CLI backend smoke (Claude, Gemini, or other local CLIs)

주요 항목:

Test: src/gateway/gateway-cli-backend.live.test.ts
Goal: validate the Gateway + agent pipeline using a local CLI backend, without touching your default config.
Backend-specific smoke defaults live with the owning plugin's cli-backend.ts definition.
Enable:
pnpm test:live (or OPENCLAW_LIVE_TEST=1 if invoking Vitest directly)
OPENCLAW_LIVE_CLI_BACKEND=1
Defaults:
Default provider/model: claude-cli/claude-sonnet-4-6
Command/args/image behavior come from the owning CLI backend plugin metadata.
Overrides (optional):
OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6"
OPENCLAW_LIVE_CLI_BACKEND_COMMAND="/full/path/to/claude"
OPENCLAW_LIVE_CLI_BACKEND_ARGS='["-p","--output-format","json"]'
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_PROBE=1 to send a real image attachment (paths are injected into the prompt). Off by default in Docker recipes.
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_ARG="--image" to pass image file paths as CLI args instead of prompt injection.
OPENCLAW_LIVE_CLI_BACKEND_IMAGE_MODE="repeat" (or "list") to control how image args are passed when IMAGE_ARG is set.
OPENCLAW_LIVE_CLI_BACKEND_RESUME_PROBE=1 to send a second turn and validate resume flow.
OPENCLAW_LIVE_CLI_BACKEND_MODEL_SWITCH_PROBE=1 to opt into the Claude Sonnet -> Opus same-session continuity probe when the selected model supports a switch target. Off by default, including in Docker recipes.
OPENCLAW_LIVE_CLI_BACKEND_MCP_PROBE=1 to opt into the MCP/tool loopback probe. Off by default in Docker recipes.
The Docker runner lives at scripts/test-live-cli-backend-docker.sh.
It runs the live CLI-backend smoke inside the repo Docker image as the non-root node user.
It resolves CLI smoke metadata from the owning plugin, then installs the matching Linux CLI package (@anthropic-ai/claude-code or @google/gemini-cli) into a cached writable prefix at OPENCLAW_DOCKER_CLI_TOOLS_DIR (default: ~/.cache/openclaw/docker-cli-tools).
codex-cli is no longer a bundled CLI backend; use openai/* with the Codex app-server runtime instead (see Live: Codex app-server harness smoke).
pnpm test:docker:live-cli-backend:claude-subscription requires portable Claude Code subscription OAuth through either ~/.claude/.credentials.json with claudeAiOauth.subscriptionType or CLAUDE_CODE_OAUTH_TOKEN from claude setup-token. It first proves direct claude -p in Docker, then runs two Gateway CLI-backend turns without preserving Anthropic API-key env vars. This subscription lane disables the Claude MCP/tool and image probes by default because it consumes the signed-in subscription's usage limits and Anthropic can change Claude Agent SDK / claude -p billing and rate-limit behavior without an OpenClaw release.
Claude and Gemini support the same probe set (text turn, image classification, MCP cron tool call, model-switch continuity) through the flags above, but none of those probes run by default - opt in per flag as needed.

OPENCLAW_LIVE_CLI_BACKEND=1 \
  OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6" \
  pnpm test:live src/gateway/gateway-cli-backend.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  pnpm test:live src/agents/cli-runner/bundle-mcp.gemini.live.test.ts

pnpm test:docker:live-cli-backend

pnpm test:docker:live-cli-backend:claude
pnpm test:docker:live-cli-backend:claude-subscription
pnpm test:docker:live-cli-backend:gemini

Live: APNs HTTP/2 proxy reachability

주요 항목:

Test: src/infra/push-apns-http2.live.test.ts
Goal: tunnel through a local HTTP CONNECT proxy to Apple's sandbox APNs endpoint, send the APNs HTTP/2 validation request, and assert Apple's real 403 InvalidProviderToken response comes back through the proxy path.
Enable:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_APNS_REACHABILITY=1 pnpm test:live src/infra/push-apns-http2.live.test.ts
Optional timeout:
OPENCLAW_LIVE_APNS_TIMEOUT_MS=30000

Live: ACP bind smoke (/acp spawn ... --bind here)

주요 항목:

Test: src/gateway/gateway-acp-bind.live.test.ts
Goal: validate the real ACP conversation-bind flow with a live ACP agent:
send /acp spawn <agent> --bind here
bind a synthetic message-channel conversation in place
send a normal follow-up on that same conversation
verify the follow-up lands in the bound ACP session transcript
Enable:
pnpm test:live src/gateway/gateway-acp-bind.live.test.ts
OPENCLAW_LIVE_ACP_BIND=1
Defaults:
ACP agents in Docker: claude,codex,gemini
ACP agent for direct pnpm test:live ...: claude
Synthetic channel: Slack DM-style conversation context
ACP backend: acpx
Overrides:
OPENCLAW_LIVE_ACP_BIND_AGENT=claude
OPENCLAW_LIVE_ACP_BIND_AGENT=codex
OPENCLAW_LIVE_ACP_BIND_AGENT=droid
OPENCLAW_LIVE_ACP_BIND_AGENT=gemini
OPENCLAW_LIVE_ACP_BIND_AGENT=opencode
OPENCLAW_LIVE_ACP_BIND_AGENTS=claude,codex,gemini
OPENCLAW_LIVE_ACP_BIND_AGENT_COMMAND='npx -y @agentclientprotocol/claude-agent-acp@<version>'
OPENCLAW_LIVE_ACP_BIND_CODEX_MODEL=gpt-5.6-luna
OPENCLAW_LIVE_ACP_BIND_OPENCODE_MODEL=opencode/kimi-k2.6
OPENCLAW_LIVE_ACP_BIND_IMAGE_PROBE=1 (or on/true/yes) to force the image probe on; any other value forces it off. Runs by default for every agent except opencode.

OPENCLAW_LIVE_ACP_BIND=1 \
  OPENCLAW_LIVE_ACP_BIND_AGENT=claude \
  pnpm test:live src/gateway/gateway-acp-bind.live.test.ts

pnpm test:docker:live-acp-bind

pnpm test:docker:live-acp-bind:claude
pnpm test:docker:live-acp-bind:codex
pnpm test:docker:live-acp-bind:droid
pnpm test:docker:live-acp-bind:gemini
pnpm test:docker:live-acp-bind:opencode

Live: Codex app-server harness smoke

command that should be approved and one fake-secret upload that should be denied so the agent asks back

주요 항목:

Goal: validate the plugin-owned Codex harness through the normal gateway
load the bundled codex plugin
select an OpenAI model through /model <ref> --runtime codex
send a first gateway agent turn with the requested thinking level
send a second turn to the same OpenClaw session and verify the app-server
run /codex status and /codex models through the same gateway command
optionally run two Guardian-reviewed escalated shell probes: one benign
Test: src/gateway/gateway-codex-harness.live.test.ts
Enable: OPENCLAW_LIVE_CODEX_HARNESS=1
Harness baseline model: openai/gpt-5.6-luna
Fresh OpenAI API-key selection default: openai/gpt-5.6
Default thinking: low
Model override: OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/<model>
Thinking override: OPENCLAW_LIVE_CODEX_HARNESS_THINKING=<level>
Non-default model effort assertion:
Matrix override: OPENCLAW_LIVE_CODEX_HARNESS_TARGETS=<model>=<thinking>,...
Auth mode: OPENCLAW_LIVE_CODEX_HARNESS_AUTH=codex-auth (default) uses the
Optional image probe: OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1
Optional MCP/tool probe: OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1
Optional Guardian probe: OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1
Optional resume stress: OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 adds
Optional fan-out stress: set OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_PROBE=1
Optional compaction stress: OPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS=1
Full direct-API context: OPENCLAW_LIVE_CODEX_HARNESS_FULL_CONTEXT=1 applies
Optional loop-relay opt-out probe:

OPENCLAW_LIVE_CODEX_HARNESS=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/gpt-5.6-luna \
  pnpm test:live -- src/gateway/gateway-codex-harness.live.test.ts

pnpm test:docker:live-codex-harness

OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 \
pnpm test:docker:live-codex-harness

OPENCLAW_LIVE_CODEX_HARNESS_AUTH=api-key \
  OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_PROBE=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_COUNT=8 \
  OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1 \
  OPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS=1 \
  pnpm test:docker:live-codex-harness

Live: OpenAI repeated compaction

least two real automatic compactions, then verify a durable marker survives.

주요 항목:

Goal: exercise the embedded OpenClaw openai-responses agent loop through at
Test: src/agents/sessions/agent-session.openai-compaction.live.test.ts
Enable: OPENCLAW_LIVE_OPENAI_COMPACTION=1
Default model: gpt-5.6-luna
Model override: OPENCLAW_LIVE_OPENAI_COMPACTION_MODEL=<model>
The normal stress mode uses a reduced client context budget to reach the same
Full-context mode sets the client budget to 922000 and compaction reserve to
The Docker runner lives at scripts/test-live-codex-harness-docker.sh.
It passes OPENAI_API_KEY, copies Codex CLI auth files when present, installs
Docker enables the image, MCP/tool, and Guardian probes by default. Set
Docker uses the same explicit Codex runtime config, so legacy aliases or OpenClaw
Matrix targets run sequentially in one container. The Docker script scales its

OPENCLAW_LIVE_TEST=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION=1 \
  pnpm test:live -- src/agents/sessions/agent-session.openai-compaction.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION_FULL=1 \
  OPENCLAW_LIVE_OPENAI_COMPACTION_MODEL=gpt-5.6-terra \
  pnpm test:live -- src/agents/sessions/agent-session.openai-compaction.live.test.ts

OPENCLAW_LIVE_GATEWAY_OPENAI_API_DEFAULT=1 \
  OPENCLAW_LIVE_GATEWAY_PROVIDERS=openai \
  OPENCLAW_LIVE_GATEWAY_THINKING=off \
  pnpm test:live -- src/gateway/gateway-models.profiles.live.test.ts

OPENCLAW_LIVE_GATEWAY_THINKING=ultra \
  OPENCLAW_LIVE_GATEWAY_PROVIDERS=openai \
  OPENCLAW_LIVE_GATEWAY_MODELS='openai/gpt-5.6-sol,openai/gpt-5.6-terra,openai/gpt-5.6-luna' \
  pnpm test:live -- src/gateway/gateway-models.profiles.live.test.ts

Recommended live recipes

Narrow, explicit allowlists are fastest and least flaky:

주요 항목:

Single model, direct (no gateway):
OPENCLAW_LIVE_MODELS="openai/gpt-5.6-luna" pnpm test:live src/agents/models.profiles.live.test.ts
Small-model direct profile:
OPENCLAW_LIVE_MODELS=small pnpm test:live src/agents/models.profiles.live.test.ts
Small-model gateway profile:
OPENCLAW_LIVE_GATEWAY_MODELS=small pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Ollama Cloud API smoke:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_OLLAMA=1 OPENCLAW_LIVE_OLLAMA_BASE_URL=https://ollama.com OPENCLAW_LIVE_OLLAMA_MODEL=glm-5.1:cloud OPENCLAW_LIVE_OLLAMA_WEB_SEARCH=0 pnpm test:live -- extensions/ollama/ollama.live.test.ts
Single model, gateway smoke:
OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Tool calling across several providers:
OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,google/gemini-3.5-flash,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Z.AI Coding Plan GLM-5.2 direct smoke:
ZAI_CODING_LIVE_TEST=1 pnpm test:live src/agents/zai.live.test.ts
Google focus (Gemini API key + Antigravity):
Gemini (API key): OPENCLAW_LIVE_GATEWAY_MODELS="google/gemini-3.5-flash" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Antigravity (OAuth): OPENCLAW_LIVE_GATEWAY_MODELS="google-antigravity/claude-opus-4-thinking,google-antigravity/gemini-3-pro-high" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts
Google adaptive thinking smoke (qa manual from the private QA CLI - requires OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 and a source checkout; see QA overview):
Gemini 3 dynamic default: OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 pnpm openclaw qa manual --provider-mode live-frontier --model google/gemini-3.1-pro-preview --alt-model google/gemini-3.1-pro-preview --message '/think adaptive Reply exactly: GEMINI_ADAPTIVE_OK' --timeout-ms 180000
Gemini 2.5 dynamic budget: OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 pnpm openclaw qa manual --provider-mode live-frontier --model google/gemini-2.5-flash --alt-model google/gemini-2.5-flash --message '/think adaptive Reply exactly: GEMINI25_ADAPTIVE_OK' --timeout-ms 180000
google/... uses the Gemini API (API key).
google-antigravity/... uses the Antigravity OAuth bridge (Cloud Code Assist-style agent endpoint).
google-gemini-cli/... uses the local Gemini CLI on your machine (separate auth + tooling quirks).
Gemini API vs Gemini CLI:
API: OpenClaw calls Google's hosted Gemini API over HTTP (API key / profile auth); this is what most users mean by "Gemini".

Live: model matrix (what we cover)

주요 항목:

codex and codex-cli providers are excluded from the default modern sweep (they cover CLI-backend/ACP behavior, tested separately above). openai/gpt-5.5 itself routes through the Codex app-server harness by default; see Live: Codex app-server harness smoke.
fireworks, google, openrouter, and xai only run their explicitly curated model ids in the modern sweep (no automatic "every model from this provider" expansion).
Include at least one image-capable model (Claude/Gemini/OpenAI-family vision variants, etc.) in OPENCLAW_LIVE_GATEWAY_MODELS to exercise the image probe.
Mistral: mistral/...
Cerebras: cerebras/... (if you have access)
LM Studio: lmstudio/... (local; tool calling depends on API mode)

Provider/model	Notes
`anthropic/claude-opus-5`
`anthropic/claude-opus-4-8`
`anthropic/claude-sonnet-5`
`anthropic/claude-sonnet-4-6`
`anthropic/claude-opus-4-7`
`google/gemini-3.1-pro-preview`	Gemini API
`google/gemini-3.5-flash`	Gemini API
`cohere/command-a-plus-05-2026`
`moonshot/kimi-k3`
`anthropic/claude-opus-4`
`deepseek/deepseek-v4-flash`
`deepseek/deepseek-v4-pro`
`minimax/MiniMax-M3`
`openai/gpt-5.5`
`openrouter/openai/gpt-5.2-chat`
`openrouter/minimax/minimax-m2.7`
`opencode-go/glm-5`
`openrouter/ai21/jamba-large-1.7`
`xai/grok-4.5`
`xai/grok-4.20-0309-reasoning`
`zai/glm-5.1`
`fireworks/accounts/fireworks/models/glm-5p1`
`minimax-portal/minimax-m3`

Provider/model
`lmstudio/qwen/qwen3.5-9b`
`vllm/qwen/qwen3-8b`
`sglang/qwen/qwen3-8b`
`ollama/gemma3:4b`
`openrouter/qwen/qwen3.5-9b`
`openrouter/z-ai/glm-5.1`
`openrouter/z-ai/glm-5`
`zai/glm-5.1`

OPENCLAW_LIVE_GATEWAY_MODELS="openai/gpt-5.6-luna,anthropic/claude-opus-4,google/gemini-3.1-pro-preview,google/gemini-3.5-flash,google-antigravity/claude-opus-4-thinking,deepseek/deepseek-v4-flash,zai/glm-5.1,minimax/MiniMax-M3" pnpm test:live src/gateway/gateway-models.profiles.live.test.ts

Aggregators / alternate gateways

If you have keys enabled, you can also test via:

주요 항목:

OpenRouter: openrouter/... (hundreds of models; use openclaw models scan to find tool+image capable candidates)
OpenCode: opencode/... for Zen and opencode-go/... for Go (auth via OPENCODE_API_KEY / OPENCODE_ZEN_API_KEY)
Built-in: anthropic, cerebras, github-copilot, google, google-antigravity, google-gemini-cli, google-vertex, groq, mistral, openai, openrouter, opencode, opencode-go, xai, zai
Via models.providers (custom endpoints): minimax (cloud/API), plus any OpenAI/Anthropic-compatible proxy (LM Studio, vLLM, LiteLLM, etc.)

Credentials (never commit)

Live tests discover credentials the same way the CLI does. Practical implications:

주요 항목:

If the CLI works, live tests should find the same keys.
If a live test says "no creds", debug the same way you'd debug openclaw models list / model selection.
Per-agent auth profiles: ~/.openclaw/agents/<agentId>/agent/auth-profiles.json (this is what "profile keys" means in the live tests)
Config: ~/.openclaw/openclaw.json (or OPENCLAW_CONFIG_PATH)
Legacy OAuth dir: ~/.openclaw/credentials/ (copied into the staged live home when present, but not the main profile-key store)
Local live runs copy the active config (with agents.*.workspace / agentDir overrides stripped) and each agent's auth-profiles.json - not the rest of that agent's directory, so workspace/ and sandboxes/ data never reaches the staged home - plus the legacy credentials/ dir and supported external CLI auth files/dirs (.claude.json, .claude/.credentials.json, .claude/settings*.json, .claude/backups, .codex/auth.json, .codex/config.toml, .gemini, .minimax) into a temp test home.

Deepgram live (audio transcription)

주요 항목:

Test: extensions/deepgram/audio.live.test.ts
Enable: DEEPGRAM_API_KEY=... DEEPGRAM_LIVE_TEST=1 pnpm test:live extensions/deepgram/audio.live.test.ts

BytePlus coding plan live

주요 항목:

Test: extensions/byteplus/live.test.ts
Enable: BYTEPLUS_API_KEY=... BYTEPLUS_LIVE_TEST=1 pnpm test:live extensions/byteplus/live.test.ts
Optional model override: BYTEPLUS_CODING_MODEL=ark-code-latest

ComfyUI workflow media live

주요 항목:

Test: extensions/comfy/comfy.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts
Scope:
Exercises the bundled comfy image, video, and music_generate paths
Skips each capability unless plugins.entries.comfy.config.<capability> is configured
Useful after changing comfy workflow submission, polling, downloads, or plugin registration

Image generation live

For the shipped CLI path, add an infer smoke after the provider/runtime live test passes:

주요 항목:

Test: test/image-generation.runtime.live.test.ts
Command: pnpm test:live test/image-generation.runtime.live.test.ts
Harness: pnpm test:live:media image
Scope:
Enumerates every registered image-generation provider plugin
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs each configured provider through the shared image-generation runtime:
<provider>:generate
<provider>:edit when the provider declares edit support
Current bundled providers covered:
deepinfra
fal
google
minimax
openai
openrouter
vydra
xai
Optional narrowing:
OPENCLAW_LIVE_IMAGE_GENERATION_PROVIDERS="openai,google,openrouter,xai"
OPENCLAW_LIVE_IMAGE_GENERATION_PROVIDERS="deepinfra"
OPENCLAW_LIVE_IMAGE_GENERATION_MODELS="openai/gpt-image-2,google/gemini-3.1-flash-image,openrouter/google/gemini-3.1-flash-image-preview,xai/grok-imagine-image"
OPENCLAW_LIVE_IMAGE_GENERATION_CASES="google:flash-generate,google:pro-edit,openrouter:generate,xai:default-generate,xai:default-edit"

OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_INFER_CLI_TEST=1 pnpm test:live -- test/image-generation.infer-cli.live.test.ts
openclaw infer image providers --json
openclaw infer image generate \
  --model google/gemini-3.1-flash-image \
  --prompt "Minimal flat test image: one blue square on a white background, no text." \
  --output ./openclaw-infer-image-smoke.png \
  --json

Music generation live

주요 항목:

Test: extensions/music-generation-providers.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts
Harness: pnpm test:live:media music
Scope:
Exercises the shared bundled music-generation provider path
Currently covers fal, google, minimax, and openrouter
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs both declared runtime modes when available:
generate with prompt-only input
edit when the provider declares capabilities.edit.enabled
comfy has its own separate live file, not this shared sweep
Optional narrowing:
OPENCLAW_LIVE_MUSIC_GENERATION_PROVIDERS="google,minimax"
OPENCLAW_LIVE_MUSIC_GENERATION_MODELS="google/lyria-3-clip-preview,minimax/music-2.6"
Optional auth behavior:
OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1 to force profile-store auth and ignore env-only overrides

Video generation live

주요 항목:

Test: extensions/video-generation-providers.live.test.ts
Enable: OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/video-generation-providers.live.test.ts
Harness: pnpm test:live:media video
Scope:
Exercises the shared bundled video-generation provider path across alibaba, byteplus, deepinfra, fal, google, minimax, openai, openrouter, pixverse, qwen, runway, together, vydra, xai
Defaults to the release-safe smoke path: one text-to-video request per provider, one-second lobster prompt, and a per-provider operation cap from OPENCLAW_LIVE_VIDEO_GENERATION_TIMEOUT_MS (180000 by default)
Skips FAL by default because provider-side queue latency can dominate release time; pass OPENCLAW_LIVE_VIDEO_GENERATION_PROVIDERS="fal" (or clear the skip list) to run it explicitly
Uses already-exported provider env vars before probing
Uses live/env API keys ahead of stored auth profiles by default, so stale test keys in auth-profiles.json do not mask real shell credentials
Skips providers with no usable auth/profile/model
Runs only generate by default
Set OPENCLAW_LIVE_VIDEO_GENERATION_FULL_MODES=1 to also run declared transform modes when available:
imageToVideo when the provider declares capabilities.imageToVideo.enabled and the selected provider/model accepts buffer-backed local image input in the shared sweep
videoToVideo when the provider declares capabilities.videoToVideo.enabled and the selected provider/model accepts buffer-backed local video input in the shared sweep
Current declared-but-skipped imageToVideo provider in the shared sweep:
vydra (buffer-backed local image input is not supported in this lane)
Provider-specific Vydra coverage:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_VYDRA_VIDEO=1 pnpm test:live -- extensions/vydra/vydra.live.test.ts
That file runs veo3 text-to-video plus a kling image-to-video lane that uses a remote image URL fixture by default (OPENCLAW_LIVE_VYDRA_KLING_IMAGE_URL to override).
Provider-specific xAI coverage:
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_XAI_VIDEO=1 pnpm test:live -- extensions/xai/xai.live.test.ts -t "classic Grok Imagine"
The classic case generates a square local PNG first frame, omits geometry, requests a one-second image-to-video clip, polls to completion, and verifies the downloaded buffer.
OPENCLAW_LIVE_TEST=1 OPENCLAW_LIVE_XAI_VIDEO=1 pnpm test:live -- extensions/xai/xai.live.test.ts -t "Grok Imagine Video 1.5"
The 1.5 case generates a local PNG first frame, requests a one-second 1080P image-to-video clip, polls to completion, and verifies the downloaded buffer.
Current videoToVideo live coverage:

Media live harness

주요 항목:

Command: pnpm test:live:media
Entrypoint: test/e2e/qa-lab/media/hosted-media-provider-live.ts, which runs pnpm test:live -- <suite-test-file> per selected suite, so heartbeat and quiet-mode behavior stay consistent with other pnpm test:live runs.
Purpose:
Runs the shared image, music, and video live suites through one repo-native entrypoint
Auto-loads missing provider env vars from ~/.profile
Auto-narrows each suite to providers that currently have usable auth by default
Flags:
--providers <csv> global provider filter; --image-providers / --music-providers / --video-providers scope a filter to one suite
--all-providers skips the auth-based auto-filter
--allow-empty exits 0 when filtering leaves no runnable providers
--quiet / --no-quiet passed through to test:live
Examples:
pnpm test:live:media
pnpm test:live:media image video --providers openai,google,minimax
pnpm test:live:media video --video-providers openai,runway --all-providers
pnpm test:live:media music --quiet

실습 체크리스트

공식 문서와 로컬 버전을 대조합니다: https://docs.openclaw.ai/help/testing-live
관련 CLI는 openclaw --help 및 하위 명령 --help로 옵션을 확인합니다.
설정 변경 시 openclaw config / openclaw doctor로 유효성을 검사합니다.
Gateway·채널·플러그인 변경 후에는 필요 시 Gateway를 재시작합니다.

자주 쓰는 명령·설정 예시

pnpm openclaw infer tts convert --local --json \
  --text "OpenClaw live smoke." \
  --output /tmp/openclaw-live-smoke.mp3

pnpm openclaw voicecall setup --json
pnpm openclaw voicecall smoke --to "+15555550123"

openclaw models list
openclaw models list --json

OPENCLAW_LIVE_CLI_BACKEND=1 \
  OPENCLAW_LIVE_CLI_BACKEND_MODEL="claude-cli/claude-sonnet-4-6" \
  pnpm test:live src/gateway/gateway-cli-backend.live.test.ts

OPENCLAW_LIVE_TEST=1 \
  pnpm test:live src/agents/cli-runner/bundle-mcp.gemini.live.test.ts

pnpm test:docker:live-cli-backend

Testing: live suites

핵심 요약

문서 구성

상세 내용

본문

Live tests vs your real gateway

Live: local smoke commands

Live: Android node capability sweep

Live: model smoke (profile keys)

Layer 1: Direct model completion (no gateway)

Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)

Live: CLI backend smoke (Claude, Gemini, or other local CLIs)

Live: APNs HTTP/2 proxy reachability

Live: ACP bind smoke (/acp spawn ... --bind here)

Live: Codex app-server harness smoke

Live: OpenAI repeated compaction

Recommended live recipes

Live: model matrix (what we cover)

Aggregators / alternate gateways

Credentials (never commit)

Deepgram live (audio transcription)

BytePlus coding plan live

ComfyUI workflow media live

Image generation live

Music generation live

Video generation live

Media live harness

관련 문서

실습 체크리스트

자주 쓰는 명령·설정 예시

관련 링크

관련 가이드

Testing: live suites

핵심 요약

문서 구성

상세 내용

본문

Live tests vs your real gateway

Live: local smoke commands

Live: Android node capability sweep

Live: model smoke (profile keys)

Layer 1: Direct model completion (no gateway)

Layer 2: Gateway + dev agent smoke (what "@openclaw" actually does)

Live: CLI backend smoke (Claude, Gemini, or other local CLIs)

Live: APNs HTTP/2 proxy reachability

Live: ACP bind smoke (/acp spawn ... --bind here)

Live: Codex app-server harness smoke

Live: OpenAI repeated compaction

Recommended live recipes

Live: model matrix (what we cover)

Aggregators / alternate gateways

Credentials (never commit)

Deepgram live (audio transcription)

BytePlus coding plan live

ComfyUI workflow media live

Image generation live

Music generation live

Video generation live

Media live harness

관련 문서

실습 체크리스트

자주 쓰는 명령·설정 예시

관련 링크

관련 가이드