Music generation

기준일: 2026-07-26
난이도: 중급
공식 기준: Music generation

개요

이 페이지는 OpenClaw Music generation 도구(파라미터, 권한, 설정, CLI)를 공식 문서 기준으로 정리합니다.

공식 요약: Generate music via music_generate across ComfyUI, fal, Google Lyria, MiniMax, and OpenRouter workflows

도구 가시성은 profile / allow·deny policy / sandbox / channel 권한에 따라 달라집니다. 최신 스키마는 항상 공식 문서를 우선합니다.

공식 문서 기반 상세

아래는 공식 tools/music-generation 문서를 정리한 내용입니다. 코드 블록, 파라미터 이름, 기본값은 원문 그대로입니다.

The music_generate tool creates music or audio through the shared music-generation capability, backed by ComfyUI, fal, Google, MiniMax, and OpenRouter.

music_generate only appears when at least one music-generation provider is available: an explicit agents.defaults.mediaModels.music config, or an auth-configured provider (a set API key, for example).

For session-backed agent runs, music_generate starts as a background task, tracks progress in the task ledger, then wakes the agent when the track is ready so it can tell the user and attach the finished audio. The completion agent follows the session's visible-reply contract: automatic final reply when configured, or message(action="send") when the session requires the message tool. If the requester session is inactive or its wake fails and generated audio is still missing from the reply, OpenClaw sends an idempotent direct fallback with just the missing audio.

빠른 시작

Configure auth

    Set an API key for at least one provider — for example
    `GEMINI_API_KEY` or `MINIMAX_API_KEY`.

Pick a default model (optional)

    ```json5
    {
      agents: {
        defaults: {
          musicGenerationModel: {
            primary: "google/lyria-3-clip-preview",
          },
        },
      },
    }
    ```

Ask the agent

    _"Generate an upbeat synthpop track about a night drive through a
    neon city."_

    The agent calls `music_generate` automatically. No tool
    allow-listing needed.
  



Without a session-backed agent run (direct/local contexts), the tool
runs inline and returns the final media path in the same tool result.

Configure the workflow

    Configure `plugins.entries.comfy.config.music` with a workflow
    JSON and prompt/output nodes.

Cloud auth (optional)

    For Comfy Cloud, set `COMFY_API_KEY` or `COMFY_CLOUD_API_KEY`.

Call the tool

    ```text
    /tool music_generate prompt="Warm ambient synth loop with soft tape texture"
    ```

Example prompts:

Generate a cinematic piano track with soft strings and no vocals.

Generate an energetic chiptune loop about launching a rocket at sunrise.

Use action: "list" to inspect available providers/models, and action: "status" to inspect the active session-backed music task:

/tool music_generate action=list
/tool music_generate action=status

Direct generation example:

/tool music_generate prompt="Dreamy lo-fi hip hop with vinyl texture and gentle rain" instrumental=true

Supported providers

Provider	Default model	Reference inputs	Supported controls	Auth
ComfyUI	`workflow`	Up to 1 image	Workflow-defined music or audio	`COMFY_API_KEY`, `COMFY_CLOUD_API_KEY`
fal	`fal-ai/minimax-music/v2.6`	None	`lyrics`, `instrumental`, `durationSeconds`, `format`	`FAL_KEY` or `FAL_API_KEY`
Google	`lyria-3-clip-preview`	Up to 10 images	`lyrics`, `instrumental`, `format`	`GEMINI_API_KEY`, `GOOGLE_API_KEY`
MiniMax	`music-2.6`	None	`lyrics`, `instrumental`, `format` (mp3 only)	`MINIMAX_API_KEY` or MiniMax OAuth
OpenRouter	`google/lyria-3-pro-preview`	Up to 1 image	`lyrics`, `instrumental`, `durationSeconds`, `format`	`OPENROUTER_API_KEY`

MiniMax registers two provider ids sharing the same models: minimax for API-key auth and minimax-portal for OAuth. Model refs follow the auth path (minimax/music-2.6 vs minimax-portal/music-2.6); see MiniMax.

fal also exposes fal-ai/ace-step/prompt-to-audio (wav, no lyrics, no instrumental toggle) and fal-ai/stable-audio-25/text-to-audio (wav, prompt-only) alongside its default MiniMax-backed model. Google's default lyria-3-clip-preview outputs mp3 only; lyria-3-pro-preview also supports wav. MiniMax also exposes music-2.6-free, music-cover, and music-cover-free. OpenRouter also exposes google/lyria-3-clip-preview.

Capability matrix

The explicit mode contract used by music_generate, contract tests, and the shared live sweep:

Provider	`generate`	`edit`	Edit limit	Shared live lanes
ComfyUI	✓	✓	1 image	Not in the shared sweep; covered by `extensions/comfy/comfy.live.test.ts`
fal	✓	—	None	`generate`
Google	✓	✓	10 images	`generate`, `edit`
MiniMax	✓	—	None	`generate`
OpenRouter	✓	✓	1 image	`generate`, `edit`

Tool parameters

prompt (string) (required): Music generation prompt. Required for action: "generate".

"status" returns the current session task; "list" inspects providers.
model (string): Provider/model override (e.g. google/lyria-3-pro-preview, comfy/workflow).
lyrics (string): Optional lyrics when the provider supports explicit lyric input.
instrumental (boolean): Request instrumental-only output when the provider supports it.
image (string): Single reference image path or URL.
images (string[]): Multiple reference images (up to 10 on supporting providers).
durationSeconds (number): Target duration in seconds when the provider supports duration hints.

Output format hint when the provider supports it.
filename (string): Output filename hint.

Not all providers support all parameters. OpenClaw still validates hard limits such as input counts before submission. When a provider supports duration but uses a shorter maximum than the requested value, OpenClaw clamps to the closest supported duration. Truly unsupported optional hints are ignored with a warning when the selected provider or model cannot honor them. Tool results report applied settings; details.normalization captures any requested-to-applied mapping.

Provider request timeouts are operator configuration only. OpenClaw uses agents.defaults.mediaModels.music.timeoutMs when configured, raises values below 120000ms to 120000ms, and otherwise defaults provider requests to 300000ms.

Async behavior

Session-backed music generation runs as a background task:

Background task: music_generate creates a background task, returns a started/task response immediately, and posts the finished track later in a follow-up agent message.
Duplicate prevention: while a task is queued or running, later music_generate calls in the same session return task status instead of starting another generation. Use action: "status" to check explicitly. A recently completed matching request is also deduplicated for 2 minutes.
Status lookup: openclaw tasks list or openclaw tasks show <taskId> inspects queued, running, and terminal status.
Completion wake: OpenClaw injects an internal completion event back into the same session so the model can write the user-facing follow-up itself.
Prompt hint: later user/manual turns in the same session get a small runtime hint when a music task is already in flight, so the model does not blindly call music_generate again.
No-session fallback: direct/local contexts without a real agent session run inline and return the final audio result in the same turn.

Task lifecycle

The music task surfaces the same states as the general task registry (see Background tasks for the full state machine, including timed_out, cancelled, and lost). Most music runs move through:

State	Meaning
`queued`	Task created, waiting for the provider to accept it.
`running`	Provider is processing (typically 30 seconds to 3 minutes depending on provider and duration).
`succeeded`	Track ready; the agent wakes and posts it to the conversation.
`failed`	Provider error or timeout; the agent wakes with error details.

Check status from the CLI:

openclaw tasks list
openclaw tasks show <taskId>
openclaw tasks cancel <taskId>

설정

모델 선택

{
  agents: {
    defaults: {
      musicGenerationModel: {
        primary: "google/lyria-3-clip-preview",
        fallbacks: ["fal/fal-ai/minimax-music/v2.6", "minimax/music-2.6"],
      },
    },
  },
}

Provider selection order

OpenClaw tries providers in this order:

model parameter from the tool call (if the agent specifies one).
musicGenerationModel.primary from config.
musicGenerationModel.fallbacks in order.
Auto-detection using auth-backed provider defaults only:
- current default text-model provider first, if it also offers music generation;
- remaining registered music-generation providers, alphabetically by provider id.

If a provider fails, the next candidate is tried automatically. If all fail, the error includes details from each attempt.

Automatic fallback across authenticated providers is always enabled. A per-call model remains authoritative.

Provider notes

ComfyUI

Workflow-driven and depends on the configured graph plus node mapping
for prompt/output fields. The bundled `comfy` plugin plugs into the
shared `music_generate` tool through the music-generation provider
registry.

fal

Uses fal model endpoints through the shared provider auth path. The
bundled provider defaults to `fal-ai/minimax-music/v2.6` and also exposes
`fal-ai/ace-step/prompt-to-audio` and
`fal-ai/stable-audio-25/text-to-audio` for prompt-to-audio requests.
Lyrics and instrumental mode are MiniMax-model-only; the other two
models are prompt-only.

Google (Lyria 3)

Uses Lyria 3 batch generation. The current bundled flow supports
prompt, optional lyrics text, and optional reference images. The
default `lyria-3-clip-preview` model outputs mp3 only; the
`lyria-3-pro-preview` model also supports wav.

MiniMax

Uses the batch `music_generation` endpoint. Supports prompt, optional
lyrics, instrumental mode, and mp3 output through either `minimax`
API-key auth or `minimax-portal` OAuth. Also exposes `music-2.6-free`,
`music-cover`, and `music-cover-free` models.

OpenRouter

Uses OpenRouter chat completions audio output with streaming enabled. The
bundled provider defaults to `google/lyria-3-pro-preview` and also exposes
`openrouter/google/lyria-3-clip-preview`.

Choosing the right path

Shared provider-backed when you want model selection, provider failover, and the built-in async task/status flow.
Plugin path (ComfyUI) when you need a custom workflow graph or a provider that is not part of the shared bundled music capability.

If you are debugging ComfyUI-specific behavior, see ComfyUI. If you are debugging shared provider behavior, start with fal, Google (Gemini), MiniMax, or OpenRouter.

Provider capability modes

The shared music-generation contract supports explicit mode declarations:

generate for prompt-only generation.
edit when the request includes one or more reference images.

New provider implementations should prefer explicit mode blocks:

capabilities: {
  generate: {
    maxTracks: 1,
    supportsLyrics: true,
    supportsFormat: true,
  },
  edit: {
    enabled: true,
    maxTracks: 1,
    maxInputImages: 1,
    supportsFormat: true,
  },
}

Legacy flat fields such as maxInputImages, supportsLyrics, and supportsFormat are not enough to advertise edit support. Providers should declare generate and edit explicitly so live tests, contract tests, and the shared music_generate tool can validate mode support deterministically.

Live tests

Opt-in live coverage for the shared bundled providers (fal, Google, MiniMax, OpenRouter):

OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts

Equivalent repo wrapper, which drives the same test file:

pnpm test:live:media:music

This live file uses already-exported provider env vars ahead of stored auth profiles by default, and runs both generate and declared edit coverage when the provider enables edit mode. Coverage today:

google: generate plus edit
fal: generate only
minimax: generate only
openrouter: generate plus edit
comfy: separate Comfy live coverage, not the shared provider sweep

Opt-in live coverage for the bundled ComfyUI music path:

OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

The Comfy live file also covers comfy image and video workflows when those sections are configured.

검증 체크리스트

해당 tool이 활성 profile/policy에서 허용되는지 확인
sandbox / elevated / host 실행 경로 정책을 이해했는지 확인
채널·에이전트 권한과 충돌하지 않는지 확인
공식 CLI/--help와 문서 옵션이 버전과 맞는지 확인

Music generation

기준일: 2026-07-26
난이도: 중급
공식 기준: Music generation

개요

이 페이지는 OpenClaw Music generation 도구(파라미터, 권한, 설정, CLI)를 공식 문서 기준으로 정리합니다.

공식 요약: Generate music via music_generate across ComfyUI, fal, Google Lyria, MiniMax, and OpenRouter workflows

도구 가시성은 profile / allow·deny policy / sandbox / channel 권한에 따라 달라집니다. 최신 스키마는 항상 공식 문서를 우선합니다.

공식 문서 기반 상세

아래는 공식 tools/music-generation 문서를 정리한 내용입니다. 코드 블록, 파라미터 이름, 기본값은 원문 그대로입니다.

The music_generate tool creates music or audio through the shared music-generation capability, backed by ComfyUI, fal, Google, MiniMax, and OpenRouter.

music_generate only appears when at least one music-generation provider is available: an explicit agents.defaults.mediaModels.music config, or an auth-configured provider (a set API key, for example).

빠른 시작

Configure auth

    Set an API key for at least one provider — for example
    `GEMINI_API_KEY` or `MINIMAX_API_KEY`.

Pick a default model (optional)

    ```json5
    {
      agents: {
        defaults: {
          musicGenerationModel: {
            primary: "google/lyria-3-clip-preview",
          },
        },
      },
    }
    ```

Ask the agent

    _"Generate an upbeat synthpop track about a night drive through a
    neon city."_

    The agent calls `music_generate` automatically. No tool
    allow-listing needed.
  



Without a session-backed agent run (direct/local contexts), the tool
runs inline and returns the final media path in the same tool result.

Configure the workflow

    Configure `plugins.entries.comfy.config.music` with a workflow
    JSON and prompt/output nodes.

Cloud auth (optional)

    For Comfy Cloud, set `COMFY_API_KEY` or `COMFY_CLOUD_API_KEY`.

Call the tool

    ```text
    /tool music_generate prompt="Warm ambient synth loop with soft tape texture"
    ```

Example prompts:

Generate a cinematic piano track with soft strings and no vocals.

Generate an energetic chiptune loop about launching a rocket at sunrise.

Use action: "list" to inspect available providers/models, and action: "status" to inspect the active session-backed music task:

/tool music_generate action=list
/tool music_generate action=status

Direct generation example:

/tool music_generate prompt="Dreamy lo-fi hip hop with vinyl texture and gentle rain" instrumental=true

Supported providers

Provider	Default model	Reference inputs	Supported controls	Auth
ComfyUI	`workflow`	Up to 1 image	Workflow-defined music or audio	`COMFY_API_KEY`, `COMFY_CLOUD_API_KEY`
fal	`fal-ai/minimax-music/v2.6`	None	`lyrics`, `instrumental`, `durationSeconds`, `format`	`FAL_KEY` or `FAL_API_KEY`
Google	`lyria-3-clip-preview`	Up to 10 images	`lyrics`, `instrumental`, `format`	`GEMINI_API_KEY`, `GOOGLE_API_KEY`
MiniMax	`music-2.6`	None	`lyrics`, `instrumental`, `format` (mp3 only)	`MINIMAX_API_KEY` or MiniMax OAuth
OpenRouter	`google/lyria-3-pro-preview`	Up to 1 image	`lyrics`, `instrumental`, `durationSeconds`, `format`	`OPENROUTER_API_KEY`

Capability matrix

The explicit mode contract used by music_generate, contract tests, and the shared live sweep:

Provider	`generate`	`edit`	Edit limit	Shared live lanes
ComfyUI	✓	✓	1 image	Not in the shared sweep; covered by `extensions/comfy/comfy.live.test.ts`
fal	✓	—	None	`generate`
Google	✓	✓	10 images	`generate`, `edit`
MiniMax	✓	—	None	`generate`
OpenRouter	✓	✓	1 image	`generate`, `edit`

Tool parameters

prompt (string) (required): Music generation prompt. Required for action: "generate".

"status" returns the current session task; "list" inspects providers.
model (string): Provider/model override (e.g. google/lyria-3-pro-preview, comfy/workflow).
lyrics (string): Optional lyrics when the provider supports explicit lyric input.
instrumental (boolean): Request instrumental-only output when the provider supports it.
image (string): Single reference image path or URL.
images (string[]): Multiple reference images (up to 10 on supporting providers).
durationSeconds (number): Target duration in seconds when the provider supports duration hints.

Output format hint when the provider supports it.
filename (string): Output filename hint.

Not all providers support all parameters. OpenClaw still validates hard limits such as input counts before submission. When a provider supports duration but uses a shorter maximum than the requested value, OpenClaw clamps to the closest supported duration. Truly unsupported optional hints are ignored with a warning when the selected provider or model cannot honor them. Tool results report applied settings; details.normalization captures any requested-to-applied mapping.

Async behavior

Session-backed music generation runs as a background task:

Background task: music_generate creates a background task, returns a started/task response immediately, and posts the finished track later in a follow-up agent message.
Duplicate prevention: while a task is queued or running, later music_generate calls in the same session return task status instead of starting another generation. Use action: "status" to check explicitly. A recently completed matching request is also deduplicated for 2 minutes.
Status lookup: openclaw tasks list or openclaw tasks show <taskId> inspects queued, running, and terminal status.
Completion wake: OpenClaw injects an internal completion event back into the same session so the model can write the user-facing follow-up itself.
Prompt hint: later user/manual turns in the same session get a small runtime hint when a music task is already in flight, so the model does not blindly call music_generate again.
No-session fallback: direct/local contexts without a real agent session run inline and return the final audio result in the same turn.

Task lifecycle

The music task surfaces the same states as the general task registry (see Background tasks for the full state machine, including timed_out, cancelled, and lost). Most music runs move through:

State	Meaning
`queued`	Task created, waiting for the provider to accept it.
`running`	Provider is processing (typically 30 seconds to 3 minutes depending on provider and duration).
`succeeded`	Track ready; the agent wakes and posts it to the conversation.
`failed`	Provider error or timeout; the agent wakes with error details.

Check status from the CLI:

openclaw tasks list
openclaw tasks show <taskId>
openclaw tasks cancel <taskId>

설정

모델 선택

{
  agents: {
    defaults: {
      musicGenerationModel: {
        primary: "google/lyria-3-clip-preview",
        fallbacks: ["fal/fal-ai/minimax-music/v2.6", "minimax/music-2.6"],
      },
    },
  },
}

Provider selection order

OpenClaw tries providers in this order:

model parameter from the tool call (if the agent specifies one).
musicGenerationModel.primary from config.
musicGenerationModel.fallbacks in order.
Auto-detection using auth-backed provider defaults only:
- current default text-model provider first, if it also offers music generation;
- remaining registered music-generation providers, alphabetically by provider id.

If a provider fails, the next candidate is tried automatically. If all fail, the error includes details from each attempt.

Automatic fallback across authenticated providers is always enabled. A per-call model remains authoritative.

Provider notes

ComfyUI

Workflow-driven and depends on the configured graph plus node mapping
for prompt/output fields. The bundled `comfy` plugin plugs into the
shared `music_generate` tool through the music-generation provider
registry.

fal

Uses fal model endpoints through the shared provider auth path. The
bundled provider defaults to `fal-ai/minimax-music/v2.6` and also exposes
`fal-ai/ace-step/prompt-to-audio` and
`fal-ai/stable-audio-25/text-to-audio` for prompt-to-audio requests.
Lyrics and instrumental mode are MiniMax-model-only; the other two
models are prompt-only.

Google (Lyria 3)

Uses Lyria 3 batch generation. The current bundled flow supports
prompt, optional lyrics text, and optional reference images. The
default `lyria-3-clip-preview` model outputs mp3 only; the
`lyria-3-pro-preview` model also supports wav.

MiniMax

Uses the batch `music_generation` endpoint. Supports prompt, optional
lyrics, instrumental mode, and mp3 output through either `minimax`
API-key auth or `minimax-portal` OAuth. Also exposes `music-2.6-free`,
`music-cover`, and `music-cover-free` models.

OpenRouter

Uses OpenRouter chat completions audio output with streaming enabled. The
bundled provider defaults to `google/lyria-3-pro-preview` and also exposes
`openrouter/google/lyria-3-clip-preview`.

Choosing the right path

Shared provider-backed when you want model selection, provider failover, and the built-in async task/status flow.
Plugin path (ComfyUI) when you need a custom workflow graph or a provider that is not part of the shared bundled music capability.

If you are debugging ComfyUI-specific behavior, see ComfyUI. If you are debugging shared provider behavior, start with fal, Google (Gemini), MiniMax, or OpenRouter.

Provider capability modes

The shared music-generation contract supports explicit mode declarations:

generate for prompt-only generation.
edit when the request includes one or more reference images.

New provider implementations should prefer explicit mode blocks:

capabilities: {
  generate: {
    maxTracks: 1,
    supportsLyrics: true,
    supportsFormat: true,
  },
  edit: {
    enabled: true,
    maxTracks: 1,
    maxInputImages: 1,
    supportsFormat: true,
  },
}

Live tests

Opt-in live coverage for the shared bundled providers (fal, Google, MiniMax, OpenRouter):

OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts

Equivalent repo wrapper, which drives the same test file:

pnpm test:live:media:music

google: generate plus edit
fal: generate only
minimax: generate only
openrouter: generate plus edit
comfy: separate Comfy live coverage, not the shared provider sweep

Opt-in live coverage for the bundled ComfyUI music path:

OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

The Comfy live file also covers comfy image and video workflows when those sections are configured.

검증 체크리스트

해당 tool이 활성 profile/policy에서 허용되는지 확인
sandbox / elevated / host 실행 경로 정책을 이해했는지 확인
채널·에이전트 권한과 충돌하지 않는지 확인
공식 CLI/--help와 문서 옵션이 버전과 맞는지 확인

Music generation

개요

공식 문서 기반 상세

빠른 시작

Configure auth

Pick a default model (optional)

Ask the agent

Configure the workflow

Cloud auth (optional)

Call the tool

Supported providers

Capability matrix

Tool parameters

Async behavior

Task lifecycle

설정

모델 선택

Provider selection order

Provider notes

ComfyUI

fal

Google (Lyria 3)

MiniMax

OpenRouter

Choosing the right path

Provider capability modes

Live tests

관련 문서

검증 체크리스트

관련

관련 가이드

Music generation

개요

공식 문서 기반 상세

빠른 시작

Configure auth

Pick a default model (optional)

Ask the agent

Configure the workflow

Cloud auth (optional)

Call the tool

Supported providers

Capability matrix

Tool parameters

Async behavior

Task lifecycle

설정

모델 선택

Provider selection order

Provider notes

ComfyUI

fal

Google (Lyria 3)

MiniMax

OpenRouter

Choosing the right path

Provider capability modes

Live tests

관련 문서

검증 체크리스트

관련

관련 가이드