Compare commits

..
Author SHA1 Message Date
Peter Steinberger 54aa0e67b9 fix: honor telegram linkPreview config (#1700) (thanks @zerone0x) 2026-01-25 05:59:43 +00:00
zerone0xandClaude 565d9fd0c7 feat(telegram): add linkPreview config option
Add channels.telegram.linkPreview config to control whether link previews
are shown in outbound messages. When set to false, uses Telegram's
link_preview_options.is_disabled to suppress URL previews.

- Add linkPreview to TelegramAccountConfig type
- Add Zod schema validation for linkPreview
- Pass link_preview_options to sendMessage in send.ts and bot/delivery.ts
- Propagate linkPreview config through deliverReplies callers
- Add tests for link preview behavior

Fixes #1675

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-25 05:53:34 +00:00
Peter Steinberger db4a786fc3 feat: add notice directive parsing 2026-01-25 05:53:22 +00:00
14 changed files with 68 additions and 579 deletions

No files matched your search

-1
View File
@@ -9,7 +9,6 @@ Docs: https://docs.clawd.bot
- Venius (Venice AI): highlight provider guide + cross-links + expanded guidance. https://docs.clawd.bot/providers/venice
### Changes
- TTS: allow custom OpenAI-compatible TTS endpoints without API keys. (#1701) Thanks @louzhixian. https://docs.clawd.bot/tts
- TTS: add Edge TTS provider fallback, defaulting to keyless Edge with MP3 retry on format failures. (#1668) Thanks @steipete. https://docs.clawd.bot/tts
- Web search: add Brave freshness filter parameter for time-scoped results. (#1688) Thanks @JonUleis. https://docs.clawd.bot/tools/web
- TTS: add auto mode enum (off/always/inbound/tagged) with per-session `/tts` override. (#1667) Thanks @sebslight. https://docs.clawd.bot/tts
-2
View File
@@ -89,8 +89,6 @@ Clawdbot ships with the pi‑ai catalog. These providers require **no**
- Gemini CLI OAuth is shipped as a bundled plugin (`google-gemini-cli-auth`, disabled by default).
- Enable: `clawdbot plugins enable google-gemini-cli-auth`
- Login: `clawdbot models auth login --provider google-gemini-cli --set-default`
- Note: you do **not** paste a client id or secret into `clawdbot.json`. The CLI login flow stores
tokens in auth profiles on the gateway host.
### Z.AI (GLM)
-3
View File
@@ -1538,7 +1538,6 @@ voice notes; other channels send MP3 audio.
},
openai: {
apiKey: "openai_api_key",
baseUrl: "https://api.openai.com/v1",
model: "gpt-4o-mini-tts",
voice: "alloy"
}
@@ -1559,8 +1558,6 @@ Notes:
- `/tts limit` and `/tts summary` control per-user summarization settings.
- `apiKey` values fall back to `ELEVENLABS_API_KEY`/`XI_API_KEY` and `OPENAI_API_KEY`.
- `elevenlabs.baseUrl` overrides the ElevenLabs API base URL.
- `openai.baseUrl` overrides the OpenAI TTS base URL (defaults to `https://api.openai.com/v1`,
or `OPENAI_TTS_BASE_URL` when set). Custom endpoints do not require an API key.
- `elevenlabs.voiceSettings` supports `stability`/`similarityBoost`/`style` (0..1),
`useSpeakerBoost`, and `speed` (0.5..2.0).
+3 -36
View File
@@ -24,7 +24,6 @@ Quick answers plus deeper troubleshooting for real-world setups (local dev, VPS,
- [How do I try the latest bits?](#how-do-i-try-the-latest-bits)
- [How long does install and onboarding usually take?](#how-long-does-install-and-onboarding-usually-take)
- [Installer stuck? How do I get more feedback?](#installer-stuck-how-do-i-get-more-feedback)
- [Windows install says git not found or clawdbot not recognized](#windows-install-says-git-not-found-or-clawdbot-not-recognized)
- [The docs didn’t answer my question - how do I get a better answer?](#the-docs-didnt-answer-my-question-how-do-i-get-a-better-answer)
- [How do I install Clawdbot on Linux?](#how-do-i-install-clawdbot-on-linux)
- [How do I install Clawdbot on a VPS?](#how-do-i-install-clawdbot-on-a-vps)
@@ -40,7 +39,6 @@ Quick answers plus deeper troubleshooting for real-world setups (local dev, VPS,
- [Is AWS Bedrock supported?](#is-aws-bedrock-supported)
- [How does Codex auth work?](#how-does-codex-auth-work)
- [Do you support OpenAI subscription auth (Codex OAuth)?](#do-you-support-openai-subscription-auth-codex-oauth)
- [How do I set up Gemini CLI OAuth](#how-do-i-set-up-gemini-cli-oauth)
- [Is a local model OK for casual chats?](#is-a-local-model-ok-for-casual-chats)
- [How do I keep hosted model traffic in a specific region?](#how-do-i-keep-hosted-model-traffic-in-a-specific-region)
- [Do I have to buy a Mac Mini to install this?](#do-i-have-to-buy-a-mac-mini-to-install-this)
@@ -513,26 +511,6 @@ curl -fsSL https://clawd.bot/install.sh | bash -s -- --install-method git --verb
More options: [Installer flags](/install/installer).
### Windows install says git not found or clawdbot not recognized
Two common Windows issues:
**1) npm error spawn git / git not found**
- Install **Git for Windows** and make sure `git` is on your PATH.
- Close and reopen PowerShell, then re-run the installer.
**2) clawdbot is not recognized after install**
- Your npm global bin folder is not on PATH.
- Check the path:
```powershell
npm config get prefix
```
- Ensure `<prefix>\\bin` is on PATH (on most systems it is `%AppData%\\npm`).
- Close and reopen PowerShell after updating PATH.
If you want the smoothest Windows setup, use **WSL2** instead of native Windows.
Docs: [Windows](/platforms/windows).
### The docs didnt answer my question how do I get a better answer
Use the **hackable (git) install** so you have the full source and docs locally, then ask
@@ -632,10 +610,9 @@ Docs: [Anthropic](/providers/anthropic), [OpenAI](/providers/openai),
Yes. You can authenticate with **Claude Code CLI OAuth** or a **setup-token**
instead of an API key. This is the subscription path.
Claude Pro/Max subscriptions **do not include an API key**, so this is the
correct approach for subscription accounts. Important: you must verify with
Anthropic that this usage is allowed under their subscription policy and terms.
If you want the most explicit, supported path, use an Anthropic API key.
Important: you must verify with Anthropic that this usage is allowed under
their subscription policy and terms. If you want the most explicit, supported
path, use an Anthropic API key.
### How does Anthropic setuptoken auth work
@@ -687,16 +664,6 @@ can import the CLI login or run the OAuth flow for you.
See [OAuth](/concepts/oauth), [Model providers](/concepts/model-providers), and [Wizard](/start/wizard).
### How do I set up Gemini CLI OAuth
Gemini CLI uses a **plugin auth flow**, not a client id or secret in `clawdbot.json`.
Steps:
1) Enable the plugin: `clawdbot plugins enable google-gemini-cli-auth`
2) Login: `clawdbot models auth login --provider google-gemini-cli --set-default`
This stores OAuth tokens in auth profiles on the gateway host. Details: [Model providers](/concepts/model-providers).
### Is a local model OK for casual chats
Usually no. Clawdbot needs large context + strong safety; small cards truncate and leak. If you must, run the **largest** MiniMax M2.1 build you can locally (LM Studio) and see [/gateway/local-models](/gateway/local-models). Smaller/quantized models increase prompt-injection risk - see [Security](/gateway/security).
+1 -34
View File
@@ -33,18 +33,11 @@ does not publish limits, so assume similar or lower limits. citeturn0searc
If you want OpenAI or ElevenLabs:
- `ELEVENLABS_API_KEY` (or `XI_API_KEY`)
- `OPENAI_API_KEY` (optional for custom OpenAI-compatible endpoints)
Optional:
- `OPENAI_TTS_BASE_URL` (defaults to `https://api.openai.com/v1`)
- `OPENAI_API_KEY`
Edge TTS does **not** require an API key. If no API keys are found, Clawdbot defaults
to Edge TTS (unless disabled via `messages.tts.edge.enabled=false`).
When `OPENAI_TTS_BASE_URL` points to a non-OpenAI endpoint, Clawdbot relaxes
OpenAI model/voice validation and does **not** require an API key. If your
endpoint needs authentication, set `messages.tts.openai.apiKey` or `OPENAI_API_KEY`.
If multiple providers are configured, the selected provider is used first and the others are fallback options.
Auto-summary uses the configured `summaryModel` (or `agents.defaults.model.primary`),
so that provider must also be authenticated if you enable summaries.
@@ -122,32 +115,6 @@ Full schema is in [Gateway configuration](/gateway/configuration).
}
```
### Custom OpenAI-compatible endpoint
Set an endpoint in the environment:
```bash
export OPENAI_TTS_BASE_URL="http://localhost:8880/v1"
```
Then configure TTS as usual (API key optional for custom endpoints):
```json5
{
messages: {
tts: {
auto: "always",
provider: "openai",
openai: {
baseUrl: "http://localhost:8880/v1",
model: "kokoro",
voice: "zm_yunxia"
}
}
}
}
```
### Edge TTS primary (no API key)
```json5
+25 -73
View File
@@ -1,5 +1,5 @@
import crypto from "node:crypto";
import type { ChildProcessWithoutNullStreams } from "node:child_process";
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
import path from "node:path";
import type { AgentTool, AgentToolResult } from "@mariozechner/pi-agent-core";
import { Type } from "@sinclair/typebox";
@@ -27,7 +27,6 @@ import {
} from "../infra/shell-env.js";
import { enqueueSystemEvent } from "../infra/system-events.js";
import { logInfo, logWarn } from "../logger.js";
import { formatSpawnError, spawnWithFallback } from "../process/spawn-utils.js";
import {
type ProcessSession,
type SessionStdin,
@@ -363,38 +362,23 @@ async function runExecProcess(opts: {
let stdin: SessionStdin | undefined;
if (opts.sandbox) {
const { child: spawned } = await spawnWithFallback({
argv: [
"docker",
...buildDockerExecArgs({
containerName: opts.sandbox.containerName,
command: opts.command,
workdir: opts.containerWorkdir ?? opts.sandbox.containerWorkdir,
env: opts.env,
tty: opts.usePty,
}),
],
options: {
child = spawn(
"docker",
buildDockerExecArgs({
containerName: opts.sandbox.containerName,
command: opts.command,
workdir: opts.containerWorkdir ?? opts.sandbox.containerWorkdir,
env: opts.env,
tty: opts.usePty,
}),
{
cwd: opts.workdir,
env: process.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
},
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (err, fallback) => {
const errText = formatSpawnError(err);
const warning = `Warning: spawn failed (${errText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${errText}); retrying with ${fallback.label}.`);
opts.warnings.push(warning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
) as ChildProcessWithoutNullStreams;
stdin = child.stdin;
} else if (opts.usePty) {
const { shell, args: shellArgs } = getShellConfig();
@@ -438,56 +422,24 @@ async function runExecProcess(opts: {
const warning = `Warning: PTY spawn failed (${errText}); retrying without PTY for \`${opts.command}\`.`;
logWarn(`exec: PTY spawn failed (${errText}); retrying without PTY for "${opts.command}".`);
opts.warnings.push(warning);
const { child: spawned } = await spawnWithFallback({
argv: [shell, ...shellArgs, opts.command],
options: {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
},
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (fallbackErr, fallback) => {
const fallbackText = formatSpawnError(fallbackErr);
const fallbackWarning = `Warning: spawn failed (${fallbackText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${fallbackText}); retrying with ${fallback.label}.`);
opts.warnings.push(fallbackWarning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
} else {
const { shell, args: shellArgs } = getShellConfig();
const { child: spawned } = await spawnWithFallback({
argv: [shell, ...shellArgs, opts.command],
options: {
child = spawn(shell, [...shellArgs, opts.command], {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
},
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (err, fallback) => {
const errText = formatSpawnError(err);
const warning = `Warning: spawn failed (${errText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${errText}); retrying with ${fallback.label}.`);
opts.warnings.push(warning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
}) as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
} else {
const { shell, args: shellArgs } = getShellConfig();
child = spawn(shell, [...shellArgs, opts.command], {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
}) as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
-1
View File
@@ -56,7 +56,6 @@ export type TtsConfig = {
/** OpenAI configuration. */
openai?: {
apiKey?: string;
baseUrl?: string;
model?: string;
voice?: string;
};
-1
View File
@@ -204,7 +204,6 @@ export const TtsConfigSchema = z
openai: z
.object({
apiKey: z.string().optional(),
baseUrl: z.string().optional(),
model: z.string().optional(),
voice: z.string().optional(),
})
+1 -1
View File
@@ -131,7 +131,7 @@ export const ttsHandlers: GatewayRequestHandlers = {
{
id: "openai",
name: "OpenAI",
configured: isTtsProviderConfigured(config, "openai"),
configured: Boolean(resolveTtsApiKey(config, "openai")),
models: [...OPENAI_TTS_MODELS],
voices: [...OPENAI_TTS_VOICES],
},
+7 -11
View File
@@ -4,7 +4,6 @@ import { promisify } from "node:util";
import { danger, shouldLogVerbose } from "../globals.js";
import { logDebug, logError } from "../logger.js";
import { resolveCommandStdio } from "./spawn-utils.js";
const execFileAsync = promisify(execFile);
@@ -79,22 +78,19 @@ export async function runCommandWithTimeout(
if (resolvedEnv.npm_config_fund == null) resolvedEnv.npm_config_fund = "false";
}
const stdio = resolveCommandStdio({ hasInput, preferInherit: true });
const child = spawn(argv[0], argv.slice(1), {
stdio,
cwd,
env: resolvedEnv,
windowsVerbatimArguments,
});
// Spawn with inherited stdin (TTY) so tools like `pi` stay interactive when needed.
return await new Promise((resolve, reject) => {
const child = spawn(argv[0], argv.slice(1), {
stdio: [hasInput ? "pipe" : "inherit", "pipe", "pipe"],
cwd,
env: resolvedEnv,
windowsVerbatimArguments,
});
let stdout = "";
let stderr = "";
let settled = false;
const timer = setTimeout(() => {
if (typeof child.kill === "function") {
child.kill("SIGKILL");
}
child.kill("SIGKILL");
}, timeoutMs);
if (hasInput && child.stdin) {
-64
View File
@@ -1,64 +0,0 @@
import { EventEmitter } from "node:events";
import { PassThrough } from "node:stream";
import type { ChildProcess } from "node:child_process";
import { describe, expect, it, vi } from "vitest";
import { spawnWithFallback } from "./spawn-utils.js";
function createStubChild() {
const child = new EventEmitter() as ChildProcess;
child.stdin = new PassThrough() as ChildProcess["stdin"];
child.stdout = new PassThrough() as ChildProcess["stdout"];
child.stderr = new PassThrough() as ChildProcess["stderr"];
child.pid = 1234;
child.killed = false;
child.kill = vi.fn(() => true) as ChildProcess["kill"];
queueMicrotask(() => {
child.emit("spawn");
});
return child;
}
describe("spawnWithFallback", () => {
it("retries on EBADF using fallback options", async () => {
const spawnMock = vi
.fn()
.mockImplementationOnce(() => {
const err = new Error("spawn EBADF");
(err as NodeJS.ErrnoException).code = "EBADF";
throw err;
})
.mockImplementationOnce(() => createStubChild());
const result = await spawnWithFallback({
argv: ["echo", "ok"],
options: { stdio: ["pipe", "pipe", "pipe"] },
fallbacks: [{ label: "safe-stdin", options: { stdio: ["ignore", "pipe", "pipe"] } }],
spawnImpl: spawnMock,
});
expect(result.usedFallback).toBe(true);
expect(result.fallbackLabel).toBe("safe-stdin");
expect(spawnMock).toHaveBeenCalledTimes(2);
expect(spawnMock.mock.calls[0]?.[2]?.stdio).toEqual(["pipe", "pipe", "pipe"]);
expect(spawnMock.mock.calls[1]?.[2]?.stdio).toEqual(["ignore", "pipe", "pipe"]);
});
it("does not retry on non-EBADF errors", async () => {
const spawnMock = vi.fn().mockImplementationOnce(() => {
const err = new Error("spawn ENOENT");
(err as NodeJS.ErrnoException).code = "ENOENT";
throw err;
});
await expect(
spawnWithFallback({
argv: ["missing"],
options: { stdio: ["pipe", "pipe", "pipe"] },
fallbacks: [{ label: "safe-stdin", options: { stdio: ["ignore", "pipe", "pipe"] } }],
spawnImpl: spawnMock,
}),
).rejects.toThrow(/ENOENT/);
expect(spawnMock).toHaveBeenCalledTimes(1);
});
});
-127
View File
@@ -1,127 +0,0 @@
import type { ChildProcess, SpawnOptions } from "node:child_process";
import { spawn } from "node:child_process";
export type SpawnFallback = {
label: string;
options: SpawnOptions;
};
export type SpawnWithFallbackResult = {
child: ChildProcess;
usedFallback: boolean;
fallbackLabel?: string;
};
type SpawnWithFallbackParams = {
argv: string[];
options: SpawnOptions;
fallbacks?: SpawnFallback[];
spawnImpl?: typeof spawn;
retryCodes?: string[];
onFallback?: (err: unknown, fallback: SpawnFallback) => void;
};
const DEFAULT_RETRY_CODES = ["EBADF"];
export function resolveCommandStdio(params: {
hasInput: boolean;
preferInherit: boolean;
}): ["pipe" | "inherit" | "ignore", "pipe", "pipe"] {
const stdin = params.hasInput ? "pipe" : params.preferInherit ? "inherit" : "pipe";
return [stdin, "pipe", "pipe"];
}
export function formatSpawnError(err: unknown): string {
if (!(err instanceof Error)) return String(err);
const details = err as NodeJS.ErrnoException;
const parts: string[] = [];
const message = err.message?.trim();
if (message) parts.push(message);
if (details.code && !message?.includes(details.code)) parts.push(details.code);
if (details.syscall) parts.push(`syscall=${details.syscall}`);
if (typeof details.errno === "number") parts.push(`errno=${details.errno}`);
return parts.join(" ");
}
function shouldRetry(err: unknown, codes: string[]): boolean {
const code =
err && typeof err === "object" && "code" in err ? String((err as { code?: unknown }).code) : "";
return code.length > 0 && codes.includes(code);
}
async function spawnAndWaitForSpawn(
spawnImpl: typeof spawn,
argv: string[],
options: SpawnOptions,
): Promise<ChildProcess> {
const child = spawnImpl(argv[0], argv.slice(1), options);
return await new Promise((resolve, reject) => {
let settled = false;
const cleanup = () => {
child.removeListener("error", onError);
child.removeListener("spawn", onSpawn);
};
const finishResolve = () => {
if (settled) return;
settled = true;
cleanup();
resolve(child);
};
const onError = (err: unknown) => {
if (settled) return;
settled = true;
cleanup();
reject(err);
};
const onSpawn = () => {
finishResolve();
};
child.once("error", onError);
child.once("spawn", onSpawn);
// Ensure mocked spawns that never emit "spawn" don't stall.
process.nextTick(() => {
if (typeof child.pid === "number") {
finishResolve();
}
});
});
}
export async function spawnWithFallback(
params: SpawnWithFallbackParams,
): Promise<SpawnWithFallbackResult> {
const spawnImpl = params.spawnImpl ?? spawn;
const retryCodes = params.retryCodes ?? DEFAULT_RETRY_CODES;
const baseOptions = { ...params.options };
const fallbacks = params.fallbacks ?? [];
const attempts: Array<{ label?: string; options: SpawnOptions }> = [
{ options: baseOptions },
...fallbacks.map((fallback) => ({
label: fallback.label,
options: { ...baseOptions, ...fallback.options },
})),
];
let lastError: unknown;
for (let index = 0; index < attempts.length; index += 1) {
const attempt = attempts[index];
try {
const child = await spawnAndWaitForSpawn(spawnImpl, params.argv, attempt.options);
return {
child,
usedFallback: index > 0,
fallbackLabel: attempt.label,
};
} catch (err) {
lastError = err;
const nextFallback = fallbacks[index];
if (!nextFallback || !shouldRetry(err, retryCodes)) {
throw err;
}
params.onFallback?.(err, nextFallback);
}
}
throw lastError;
}
+6 -114
View File
@@ -106,36 +106,26 @@ describe("tts", () => {
expect(isValidOpenAIVoice("alloy ")).toBe(false);
expect(isValidOpenAIVoice(" alloy")).toBe(false);
});
it("accepts custom voices for custom endpoints", () => {
expect(isValidOpenAIVoice("zm_yunxia", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
});
});
describe("isValidOpenAIModel", () => {
it("accepts supported models", () => {
it("accepts gpt-4o-mini-tts model", () => {
expect(isValidOpenAIModel("gpt-4o-mini-tts")).toBe(true);
expect(isValidOpenAIModel("tts-1")).toBe(true);
expect(isValidOpenAIModel("tts-1-hd")).toBe(true);
});
it("rejects other models", () => {
expect(isValidOpenAIModel("tts-1")).toBe(false);
expect(isValidOpenAIModel("tts-1-hd")).toBe(false);
expect(isValidOpenAIModel("invalid")).toBe(false);
expect(isValidOpenAIModel("")).toBe(false);
expect(isValidOpenAIModel("gpt-4")).toBe(false);
});
it("accepts custom models for custom endpoints", () => {
expect(isValidOpenAIModel("kokoro", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
});
});
describe("OPENAI_TTS_MODELS", () => {
it("contains supported models", () => {
it("contains only gpt-4o-mini-tts", () => {
expect(OPENAI_TTS_MODELS).toContain("gpt-4o-mini-tts");
expect(OPENAI_TTS_MODELS).toContain("tts-1");
expect(OPENAI_TTS_MODELS).toContain("tts-1-hd");
expect(OPENAI_TTS_MODELS).toHaveLength(3);
expect(OPENAI_TTS_MODELS).toHaveLength(1);
});
it("is a non-empty array", () => {
@@ -210,30 +200,6 @@ describe("tts", () => {
expect(result.overrides.provider).toBe("edge");
});
it("routes custom OpenAI models when custom endpoints are used", () => {
const policy = resolveModelOverridePolicy({ enabled: true });
const input = "Hello [[tts:provider=openai model=kokoro voice=zm_yunxia]] world";
const result = parseTtsDirectives(input, policy, {
defaultProvider: "openai",
openaiBaseUrl: "http://localhost:8880/v1",
});
expect(result.overrides.openai?.model).toBe("kokoro");
expect(result.overrides.openai?.voice).toBe("zm_yunxia");
});
it("routes model overrides to ElevenLabs when provider is ElevenLabs", () => {
const policy = resolveModelOverridePolicy({ enabled: true });
const input = "Hello [[tts:provider=elevenlabs model=eleven_multilingual_v2]] world";
const result = parseTtsDirectives(input, policy, {
defaultProvider: "openai",
openaiBaseUrl: "http://localhost:8880/v1",
});
expect(result.overrides.elevenlabs?.modelId).toBe("eleven_multilingual_v2");
expect(result.overrides.openai?.model).toBeUndefined();
});
it("keeps text intact when overrides are disabled", () => {
const policy = resolveModelOverridePolicy({ enabled: false });
const input = "Hello [[tts:voice=alloy]] world";
@@ -391,12 +357,7 @@ describe("tts", () => {
};
const restoreEnv = (snapshot: Record<string, string | undefined>) => {
const keys = [
"OPENAI_API_KEY",
"OPENAI_TTS_BASE_URL",
"ELEVENLABS_API_KEY",
"XI_API_KEY",
] as const;
const keys = ["OPENAI_API_KEY", "ELEVENLABS_API_KEY", "XI_API_KEY"] as const;
for (const key of keys) {
const value = snapshot[key];
if (value === undefined) {
@@ -410,7 +371,6 @@ describe("tts", () => {
const withEnv = (env: Record<string, string | undefined>, run: () => void) => {
const snapshot = {
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
OPENAI_TTS_BASE_URL: process.env.OPENAI_TTS_BASE_URL,
ELEVENLABS_API_KEY: process.env.ELEVENLABS_API_KEY,
XI_API_KEY: process.env.XI_API_KEY,
};
@@ -432,7 +392,6 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: "test-openai-key",
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
@@ -448,7 +407,6 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: "test-elevenlabs-key",
XI_API_KEY: undefined,
},
@@ -464,7 +422,6 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
@@ -475,22 +432,6 @@ describe("tts", () => {
},
);
});
it("prefers OpenAI when a custom endpoint is configured without API keys", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: "http://localhost:8880/v1",
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
() => {
const config = resolveTtsConfig(baseCfg);
const provider = getTtsProvider(config, "/tmp/tts-prefs-openai-custom.json");
expect(provider).toBe("openai");
},
);
});
});
describe("maybeApplyTtsToPayload", () => {
@@ -554,55 +495,6 @@ describe("tts", () => {
process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
});
it("allows custom OpenAI endpoints without API keys", async () => {
const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
const prevBaseUrl = process.env.OPENAI_TTS_BASE_URL;
const prevOpenAiKey = process.env.OPENAI_API_KEY;
process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
process.env.OPENAI_TTS_BASE_URL = "http://localhost:8880/v1";
delete process.env.OPENAI_API_KEY;
const originalFetch = globalThis.fetch;
const fetchMock = vi.fn(async () => ({
ok: true,
arrayBuffer: async () => new ArrayBuffer(1),
}));
globalThis.fetch = fetchMock as unknown as typeof fetch;
const cfg = {
agents: { defaults: { model: { primary: "openai/gpt-4o-mini" } } },
messages: {
tts: {
auto: "always",
provider: "openai",
openai: { model: "kokoro", voice: "zm_yunxia" },
},
},
};
const result = await maybeApplyTtsToPayload({
payload: { text: "Hello world" },
cfg,
kind: "final",
});
expect(result.mediaUrl).toBeDefined();
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, options] = fetchMock.mock.calls[0] ?? [];
expect(url).toBe("http://localhost:8880/v1/audio/speech");
expect(
(options as { headers?: Record<string, string> })?.headers?.Authorization,
).toBeUndefined();
globalThis.fetch = originalFetch;
process.env.OPENAI_TTS_BASE_URL = prevBaseUrl;
if (prevOpenAiKey === undefined) {
delete process.env.OPENAI_API_KEY;
} else {
process.env.OPENAI_API_KEY = prevOpenAiKey;
}
process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
});
it("skips auto-TTS in tagged mode unless a tts tag is present", async () => {
const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
+25 -111
View File
@@ -48,7 +48,6 @@ const DEFAULT_ELEVENLABS_VOICE_ID = "pMsXgVXv3BLzUgSXRplE";
const DEFAULT_ELEVENLABS_MODEL_ID = "eleven_multilingual_v2";
const DEFAULT_OPENAI_MODEL = "gpt-4o-mini-tts";
const DEFAULT_OPENAI_VOICE = "alloy";
const DEFAULT_OPENAI_TTS_BASE_URL = "https://api.openai.com/v1";
const DEFAULT_EDGE_VOICE = "en-US-MichelleNeural";
const DEFAULT_EDGE_LANG = "en-US";
const DEFAULT_EDGE_OUTPUT_FORMAT = "audio-24khz-48kbitrate-mono-mp3";
@@ -104,7 +103,6 @@ export type ResolvedTtsConfig = {
};
openai: {
apiKey?: string;
baseUrl: string;
model: string;
voice: string;
};
@@ -236,9 +234,6 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
const raw: TtsConfig = cfg.messages?.tts ?? {};
const providerSource = raw.provider ? "config" : "default";
const edgeOutputFormat = raw.edge?.outputFormat?.trim();
const openaiBaseUrl = normalizeOpenAiTtsBaseUrl(
raw.openai?.baseUrl?.trim() || process.env.OPENAI_TTS_BASE_URL,
);
const auto = normalizeTtsAutoMode(raw.auto) ?? (raw.enabled ? "always" : "off");
return {
auto,
@@ -270,7 +265,6 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
},
openai: {
apiKey: raw.openai?.apiKey,
baseUrl: openaiBaseUrl,
model: raw.openai?.model ?? DEFAULT_OPENAI_MODEL,
voice: raw.openai?.voice ?? DEFAULT_OPENAI_VOICE,
},
@@ -401,9 +395,7 @@ export function getTtsProvider(config: ResolvedTtsConfig, prefsPath: string): Tt
if (prefs.tts?.provider) return prefs.tts.provider;
if (config.providerSource === "config") return config.provider;
if (resolveTtsApiKey(config, "openai") || isCustomOpenAiTtsEndpoint(config.openai.baseUrl)) {
return "openai";
}
if (resolveTtsApiKey(config, "openai")) return "openai";
if (resolveTtsApiKey(config, "elevenlabs")) return "elevenlabs";
return "edge";
}
@@ -478,12 +470,6 @@ export function resolveTtsProviderOrder(primary: TtsProvider): TtsProvider[] {
export function isTtsProviderConfigured(config: ResolvedTtsConfig, provider: TtsProvider): boolean {
if (provider === "edge") return config.edge.enabled;
if (provider === "openai") {
return (
Boolean(resolveTtsApiKey(config, provider)) ||
isCustomOpenAiTtsEndpoint(config.openai.baseUrl)
);
}
return Boolean(resolveTtsApiKey(config, provider));
}
@@ -497,16 +483,6 @@ function normalizeElevenLabsBaseUrl(baseUrl: string): string {
return trimmed.replace(/\/+$/, "");
}
function normalizeOpenAiTtsBaseUrl(baseUrl?: string): string {
const trimmed = baseUrl?.trim();
if (!trimmed) return DEFAULT_OPENAI_TTS_BASE_URL;
return trimmed.replace(/\/+$/, "");
}
function isCustomOpenAiTtsEndpoint(baseUrl: string): boolean {
return normalizeOpenAiTtsBaseUrl(baseUrl) !== DEFAULT_OPENAI_TTS_BASE_URL;
}
function requireInRange(value: number, min: number, max: number, label: string): void {
if (!Number.isFinite(value) || value < min || value > max) {
throw new Error(`${label} must be between ${min} and ${max}`);
@@ -562,7 +538,6 @@ function parseNumberValue(value: string): number | undefined {
function parseTtsDirectives(
text: string,
policy: ResolvedTtsModelOverrides,
options: { defaultProvider?: TtsProvider; openaiBaseUrl?: string } = {},
): TtsDirectiveParseResult {
if (!policy.enabled) {
return { cleanedText: text, overrides: {}, warnings: [], hasDirective: false };
@@ -586,30 +561,6 @@ function parseTtsDirectives(
cleanedText = cleanedText.replace(directiveRegex, (_match, body: string) => {
hasDirective = true;
const tokens = body.split(/\s+/).filter(Boolean);
let providerOverride: TtsProvider | undefined;
for (const token of tokens) {
const eqIndex = token.indexOf("=");
if (eqIndex === -1) continue;
const rawKey = token.slice(0, eqIndex).trim();
const rawValue = token.slice(eqIndex + 1).trim();
if (!rawKey || !rawValue) continue;
if (rawKey.toLowerCase() !== "provider") continue;
if (!policy.allowProvider) break;
if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
providerOverride = rawValue;
} else {
warnings.push(`unsupported provider "${rawValue}"`);
}
break;
}
if (providerOverride) {
overrides.provider = providerOverride;
}
const providerHint = overrides.provider ?? options.defaultProvider;
const openAiContext = { baseUrl: options.openaiBaseUrl };
for (const token of tokens) {
const eqIndex = token.indexOf("=");
if (eqIndex === -1) continue;
@@ -620,12 +571,18 @@ function parseTtsDirectives(
try {
switch (key) {
case "provider":
if (!policy.allowProvider) break;
if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
overrides.provider = rawValue;
} else {
warnings.push(`unsupported provider "${rawValue}"`);
}
break;
case "voice":
case "openai_voice":
case "openaivoice":
if (!policy.allowVoice) break;
if (isValidOpenAIVoice(rawValue, openAiContext)) {
if (isValidOpenAIVoice(rawValue)) {
overrides.openai = { ...overrides.openai, voice: rawValue };
} else {
warnings.push(`invalid OpenAI voice "${rawValue}"`);
@@ -645,40 +602,17 @@ function parseTtsDirectives(
case "model":
case "modelid":
case "model_id":
case "elevenlabs_model":
case "elevenlabsmodel":
case "openai_model":
case "openaimodel":
if (!policy.allowModelId) break;
if (key === "openai_model" || key === "openaimodel") {
if (isValidOpenAIModel(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
warnings.push(`invalid OpenAI model "${rawValue}"`);
}
break;
}
if (providerHint === "openai") {
if (isValidOpenAIModel(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
warnings.push(`invalid OpenAI model "${rawValue}"`);
}
break;
}
if (providerHint === "elevenlabs") {
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
break;
}
if (isValidOpenAIModel(rawValue, openAiContext)) {
if (isValidOpenAIModel(rawValue)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
}
break;
case "elevenlabs_model":
case "elevenlabsmodel":
if (!policy.allowModelId) break;
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
break;
case "stability":
if (!policy.allowVoiceSettings) break;
{
@@ -802,7 +736,7 @@ function parseTtsDirectives(
};
}
export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts", "tts-1", "tts-1-hd"] as const;
export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts"] as const;
export const OPENAI_TTS_VOICES = [
"alloy",
"ash",
@@ -816,15 +750,12 @@ export const OPENAI_TTS_VOICES = [
] as const;
type OpenAiTtsVoice = (typeof OPENAI_TTS_VOICES)[number];
type OpenAiTtsValidationContext = { baseUrl?: string };
function isValidOpenAIModel(model: string, context: OpenAiTtsValidationContext = {}): boolean {
if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
function isValidOpenAIModel(model: string): boolean {
return OPENAI_TTS_MODELS.includes(model as (typeof OPENAI_TTS_MODELS)[number]);
}
function isValidOpenAIVoice(voice: string, context: OpenAiTtsValidationContext = {}): boolean {
if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
function isValidOpenAIVoice(voice: string): voice is OpenAiTtsVoice {
return OPENAI_TTS_VOICES.includes(voice as OpenAiTtsVoice);
}
@@ -1032,21 +963,18 @@ async function elevenLabsTTS(params: {
async function openaiTTS(params: {
text: string;
apiKey?: string;
baseUrl: string;
apiKey: string;
model: string;
voice: string;
responseFormat: "mp3" | "opus";
timeoutMs: number;
}): Promise<Buffer> {
const { text, model, voice, responseFormat, timeoutMs } = params;
const apiKey = params.apiKey?.trim();
const baseUrl = normalizeOpenAiTtsBaseUrl(params.baseUrl);
const { text, apiKey, model, voice, responseFormat, timeoutMs } = params;
if (!isValidOpenAIModel(model, { baseUrl })) {
if (!isValidOpenAIModel(model)) {
throw new Error(`Invalid model: ${model}`);
}
if (!isValidOpenAIVoice(voice, { baseUrl })) {
if (!isValidOpenAIVoice(voice)) {
throw new Error(`Invalid voice: ${voice}`);
}
@@ -1054,15 +982,12 @@ async function openaiTTS(params: {
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const headers: Record<string, string> = {
"Content-Type": "application/json",
};
if (apiKey) {
headers.Authorization = `Bearer ${apiKey}`;
}
const response = await fetch(`${baseUrl}/audio/speech`, {
const response = await fetch("https://api.openai.com/v1/audio/speech", {
method: "POST",
headers,
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model,
input: text,
@@ -1212,19 +1137,13 @@ export async function textToSpeech(params: {
}
const apiKey = resolveTtsApiKey(config, provider);
const allowMissingKey =
provider === "openai" && isCustomOpenAiTtsEndpoint(config.openai.baseUrl);
if (!apiKey && !allowMissingKey) {
if (!apiKey) {
lastError = `No API key for ${provider}`;
continue;
}
let audioBuffer: Buffer;
if (provider === "elevenlabs") {
if (!apiKey) {
lastError = "No API key for elevenlabs";
continue;
}
const voiceIdOverride = params.overrides?.elevenlabs?.voiceId;
const modelIdOverride = params.overrides?.elevenlabs?.modelId;
const voiceSettings = {
@@ -1253,7 +1172,6 @@ export async function textToSpeech(params: {
audioBuffer = await openaiTTS({
text: params.text,
apiKey,
baseUrl: config.openai.baseUrl,
model: openaiModelOverride ?? config.openai.model,
voice: openaiVoiceOverride ?? config.openai.voice,
responseFormat: output.openai,
@@ -1309,12 +1227,8 @@ export async function maybeApplyTtsToPayload(params: {
});
if (autoMode === "off") return params.payload;
const defaultProvider = getTtsProvider(config, prefsPath);
const text = params.payload.text ?? "";
const directives = parseTtsDirectives(text, config.modelOverrides, {
defaultProvider,
openaiBaseUrl: config.openai.baseUrl,
});
const directives = parseTtsDirectives(text, config.modelOverrides);
if (directives.warnings.length > 0) {
logVerbose(`TTS: ignored directive overrides (${directives.warnings.join("; ")})`);
}