fix: support custom TTS endpoints (#1701 ) (thanks @louzhixian)

test: update tests for expanded OpenAI TTS model list
- Accept tts-1 and tts-1-hd as valid models - Update OPENAI_TTS_MODELS length expectation to 3
2026-01-25 08:01:11 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00
31 changed files with 340 additions and 847 deletions
@@ -9,6 +9,7 @@ Docs: https://docs.clawd.bot
 - Venius (Venice AI): highlight provider guide + cross-links + expanded guidance. https://docs.clawd.bot/providers/venice
 ### Changes
 - TTS: allow custom OpenAI-compatible TTS endpoints without API keys. (#1701) Thanks @louzhixian. https://docs.clawd.bot/tts
 - TTS: add Edge TTS provider fallback, defaulting to keyless Edge with MP3 retry on format failures. (#1668) Thanks @steipete. https://docs.clawd.bot/tts
 - Web search: add Brave freshness filter parameter for time-scoped results. (#1688) Thanks @JonUleis. https://docs.clawd.bot/tools/web
 - TTS: add auto mode enum (off/always/inbound/tagged) with per-session `/tts` override. (#1667) Thanks @sebslight. https://docs.clawd.bot/tts
@@ -28,7 +29,6 @@ Docs: https://docs.clawd.bot
 - Web UI: hide internal `message_id` hints in chat bubbles.
 - Web UI: show Stop button during active runs, swap back to New session when idle. (#1664) Thanks @ndbroadbent.
 - Web UI: clear stale disconnect banners on reconnect; allow form saves with unsupported schema paths but block missing schema. (#1707) Thanks @Glucksberg.
 - Auto-reply: don't treat `/models` as a `/model` directive. (#1753) Thanks @uos-status.
 - Heartbeat: normalize target identifiers for consistent routing.
 - TUI: reload history after gateway reconnect to restore session state. (#1663)
 - Telegram: use wrapped fetch for long-polling on Node to normalize AbortSignal handling. (#1639)
@@ -43,7 +43,6 @@ Docs: https://docs.clawd.bot
 - Gateway: reduce log noise for late invokes + remote node probes; debounce skills refresh. (#1607) Thanks @petter-b.
 - Gateway: clarify Control UI/WebChat auth error hints for missing tokens. (#1690)
 - Gateway: listen on IPv6 loopback when bound to 127.0.0.1 so localhost webhooks work.
 - Gateway: store lock files in the temp directory to avoid stale locks on persistent volumes. (#1676)
 - macOS: default direct-transport `ws://` URLs to port 18789; document `gateway.remote.transport`. (#1603) Thanks @ngutman.
 - Voice Call: return stream TwiML for outbound conversation calls on initial Twilio webhook. (#1634)
 - Google Chat: tighten email allowlist matching, typing cleanup, media caps, and onboarding/docs/tests. (#1635) Thanks @iHildy.
@@ -1538,6 +1538,7 @@ voice notes; other channels send MP3 audio.
      },
      openai: {
        apiKey: "openai_api_key",
        baseUrl: "https://api.openai.com/v1",
        model: "gpt-4o-mini-tts",
        voice: "alloy"
      }
@@ -1558,6 +1559,8 @@ Notes:
 - `/tts limit` and `/tts summary` control per-user summarization settings.
 - `apiKey` values fall back to `ELEVENLABS_API_KEY`/`XI_API_KEY` and `OPENAI_API_KEY`.
 - `elevenlabs.baseUrl` overrides the ElevenLabs API base URL.
 - `openai.baseUrl` overrides the OpenAI TTS base URL (defaults to `https://api.openai.com/v1`,
  or `OPENAI_TTS_BASE_URL` when set). Custom endpoints do not require an API key.
 - `elevenlabs.voiceSettings` supports `stability`/`similarityBoost`/`style` (0..1),
  `useSpeakerBoost`, and `speed` (0.5..2.0).
@@ -67,22 +67,6 @@ Plugins can register:
 Plugins run **in‑process** with the Gateway, so treat them as trusted code.
 Tool authoring guide: [Plugin agent tools](/plugins/agent-tools).
 ## Runtime helpers
 Plugins can access selected core helpers via `api.runtime`. For telephony TTS:
 ```ts
 const result = await api.runtime.tts.textToSpeechTelephony({
  text: "Hello from Clawdbot",
  cfg: api.config,
 });
 ```
 Notes:
 - Uses core `messages.tts` configuration (OpenAI or ElevenLabs).
 - Returns PCM audio buffer + sample rate. Plugins must resample/encode for providers.
 - Edge TTS is not supported for telephony.
 ## Discovery & precedence
 Clawdbot scans, in order:
@@ -104,87 +104,6 @@ Notes:
 - `mock` is a local dev provider (no network calls).
 - `skipSignatureVerification` is for local testing only.
 ## TTS for calls
 Voice Call uses the core `messages.tts` configuration (OpenAI or ElevenLabs) for
 streaming speech on calls. You can override it under the plugin config with the
 **same shape** — it deep‑merges with `messages.tts`.
 ```json5
 {
  tts: {
    provider: "elevenlabs",
    elevenlabs: {
      voiceId: "pMsXgVXv3BLzUgSXRplE",
      modelId: "eleven_multilingual_v2"
    }
  }
 }
 ```
 Notes:
 - **Edge TTS is ignored for voice calls** (telephony audio needs PCM; Edge output is unreliable).
 - Core TTS is used when Twilio media streaming is enabled; otherwise calls fall back to provider native voices.
 ### More examples
 Use core TTS only (no override):
 ```json5
 {
  messages: {
    tts: {
      provider: "openai",
      openai: { voice: "alloy" }
    }
  }
 }
 ```
 Override to ElevenLabs just for calls (keep core default elsewhere):
 ```json5
 {
  plugins: {
    entries: {
      "voice-call": {
        config: {
          tts: {
            provider: "elevenlabs",
            elevenlabs: {
              apiKey: "elevenlabs_key",
              voiceId: "pMsXgVXv3BLzUgSXRplE",
              modelId: "eleven_multilingual_v2"
            }
          }
        }
      }
    }
  }
 }
 ```
 Override only the OpenAI model for calls (deep‑merge example):
 ```json5
 {
  plugins: {
    entries: {
      "voice-call": {
        config: {
          tts: {
            openai: {
              model: "gpt-4o-mini-tts",
              voice: "marin"
            }
          }
        }
      }
    }
  }
 }
 ```
 ## Inbound calls
 Inbound policy defaults to `disabled`. To enable inbound calls, set:
@@ -16,7 +16,7 @@ and you configure everything via the `/setup` web wizard.
 ## One-click deploy
-<a href="https://railway.app/new/template?template=https://github.com/vignesh07/clawdbot-railway-template" target="_blank" rel="noreferrer">Deploy on Railway</a>
+<a href="https://railway.com/deploy/clawdbot-railway-template" target="_blank" rel="noreferrer">Deploy on Railway</a>
 After deploy, find your public URL in **Railway → your service → Settings → Domains**.
@@ -55,7 +55,6 @@ Attach a volume mounted at:
 Set these variables on the service:
 - `SETUP_PASSWORD` (required)
 - `PORT=8080` (required — must match the port in Public Networking)
 - `CLAWDBOT_STATE_DIR=/data/.clawdbot` (recommended)
 - `CLAWDBOT_WORKSPACE_DIR=/data/workspace` (recommended)
 - `CLAWDBOT_GATEWAY_TOKEN` (recommended; treat as an admin secret)
@@ -83,9 +82,8 @@ If Telegram DMs are set to pairing, the setup wizard can approve the pairing cod
 1) Go to https://discord.com/developers/applications
 2) **New Application** → choose a name
 3) **Bot** → **Add Bot**
-4) **Enable MESSAGE CONTENT INTENT** under Bot → Privileged Gateway Intents (required or the bot will crash on startup)
+4) Copy the **Bot Token** and paste into `/setup`
-5) Copy the **Bot Token** and paste into `/setup`
+5) Invite the bot to your server (OAuth2 URL Generator; scopes: `bot`, `applications.commands`)
 6) Invite the bot to your server (OAuth2 URL Generator; scopes: `bot`, `applications.commands`)
 ## Backups & migration
@@ -33,11 +33,18 @@ does not publish limits, so assume similar or lower limits. citeturn0searc
 If you want OpenAI or ElevenLabs:
 - `ELEVENLABS_API_KEY` (or `XI_API_KEY`)
- `OPENAI_API_KEY`
+- `OPENAI_API_KEY` (optional for custom OpenAI-compatible endpoints)
 Optional:
 - `OPENAI_TTS_BASE_URL` (defaults to `https://api.openai.com/v1`)
 Edge TTS does **not** require an API key. If no API keys are found, Clawdbot defaults
 to Edge TTS (unless disabled via `messages.tts.edge.enabled=false`).
 When `OPENAI_TTS_BASE_URL` points to a non-OpenAI endpoint, Clawdbot relaxes
 OpenAI model/voice validation and does **not** require an API key. If your
 endpoint needs authentication, set `messages.tts.openai.apiKey` or `OPENAI_API_KEY`.
 If multiple providers are configured, the selected provider is used first and the others are fallback options.
 Auto-summary uses the configured `summaryModel` (or `agents.defaults.model.primary`),
 so that provider must also be authenticated if you enable summaries.
@@ -115,6 +122,32 @@ Full schema is in [Gateway configuration](/gateway/configuration).
 }
 ```
 ### Custom OpenAI-compatible endpoint
 Set an endpoint in the environment:
 ```bash
 export OPENAI_TTS_BASE_URL="http://localhost:8880/v1"
 ```
 Then configure TTS as usual (API key optional for custom endpoints):
 ```json5
 {
  messages: {
    tts: {
      auto: "always",
      provider: "openai",
      openai: {
        baseUrl: "http://localhost:8880/v1",
        model: "kokoro",
        voice: "zm_yunxia"
      }
    }
  }
 }
 ```
 ### Edge TTS primary (no API key)
 ```json5
@@ -1,12 +1,5 @@
 # Changelog
 ## 2026.1.24
 ### Changes
 - Breaking: voice-call TTS now uses core `messages.tts` (plugin TTS config deep‑merges with core).
 - Telephony TTS supports OpenAI + ElevenLabs; Edge TTS is ignored for calls.
 - Removed legacy `tts.model`/`tts.voice`/`tts.instructions` plugin fields.
 ## 2026.1.23
 ### Changes
@@ -75,27 +75,6 @@ Notes:
 - Twilio/Telnyx/Plivo require a **publicly reachable** webhook URL.
 - `mock` is a local dev provider (no network calls).
 ## TTS for calls
 Voice Call uses the core `messages.tts` configuration (OpenAI or ElevenLabs) for
 streaming speech on calls. You can override it under the plugin config with the
 same shape — overrides deep-merge with `messages.tts`.
 ```json5
 {
  tts: {
    provider: "openai",
    openai: {
      voice: "alloy"
    }
  }
 }
 ```
 Notes:
 - Edge TTS is ignored for voice calls (telephony audio needs PCM; Edge output is unreliable).
 - Core TTS is used when Twilio media streaming is enabled; otherwise calls fall back to provider native voices.
 ## CLI
 ```bash
@@ -99,39 +99,16 @@
      "label": "Media Stream Path",
      "advanced": true
    },
-    "tts.provider": {
+    "tts.model": {
-      "label": "TTS Provider Override",
+      "label": "TTS Model",
      "help": "Deep-merges with messages.tts (Edge is ignored for calls).",
      "advanced": true
    },
-    "tts.openai.model": {
+    "tts.voice": {
-      "label": "OpenAI TTS Model",
+      "label": "TTS Voice",
      "advanced": true
    },
-    "tts.openai.voice": {
+    "tts.instructions": {
-      "label": "OpenAI TTS Voice",
+      "label": "TTS Instructions",
      "advanced": true
    },
    "tts.openai.apiKey": {
      "label": "OpenAI API Key",
      "sensitive": true,
      "advanced": true
    },
    "tts.elevenlabs.modelId": {
      "label": "ElevenLabs Model ID",
      "advanced": true
    },
    "tts.elevenlabs.voiceId": {
      "label": "ElevenLabs Voice ID",
      "advanced": true
    },
    "tts.elevenlabs.apiKey": {
      "label": "ElevenLabs API Key",
      "sensitive": true,
      "advanced": true
    },
    "tts.elevenlabs.baseUrl": {
      "label": "ElevenLabs Base URL",
      "advanced": true
    },
    "publicUrl": {
@@ -393,193 +370,20 @@
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "auto": {
            "type": "string",
            "enum": [
              "off",
              "always",
              "inbound",
              "tagged"
            ]
          },
          "enabled": {
            "type": "boolean"
          },
          "mode": {
            "type": "string",
            "enum": [
              "final",
              "all"
            ]
          },
          "provider": {
            "type": "string",
            "enum": [
-              "openai",
+              "openai"
              "elevenlabs",
              "edge"
            ]
          },
-          "summaryModel": {
+          "model": {
            "type": "string"
          },
-          "modelOverrides": {
+          "voice": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "enabled": {
                "type": "boolean"
              },
              "allowText": {
                "type": "boolean"
              },
              "allowProvider": {
                "type": "boolean"
              },
              "allowVoice": {
                "type": "boolean"
              },
              "allowModelId": {
                "type": "boolean"
              },
              "allowVoiceSettings": {
                "type": "boolean"
              },
              "allowNormalization": {
                "type": "boolean"
              },
              "allowSeed": {
                "type": "boolean"
              }
            }
          },
          "elevenlabs": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "apiKey": {
                "type": "string"
              },
              "baseUrl": {
                "type": "string"
              },
              "voiceId": {
                "type": "string"
              },
              "modelId": {
                "type": "string"
              },
              "seed": {
                "type": "integer",
                "minimum": 0,
                "maximum": 4294967295
              },
              "applyTextNormalization": {
                "type": "string",
                "enum": [
                  "auto",
                  "on",
                  "off"
                ]
              },
              "languageCode": {
                "type": "string"
              },
              "voiceSettings": {
                "type": "object",
                "additionalProperties": false,
                "properties": {
                  "stability": {
                    "type": "number",
                    "minimum": 0,
                    "maximum": 1
                  },
                  "similarityBoost": {
                    "type": "number",
                    "minimum": 0,
                    "maximum": 1
                  },
                  "style": {
                    "type": "number",
                    "minimum": 0,
                    "maximum": 1
                  },
                  "useSpeakerBoost": {
                    "type": "boolean"
                  },
                  "speed": {
                    "type": "number",
                    "minimum": 0.5,
                    "maximum": 2
                  }
                }
              }
            }
          },
          "openai": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "apiKey": {
                "type": "string"
              },
              "model": {
                "type": "string"
              },
              "voice": {
                "type": "string"
              }
            }
          },
          "edge": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
              "enabled": {
                "type": "boolean"
              },
              "voice": {
                "type": "string"
              },
              "lang": {
                "type": "string"
              },
              "outputFormat": {
                "type": "string"
              },
              "pitch": {
                "type": "string"
              },
              "rate": {
                "type": "string"
              },
              "volume": {
                "type": "string"
              },
              "saveSubtitles": {
                "type": "boolean"
              },
              "proxy": {
                "type": "string"
              },
              "timeoutMs": {
                "type": "integer",
                "minimum": 1000,
                "maximum": 120000
              }
            }
          },
          "prefsPath": {
            "type": "string"
          },
-          "maxTextLength": {
+          "instructions": {
-            "type": "integer",
+            "type": "string"
            "minimum": 1
          },
          "timeoutMs": {
            "type": "integer",
            "minimum": 1000,
            "maximum": 120000
          }
        }
      },
@@ -74,26 +74,9 @@ const voiceCallConfigSchema = {
    },
    "streaming.sttModel": { label: "Realtime STT Model", advanced: true },
    "streaming.streamPath": { label: "Media Stream Path", advanced: true },
-    "tts.provider": {
+    "tts.model": { label: "TTS Model", advanced: true },
-      label: "TTS Provider Override",
+    "tts.voice": { label: "TTS Voice", advanced: true },
-      help: "Deep-merges with messages.tts (Edge is ignored for calls).",
+    "tts.instructions": { label: "TTS Instructions", advanced: true },
      advanced: true,
    },
    "tts.openai.model": { label: "OpenAI TTS Model", advanced: true },
    "tts.openai.voice": { label: "OpenAI TTS Voice", advanced: true },
    "tts.openai.apiKey": {
      label: "OpenAI API Key",
      sensitive: true,
      advanced: true,
    },
    "tts.elevenlabs.modelId": { label: "ElevenLabs Model ID", advanced: true },
    "tts.elevenlabs.voiceId": { label: "ElevenLabs Voice ID", advanced: true },
    "tts.elevenlabs.apiKey": {
      label: "ElevenLabs API Key",
      sensitive: true,
      advanced: true,
    },
    "tts.elevenlabs.baseUrl": { label: "ElevenLabs Base URL", advanced: true },
    publicUrl: { label: "Public Webhook URL", advanced: true },
    skipSignatureVerification: {
      label: "Skip Signature Verification",
@@ -178,7 +161,6 @@ const voiceCallPlugin = {
        runtimePromise = createVoiceCallRuntime({
          config: cfg,
          coreConfig: api.config as CoreConfig,
          ttsRuntime: api.runtime.tts,
          logger: api.logger,
        });
      }
@@ -82,82 +82,31 @@ export const SttConfigSchema = z
  .default({ provider: "openai", model: "whisper-1" });
 export type SttConfig = z.infer<typeof SttConfigSchema>;
 export const TtsProviderSchema = z.enum(["openai", "elevenlabs", "edge"]);
 export const TtsModeSchema = z.enum(["final", "all"]);
 export const TtsAutoSchema = z.enum(["off", "always", "inbound", "tagged"]);
 export const TtsConfigSchema = z
  .object({
-    auto: TtsAutoSchema.optional(),
+    /** TTS provider (currently only OpenAI supported) */
-    enabled: z.boolean().optional(),
+    provider: z.literal("openai").default("openai"),
-    mode: TtsModeSchema.optional(),
+    /**
-    provider: TtsProviderSchema.optional(),
+     * TTS model to use:
-    summaryModel: z.string().optional(),
+     * - gpt-4o-mini-tts: newest, supports instructions for tone/style control (recommended)
-    modelOverrides: z
+     * - tts-1: lower latency
-      .object({
+     * - tts-1-hd: higher quality
-        enabled: z.boolean().optional(),
+     */
-        allowText: z.boolean().optional(),
+    model: z.string().min(1).default("gpt-4o-mini-tts"),
-        allowProvider: z.boolean().optional(),
+    /**
-        allowVoice: z.boolean().optional(),
+     * Voice ID. For best quality, use marin or cedar.
-        allowModelId: z.boolean().optional(),
+     * All voices: alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer, verse, marin, cedar
-        allowVoiceSettings: z.boolean().optional(),
+     */
-        allowNormalization: z.boolean().optional(),
+    voice: z.string().min(1).default("coral"),
-        allowSeed: z.boolean().optional(),
+    /**
-      })
+     * Instructions for speech style (only works with gpt-4o-mini-tts).
-      .strict()
+     * Examples: "Speak in a cheerful tone", "Talk like a sympathetic customer service agent"
-      .optional(),
+     */
-    elevenlabs: z
+    instructions: z.string().optional(),
      .object({
        apiKey: z.string().optional(),
        baseUrl: z.string().optional(),
        voiceId: z.string().optional(),
        modelId: z.string().optional(),
        seed: z.number().int().min(0).max(4294967295).optional(),
        applyTextNormalization: z.enum(["auto", "on", "off"]).optional(),
        languageCode: z.string().optional(),
        voiceSettings: z
          .object({
            stability: z.number().min(0).max(1).optional(),
            similarityBoost: z.number().min(0).max(1).optional(),
            style: z.number().min(0).max(1).optional(),
            useSpeakerBoost: z.boolean().optional(),
            speed: z.number().min(0.5).max(2).optional(),
          })
          .strict()
          .optional(),
      })
      .strict()
      .optional(),
    openai: z
      .object({
        apiKey: z.string().optional(),
        model: z.string().optional(),
        voice: z.string().optional(),
      })
      .strict()
      .optional(),
    edge: z
      .object({
        enabled: z.boolean().optional(),
        voice: z.string().optional(),
        lang: z.string().optional(),
        outputFormat: z.string().optional(),
        pitch: z.string().optional(),
        rate: z.string().optional(),
        volume: z.string().optional(),
        saveSubtitles: z.boolean().optional(),
        proxy: z.string().optional(),
        timeoutMs: z.number().int().min(1000).max(120000).optional(),
      })
      .strict()
      .optional(),
    prefsPath: z.string().optional(),
    maxTextLength: z.number().int().min(1).optional(),
    timeoutMs: z.number().int().min(1000).max(120000).optional(),
  })
  .strict()
-  .optional();
+  .default({ provider: "openai", model: "gpt-4o-mini-tts", voice: "coral" });
-export type VoiceCallTtsConfig = z.infer<typeof TtsConfigSchema>;
+export type TtsConfig = z.infer<typeof TtsConfigSchema>;
 // -----------------------------------------------------------------------------
 // Webhook Server Configuration
@@ -358,7 +307,7 @@ export const VoiceCallConfigSchema = z
  /** STT configuration */
  stt: SttConfigSchema,
-  /** TTS override (deep-merges with core messages.tts) */
+  /** TTS configuration */
  tts: TtsConfigSchema,
  /** Store path for call logs */
@@ -2,16 +2,10 @@ import fs from "node:fs";
 import path from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 import type { VoiceCallTtsConfig } from "./config.js";
 export type CoreConfig = {
  session?: {
    store?: string;
  };
  messages?: {
    tts?: VoiceCallTtsConfig;
  };
  [key: string]: unknown;
 };
 type CoreAgentDeps = {
@@ -143,7 +143,7 @@ export class CallManager {
      // For notify mode with a message, use inline TwiML with <Say>
      let inlineTwiml: string | undefined;
      if (mode === "notify" && initialMessage) {
-        const pollyVoice = mapVoiceToPolly(this.config.tts?.openai?.voice);
+        const pollyVoice = mapVoiceToPolly(this.config.tts.voice);
        inlineTwiml = this.generateNotifyTwiml(initialMessage, pollyVoice);
        console.log(
          `[voice-call] Using inline TwiML for notify mode (voice: ${pollyVoice})`,
@@ -210,13 +210,11 @@ export class CallManager {
      this.addTranscriptEntry(call, "bot", text);
      // Play TTS
      const voice =
        this.provider?.name === "twilio" ? this.config.tts?.openai?.voice : undefined;
      await this.provider.playTts({
        callId,
        providerCallId: call.providerCallId,
        text,
-        voice,
+        voice: this.config.tts.voice,
      });
      return { success: true };
@@ -68,7 +68,7 @@ export async function initiateCall(
    // For notify mode with a message, use inline TwiML with <Say>.
    let inlineTwiml: string | undefined;
    if (mode === "notify" && initialMessage) {
-      const pollyVoice = mapVoiceToPolly(ctx.config.tts?.openai?.voice);
+      const pollyVoice = mapVoiceToPolly(ctx.config.tts.voice);
      inlineTwiml = generateNotifyTwiml(initialMessage, pollyVoice);
      console.log(`[voice-call] Using inline TwiML for notify mode (voice: ${pollyVoice})`);
    }
@@ -120,13 +120,11 @@ export async function speak(
    addTranscriptEntry(call, "bot", text);
    const voice =
      ctx.provider?.name === "twilio" ? ctx.config.tts?.openai?.voice : undefined;
    await ctx.provider.playTts({
      callId,
      providerCallId: call.providerCallId,
      text,
-      voice,
+      voice: ctx.config.tts.voice,
    });
    return { success: true };
@@ -246,3 +244,4 @@ export async function endCall(
    return { success: false, error: err instanceof Error ? err.message : String(err) };
  }
 }
@@ -15,9 +15,9 @@ import type {
  WebhookVerificationResult,
 } from "../types.js";
 import { escapeXml, mapVoiceToPolly } from "../voice-mapping.js";
 import { chunkAudio } from "../telephony-audio.js";
 import type { TelephonyTtsProvider } from "../telephony-tts.js";
 import type { VoiceCallProvider } from "./base.js";
 import type { OpenAITTSProvider } from "./tts-openai.js";
 import { chunkAudio } from "./tts-openai.js";
 import { twilioApiRequest } from "./twilio/api.js";
 import { verifyTwilioProviderWebhook } from "./twilio/webhook.js";
@@ -53,8 +53,8 @@ export class TwilioProvider implements VoiceCallProvider {
  /** Current public webhook URL (set when tunnel starts or from config) */
  private currentPublicUrl: string | null = null;
-  /** Optional telephony TTS provider for streaming TTS */
+  /** Optional OpenAI TTS provider for streaming TTS */
-  private ttsProvider: TelephonyTtsProvider | null = null;
+  private ttsProvider: OpenAITTSProvider | null = null;
  /** Optional media stream handler for sending audio */
  private mediaStreamHandler: MediaStreamHandler | null = null;
@@ -119,7 +119,7 @@ export class TwilioProvider implements VoiceCallProvider {
    return this.currentPublicUrl;
  }
-  setTTSProvider(provider: TelephonyTtsProvider): void {
+  setTTSProvider(provider: OpenAITTSProvider): void {
    this.ttsProvider = provider;
  }
@@ -454,13 +454,13 @@ export class TwilioProvider implements VoiceCallProvider {
   * Play TTS audio via Twilio.
   *
   * Two modes:
-   * 1. Core TTS + Media Streams: If TTS provider and media stream are available,
+   * 1. OpenAI TTS + Media Streams: If TTS provider and media stream are available,
-   *    generates audio via core TTS and streams it through WebSocket (preferred).
+   *    generates audio via OpenAI and streams it through WebSocket (preferred).
   * 2. TwiML <Say>: Falls back to Twilio's native TTS with Polly voices.
   *    Note: This may not work on all Twilio accounts.
   */
  async playTts(input: PlayTtsInput): Promise<void> {
-    // Try telephony TTS via media stream first (if configured)
+    // Try OpenAI TTS via media stream first (if configured)
    const streamSid = this.callStreamMap.get(input.providerCallId);
    if (this.ttsProvider && this.mediaStreamHandler && streamSid) {
      try {
@@ -468,7 +468,7 @@ export class TwilioProvider implements VoiceCallProvider {
        return;
      } catch (err) {
        console.warn(
-          `[voice-call] Telephony TTS failed, falling back to Twilio <Say>:`,
+          `[voice-call] OpenAI TTS failed, falling back to Twilio <Say>:`,
          err instanceof Error ? err.message : err,
        );
        // Fall through to TwiML <Say> fallback
@@ -484,7 +484,7 @@ export class TwilioProvider implements VoiceCallProvider {
    }
    console.warn(
-      "[voice-call] Using TwiML <Say> fallback - telephony TTS not configured or media stream not active",
+      "[voice-call] Using TwiML <Say> fallback - OpenAI TTS not configured or media stream not active",
    );
    const pollyVoice = mapVoiceToPolly(input.voice);
@@ -502,8 +502,8 @@ export class TwilioProvider implements VoiceCallProvider {
  }
  /**
-   * Play TTS via core TTS and Twilio Media Streams.
+   * Play TTS via OpenAI and Twilio Media Streams.
-   * Generates audio with core TTS, converts to mu-law, and streams via WebSocket.
+   * Generates audio with OpenAI TTS, converts to mu-law, and streams via WebSocket.
   * Uses a jitter buffer to smooth out timing variations.
   */
  private async playTtsViaStream(
@@ -514,8 +514,8 @@ export class TwilioProvider implements VoiceCallProvider {
      throw new Error("TTS provider and media stream handler required");
    }
-    // Generate audio with core TTS (returns mu-law at 8kHz)
+    // Generate audio with OpenAI TTS (returns mu-law at 8kHz)
-    const muLawAudio = await this.ttsProvider.synthesizeForTelephony(text);
+    const muLawAudio = await this.ttsProvider.synthesizeForTwilio(text);
    // Stream audio in 20ms chunks (160 bytes at 8kHz mu-law)
    const CHUNK_SIZE = 160;
@@ -6,9 +6,8 @@ import type { VoiceCallProvider } from "./providers/base.js";
 import { MockProvider } from "./providers/mock.js";
 import { PlivoProvider } from "./providers/plivo.js";
 import { TelnyxProvider } from "./providers/telnyx.js";
 import { OpenAITTSProvider } from "./providers/tts-openai.js";
 import { TwilioProvider } from "./providers/twilio.js";
 import type { TelephonyTtsRuntime } from "./telephony-tts.js";
 import { createTelephonyTtsProvider } from "./telephony-tts.js";
 import { startTunnel, type TunnelResult } from "./tunnel.js";
 import {
  cleanupTailscaleExposure,
@@ -82,10 +81,9 @@ function resolveProvider(config: VoiceCallConfig): VoiceCallProvider {
 export async function createVoiceCallRuntime(params: {
  config: VoiceCallConfig;
  coreConfig: CoreConfig;
  ttsRuntime?: TelephonyTtsRuntime;
  logger?: Logger;
 }): Promise<VoiceCallRuntime> {
-  const { config, coreConfig, ttsRuntime, logger } = params;
+  const { config, coreConfig, logger } = params;
  const log = logger ?? {
    info: console.log,
    warn: console.warn,
@@ -151,24 +149,27 @@ export async function createVoiceCallRuntime(params: {
  if (provider.name === "twilio" && config.streaming?.enabled) {
    const twilioProvider = provider as TwilioProvider;
-    if (ttsRuntime?.textToSpeechTelephony) {
+    const openaiApiKey =
      config.streaming.openaiApiKey || process.env.OPENAI_API_KEY;
    if (openaiApiKey) {
      try {
-        const ttsProvider = createTelephonyTtsProvider({
+        const ttsProvider = new OpenAITTSProvider({
-          coreConfig,
+          apiKey: openaiApiKey,
-          ttsOverride: config.tts,
+          voice: config.tts.voice,
-          runtime: ttsRuntime,
+          model: config.tts.model,
          instructions: config.tts.instructions,
        });
        twilioProvider.setTTSProvider(ttsProvider);
-        log.info("[voice-call] Telephony TTS provider configured");
+        log.info("[voice-call] OpenAI TTS provider configured");
      } catch (err) {
        log.warn(
-          `[voice-call] Failed to initialize telephony TTS: ${
+          `[voice-call] Failed to initialize OpenAI TTS: ${
            err instanceof Error ? err.message : String(err)
          }`,
        );
      }
    } else {
-      log.warn("[voice-call] Telephony TTS unavailable; streaming TTS disabled");
+      log.warn("[voice-call] OpenAI TTS key missing; streaming TTS disabled");
    }
    const mediaHandler = webhookServer.getMediaStreamHandler();
@@ -1,88 +0,0 @@
 const TELEPHONY_SAMPLE_RATE = 8000;
 function clamp16(value: number): number {
  return Math.max(-32768, Math.min(32767, value));
 }
 /**
 * Resample 16-bit PCM (little-endian mono) to 8kHz using linear interpolation.
 */
 export function resamplePcmTo8k(input: Buffer, inputSampleRate: number): Buffer {
  if (inputSampleRate === TELEPHONY_SAMPLE_RATE) return input;
  const inputSamples = Math.floor(input.length / 2);
  if (inputSamples === 0) return Buffer.alloc(0);
  const ratio = inputSampleRate / TELEPHONY_SAMPLE_RATE;
  const outputSamples = Math.floor(inputSamples / ratio);
  const output = Buffer.alloc(outputSamples * 2);
  for (let i = 0; i < outputSamples; i++) {
    const srcPos = i * ratio;
    const srcIndex = Math.floor(srcPos);
    const frac = srcPos - srcIndex;
    const s0 = input.readInt16LE(srcIndex * 2);
    const s1Index = Math.min(srcIndex + 1, inputSamples - 1);
    const s1 = input.readInt16LE(s1Index * 2);
    const sample = Math.round(s0 + frac * (s1 - s0));
    output.writeInt16LE(clamp16(sample), i * 2);
  }
  return output;
 }
 /**
 * Convert 16-bit PCM to 8-bit mu-law (G.711).
 */
 export function pcmToMulaw(pcm: Buffer): Buffer {
  const samples = Math.floor(pcm.length / 2);
  const mulaw = Buffer.alloc(samples);
  for (let i = 0; i < samples; i++) {
    const sample = pcm.readInt16LE(i * 2);
    mulaw[i] = linearToMulaw(sample);
  }
  return mulaw;
 }
 export function convertPcmToMulaw8k(
  pcm: Buffer,
  inputSampleRate: number,
 ): Buffer {
  const pcm8k = resamplePcmTo8k(pcm, inputSampleRate);
  return pcmToMulaw(pcm8k);
 }
 /**
 * Chunk audio buffer into 20ms frames for streaming (8kHz mono mu-law).
 */
 export function chunkAudio(
  audio: Buffer,
  chunkSize = 160,
 ): Generator<Buffer, void, unknown> {
  return (function* () {
    for (let i = 0; i < audio.length; i += chunkSize) {
      yield audio.subarray(i, Math.min(i + chunkSize, audio.length));
    }
  })();
 }
 function linearToMulaw(sample: number): number {
  const BIAS = 132;
  const CLIP = 32635;
  const sign = sample < 0 ? 0x80 : 0;
  if (sample < 0) sample = -sample;
  if (sample > CLIP) sample = CLIP;
  sample += BIAS;
  let exponent = 7;
  for (let expMask = 0x4000; (sample & expMask) === 0 && exponent > 0; exponent--) {
    expMask >>= 1;
  }
  const mantissa = (sample >> (exponent + 3)) & 0x0f;
  return ~(sign | (exponent << 4) | mantissa) & 0xff;
 }
@@ -1,95 +0,0 @@
 import type { CoreConfig } from "./core-bridge.js";
 import type { VoiceCallTtsConfig } from "./config.js";
 import { convertPcmToMulaw8k } from "./telephony-audio.js";
 export type TelephonyTtsRuntime = {
  textToSpeechTelephony: (params: {
    text: string;
    cfg: CoreConfig;
    prefsPath?: string;
  }) => Promise<{
    success: boolean;
    audioBuffer?: Buffer;
    sampleRate?: number;
    provider?: string;
    error?: string;
  }>;
 };
 export type TelephonyTtsProvider = {
  synthesizeForTelephony: (text: string) => Promise<Buffer>;
 };
 export function createTelephonyTtsProvider(params: {
  coreConfig: CoreConfig;
  ttsOverride?: VoiceCallTtsConfig;
  runtime: TelephonyTtsRuntime;
 }): TelephonyTtsProvider {
  const { coreConfig, ttsOverride, runtime } = params;
  const mergedConfig = applyTtsOverride(coreConfig, ttsOverride);
  return {
    synthesizeForTelephony: async (text: string) => {
      const result = await runtime.textToSpeechTelephony({
        text,
        cfg: mergedConfig,
      });
      if (!result.success || !result.audioBuffer || !result.sampleRate) {
        throw new Error(result.error ?? "TTS conversion failed");
      }
      return convertPcmToMulaw8k(result.audioBuffer, result.sampleRate);
    },
  };
 }
 function applyTtsOverride(
  coreConfig: CoreConfig,
  override?: VoiceCallTtsConfig,
 ): CoreConfig {
  if (!override) return coreConfig;
  const base = coreConfig.messages?.tts;
  const merged = mergeTtsConfig(base, override);
  if (!merged) return coreConfig;
  return {
    ...coreConfig,
    messages: {
      ...(coreConfig.messages ?? {}),
      tts: merged,
    },
  };
 }
 function mergeTtsConfig(
  base?: VoiceCallTtsConfig,
  override?: VoiceCallTtsConfig,
 ): VoiceCallTtsConfig | undefined {
  if (!base && !override) return undefined;
  if (!override) return base;
  if (!base) return override;
  return deepMerge(base, override);
 }
 function deepMerge<T>(base: T, override: T): T {
  if (!isPlainObject(base) || !isPlainObject(override)) {
    return override;
  }
  const result: Record<string, unknown> = { ...base };
  for (const [key, value] of Object.entries(override)) {
    if (value === undefined) continue;
    const existing = (base as Record<string, unknown>)[key];
    if (isPlainObject(existing) && isPlainObject(value)) {
      result[key] = deepMerge(existing, value);
    } else {
      result[key] = value;
    }
  }
  return result as T;
 }
 function isPlainObject(value: unknown): value is Record<string, unknown> {
  return Boolean(value) && typeof value === "object" && !Array.isArray(value);
 }
@@ -10,17 +10,11 @@ describe("extractModelDirective", () => {
      expect(result.cleaned).toBe("");
    });
-    it("does not treat /models as a /model directive", () => {
+    it("extracts /models with argument", () => {
      const result = extractModelDirective("/models gpt-5");
-      expect(result.hasDirective).toBe(false);
+      expect(result.hasDirective).toBe(true);
-      expect(result.rawModel).toBeUndefined();
+      expect(result.rawModel).toBe("gpt-5");
-      expect(result.cleaned).toBe("/models gpt-5");
+      expect(result.cleaned).toBe("");
    });
    it("does not parse /models as a /model directive (no args)", () => {
      const result = extractModelDirective("/models");
      expect(result.hasDirective).toBe(false);
      expect(result.cleaned).toBe("/models");
    });
    it("extracts /model with provider/model format", () => {
@@ -14,7 +14,7 @@ export function extractModelDirective(
  if (!body) return { cleaned: "", hasDirective: false };
  const modelMatch = body.match(
-    /(?:^|\s)\/model(?=$|\s|:)\s*:?\s*([A-Za-z0-9_.:@-]+(?:\/[A-Za-z0-9_.:@-]+)*)?/i,
+    /(?:^|\s)\/models?(?=$|\s|:)\s*:?\s*([A-Za-z0-9_.:@-]+(?:\/[A-Za-z0-9_.:@-]+)*)?/i,
  );
  const aliases = (options?.aliases ?? []).map((alias) => alias.trim()).filter(Boolean);
@@ -59,17 +59,6 @@ export const CONFIG_PATH_CLAWDBOT = resolveConfigPath();
 export const DEFAULT_GATEWAY_PORT = 18789;
 /**
 * Gateway lock directory (ephemeral).
 * Default: os.tmpdir()/clawdbot-<uid> (uid suffix when available).
 */
 export function resolveGatewayLockDir(tmpdir: () => string = os.tmpdir): string {
  const base = tmpdir();
  const uid = typeof process.getuid === "function" ? process.getuid() : undefined;
  const suffix = uid != null ? `clawdbot-${uid}` : "clawdbot";
  return path.join(base, suffix);
 }
 const OAUTH_FILENAME = "oauth.json";
 /**
@@ -56,6 +56,7 @@ export type TtsConfig = {
  /** OpenAI configuration. */
  openai?: {
    apiKey?: string;
    baseUrl?: string;
    model?: string;
    voice?: string;
  };
@@ -204,6 +204,7 @@ export const TtsConfigSchema = z
    openai: z
      .object({
        apiKey: z.string().optional(),
        baseUrl: z.string().optional(),
        model: z.string().optional(),
        voice: z.string().optional(),
      })
@@ -131,7 +131,7 @@ export const ttsHandlers: GatewayRequestHandlers = {
          {
            id: "openai",
            name: "OpenAI",
-            configured: Boolean(resolveTtsApiKey(config, "openai")),
+            configured: isTtsProviderConfigured(config, "openai"),
            models: [...OPENAI_TTS_MODELS],
            voices: [...OPENAI_TTS_VOICES],
          },
@@ -7,13 +7,12 @@ import path from "node:path";
 import { describe, expect, it, vi } from "vitest";
 import { acquireGatewayLock, GatewayLockError } from "./gateway-lock.js";
-import { resolveConfigPath, resolveGatewayLockDir, resolveStateDir } from "../config/paths.js";
+import { resolveConfigPath, resolveStateDir } from "../config/paths.js";
 async function makeEnv() {
  const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-gateway-lock-"));
  const configPath = path.join(dir, "clawdbot.json");
  await fs.writeFile(configPath, "{}", "utf8");
  await fs.mkdir(resolveGatewayLockDir(), { recursive: true });
  return {
    env: {
      ...process.env,
@@ -30,8 +29,7 @@ function resolveLockPath(env: NodeJS.ProcessEnv) {
  const stateDir = resolveStateDir(env);
  const configPath = resolveConfigPath(env, stateDir);
  const hash = createHash("sha1").update(configPath).digest("hex").slice(0, 8);
-  const lockDir = resolveGatewayLockDir();
+  return { lockPath: path.join(stateDir, `gateway.${hash}.lock`), configPath };
  return { lockPath: path.join(lockDir, `gateway.${hash}.lock`), configPath };
 }
 function makeProcStat(pid: number, startTime: number) {
@@ -3,7 +3,7 @@ import fs from "node:fs/promises";
 import fsSync from "node:fs";
 import path from "node:path";
-import { resolveConfigPath, resolveGatewayLockDir, resolveStateDir } from "../config/paths.js";
+import { resolveConfigPath, resolveStateDir } from "../config/paths.js";
 const DEFAULT_TIMEOUT_MS = 5000;
 const DEFAULT_POLL_INTERVAL_MS = 100;
@@ -150,8 +150,7 @@ function resolveGatewayLockPath(env: NodeJS.ProcessEnv) {
  const stateDir = resolveStateDir(env);
  const configPath = resolveConfigPath(env, stateDir);
  const hash = createHash("sha1").update(configPath).digest("hex").slice(0, 8);
-  const lockDir = resolveGatewayLockDir();
+  const lockPath = path.join(stateDir, `gateway.${hash}.lock`);
  const lockPath = path.join(lockDir, `gateway.${hash}.lock`);
  return { lockPath, configPath };
 }
@@ -124,7 +124,6 @@ import { startWebLoginWithQr, waitForWebLogin } from "../../web/login-qr.js";
 import { sendMessageWhatsApp, sendPollWhatsApp } from "../../web/outbound.js";
 import { registerMemoryCli } from "../../cli/memory-cli.js";
 import { formatNativeDependencyHint } from "./native-deps.js";
 import { textToSpeechTelephony } from "../../tts/tts.js";
 import type { PluginRuntime } from "./types.js";
@@ -163,9 +162,6 @@ export function createPluginRuntime(): PluginRuntime {
      getImageMetadata,
      resizeToJpeg,
    },
    tts: {
      textToSpeechTelephony,
    },
    tools: {
      createMemoryGetTool,
      createMemorySearchTool,
@@ -16,7 +16,6 @@ type UpsertChannelPairingRequest =
  typeof import("../../pairing/pairing-store.js").upsertChannelPairingRequest;
 type FetchRemoteMedia = typeof import("../../media/fetch.js").fetchRemoteMedia;
 type SaveMediaBuffer = typeof import("../../media/store.js").saveMediaBuffer;
 type TextToSpeechTelephony = typeof import("../../tts/tts.js").textToSpeechTelephony;
 type BuildMentionRegexes = typeof import("../../auto-reply/reply/mentions.js").buildMentionRegexes;
 type MatchesMentionPatterns =
  typeof import("../../auto-reply/reply/mentions.js").matchesMentionPatterns;
@@ -174,9 +173,6 @@ export type PluginRuntime = {
    getImageMetadata: GetImageMetadata;
    resizeToJpeg: ResizeToJpeg;
  };
  tts: {
    textToSpeechTelephony: TextToSpeechTelephony;
  };
  tools: {
    createMemoryGetTool: CreateMemoryGetTool;
    createMemorySearchTool: CreateMemorySearchTool;
@@ -43,7 +43,6 @@ function setup(config: Record<string, unknown>): Registered {
    source: "test",
    config: {},
    pluginConfig: config,
    runtime: { tts: { textToSpeechTelephony: vi.fn() } },
    logger: noopLogger,
    registerGatewayMethod: (method, handler) => methods.set(method, handler),
    registerTool: (tool) => tools.push(tool),
@@ -143,7 +142,6 @@ describe("voice-call plugin", () => {
      source: "test",
      config: {},
      pluginConfig: { provider: "mock" },
      runtime: { tts: { textToSpeechTelephony: vi.fn() } },
      logger: noopLogger,
      registerGatewayMethod: () => {},
      registerTool: () => {},
@@ -106,6 +106,10 @@ describe("tts", () => {
      expect(isValidOpenAIVoice("alloy ")).toBe(false);
      expect(isValidOpenAIVoice(" alloy")).toBe(false);
    });
    it("accepts custom voices for custom endpoints", () => {
      expect(isValidOpenAIVoice("zm_yunxia", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
    });
  });
  describe("isValidOpenAIModel", () => {
@@ -115,11 +119,15 @@ describe("tts", () => {
      expect(isValidOpenAIModel("tts-1-hd")).toBe(true);
    });
-    it("rejects unsupported models", () => {
+    it("rejects other models", () => {
      expect(isValidOpenAIModel("invalid")).toBe(false);
      expect(isValidOpenAIModel("")).toBe(false);
      expect(isValidOpenAIModel("gpt-4")).toBe(false);
    });
    it("accepts custom models for custom endpoints", () => {
      expect(isValidOpenAIModel("kokoro", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
    });
  });
  describe("OPENAI_TTS_MODELS", () => {
@@ -202,6 +210,30 @@ describe("tts", () => {
      expect(result.overrides.provider).toBe("edge");
    });
    it("routes custom OpenAI models when custom endpoints are used", () => {
      const policy = resolveModelOverridePolicy({ enabled: true });
      const input = "Hello [[tts:provider=openai model=kokoro voice=zm_yunxia]] world";
      const result = parseTtsDirectives(input, policy, {
        defaultProvider: "openai",
        openaiBaseUrl: "http://localhost:8880/v1",
      });
      expect(result.overrides.openai?.model).toBe("kokoro");
      expect(result.overrides.openai?.voice).toBe("zm_yunxia");
    });
    it("routes model overrides to ElevenLabs when provider is ElevenLabs", () => {
      const policy = resolveModelOverridePolicy({ enabled: true });
      const input = "Hello [[tts:provider=elevenlabs model=eleven_multilingual_v2]] world";
      const result = parseTtsDirectives(input, policy, {
        defaultProvider: "openai",
        openaiBaseUrl: "http://localhost:8880/v1",
      });
      expect(result.overrides.elevenlabs?.modelId).toBe("eleven_multilingual_v2");
      expect(result.overrides.openai?.model).toBeUndefined();
    });
    it("keeps text intact when overrides are disabled", () => {
      const policy = resolveModelOverridePolicy({ enabled: false });
      const input = "Hello [[tts:voice=alloy]] world";
@@ -359,7 +391,12 @@ describe("tts", () => {
    };
    const restoreEnv = (snapshot: Record<string, string | undefined>) => {
-      const keys = ["OPENAI_API_KEY", "ELEVENLABS_API_KEY", "XI_API_KEY"] as const;
+      const keys = [
        "OPENAI_API_KEY",
        "OPENAI_TTS_BASE_URL",
        "ELEVENLABS_API_KEY",
        "XI_API_KEY",
      ] as const;
      for (const key of keys) {
        const value = snapshot[key];
        if (value === undefined) {
@@ -373,6 +410,7 @@ describe("tts", () => {
    const withEnv = (env: Record<string, string | undefined>, run: () => void) => {
      const snapshot = {
        OPENAI_API_KEY: process.env.OPENAI_API_KEY,
        OPENAI_TTS_BASE_URL: process.env.OPENAI_TTS_BASE_URL,
        ELEVENLABS_API_KEY: process.env.ELEVENLABS_API_KEY,
        XI_API_KEY: process.env.XI_API_KEY,
      };
@@ -394,6 +432,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: "test-openai-key",
          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: undefined,
          XI_API_KEY: undefined,
        },
@@ -409,6 +448,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: undefined,
          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: "test-elevenlabs-key",
          XI_API_KEY: undefined,
        },
@@ -424,6 +464,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: undefined,
          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: undefined,
          XI_API_KEY: undefined,
        },
@@ -434,6 +475,22 @@ describe("tts", () => {
        },
      );
    });
    it("prefers OpenAI when a custom endpoint is configured without API keys", () => {
      withEnv(
        {
          OPENAI_API_KEY: undefined,
          OPENAI_TTS_BASE_URL: "http://localhost:8880/v1",
          ELEVENLABS_API_KEY: undefined,
          XI_API_KEY: undefined,
        },
        () => {
          const config = resolveTtsConfig(baseCfg);
          const provider = getTtsProvider(config, "/tmp/tts-prefs-openai-custom.json");
          expect(provider).toBe("openai");
        },
      );
    });
  });
  describe("maybeApplyTtsToPayload", () => {
@@ -497,6 +554,55 @@ describe("tts", () => {
      process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
    });
    it("allows custom OpenAI endpoints without API keys", async () => {
      const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
      const prevBaseUrl = process.env.OPENAI_TTS_BASE_URL;
      const prevOpenAiKey = process.env.OPENAI_API_KEY;
      process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
      process.env.OPENAI_TTS_BASE_URL = "http://localhost:8880/v1";
      delete process.env.OPENAI_API_KEY;
      const originalFetch = globalThis.fetch;
      const fetchMock = vi.fn(async () => ({
        ok: true,
        arrayBuffer: async () => new ArrayBuffer(1),
      }));
      globalThis.fetch = fetchMock as unknown as typeof fetch;
      const cfg = {
        agents: { defaults: { model: { primary: "openai/gpt-4o-mini" } } },
        messages: {
          tts: {
            auto: "always",
            provider: "openai",
            openai: { model: "kokoro", voice: "zm_yunxia" },
          },
        },
      };
      const result = await maybeApplyTtsToPayload({
        payload: { text: "Hello world" },
        cfg,
        kind: "final",
      });
      expect(result.mediaUrl).toBeDefined();
      expect(fetchMock).toHaveBeenCalledTimes(1);
      const [url, options] = fetchMock.mock.calls[0] ?? [];
      expect(url).toBe("http://localhost:8880/v1/audio/speech");
      expect(
        (options as { headers?: Record<string, string> })?.headers?.Authorization,
      ).toBeUndefined();
      globalThis.fetch = originalFetch;
      process.env.OPENAI_TTS_BASE_URL = prevBaseUrl;
      if (prevOpenAiKey === undefined) {
        delete process.env.OPENAI_API_KEY;
      } else {
        process.env.OPENAI_API_KEY = prevOpenAiKey;
      }
      process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
    });
    it("skips auto-TTS in tagged mode unless a tts tag is present", async () => {
      const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
      process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
@@ -48,6 +48,7 @@ const DEFAULT_ELEVENLABS_VOICE_ID = "pMsXgVXv3BLzUgSXRplE";
 const DEFAULT_ELEVENLABS_MODEL_ID = "eleven_multilingual_v2";
 const DEFAULT_OPENAI_MODEL = "gpt-4o-mini-tts";
 const DEFAULT_OPENAI_VOICE = "alloy";
 const DEFAULT_OPENAI_TTS_BASE_URL = "https://api.openai.com/v1";
 const DEFAULT_EDGE_VOICE = "en-US-MichelleNeural";
 const DEFAULT_EDGE_LANG = "en-US";
 const DEFAULT_EDGE_OUTPUT_FORMAT = "audio-24khz-48kbitrate-mono-mp3";
@@ -76,11 +77,6 @@ const DEFAULT_OUTPUT = {
  voiceCompatible: false,
 };
 const TELEPHONY_OUTPUT = {
  openai: { format: "pcm" as const, sampleRate: 24000 },
  elevenlabs: { format: "pcm_22050", sampleRate: 22050 },
 };
 const TTS_AUTO_MODES = new Set<TtsAutoMode>(["off", "always", "inbound", "tagged"]);
 export type ResolvedTtsConfig = {
@@ -108,6 +104,7 @@ export type ResolvedTtsConfig = {
  };
  openai: {
    apiKey?: string;
    baseUrl: string;
    model: string;
    voice: string;
  };
@@ -185,16 +182,6 @@ export type TtsResult = {
  voiceCompatible?: boolean;
 };
 export type TtsTelephonyResult = {
  success: boolean;
  audioBuffer?: Buffer;
  error?: string;
  latencyMs?: number;
  provider?: string;
  outputFormat?: string;
  sampleRate?: number;
 };
 type TtsStatusEntry = {
  timestamp: number;
  success: boolean;
@@ -249,6 +236,9 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
  const raw: TtsConfig = cfg.messages?.tts ?? {};
  const providerSource = raw.provider ? "config" : "default";
  const edgeOutputFormat = raw.edge?.outputFormat?.trim();
  const openaiBaseUrl = normalizeOpenAiTtsBaseUrl(
    raw.openai?.baseUrl?.trim() || process.env.OPENAI_TTS_BASE_URL,
  );
  const auto = normalizeTtsAutoMode(raw.auto) ?? (raw.enabled ? "always" : "off");
  return {
    auto,
@@ -280,6 +270,7 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
    },
    openai: {
      apiKey: raw.openai?.apiKey,
      baseUrl: openaiBaseUrl,
      model: raw.openai?.model ?? DEFAULT_OPENAI_MODEL,
      voice: raw.openai?.voice ?? DEFAULT_OPENAI_VOICE,
    },
@@ -410,7 +401,9 @@ export function getTtsProvider(config: ResolvedTtsConfig, prefsPath: string): Tt
  if (prefs.tts?.provider) return prefs.tts.provider;
  if (config.providerSource === "config") return config.provider;
-  if (resolveTtsApiKey(config, "openai")) return "openai";
+  if (resolveTtsApiKey(config, "openai") || isCustomOpenAiTtsEndpoint(config.openai.baseUrl)) {
    return "openai";
  }
  if (resolveTtsApiKey(config, "elevenlabs")) return "elevenlabs";
  return "edge";
 }
@@ -485,6 +478,12 @@ export function resolveTtsProviderOrder(primary: TtsProvider): TtsProvider[] {
 export function isTtsProviderConfigured(config: ResolvedTtsConfig, provider: TtsProvider): boolean {
  if (provider === "edge") return config.edge.enabled;
  if (provider === "openai") {
    return (
      Boolean(resolveTtsApiKey(config, provider)) ||
      isCustomOpenAiTtsEndpoint(config.openai.baseUrl)
    );
  }
  return Boolean(resolveTtsApiKey(config, provider));
 }
@@ -498,6 +497,16 @@ function normalizeElevenLabsBaseUrl(baseUrl: string): string {
  return trimmed.replace(/\/+$/, "");
 }
 function normalizeOpenAiTtsBaseUrl(baseUrl?: string): string {
  const trimmed = baseUrl?.trim();
  if (!trimmed) return DEFAULT_OPENAI_TTS_BASE_URL;
  return trimmed.replace(/\/+$/, "");
 }
 function isCustomOpenAiTtsEndpoint(baseUrl: string): boolean {
  return normalizeOpenAiTtsBaseUrl(baseUrl) !== DEFAULT_OPENAI_TTS_BASE_URL;
 }
 function requireInRange(value: number, min: number, max: number, label: string): void {
  if (!Number.isFinite(value) || value < min || value > max) {
    throw new Error(`${label} must be between ${min} and ${max}`);
@@ -553,6 +562,7 @@ function parseNumberValue(value: string): number | undefined {
 function parseTtsDirectives(
  text: string,
  policy: ResolvedTtsModelOverrides,
  options: { defaultProvider?: TtsProvider; openaiBaseUrl?: string } = {},
 ): TtsDirectiveParseResult {
  if (!policy.enabled) {
    return { cleanedText: text, overrides: {}, warnings: [], hasDirective: false };
@@ -576,6 +586,30 @@ function parseTtsDirectives(
  cleanedText = cleanedText.replace(directiveRegex, (_match, body: string) => {
    hasDirective = true;
    const tokens = body.split(/\s+/).filter(Boolean);
    let providerOverride: TtsProvider | undefined;
    for (const token of tokens) {
      const eqIndex = token.indexOf("=");
      if (eqIndex === -1) continue;
      const rawKey = token.slice(0, eqIndex).trim();
      const rawValue = token.slice(eqIndex + 1).trim();
      if (!rawKey || !rawValue) continue;
      if (rawKey.toLowerCase() !== "provider") continue;
      if (!policy.allowProvider) break;
      if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
        providerOverride = rawValue;
      } else {
        warnings.push(`unsupported provider "${rawValue}"`);
      }
      break;
    }
    if (providerOverride) {
      overrides.provider = providerOverride;
    }
    const providerHint = overrides.provider ?? options.defaultProvider;
    const openAiContext = { baseUrl: options.openaiBaseUrl };
    for (const token of tokens) {
      const eqIndex = token.indexOf("=");
      if (eqIndex === -1) continue;
@@ -586,18 +620,12 @@ function parseTtsDirectives(
      try {
        switch (key) {
          case "provider":
            if (!policy.allowProvider) break;
            if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
              overrides.provider = rawValue;
            } else {
              warnings.push(`unsupported provider "${rawValue}"`);
            }
            break;
          case "voice":
          case "openai_voice":
          case "openaivoice":
            if (!policy.allowVoice) break;
-            if (isValidOpenAIVoice(rawValue)) {
+            if (isValidOpenAIVoice(rawValue, openAiContext)) {
              overrides.openai = { ...overrides.openai, voice: rawValue };
            } else {
              warnings.push(`invalid OpenAI voice "${rawValue}"`);
@@ -617,17 +645,40 @@ function parseTtsDirectives(
          case "model":
          case "modelid":
          case "model_id":
          case "elevenlabs_model":
          case "elevenlabsmodel":
          case "openai_model":
          case "openaimodel":
            if (!policy.allowModelId) break;
-            if (isValidOpenAIModel(rawValue)) {
+            if (key === "openai_model" || key === "openaimodel") {
              if (isValidOpenAIModel(rawValue, openAiContext)) {
                overrides.openai = { ...overrides.openai, model: rawValue };
              } else {
                warnings.push(`invalid OpenAI model "${rawValue}"`);
              }
              break;
            }
            if (providerHint === "openai") {
              if (isValidOpenAIModel(rawValue, openAiContext)) {
                overrides.openai = { ...overrides.openai, model: rawValue };
              } else {
                warnings.push(`invalid OpenAI model "${rawValue}"`);
              }
              break;
            }
            if (providerHint === "elevenlabs") {
              overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
              break;
            }
            if (isValidOpenAIModel(rawValue, openAiContext)) {
              overrides.openai = { ...overrides.openai, model: rawValue };
            } else {
              overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
            }
            break;
          case "elevenlabs_model":
          case "elevenlabsmodel":
            if (!policy.allowModelId) break;
            overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
            break;
          case "stability":
            if (!policy.allowVoiceSettings) break;
            {
@@ -752,16 +803,6 @@ function parseTtsDirectives(
 }
 export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts", "tts-1", "tts-1-hd"] as const;
 /**
 * Custom OpenAI-compatible TTS endpoint.
 * When set, model/voice validation is relaxed to allow non-OpenAI models.
 * Example: OPENAI_TTS_BASE_URL=http://localhost:8880/v1
 */
 const OPENAI_TTS_BASE_URL = (
  process.env.OPENAI_TTS_BASE_URL?.trim() || "https://api.openai.com/v1"
 ).replace(/\/+$/, "");
 const isCustomOpenAIEndpoint = OPENAI_TTS_BASE_URL !== "https://api.openai.com/v1";
 export const OPENAI_TTS_VOICES = [
  "alloy",
  "ash",
@@ -775,16 +816,15 @@ export const OPENAI_TTS_VOICES = [
 ] as const;
 type OpenAiTtsVoice = (typeof OPENAI_TTS_VOICES)[number];
 type OpenAiTtsValidationContext = { baseUrl?: string };
-function isValidOpenAIModel(model: string): boolean {
+function isValidOpenAIModel(model: string, context: OpenAiTtsValidationContext = {}): boolean {
-  // Allow any model when using custom endpoint (e.g., Kokoro, LocalAI)
+  if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
  if (isCustomOpenAIEndpoint) return true;
  return OPENAI_TTS_MODELS.includes(model as (typeof OPENAI_TTS_MODELS)[number]);
 }
-function isValidOpenAIVoice(voice: string): voice is OpenAiTtsVoice {
+function isValidOpenAIVoice(voice: string, context: OpenAiTtsValidationContext = {}): boolean {
-  // Allow any voice when using custom endpoint (e.g., Kokoro Chinese voices)
+  if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
  if (isCustomOpenAIEndpoint) return true;
  return OPENAI_TTS_VOICES.includes(voice as OpenAiTtsVoice);
 }
@@ -992,18 +1032,21 @@ async function elevenLabsTTS(params: {
 async function openaiTTS(params: {
  text: string;
-  apiKey: string;
+  apiKey?: string;
  baseUrl: string;
  model: string;
  voice: string;
-  responseFormat: "mp3" | "opus" | "pcm";
+  responseFormat: "mp3" | "opus";
  timeoutMs: number;
 }): Promise<Buffer> {
-  const { text, apiKey, model, voice, responseFormat, timeoutMs } = params;
+  const { text, model, voice, responseFormat, timeoutMs } = params;
  const apiKey = params.apiKey?.trim();
  const baseUrl = normalizeOpenAiTtsBaseUrl(params.baseUrl);
-  if (!isValidOpenAIModel(model)) {
+  if (!isValidOpenAIModel(model, { baseUrl })) {
    throw new Error(`Invalid model: ${model}`);
  }
-  if (!isValidOpenAIVoice(voice)) {
+  if (!isValidOpenAIVoice(voice, { baseUrl })) {
    throw new Error(`Invalid voice: ${voice}`);
  }
@@ -1011,12 +1054,15 @@ async function openaiTTS(params: {
  const timeout = setTimeout(() => controller.abort(), timeoutMs);
  try {
-    const response = await fetch(`${OPENAI_TTS_BASE_URL}/audio/speech`, {
+    const headers: Record<string, string> = {
      "Content-Type": "application/json",
    };
    if (apiKey) {
      headers.Authorization = `Bearer ${apiKey}`;
    }
    const response = await fetch(`${baseUrl}/audio/speech`, {
      method: "POST",
-      headers: {
+      headers,
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model,
        input: text,
@@ -1166,13 +1212,19 @@ export async function textToSpeech(params: {
      }
      const apiKey = resolveTtsApiKey(config, provider);
-      if (!apiKey) {
+      const allowMissingKey =
        provider === "openai" && isCustomOpenAiTtsEndpoint(config.openai.baseUrl);
      if (!apiKey && !allowMissingKey) {
        lastError = `No API key for ${provider}`;
        continue;
      }
      let audioBuffer: Buffer;
      if (provider === "elevenlabs") {
        if (!apiKey) {
          lastError = "No API key for elevenlabs";
          continue;
        }
        const voiceIdOverride = params.overrides?.elevenlabs?.voiceId;
        const modelIdOverride = params.overrides?.elevenlabs?.modelId;
        const voiceSettings = {
@@ -1201,6 +1253,7 @@ export async function textToSpeech(params: {
        audioBuffer = await openaiTTS({
          text: params.text,
          apiKey,
          baseUrl: config.openai.baseUrl,
          model: openaiModelOverride ?? config.openai.model,
          voice: openaiVoiceOverride ?? config.openai.voice,
          responseFormat: output.openai,
@@ -1239,100 +1292,6 @@ export async function textToSpeech(params: {
  };
 }
 export async function textToSpeechTelephony(params: {
  text: string;
  cfg: ClawdbotConfig;
  prefsPath?: string;
 }): Promise<TtsTelephonyResult> {
  const config = resolveTtsConfig(params.cfg);
  const prefsPath = params.prefsPath ?? resolveTtsPrefsPath(config);
  if (params.text.length > config.maxTextLength) {
    return {
      success: false,
      error: `Text too long (${params.text.length} chars, max ${config.maxTextLength})`,
    };
  }
  const userProvider = getTtsProvider(config, prefsPath);
  const providers = resolveTtsProviderOrder(userProvider);
  let lastError: string | undefined;
  for (const provider of providers) {
    const providerStart = Date.now();
    try {
      if (provider === "edge") {
        lastError = "edge: unsupported for telephony";
        continue;
      }
      const apiKey = resolveTtsApiKey(config, provider);
      if (!apiKey) {
        lastError = `No API key for ${provider}`;
        continue;
      }
      if (provider === "elevenlabs") {
        const output = TELEPHONY_OUTPUT.elevenlabs;
        const audioBuffer = await elevenLabsTTS({
          text: params.text,
          apiKey,
          baseUrl: config.elevenlabs.baseUrl,
          voiceId: config.elevenlabs.voiceId,
          modelId: config.elevenlabs.modelId,
          outputFormat: output.format,
          seed: config.elevenlabs.seed,
          applyTextNormalization: config.elevenlabs.applyTextNormalization,
          languageCode: config.elevenlabs.languageCode,
          voiceSettings: config.elevenlabs.voiceSettings,
          timeoutMs: config.timeoutMs,
        });
        return {
          success: true,
          audioBuffer,
          latencyMs: Date.now() - providerStart,
          provider,
          outputFormat: output.format,
          sampleRate: output.sampleRate,
        };
      }
      const output = TELEPHONY_OUTPUT.openai;
      const audioBuffer = await openaiTTS({
        text: params.text,
        apiKey,
        model: config.openai.model,
        voice: config.openai.voice,
        responseFormat: output.format,
        timeoutMs: config.timeoutMs,
      });
      return {
        success: true,
        audioBuffer,
        latencyMs: Date.now() - providerStart,
        provider,
        outputFormat: output.format,
        sampleRate: output.sampleRate,
      };
    } catch (err) {
      const error = err as Error;
      if (error.name === "AbortError") {
        lastError = `${provider}: request timed out`;
      } else {
        lastError = `${provider}: ${error.message}`;
      }
    }
  }
  return {
    success: false,
    error: `TTS conversion failed: ${lastError || "no providers available"}`,
  };
 }
 export async function maybeApplyTtsToPayload(params: {
  payload: ReplyPayload;
  cfg: ClawdbotConfig;
@@ -1350,8 +1309,12 @@ export async function maybeApplyTtsToPayload(params: {
  });
  if (autoMode === "off") return params.payload;
  const defaultProvider = getTtsProvider(config, prefsPath);
  const text = params.payload.text ?? "";
-  const directives = parseTtsDirectives(text, config.modelOverrides);
+  const directives = parseTtsDirectives(text, config.modelOverrides, {
    defaultProvider,
    openaiBaseUrl: config.openai.baseUrl,
  });
  if (directives.warnings.length > 0) {
    logVerbose(`TTS: ignored directive overrides (${directives.warnings.join("; ")})`);
  }
Author	SHA1	Message	Date
Peter Steinberger	d649710627	fix: support custom TTS endpoints (#1701 ) (thanks @louzhixian)	2026-01-25 08:01:11 +00:00
zhixian	498c15555e	test: update tests for expanded OpenAI TTS model list - Accept tts-1 and tts-1-hd as valid models - Update OPENAI_TTS_MODELS length expectation to 3	2026-01-25 07:59:20 +00:00
zhixian	25345a909a	style: format code with oxfmt	2026-01-25 07:59:20 +00:00
zhixian	bfd8321b42	fix: strip trailing slashes from OPENAI_TTS_BASE_URL Address review feedback: normalize the base URL by removing trailing slashes to prevent double-slash paths like /v1//audio/speech which cause 404 errors on some OpenAI-compatible servers.	2026-01-25 07:59:20 +00:00
zhixian	fbbed439bd	feat(tts): support custom OpenAI-compatible TTS endpoints Add OPENAI_TTS_BASE_URL environment variable to allow using self-hosted or third-party OpenAI-compatible TTS services like Kokoro, LocalAI, or OpenedAI-Speech. Changes: - Add OPENAI_TTS_BASE_URL env var (defaults to OpenAI official API) - Relax model/voice validation when using custom endpoints - Add tts-1 and tts-1-hd to the model allowlist This enables users to: - Use local TTS for privacy and cost savings - Use models with better non-English language support (Chinese, Japanese) - Reduce latency with local inference Example usage: OPENAI_TTS_BASE_URL=http://localhost:8880/v1 Tested with Kokoro-FastAPI.	2026-01-25 07:59:20 +00:00