fix: support custom TTS endpoints (#1701 ) (thanks @louzhixian)

test: update tests for expanded OpenAI TTS model list
- Accept tts-1 and tts-1-hd as valid models - Update OPENAI_TTS_MODELS length expectation to 3
2026-01-25 08:01:11 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00 · 2026-01-25 07:59:20 +00:00
31 changed files with 340 additions and 847 deletions
@@ -9,6 +9,7 @@ Docs: https://docs.clawd.bot
 - Venius (Venice AI): highlight provider guide + cross-links + expanded guidance. https://docs.clawd.bot/providers/venice

 ### Changes
+- TTS: allow custom OpenAI-compatible TTS endpoints without API keys. (#1701) Thanks @louzhixian. https://docs.clawd.bot/tts
 - TTS: add Edge TTS provider fallback, defaulting to keyless Edge with MP3 retry on format failures. (#1668) Thanks @steipete. https://docs.clawd.bot/tts
 - Web search: add Brave freshness filter parameter for time-scoped results. (#1688) Thanks @JonUleis. https://docs.clawd.bot/tools/web
 - TTS: add auto mode enum (off/always/inbound/tagged) with per-session `/tts` override. (#1667) Thanks @sebslight. https://docs.clawd.bot/tts
@@ -28,7 +29,6 @@ Docs: https://docs.clawd.bot
 - Web UI: hide internal `message_id` hints in chat bubbles.
 - Web UI: show Stop button during active runs, swap back to New session when idle. (#1664) Thanks @ndbroadbent.
 - Web UI: clear stale disconnect banners on reconnect; allow form saves with unsupported schema paths but block missing schema. (#1707) Thanks @Glucksberg.
- Auto-reply: don't treat `/models` as a `/model` directive. (#1753) Thanks @uos-status.
 - Heartbeat: normalize target identifiers for consistent routing.
 - TUI: reload history after gateway reconnect to restore session state. (#1663)
 - Telegram: use wrapped fetch for long-polling on Node to normalize AbortSignal handling. (#1639)
@@ -43,7 +43,6 @@ Docs: https://docs.clawd.bot
 - Gateway: reduce log noise for late invokes + remote node probes; debounce skills refresh. (#1607) Thanks @petter-b.
 - Gateway: clarify Control UI/WebChat auth error hints for missing tokens. (#1690)
 - Gateway: listen on IPv6 loopback when bound to 127.0.0.1 so localhost webhooks work.
- Gateway: store lock files in the temp directory to avoid stale locks on persistent volumes. (#1676)
 - macOS: default direct-transport `ws://` URLs to port 18789; document `gateway.remote.transport`. (#1603) Thanks @ngutman.
 - Voice Call: return stream TwiML for outbound conversation calls on initial Twilio webhook. (#1634)
 - Google Chat: tighten email allowlist matching, typing cleanup, media caps, and onboarding/docs/tests. (#1635) Thanks @iHildy.
@@ -1538,6 +1538,7 @@ voice notes; other channels send MP3 audio.
      },
      openai: {
        apiKey: "openai_api_key",
+        baseUrl: "https://api.openai.com/v1",
        model: "gpt-4o-mini-tts",
        voice: "alloy"
      }
@@ -1558,6 +1559,8 @@ Notes:
 - `/tts limit` and `/tts summary` control per-user summarization settings.
 - `apiKey` values fall back to `ELEVENLABS_API_KEY`/`XI_API_KEY` and `OPENAI_API_KEY`.
 - `elevenlabs.baseUrl` overrides the ElevenLabs API base URL.
+- `openai.baseUrl` overrides the OpenAI TTS base URL (defaults to `https://api.openai.com/v1`,
+  or `OPENAI_TTS_BASE_URL` when set). Custom endpoints do not require an API key.
 - `elevenlabs.voiceSettings` supports `stability`/`similarityBoost`/`style` (0..1),
  `useSpeakerBoost`, and `speed` (0.5..2.0).

@@ -67,22 +67,6 @@ Plugins can register:
 Plugins run **in‑process** with the Gateway, so treat them as trusted code.
 Tool authoring guide: [Plugin agent tools](/plugins/agent-tools).

-## Runtime helpers
-
-Plugins can access selected core helpers via `api.runtime`. For telephony TTS:
-
-```ts
-const result = await api.runtime.tts.textToSpeechTelephony({
-  text: "Hello from Clawdbot",
-  cfg: api.config,
-});
-```
-
-Notes:
- Uses core `messages.tts` configuration (OpenAI or ElevenLabs).
- Returns PCM audio buffer + sample rate. Plugins must resample/encode for providers.
- Edge TTS is not supported for telephony.
-
 ## Discovery & precedence

 Clawdbot scans, in order:
@@ -104,87 +104,6 @@ Notes:
 - `mock` is a local dev provider (no network calls).
 - `skipSignatureVerification` is for local testing only.

-## TTS for calls
-
-Voice Call uses the core `messages.tts` configuration (OpenAI or ElevenLabs) for
-streaming speech on calls. You can override it under the plugin config with the
-**same shape** — it deep‑merges with `messages.tts`.
-
-```json5
-{
-  tts: {
-    provider: "elevenlabs",
-    elevenlabs: {
-      voiceId: "pMsXgVXv3BLzUgSXRplE",
-      modelId: "eleven_multilingual_v2"
-    }
-  }
-}
-```
-
-Notes:
- **Edge TTS is ignored for voice calls** (telephony audio needs PCM; Edge output is unreliable).
- Core TTS is used when Twilio media streaming is enabled; otherwise calls fall back to provider native voices.
-
-### More examples
-
-Use core TTS only (no override):
-
-```json5
-{
-  messages: {
-    tts: {
-      provider: "openai",
-      openai: { voice: "alloy" }
-    }
-  }
-}
-```
-
-Override to ElevenLabs just for calls (keep core default elsewhere):
-
-```json5
-{
-  plugins: {
-    entries: {
-      "voice-call": {
-        config: {
-          tts: {
-            provider: "elevenlabs",
-            elevenlabs: {
-              apiKey: "elevenlabs_key",
-              voiceId: "pMsXgVXv3BLzUgSXRplE",
-              modelId: "eleven_multilingual_v2"
-            }
-          }
-        }
-      }
-    }
-  }
-}
-```
-
-Override only the OpenAI model for calls (deep‑merge example):
-
-```json5
-{
-  plugins: {
-    entries: {
-      "voice-call": {
-        config: {
-          tts: {
-            openai: {
-              model: "gpt-4o-mini-tts",
-              voice: "marin"
-            }
-          }
-        }
-      }
-    }
-  }
-}
-```
-
 ## Inbound calls

 Inbound policy defaults to `disabled`. To enable inbound calls, set:
@@ -16,7 +16,7 @@ and you configure everything via the `/setup` web wizard.

 ## One-click deploy

-<a href="https://railway.app/new/template?template=https://github.com/vignesh07/clawdbot-railway-template" target="_blank" rel="noreferrer">Deploy on Railway</a>
+<a href="https://railway.com/deploy/clawdbot-railway-template" target="_blank" rel="noreferrer">Deploy on Railway</a>

 After deploy, find your public URL in **Railway → your service → Settings → Domains**.

@@ -55,7 +55,6 @@ Attach a volume mounted at:
 Set these variables on the service:

 - `SETUP_PASSWORD` (required)
- `PORT=8080` (required — must match the port in Public Networking)
 - `CLAWDBOT_STATE_DIR=/data/.clawdbot` (recommended)
 - `CLAWDBOT_WORKSPACE_DIR=/data/workspace` (recommended)
 - `CLAWDBOT_GATEWAY_TOKEN` (recommended; treat as an admin secret)
@@ -83,9 +82,8 @@ If Telegram DMs are set to pairing, the setup wizard can approve the pairing cod
 1) Go to https://discord.com/developers/applications
 2) **New Application** → choose a name
 3) **Bot** → **Add Bot**
-4) **Enable MESSAGE CONTENT INTENT** under Bot → Privileged Gateway Intents (required or the bot will crash on startup)
-5) Copy the **Bot Token** and paste into `/setup`
-6) Invite the bot to your server (OAuth2 URL Generator; scopes: `bot`, `applications.commands`)
+4) Copy the **Bot Token** and paste into `/setup`
+5) Invite the bot to your server (OAuth2 URL Generator; scopes: `bot`, `applications.commands`)

 ## Backups & migration

@@ -33,11 +33,18 @@ does not publish limits, so assume similar or lower limits. citeturn0searc

 If you want OpenAI or ElevenLabs:
 - `ELEVENLABS_API_KEY` (or `XI_API_KEY`)
- `OPENAI_API_KEY`
+- `OPENAI_API_KEY` (optional for custom OpenAI-compatible endpoints)
+
+Optional:
+- `OPENAI_TTS_BASE_URL` (defaults to `https://api.openai.com/v1`)

 Edge TTS does **not** require an API key. If no API keys are found, Clawdbot defaults
 to Edge TTS (unless disabled via `messages.tts.edge.enabled=false`).

+When `OPENAI_TTS_BASE_URL` points to a non-OpenAI endpoint, Clawdbot relaxes
+OpenAI model/voice validation and does **not** require an API key. If your
+endpoint needs authentication, set `messages.tts.openai.apiKey` or `OPENAI_API_KEY`.
+
 If multiple providers are configured, the selected provider is used first and the others are fallback options.
 Auto-summary uses the configured `summaryModel` (or `agents.defaults.model.primary`),
 so that provider must also be authenticated if you enable summaries.
@@ -115,6 +122,32 @@ Full schema is in [Gateway configuration](/gateway/configuration).
 }
 ```

+### Custom OpenAI-compatible endpoint
+
+Set an endpoint in the environment:
+
+```bash
+export OPENAI_TTS_BASE_URL="http://localhost:8880/v1"
+```
+
+Then configure TTS as usual (API key optional for custom endpoints):
+
+```json5
+{
+  messages: {
+    tts: {
+      auto: "always",
+      provider: "openai",
+      openai: {
+        baseUrl: "http://localhost:8880/v1",
+        model: "kokoro",
+        voice: "zm_yunxia"
+      }
+    }
+  }
+}
+```
+
 ### Edge TTS primary (no API key)

 ```json5
@@ -1,12 +1,5 @@
 # Changelog

-## 2026.1.24
-
-### Changes
- Breaking: voice-call TTS now uses core `messages.tts` (plugin TTS config deep‑merges with core).
- Telephony TTS supports OpenAI + ElevenLabs; Edge TTS is ignored for calls.
- Removed legacy `tts.model`/`tts.voice`/`tts.instructions` plugin fields.
-
 ## 2026.1.23

 ### Changes
@@ -75,27 +75,6 @@ Notes:
 - Twilio/Telnyx/Plivo require a **publicly reachable** webhook URL.
 - `mock` is a local dev provider (no network calls).

-## TTS for calls
-
-Voice Call uses the core `messages.tts` configuration (OpenAI or ElevenLabs) for
-streaming speech on calls. You can override it under the plugin config with the
-same shape — overrides deep-merge with `messages.tts`.
-
-```json5
-{
-  tts: {
-    provider: "openai",
-    openai: {
-      voice: "alloy"
-    }
-  }
-}
-```
-
-Notes:
- Edge TTS is ignored for voice calls (telephony audio needs PCM; Edge output is unreliable).
- Core TTS is used when Twilio media streaming is enabled; otherwise calls fall back to provider native voices.
-
 ## CLI

 ```bash
@@ -99,39 +99,16 @@
      "label": "Media Stream Path",
      "advanced": true
    },
-    "tts.provider": {
-      "label": "TTS Provider Override",
-      "help": "Deep-merges with messages.tts (Edge is ignored for calls).",
+    "tts.model": {
+      "label": "TTS Model",
      "advanced": true
    },
-    "tts.openai.model": {
-      "label": "OpenAI TTS Model",
+    "tts.voice": {
+      "label": "TTS Voice",
      "advanced": true
    },
-    "tts.openai.voice": {
-      "label": "OpenAI TTS Voice",
-      "advanced": true
-    },
-    "tts.openai.apiKey": {
-      "label": "OpenAI API Key",
-      "sensitive": true,
-      "advanced": true
-    },
-    "tts.elevenlabs.modelId": {
-      "label": "ElevenLabs Model ID",
-      "advanced": true
-    },
-    "tts.elevenlabs.voiceId": {
-      "label": "ElevenLabs Voice ID",
-      "advanced": true
-    },
-    "tts.elevenlabs.apiKey": {
-      "label": "ElevenLabs API Key",
-      "sensitive": true,
-      "advanced": true
-    },
-    "tts.elevenlabs.baseUrl": {
-      "label": "ElevenLabs Base URL",
+    "tts.instructions": {
+      "label": "TTS Instructions",
      "advanced": true
    },
    "publicUrl": {
@@ -393,193 +370,20 @@
        "type": "object",
        "additionalProperties": false,
        "properties": {
-          "auto": {
-            "type": "string",
-            "enum": [
-              "off",
-              "always",
-              "inbound",
-              "tagged"
-            ]
-          },
-          "enabled": {
-            "type": "boolean"
-          },
-          "mode": {
-            "type": "string",
-            "enum": [
-              "final",
-              "all"
-            ]
-          },
          "provider": {
            "type": "string",
            "enum": [
-              "openai",
-              "elevenlabs",
-              "edge"
+              "openai"
            ]
          },
-          "summaryModel": {
+          "model": {
            "type": "string"
          },
-          "modelOverrides": {
-            "type": "object",
-            "additionalProperties": false,
-            "properties": {
-              "enabled": {
-                "type": "boolean"
-              },
-              "allowText": {
-                "type": "boolean"
-              },
-              "allowProvider": {
-                "type": "boolean"
-              },
-              "allowVoice": {
-                "type": "boolean"
-              },
-              "allowModelId": {
-                "type": "boolean"
-              },
-              "allowVoiceSettings": {
-                "type": "boolean"
-              },
-              "allowNormalization": {
-                "type": "boolean"
-              },
-              "allowSeed": {
-                "type": "boolean"
-              }
-            }
-          },
-          "elevenlabs": {
-            "type": "object",
-            "additionalProperties": false,
-            "properties": {
-              "apiKey": {
-                "type": "string"
-              },
-              "baseUrl": {
-                "type": "string"
-              },
-              "voiceId": {
-                "type": "string"
-              },
-              "modelId": {
-                "type": "string"
-              },
-              "seed": {
-                "type": "integer",
-                "minimum": 0,
-                "maximum": 4294967295
-              },
-              "applyTextNormalization": {
-                "type": "string",
-                "enum": [
-                  "auto",
-                  "on",
-                  "off"
-                ]
-              },
-              "languageCode": {
-                "type": "string"
-              },
-              "voiceSettings": {
-                "type": "object",
-                "additionalProperties": false,
-                "properties": {
-                  "stability": {
-                    "type": "number",
-                    "minimum": 0,
-                    "maximum": 1
-                  },
-                  "similarityBoost": {
-                    "type": "number",
-                    "minimum": 0,
-                    "maximum": 1
-                  },
-                  "style": {
-                    "type": "number",
-                    "minimum": 0,
-                    "maximum": 1
-                  },
-                  "useSpeakerBoost": {
-                    "type": "boolean"
-                  },
-                  "speed": {
-                    "type": "number",
-                    "minimum": 0.5,
-                    "maximum": 2
-                  }
-                }
-              }
-            }
-          },
-          "openai": {
-            "type": "object",
-            "additionalProperties": false,
-            "properties": {
-              "apiKey": {
-                "type": "string"
-              },
-              "model": {
-                "type": "string"
-              },
-              "voice": {
-                "type": "string"
-              }
-            }
-          },
-          "edge": {
-            "type": "object",
-            "additionalProperties": false,
-            "properties": {
-              "enabled": {
-                "type": "boolean"
-              },
-              "voice": {
-                "type": "string"
-              },
-              "lang": {
-                "type": "string"
-              },
-              "outputFormat": {
-                "type": "string"
-              },
-              "pitch": {
-                "type": "string"
-              },
-              "rate": {
-                "type": "string"
-              },
-              "volume": {
-                "type": "string"
-              },
-              "saveSubtitles": {
-                "type": "boolean"
-              },
-              "proxy": {
-                "type": "string"
-              },
-              "timeoutMs": {
-                "type": "integer",
-                "minimum": 1000,
-                "maximum": 120000
-              }
-            }
-          },
-          "prefsPath": {
+          "voice": {
            "type": "string"
          },
-          "maxTextLength": {
-            "type": "integer",
-            "minimum": 1
-          },
-          "timeoutMs": {
-            "type": "integer",
-            "minimum": 1000,
-            "maximum": 120000
+          "instructions": {
+            "type": "string"
          }
        }
      },
@@ -74,26 +74,9 @@ const voiceCallConfigSchema = {
    },
    "streaming.sttModel": { label: "Realtime STT Model", advanced: true },
    "streaming.streamPath": { label: "Media Stream Path", advanced: true },
-    "tts.provider": {
-      label: "TTS Provider Override",
-      help: "Deep-merges with messages.tts (Edge is ignored for calls).",
-      advanced: true,
-    },
-    "tts.openai.model": { label: "OpenAI TTS Model", advanced: true },
-    "tts.openai.voice": { label: "OpenAI TTS Voice", advanced: true },
-    "tts.openai.apiKey": {
-      label: "OpenAI API Key",
-      sensitive: true,
-      advanced: true,
-    },
-    "tts.elevenlabs.modelId": { label: "ElevenLabs Model ID", advanced: true },
-    "tts.elevenlabs.voiceId": { label: "ElevenLabs Voice ID", advanced: true },
-    "tts.elevenlabs.apiKey": {
-      label: "ElevenLabs API Key",
-      sensitive: true,
-      advanced: true,
-    },
-    "tts.elevenlabs.baseUrl": { label: "ElevenLabs Base URL", advanced: true },
+    "tts.model": { label: "TTS Model", advanced: true },
+    "tts.voice": { label: "TTS Voice", advanced: true },
+    "tts.instructions": { label: "TTS Instructions", advanced: true },
    publicUrl: { label: "Public Webhook URL", advanced: true },
    skipSignatureVerification: {
      label: "Skip Signature Verification",
@@ -178,7 +161,6 @@ const voiceCallPlugin = {
        runtimePromise = createVoiceCallRuntime({
          config: cfg,
          coreConfig: api.config as CoreConfig,
-          ttsRuntime: api.runtime.tts,
          logger: api.logger,
        });
      }
@@ -82,82 +82,31 @@ export const SttConfigSchema = z
  .default({ provider: "openai", model: "whisper-1" });
 export type SttConfig = z.infer<typeof SttConfigSchema>;

-export const TtsProviderSchema = z.enum(["openai", "elevenlabs", "edge"]);
-export const TtsModeSchema = z.enum(["final", "all"]);
-export const TtsAutoSchema = z.enum(["off", "always", "inbound", "tagged"]);
-
 export const TtsConfigSchema = z
  .object({
-    auto: TtsAutoSchema.optional(),
-    enabled: z.boolean().optional(),
-    mode: TtsModeSchema.optional(),
-    provider: TtsProviderSchema.optional(),
-    summaryModel: z.string().optional(),
-    modelOverrides: z
-      .object({
-        enabled: z.boolean().optional(),
-        allowText: z.boolean().optional(),
-        allowProvider: z.boolean().optional(),
-        allowVoice: z.boolean().optional(),
-        allowModelId: z.boolean().optional(),
-        allowVoiceSettings: z.boolean().optional(),
-        allowNormalization: z.boolean().optional(),
-        allowSeed: z.boolean().optional(),
-      })
-      .strict()
-      .optional(),
-    elevenlabs: z
-      .object({
-        apiKey: z.string().optional(),
-        baseUrl: z.string().optional(),
-        voiceId: z.string().optional(),
-        modelId: z.string().optional(),
-        seed: z.number().int().min(0).max(4294967295).optional(),
-        applyTextNormalization: z.enum(["auto", "on", "off"]).optional(),
-        languageCode: z.string().optional(),
-        voiceSettings: z
-          .object({
-            stability: z.number().min(0).max(1).optional(),
-            similarityBoost: z.number().min(0).max(1).optional(),
-            style: z.number().min(0).max(1).optional(),
-            useSpeakerBoost: z.boolean().optional(),
-            speed: z.number().min(0.5).max(2).optional(),
-          })
-          .strict()
-          .optional(),
-      })
-      .strict()
-      .optional(),
-    openai: z
-      .object({
-        apiKey: z.string().optional(),
-        model: z.string().optional(),
-        voice: z.string().optional(),
-      })
-      .strict()
-      .optional(),
-    edge: z
-      .object({
-        enabled: z.boolean().optional(),
-        voice: z.string().optional(),
-        lang: z.string().optional(),
-        outputFormat: z.string().optional(),
-        pitch: z.string().optional(),
-        rate: z.string().optional(),
-        volume: z.string().optional(),
-        saveSubtitles: z.boolean().optional(),
-        proxy: z.string().optional(),
-        timeoutMs: z.number().int().min(1000).max(120000).optional(),
-      })
-      .strict()
-      .optional(),
-    prefsPath: z.string().optional(),
-    maxTextLength: z.number().int().min(1).optional(),
-    timeoutMs: z.number().int().min(1000).max(120000).optional(),
+    /** TTS provider (currently only OpenAI supported) */
+    provider: z.literal("openai").default("openai"),
+    /**
+     * TTS model to use:
+     * - gpt-4o-mini-tts: newest, supports instructions for tone/style control (recommended)
+     * - tts-1: lower latency
+     * - tts-1-hd: higher quality
+     */
+    model: z.string().min(1).default("gpt-4o-mini-tts"),
+    /**
+     * Voice ID. For best quality, use marin or cedar.
+     * All voices: alloy, ash, ballad, coral, echo, fable, nova, onyx, sage, shimmer, verse, marin, cedar
+     */
+    voice: z.string().min(1).default("coral"),
+    /**
+     * Instructions for speech style (only works with gpt-4o-mini-tts).
+     * Examples: "Speak in a cheerful tone", "Talk like a sympathetic customer service agent"
+     */
+    instructions: z.string().optional(),
  })
  .strict()
-  .optional();
-export type VoiceCallTtsConfig = z.infer<typeof TtsConfigSchema>;
+  .default({ provider: "openai", model: "gpt-4o-mini-tts", voice: "coral" });
+export type TtsConfig = z.infer<typeof TtsConfigSchema>;

 // -----------------------------------------------------------------------------
 // Webhook Server Configuration
@@ -358,7 +307,7 @@ export const VoiceCallConfigSchema = z
  /** STT configuration */
  stt: SttConfigSchema,

-  /** TTS override (deep-merges with core messages.tts) */
+  /** TTS configuration */
  tts: TtsConfigSchema,

  /** Store path for call logs */
@@ -2,16 +2,10 @@ import fs from "node:fs";
 import path from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";

-import type { VoiceCallTtsConfig } from "./config.js";
-
 export type CoreConfig = {
  session?: {
    store?: string;
  };
-  messages?: {
-    tts?: VoiceCallTtsConfig;
-  };
-  [key: string]: unknown;
 };

 type CoreAgentDeps = {
@@ -143,7 +143,7 @@ export class CallManager {
      // For notify mode with a message, use inline TwiML with <Say>
      let inlineTwiml: string | undefined;
      if (mode === "notify" && initialMessage) {
-        const pollyVoice = mapVoiceToPolly(this.config.tts?.openai?.voice);
+        const pollyVoice = mapVoiceToPolly(this.config.tts.voice);
        inlineTwiml = this.generateNotifyTwiml(initialMessage, pollyVoice);
        console.log(
          `[voice-call] Using inline TwiML for notify mode (voice: ${pollyVoice})`,
@@ -210,13 +210,11 @@ export class CallManager {
      this.addTranscriptEntry(call, "bot", text);

      // Play TTS
-      const voice =
-        this.provider?.name === "twilio" ? this.config.tts?.openai?.voice : undefined;
      await this.provider.playTts({
        callId,
        providerCallId: call.providerCallId,
        text,
-        voice,
+        voice: this.config.tts.voice,
      });

      return { success: true };
@@ -68,7 +68,7 @@ export async function initiateCall(
    // For notify mode with a message, use inline TwiML with <Say>.
    let inlineTwiml: string | undefined;
    if (mode === "notify" && initialMessage) {
-      const pollyVoice = mapVoiceToPolly(ctx.config.tts?.openai?.voice);
+      const pollyVoice = mapVoiceToPolly(ctx.config.tts.voice);
      inlineTwiml = generateNotifyTwiml(initialMessage, pollyVoice);
      console.log(`[voice-call] Using inline TwiML for notify mode (voice: ${pollyVoice})`);
    }
@@ -120,13 +120,11 @@ export async function speak(

    addTranscriptEntry(call, "bot", text);

-    const voice =
-      ctx.provider?.name === "twilio" ? ctx.config.tts?.openai?.voice : undefined;
    await ctx.provider.playTts({
      callId,
      providerCallId: call.providerCallId,
      text,
-      voice,
+      voice: ctx.config.tts.voice,
    });

    return { success: true };
@@ -246,3 +244,4 @@ export async function endCall(
    return { success: false, error: err instanceof Error ? err.message : String(err) };
  }
 }
+
@@ -15,9 +15,9 @@ import type {
  WebhookVerificationResult,
 } from "../types.js";
 import { escapeXml, mapVoiceToPolly } from "../voice-mapping.js";
-import { chunkAudio } from "../telephony-audio.js";
-import type { TelephonyTtsProvider } from "../telephony-tts.js";
 import type { VoiceCallProvider } from "./base.js";
+import type { OpenAITTSProvider } from "./tts-openai.js";
+import { chunkAudio } from "./tts-openai.js";
 import { twilioApiRequest } from "./twilio/api.js";
 import { verifyTwilioProviderWebhook } from "./twilio/webhook.js";

@@ -53,8 +53,8 @@ export class TwilioProvider implements VoiceCallProvider {
  /** Current public webhook URL (set when tunnel starts or from config) */
  private currentPublicUrl: string | null = null;

-  /** Optional telephony TTS provider for streaming TTS */
-  private ttsProvider: TelephonyTtsProvider | null = null;
+  /** Optional OpenAI TTS provider for streaming TTS */
+  private ttsProvider: OpenAITTSProvider | null = null;

  /** Optional media stream handler for sending audio */
  private mediaStreamHandler: MediaStreamHandler | null = null;
@@ -119,7 +119,7 @@ export class TwilioProvider implements VoiceCallProvider {
    return this.currentPublicUrl;
  }

-  setTTSProvider(provider: TelephonyTtsProvider): void {
+  setTTSProvider(provider: OpenAITTSProvider): void {
    this.ttsProvider = provider;
  }

@@ -454,13 +454,13 @@ export class TwilioProvider implements VoiceCallProvider {
   * Play TTS audio via Twilio.
   *
   * Two modes:
-   * 1. Core TTS + Media Streams: If TTS provider and media stream are available,
-   *    generates audio via core TTS and streams it through WebSocket (preferred).
+   * 1. OpenAI TTS + Media Streams: If TTS provider and media stream are available,
+   *    generates audio via OpenAI and streams it through WebSocket (preferred).
   * 2. TwiML <Say>: Falls back to Twilio's native TTS with Polly voices.
   *    Note: This may not work on all Twilio accounts.
   */
  async playTts(input: PlayTtsInput): Promise<void> {
-    // Try telephony TTS via media stream first (if configured)
+    // Try OpenAI TTS via media stream first (if configured)
    const streamSid = this.callStreamMap.get(input.providerCallId);
    if (this.ttsProvider && this.mediaStreamHandler && streamSid) {
      try {
@@ -468,7 +468,7 @@ export class TwilioProvider implements VoiceCallProvider {
        return;
      } catch (err) {
        console.warn(
-          `[voice-call] Telephony TTS failed, falling back to Twilio <Say>:`,
+          `[voice-call] OpenAI TTS failed, falling back to Twilio <Say>:`,
          err instanceof Error ? err.message : err,
        );
        // Fall through to TwiML <Say> fallback
@@ -484,7 +484,7 @@ export class TwilioProvider implements VoiceCallProvider {
    }

    console.warn(
-      "[voice-call] Using TwiML <Say> fallback - telephony TTS not configured or media stream not active",
+      "[voice-call] Using TwiML <Say> fallback - OpenAI TTS not configured or media stream not active",
    );

    const pollyVoice = mapVoiceToPolly(input.voice);
@@ -502,8 +502,8 @@ export class TwilioProvider implements VoiceCallProvider {
  }

  /**
-   * Play TTS via core TTS and Twilio Media Streams.
-   * Generates audio with core TTS, converts to mu-law, and streams via WebSocket.
+   * Play TTS via OpenAI and Twilio Media Streams.
+   * Generates audio with OpenAI TTS, converts to mu-law, and streams via WebSocket.
   * Uses a jitter buffer to smooth out timing variations.
   */
  private async playTtsViaStream(
@@ -514,8 +514,8 @@ export class TwilioProvider implements VoiceCallProvider {
      throw new Error("TTS provider and media stream handler required");
    }

-    // Generate audio with core TTS (returns mu-law at 8kHz)
-    const muLawAudio = await this.ttsProvider.synthesizeForTelephony(text);
+    // Generate audio with OpenAI TTS (returns mu-law at 8kHz)
+    const muLawAudio = await this.ttsProvider.synthesizeForTwilio(text);

    // Stream audio in 20ms chunks (160 bytes at 8kHz mu-law)
    const CHUNK_SIZE = 160;
@@ -6,9 +6,8 @@ import type { VoiceCallProvider } from "./providers/base.js";
 import { MockProvider } from "./providers/mock.js";
 import { PlivoProvider } from "./providers/plivo.js";
 import { TelnyxProvider } from "./providers/telnyx.js";
+import { OpenAITTSProvider } from "./providers/tts-openai.js";
 import { TwilioProvider } from "./providers/twilio.js";
-import type { TelephonyTtsRuntime } from "./telephony-tts.js";
-import { createTelephonyTtsProvider } from "./telephony-tts.js";
 import { startTunnel, type TunnelResult } from "./tunnel.js";
 import {
  cleanupTailscaleExposure,
@@ -82,10 +81,9 @@ function resolveProvider(config: VoiceCallConfig): VoiceCallProvider {
 export async function createVoiceCallRuntime(params: {
  config: VoiceCallConfig;
  coreConfig: CoreConfig;
-  ttsRuntime?: TelephonyTtsRuntime;
  logger?: Logger;
 }): Promise<VoiceCallRuntime> {
-  const { config, coreConfig, ttsRuntime, logger } = params;
+  const { config, coreConfig, logger } = params;
  const log = logger ?? {
    info: console.log,
    warn: console.warn,
@@ -151,24 +149,27 @@ export async function createVoiceCallRuntime(params: {

  if (provider.name === "twilio" && config.streaming?.enabled) {
    const twilioProvider = provider as TwilioProvider;
-    if (ttsRuntime?.textToSpeechTelephony) {
+    const openaiApiKey =
+      config.streaming.openaiApiKey || process.env.OPENAI_API_KEY;
+    if (openaiApiKey) {
      try {
-        const ttsProvider = createTelephonyTtsProvider({
-          coreConfig,
-          ttsOverride: config.tts,
-          runtime: ttsRuntime,
+        const ttsProvider = new OpenAITTSProvider({
+          apiKey: openaiApiKey,
+          voice: config.tts.voice,
+          model: config.tts.model,
+          instructions: config.tts.instructions,
        });
        twilioProvider.setTTSProvider(ttsProvider);
-        log.info("[voice-call] Telephony TTS provider configured");
+        log.info("[voice-call] OpenAI TTS provider configured");
      } catch (err) {
        log.warn(
-          `[voice-call] Failed to initialize telephony TTS: ${
+          `[voice-call] Failed to initialize OpenAI TTS: ${
            err instanceof Error ? err.message : String(err)
          }`,
        );
      }
    } else {
-      log.warn("[voice-call] Telephony TTS unavailable; streaming TTS disabled");
+      log.warn("[voice-call] OpenAI TTS key missing; streaming TTS disabled");
    }

    const mediaHandler = webhookServer.getMediaStreamHandler();
@@ -1,88 +0,0 @@
-const TELEPHONY_SAMPLE_RATE = 8000;
-
-function clamp16(value: number): number {
-  return Math.max(-32768, Math.min(32767, value));
-}
-
-/**
- * Resample 16-bit PCM (little-endian mono) to 8kHz using linear interpolation.
- */
-export function resamplePcmTo8k(input: Buffer, inputSampleRate: number): Buffer {
-  if (inputSampleRate === TELEPHONY_SAMPLE_RATE) return input;
-  const inputSamples = Math.floor(input.length / 2);
-  if (inputSamples === 0) return Buffer.alloc(0);
-
-  const ratio = inputSampleRate / TELEPHONY_SAMPLE_RATE;
-  const outputSamples = Math.floor(inputSamples / ratio);
-  const output = Buffer.alloc(outputSamples * 2);
-
-  for (let i = 0; i < outputSamples; i++) {
-    const srcPos = i * ratio;
-    const srcIndex = Math.floor(srcPos);
-    const frac = srcPos - srcIndex;
-
-    const s0 = input.readInt16LE(srcIndex * 2);
-    const s1Index = Math.min(srcIndex + 1, inputSamples - 1);
-    const s1 = input.readInt16LE(s1Index * 2);
-
-    const sample = Math.round(s0 + frac * (s1 - s0));
-    output.writeInt16LE(clamp16(sample), i * 2);
-  }
-
-  return output;
-}
-
-/**
- * Convert 16-bit PCM to 8-bit mu-law (G.711).
- */
-export function pcmToMulaw(pcm: Buffer): Buffer {
-  const samples = Math.floor(pcm.length / 2);
-  const mulaw = Buffer.alloc(samples);
-
-  for (let i = 0; i < samples; i++) {
-    const sample = pcm.readInt16LE(i * 2);
-    mulaw[i] = linearToMulaw(sample);
-  }
-
-  return mulaw;
-}
-
-export function convertPcmToMulaw8k(
-  pcm: Buffer,
-  inputSampleRate: number,
-): Buffer {
-  const pcm8k = resamplePcmTo8k(pcm, inputSampleRate);
-  return pcmToMulaw(pcm8k);
-}
-
-/**
- * Chunk audio buffer into 20ms frames for streaming (8kHz mono mu-law).
- */
-export function chunkAudio(
-  audio: Buffer,
-  chunkSize = 160,
-): Generator<Buffer, void, unknown> {
-  return (function* () {
-    for (let i = 0; i < audio.length; i += chunkSize) {
-      yield audio.subarray(i, Math.min(i + chunkSize, audio.length));
-    }
-  })();
-}
-
-function linearToMulaw(sample: number): number {
-  const BIAS = 132;
-  const CLIP = 32635;
-
-  const sign = sample < 0 ? 0x80 : 0;
-  if (sample < 0) sample = -sample;
-  if (sample > CLIP) sample = CLIP;
-
-  sample += BIAS;
-  let exponent = 7;
-  for (let expMask = 0x4000; (sample & expMask) === 0 && exponent > 0; exponent--) {
-    expMask >>= 1;
-  }
-
-  const mantissa = (sample >> (exponent + 3)) & 0x0f;
-  return ~(sign | (exponent << 4) | mantissa) & 0xff;
-}
@@ -1,95 +0,0 @@
-import type { CoreConfig } from "./core-bridge.js";
-import type { VoiceCallTtsConfig } from "./config.js";
-import { convertPcmToMulaw8k } from "./telephony-audio.js";
-
-export type TelephonyTtsRuntime = {
-  textToSpeechTelephony: (params: {
-    text: string;
-    cfg: CoreConfig;
-    prefsPath?: string;
-  }) => Promise<{
-    success: boolean;
-    audioBuffer?: Buffer;
-    sampleRate?: number;
-    provider?: string;
-    error?: string;
-  }>;
-};
-
-export type TelephonyTtsProvider = {
-  synthesizeForTelephony: (text: string) => Promise<Buffer>;
-};
-
-export function createTelephonyTtsProvider(params: {
-  coreConfig: CoreConfig;
-  ttsOverride?: VoiceCallTtsConfig;
-  runtime: TelephonyTtsRuntime;
-}): TelephonyTtsProvider {
-  const { coreConfig, ttsOverride, runtime } = params;
-  const mergedConfig = applyTtsOverride(coreConfig, ttsOverride);
-
-  return {
-    synthesizeForTelephony: async (text: string) => {
-      const result = await runtime.textToSpeechTelephony({
-        text,
-        cfg: mergedConfig,
-      });
-
-      if (!result.success || !result.audioBuffer || !result.sampleRate) {
-        throw new Error(result.error ?? "TTS conversion failed");
-      }
-
-      return convertPcmToMulaw8k(result.audioBuffer, result.sampleRate);
-    },
-  };
-}
-
-function applyTtsOverride(
-  coreConfig: CoreConfig,
-  override?: VoiceCallTtsConfig,
-): CoreConfig {
-  if (!override) return coreConfig;
-
-  const base = coreConfig.messages?.tts;
-  const merged = mergeTtsConfig(base, override);
-  if (!merged) return coreConfig;
-
-  return {
-    ...coreConfig,
-    messages: {
-      ...(coreConfig.messages ?? {}),
-      tts: merged,
-    },
-  };
-}
-
-function mergeTtsConfig(
-  base?: VoiceCallTtsConfig,
-  override?: VoiceCallTtsConfig,
-): VoiceCallTtsConfig | undefined {
-  if (!base && !override) return undefined;
-  if (!override) return base;
-  if (!base) return override;
-  return deepMerge(base, override);
-}
-
-function deepMerge<T>(base: T, override: T): T {
-  if (!isPlainObject(base) || !isPlainObject(override)) {
-    return override;
-  }
-  const result: Record<string, unknown> = { ...base };
-  for (const [key, value] of Object.entries(override)) {
-    if (value === undefined) continue;
-    const existing = (base as Record<string, unknown>)[key];
-    if (isPlainObject(existing) && isPlainObject(value)) {
-      result[key] = deepMerge(existing, value);
-    } else {
-      result[key] = value;
-    }
-  }
-  return result as T;
-}
-
-function isPlainObject(value: unknown): value is Record<string, unknown> {
-  return Boolean(value) && typeof value === "object" && !Array.isArray(value);
-}
@@ -10,17 +10,11 @@ describe("extractModelDirective", () => {
      expect(result.cleaned).toBe("");
    });

-    it("does not treat /models as a /model directive", () => {
+    it("extracts /models with argument", () => {
      const result = extractModelDirective("/models gpt-5");
-      expect(result.hasDirective).toBe(false);
-      expect(result.rawModel).toBeUndefined();
-      expect(result.cleaned).toBe("/models gpt-5");
-    });
-
-    it("does not parse /models as a /model directive (no args)", () => {
-      const result = extractModelDirective("/models");
-      expect(result.hasDirective).toBe(false);
-      expect(result.cleaned).toBe("/models");
+      expect(result.hasDirective).toBe(true);
+      expect(result.rawModel).toBe("gpt-5");
+      expect(result.cleaned).toBe("");
    });

    it("extracts /model with provider/model format", () => {
@@ -14,7 +14,7 @@ export function extractModelDirective(
  if (!body) return { cleaned: "", hasDirective: false };

  const modelMatch = body.match(
-    /(?:^|\s)\/model(?=$|\s|:)\s*:?\s*([A-Za-z0-9_.:@-]+(?:\/[A-Za-z0-9_.:@-]+)*)?/i,
+    /(?:^|\s)\/models?(?=$|\s|:)\s*:?\s*([A-Za-z0-9_.:@-]+(?:\/[A-Za-z0-9_.:@-]+)*)?/i,
  );

  const aliases = (options?.aliases ?? []).map((alias) => alias.trim()).filter(Boolean);
@@ -59,17 +59,6 @@ export const CONFIG_PATH_CLAWDBOT = resolveConfigPath();

 export const DEFAULT_GATEWAY_PORT = 18789;

-/**
- * Gateway lock directory (ephemeral).
- * Default: os.tmpdir()/clawdbot-<uid> (uid suffix when available).
- */
-export function resolveGatewayLockDir(tmpdir: () => string = os.tmpdir): string {
-  const base = tmpdir();
-  const uid = typeof process.getuid === "function" ? process.getuid() : undefined;
-  const suffix = uid != null ? `clawdbot-${uid}` : "clawdbot";
-  return path.join(base, suffix);
-}
-
 const OAUTH_FILENAME = "oauth.json";

 /**
@@ -56,6 +56,7 @@ export type TtsConfig = {
  /** OpenAI configuration. */
  openai?: {
    apiKey?: string;
+    baseUrl?: string;
    model?: string;
    voice?: string;
  };
@@ -204,6 +204,7 @@ export const TtsConfigSchema = z
    openai: z
      .object({
        apiKey: z.string().optional(),
+        baseUrl: z.string().optional(),
        model: z.string().optional(),
        voice: z.string().optional(),
      })
@@ -131,7 +131,7 @@ export const ttsHandlers: GatewayRequestHandlers = {
          {
            id: "openai",
            name: "OpenAI",
-            configured: Boolean(resolveTtsApiKey(config, "openai")),
+            configured: isTtsProviderConfigured(config, "openai"),
            models: [...OPENAI_TTS_MODELS],
            voices: [...OPENAI_TTS_VOICES],
          },
@@ -7,13 +7,12 @@ import path from "node:path";
 import { describe, expect, it, vi } from "vitest";

 import { acquireGatewayLock, GatewayLockError } from "./gateway-lock.js";
-import { resolveConfigPath, resolveGatewayLockDir, resolveStateDir } from "../config/paths.js";
+import { resolveConfigPath, resolveStateDir } from "../config/paths.js";

 async function makeEnv() {
  const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-gateway-lock-"));
  const configPath = path.join(dir, "clawdbot.json");
  await fs.writeFile(configPath, "{}", "utf8");
-  await fs.mkdir(resolveGatewayLockDir(), { recursive: true });
  return {
    env: {
      ...process.env,
@@ -30,8 +29,7 @@ function resolveLockPath(env: NodeJS.ProcessEnv) {
  const stateDir = resolveStateDir(env);
  const configPath = resolveConfigPath(env, stateDir);
  const hash = createHash("sha1").update(configPath).digest("hex").slice(0, 8);
-  const lockDir = resolveGatewayLockDir();
-  return { lockPath: path.join(lockDir, `gateway.${hash}.lock`), configPath };
+  return { lockPath: path.join(stateDir, `gateway.${hash}.lock`), configPath };
 }

 function makeProcStat(pid: number, startTime: number) {
@@ -3,7 +3,7 @@ import fs from "node:fs/promises";
 import fsSync from "node:fs";
 import path from "node:path";

-import { resolveConfigPath, resolveGatewayLockDir, resolveStateDir } from "../config/paths.js";
+import { resolveConfigPath, resolveStateDir } from "../config/paths.js";

 const DEFAULT_TIMEOUT_MS = 5000;
 const DEFAULT_POLL_INTERVAL_MS = 100;
@@ -150,8 +150,7 @@ function resolveGatewayLockPath(env: NodeJS.ProcessEnv) {
  const stateDir = resolveStateDir(env);
  const configPath = resolveConfigPath(env, stateDir);
  const hash = createHash("sha1").update(configPath).digest("hex").slice(0, 8);
-  const lockDir = resolveGatewayLockDir();
-  const lockPath = path.join(lockDir, `gateway.${hash}.lock`);
+  const lockPath = path.join(stateDir, `gateway.${hash}.lock`);
  return { lockPath, configPath };
 }

@@ -124,7 +124,6 @@ import { startWebLoginWithQr, waitForWebLogin } from "../../web/login-qr.js";
 import { sendMessageWhatsApp, sendPollWhatsApp } from "../../web/outbound.js";
 import { registerMemoryCli } from "../../cli/memory-cli.js";
 import { formatNativeDependencyHint } from "./native-deps.js";
-import { textToSpeechTelephony } from "../../tts/tts.js";

 import type { PluginRuntime } from "./types.js";

@@ -163,9 +162,6 @@ export function createPluginRuntime(): PluginRuntime {
      getImageMetadata,
      resizeToJpeg,
    },
-    tts: {
-      textToSpeechTelephony,
-    },
    tools: {
      createMemoryGetTool,
      createMemorySearchTool,
@@ -16,7 +16,6 @@ type UpsertChannelPairingRequest =
  typeof import("../../pairing/pairing-store.js").upsertChannelPairingRequest;
 type FetchRemoteMedia = typeof import("../../media/fetch.js").fetchRemoteMedia;
 type SaveMediaBuffer = typeof import("../../media/store.js").saveMediaBuffer;
-type TextToSpeechTelephony = typeof import("../../tts/tts.js").textToSpeechTelephony;
 type BuildMentionRegexes = typeof import("../../auto-reply/reply/mentions.js").buildMentionRegexes;
 type MatchesMentionPatterns =
  typeof import("../../auto-reply/reply/mentions.js").matchesMentionPatterns;
@@ -174,9 +173,6 @@ export type PluginRuntime = {
    getImageMetadata: GetImageMetadata;
    resizeToJpeg: ResizeToJpeg;
  };
-  tts: {
-    textToSpeechTelephony: TextToSpeechTelephony;
-  };
  tools: {
    createMemoryGetTool: CreateMemoryGetTool;
    createMemorySearchTool: CreateMemorySearchTool;
@@ -43,7 +43,6 @@ function setup(config: Record<string, unknown>): Registered {
    source: "test",
    config: {},
    pluginConfig: config,
-    runtime: { tts: { textToSpeechTelephony: vi.fn() } },
    logger: noopLogger,
    registerGatewayMethod: (method, handler) => methods.set(method, handler),
    registerTool: (tool) => tools.push(tool),
@@ -143,7 +142,6 @@ describe("voice-call plugin", () => {
      source: "test",
      config: {},
      pluginConfig: { provider: "mock" },
-      runtime: { tts: { textToSpeechTelephony: vi.fn() } },
      logger: noopLogger,
      registerGatewayMethod: () => {},
      registerTool: () => {},
@@ -106,6 +106,10 @@ describe("tts", () => {
      expect(isValidOpenAIVoice("alloy ")).toBe(false);
      expect(isValidOpenAIVoice(" alloy")).toBe(false);
    });
+
+    it("accepts custom voices for custom endpoints", () => {
+      expect(isValidOpenAIVoice("zm_yunxia", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
+    });
  });

  describe("isValidOpenAIModel", () => {
@@ -115,11 +119,15 @@ describe("tts", () => {
      expect(isValidOpenAIModel("tts-1-hd")).toBe(true);
    });

-    it("rejects unsupported models", () => {
+    it("rejects other models", () => {
      expect(isValidOpenAIModel("invalid")).toBe(false);
      expect(isValidOpenAIModel("")).toBe(false);
      expect(isValidOpenAIModel("gpt-4")).toBe(false);
    });
+
+    it("accepts custom models for custom endpoints", () => {
+      expect(isValidOpenAIModel("kokoro", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
+    });
  });

  describe("OPENAI_TTS_MODELS", () => {
@@ -202,6 +210,30 @@ describe("tts", () => {
      expect(result.overrides.provider).toBe("edge");
    });

+    it("routes custom OpenAI models when custom endpoints are used", () => {
+      const policy = resolveModelOverridePolicy({ enabled: true });
+      const input = "Hello [[tts:provider=openai model=kokoro voice=zm_yunxia]] world";
+      const result = parseTtsDirectives(input, policy, {
+        defaultProvider: "openai",
+        openaiBaseUrl: "http://localhost:8880/v1",
+      });
+
+      expect(result.overrides.openai?.model).toBe("kokoro");
+      expect(result.overrides.openai?.voice).toBe("zm_yunxia");
+    });
+
+    it("routes model overrides to ElevenLabs when provider is ElevenLabs", () => {
+      const policy = resolveModelOverridePolicy({ enabled: true });
+      const input = "Hello [[tts:provider=elevenlabs model=eleven_multilingual_v2]] world";
+      const result = parseTtsDirectives(input, policy, {
+        defaultProvider: "openai",
+        openaiBaseUrl: "http://localhost:8880/v1",
+      });
+
+      expect(result.overrides.elevenlabs?.modelId).toBe("eleven_multilingual_v2");
+      expect(result.overrides.openai?.model).toBeUndefined();
+    });
+
    it("keeps text intact when overrides are disabled", () => {
      const policy = resolveModelOverridePolicy({ enabled: false });
      const input = "Hello [[tts:voice=alloy]] world";
@@ -359,7 +391,12 @@ describe("tts", () => {
    };

    const restoreEnv = (snapshot: Record<string, string | undefined>) => {
-      const keys = ["OPENAI_API_KEY", "ELEVENLABS_API_KEY", "XI_API_KEY"] as const;
+      const keys = [
+        "OPENAI_API_KEY",
+        "OPENAI_TTS_BASE_URL",
+        "ELEVENLABS_API_KEY",
+        "XI_API_KEY",
+      ] as const;
      for (const key of keys) {
        const value = snapshot[key];
        if (value === undefined) {
@@ -373,6 +410,7 @@ describe("tts", () => {
    const withEnv = (env: Record<string, string | undefined>, run: () => void) => {
      const snapshot = {
        OPENAI_API_KEY: process.env.OPENAI_API_KEY,
+        OPENAI_TTS_BASE_URL: process.env.OPENAI_TTS_BASE_URL,
        ELEVENLABS_API_KEY: process.env.ELEVENLABS_API_KEY,
        XI_API_KEY: process.env.XI_API_KEY,
      };
@@ -394,6 +432,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: "test-openai-key",
+          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: undefined,
          XI_API_KEY: undefined,
        },
@@ -409,6 +448,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: undefined,
+          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: "test-elevenlabs-key",
          XI_API_KEY: undefined,
        },
@@ -424,6 +464,7 @@ describe("tts", () => {
      withEnv(
        {
          OPENAI_API_KEY: undefined,
+          OPENAI_TTS_BASE_URL: undefined,
          ELEVENLABS_API_KEY: undefined,
          XI_API_KEY: undefined,
        },
@@ -434,6 +475,22 @@ describe("tts", () => {
        },
      );
    });
+
+    it("prefers OpenAI when a custom endpoint is configured without API keys", () => {
+      withEnv(
+        {
+          OPENAI_API_KEY: undefined,
+          OPENAI_TTS_BASE_URL: "http://localhost:8880/v1",
+          ELEVENLABS_API_KEY: undefined,
+          XI_API_KEY: undefined,
+        },
+        () => {
+          const config = resolveTtsConfig(baseCfg);
+          const provider = getTtsProvider(config, "/tmp/tts-prefs-openai-custom.json");
+          expect(provider).toBe("openai");
+        },
+      );
+    });
  });

  describe("maybeApplyTtsToPayload", () => {
@@ -497,6 +554,55 @@ describe("tts", () => {
      process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
    });

+    it("allows custom OpenAI endpoints without API keys", async () => {
+      const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
+      const prevBaseUrl = process.env.OPENAI_TTS_BASE_URL;
+      const prevOpenAiKey = process.env.OPENAI_API_KEY;
+      process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
+      process.env.OPENAI_TTS_BASE_URL = "http://localhost:8880/v1";
+      delete process.env.OPENAI_API_KEY;
+      const originalFetch = globalThis.fetch;
+      const fetchMock = vi.fn(async () => ({
+        ok: true,
+        arrayBuffer: async () => new ArrayBuffer(1),
+      }));
+      globalThis.fetch = fetchMock as unknown as typeof fetch;
+
+      const cfg = {
+        agents: { defaults: { model: { primary: "openai/gpt-4o-mini" } } },
+        messages: {
+          tts: {
+            auto: "always",
+            provider: "openai",
+            openai: { model: "kokoro", voice: "zm_yunxia" },
+          },
+        },
+      };
+
+      const result = await maybeApplyTtsToPayload({
+        payload: { text: "Hello world" },
+        cfg,
+        kind: "final",
+      });
+
+      expect(result.mediaUrl).toBeDefined();
+      expect(fetchMock).toHaveBeenCalledTimes(1);
+      const [url, options] = fetchMock.mock.calls[0] ?? [];
+      expect(url).toBe("http://localhost:8880/v1/audio/speech");
+      expect(
+        (options as { headers?: Record<string, string> })?.headers?.Authorization,
+      ).toBeUndefined();
+
+      globalThis.fetch = originalFetch;
+      process.env.OPENAI_TTS_BASE_URL = prevBaseUrl;
+      if (prevOpenAiKey === undefined) {
+        delete process.env.OPENAI_API_KEY;
+      } else {
+        process.env.OPENAI_API_KEY = prevOpenAiKey;
+      }
+      process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
+    });
+
    it("skips auto-TTS in tagged mode unless a tts tag is present", async () => {
      const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
      process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
@@ -48,6 +48,7 @@ const DEFAULT_ELEVENLABS_VOICE_ID = "pMsXgVXv3BLzUgSXRplE";
 const DEFAULT_ELEVENLABS_MODEL_ID = "eleven_multilingual_v2";
 const DEFAULT_OPENAI_MODEL = "gpt-4o-mini-tts";
 const DEFAULT_OPENAI_VOICE = "alloy";
+const DEFAULT_OPENAI_TTS_BASE_URL = "https://api.openai.com/v1";
 const DEFAULT_EDGE_VOICE = "en-US-MichelleNeural";
 const DEFAULT_EDGE_LANG = "en-US";
 const DEFAULT_EDGE_OUTPUT_FORMAT = "audio-24khz-48kbitrate-mono-mp3";
@@ -76,11 +77,6 @@ const DEFAULT_OUTPUT = {
  voiceCompatible: false,
 };

-const TELEPHONY_OUTPUT = {
-  openai: { format: "pcm" as const, sampleRate: 24000 },
-  elevenlabs: { format: "pcm_22050", sampleRate: 22050 },
-};
-
 const TTS_AUTO_MODES = new Set<TtsAutoMode>(["off", "always", "inbound", "tagged"]);

 export type ResolvedTtsConfig = {
@@ -108,6 +104,7 @@ export type ResolvedTtsConfig = {
  };
  openai: {
    apiKey?: string;
+    baseUrl: string;
    model: string;
    voice: string;
  };
@@ -185,16 +182,6 @@ export type TtsResult = {
  voiceCompatible?: boolean;
 };

-export type TtsTelephonyResult = {
-  success: boolean;
-  audioBuffer?: Buffer;
-  error?: string;
-  latencyMs?: number;
-  provider?: string;
-  outputFormat?: string;
-  sampleRate?: number;
-};
-
 type TtsStatusEntry = {
  timestamp: number;
  success: boolean;
@@ -249,6 +236,9 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
  const raw: TtsConfig = cfg.messages?.tts ?? {};
  const providerSource = raw.provider ? "config" : "default";
  const edgeOutputFormat = raw.edge?.outputFormat?.trim();
+  const openaiBaseUrl = normalizeOpenAiTtsBaseUrl(
+    raw.openai?.baseUrl?.trim() || process.env.OPENAI_TTS_BASE_URL,
+  );
  const auto = normalizeTtsAutoMode(raw.auto) ?? (raw.enabled ? "always" : "off");
  return {
    auto,
@@ -280,6 +270,7 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
    },
    openai: {
      apiKey: raw.openai?.apiKey,
+      baseUrl: openaiBaseUrl,
      model: raw.openai?.model ?? DEFAULT_OPENAI_MODEL,
      voice: raw.openai?.voice ?? DEFAULT_OPENAI_VOICE,
    },
@@ -410,7 +401,9 @@ export function getTtsProvider(config: ResolvedTtsConfig, prefsPath: string): Tt
  if (prefs.tts?.provider) return prefs.tts.provider;
  if (config.providerSource === "config") return config.provider;

-  if (resolveTtsApiKey(config, "openai")) return "openai";
+  if (resolveTtsApiKey(config, "openai") || isCustomOpenAiTtsEndpoint(config.openai.baseUrl)) {
+    return "openai";
+  }
  if (resolveTtsApiKey(config, "elevenlabs")) return "elevenlabs";
  return "edge";
 }
@@ -485,6 +478,12 @@ export function resolveTtsProviderOrder(primary: TtsProvider): TtsProvider[] {

 export function isTtsProviderConfigured(config: ResolvedTtsConfig, provider: TtsProvider): boolean {
  if (provider === "edge") return config.edge.enabled;
+  if (provider === "openai") {
+    return (
+      Boolean(resolveTtsApiKey(config, provider)) ||
+      isCustomOpenAiTtsEndpoint(config.openai.baseUrl)
+    );
+  }
  return Boolean(resolveTtsApiKey(config, provider));
 }

@@ -498,6 +497,16 @@ function normalizeElevenLabsBaseUrl(baseUrl: string): string {
  return trimmed.replace(/\/+$/, "");
 }

+function normalizeOpenAiTtsBaseUrl(baseUrl?: string): string {
+  const trimmed = baseUrl?.trim();
+  if (!trimmed) return DEFAULT_OPENAI_TTS_BASE_URL;
+  return trimmed.replace(/\/+$/, "");
+}
+
+function isCustomOpenAiTtsEndpoint(baseUrl: string): boolean {
+  return normalizeOpenAiTtsBaseUrl(baseUrl) !== DEFAULT_OPENAI_TTS_BASE_URL;
+}
+
 function requireInRange(value: number, min: number, max: number, label: string): void {
  if (!Number.isFinite(value) || value < min || value > max) {
    throw new Error(`${label} must be between ${min} and ${max}`);
@@ -553,6 +562,7 @@ function parseNumberValue(value: string): number | undefined {
 function parseTtsDirectives(
  text: string,
  policy: ResolvedTtsModelOverrides,
+  options: { defaultProvider?: TtsProvider; openaiBaseUrl?: string } = {},
 ): TtsDirectiveParseResult {
  if (!policy.enabled) {
    return { cleanedText: text, overrides: {}, warnings: [], hasDirective: false };
@@ -576,6 +586,30 @@ function parseTtsDirectives(
  cleanedText = cleanedText.replace(directiveRegex, (_match, body: string) => {
    hasDirective = true;
    const tokens = body.split(/\s+/).filter(Boolean);
+    let providerOverride: TtsProvider | undefined;
+
+    for (const token of tokens) {
+      const eqIndex = token.indexOf("=");
+      if (eqIndex === -1) continue;
+      const rawKey = token.slice(0, eqIndex).trim();
+      const rawValue = token.slice(eqIndex + 1).trim();
+      if (!rawKey || !rawValue) continue;
+      if (rawKey.toLowerCase() !== "provider") continue;
+      if (!policy.allowProvider) break;
+      if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
+        providerOverride = rawValue;
+      } else {
+        warnings.push(`unsupported provider "${rawValue}"`);
+      }
+      break;
+    }
+
+    if (providerOverride) {
+      overrides.provider = providerOverride;
+    }
+
+    const providerHint = overrides.provider ?? options.defaultProvider;
+    const openAiContext = { baseUrl: options.openaiBaseUrl };
    for (const token of tokens) {
      const eqIndex = token.indexOf("=");
      if (eqIndex === -1) continue;
@@ -586,18 +620,12 @@ function parseTtsDirectives(
      try {
        switch (key) {
          case "provider":
-            if (!policy.allowProvider) break;
-            if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
-              overrides.provider = rawValue;
-            } else {
-              warnings.push(`unsupported provider "${rawValue}"`);
-            }
            break;
          case "voice":
          case "openai_voice":
          case "openaivoice":
            if (!policy.allowVoice) break;
-            if (isValidOpenAIVoice(rawValue)) {
+            if (isValidOpenAIVoice(rawValue, openAiContext)) {
              overrides.openai = { ...overrides.openai, voice: rawValue };
            } else {
              warnings.push(`invalid OpenAI voice "${rawValue}"`);
@@ -617,17 +645,40 @@ function parseTtsDirectives(
          case "model":
          case "modelid":
          case "model_id":
-          case "elevenlabs_model":
-          case "elevenlabsmodel":
          case "openai_model":
          case "openaimodel":
            if (!policy.allowModelId) break;
-            if (isValidOpenAIModel(rawValue)) {
+            if (key === "openai_model" || key === "openaimodel") {
+              if (isValidOpenAIModel(rawValue, openAiContext)) {
+                overrides.openai = { ...overrides.openai, model: rawValue };
+              } else {
+                warnings.push(`invalid OpenAI model "${rawValue}"`);
+              }
+              break;
+            }
+            if (providerHint === "openai") {
+              if (isValidOpenAIModel(rawValue, openAiContext)) {
+                overrides.openai = { ...overrides.openai, model: rawValue };
+              } else {
+                warnings.push(`invalid OpenAI model "${rawValue}"`);
+              }
+              break;
+            }
+            if (providerHint === "elevenlabs") {
+              overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
+              break;
+            }
+            if (isValidOpenAIModel(rawValue, openAiContext)) {
              overrides.openai = { ...overrides.openai, model: rawValue };
            } else {
              overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
            }
            break;
+          case "elevenlabs_model":
+          case "elevenlabsmodel":
+            if (!policy.allowModelId) break;
+            overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
+            break;
          case "stability":
            if (!policy.allowVoiceSettings) break;
            {
@@ -752,16 +803,6 @@ function parseTtsDirectives(
 }

 export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts", "tts-1", "tts-1-hd"] as const;
-
-/**
- * Custom OpenAI-compatible TTS endpoint.
- * When set, model/voice validation is relaxed to allow non-OpenAI models.
- * Example: OPENAI_TTS_BASE_URL=http://localhost:8880/v1
- */
-const OPENAI_TTS_BASE_URL = (
-  process.env.OPENAI_TTS_BASE_URL?.trim() || "https://api.openai.com/v1"
-).replace(/\/+$/, "");
-const isCustomOpenAIEndpoint = OPENAI_TTS_BASE_URL !== "https://api.openai.com/v1";
 export const OPENAI_TTS_VOICES = [
  "alloy",
  "ash",
@@ -775,16 +816,15 @@ export const OPENAI_TTS_VOICES = [
 ] as const;

 type OpenAiTtsVoice = (typeof OPENAI_TTS_VOICES)[number];
+type OpenAiTtsValidationContext = { baseUrl?: string };

-function isValidOpenAIModel(model: string): boolean {
-  // Allow any model when using custom endpoint (e.g., Kokoro, LocalAI)
-  if (isCustomOpenAIEndpoint) return true;
+function isValidOpenAIModel(model: string, context: OpenAiTtsValidationContext = {}): boolean {
+  if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
  return OPENAI_TTS_MODELS.includes(model as (typeof OPENAI_TTS_MODELS)[number]);
 }

-function isValidOpenAIVoice(voice: string): voice is OpenAiTtsVoice {
-  // Allow any voice when using custom endpoint (e.g., Kokoro Chinese voices)
-  if (isCustomOpenAIEndpoint) return true;
+function isValidOpenAIVoice(voice: string, context: OpenAiTtsValidationContext = {}): boolean {
+  if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
  return OPENAI_TTS_VOICES.includes(voice as OpenAiTtsVoice);
 }

@@ -992,18 +1032,21 @@ async function elevenLabsTTS(params: {

 async function openaiTTS(params: {
  text: string;
-  apiKey: string;
+  apiKey?: string;
+  baseUrl: string;
  model: string;
  voice: string;
-  responseFormat: "mp3" | "opus" | "pcm";
+  responseFormat: "mp3" | "opus";
  timeoutMs: number;
 }): Promise<Buffer> {
-  const { text, apiKey, model, voice, responseFormat, timeoutMs } = params;
+  const { text, model, voice, responseFormat, timeoutMs } = params;
+  const apiKey = params.apiKey?.trim();
+  const baseUrl = normalizeOpenAiTtsBaseUrl(params.baseUrl);

-  if (!isValidOpenAIModel(model)) {
+  if (!isValidOpenAIModel(model, { baseUrl })) {
    throw new Error(`Invalid model: ${model}`);
  }
-  if (!isValidOpenAIVoice(voice)) {
+  if (!isValidOpenAIVoice(voice, { baseUrl })) {
    throw new Error(`Invalid voice: ${voice}`);
  }

@@ -1011,12 +1054,15 @@ async function openaiTTS(params: {
  const timeout = setTimeout(() => controller.abort(), timeoutMs);

  try {
-    const response = await fetch(`${OPENAI_TTS_BASE_URL}/audio/speech`, {
+    const headers: Record<string, string> = {
+      "Content-Type": "application/json",
+    };
+    if (apiKey) {
+      headers.Authorization = `Bearer ${apiKey}`;
+    }
+    const response = await fetch(`${baseUrl}/audio/speech`, {
      method: "POST",
-      headers: {
-        Authorization: `Bearer ${apiKey}`,
-        "Content-Type": "application/json",
-      },
+      headers,
      body: JSON.stringify({
        model,
        input: text,
@@ -1166,13 +1212,19 @@ export async function textToSpeech(params: {
      }

      const apiKey = resolveTtsApiKey(config, provider);
-      if (!apiKey) {
+      const allowMissingKey =
+        provider === "openai" && isCustomOpenAiTtsEndpoint(config.openai.baseUrl);
+      if (!apiKey && !allowMissingKey) {
        lastError = `No API key for ${provider}`;
        continue;
      }

      let audioBuffer: Buffer;
      if (provider === "elevenlabs") {
+        if (!apiKey) {
+          lastError = "No API key for elevenlabs";
+          continue;
+        }
        const voiceIdOverride = params.overrides?.elevenlabs?.voiceId;
        const modelIdOverride = params.overrides?.elevenlabs?.modelId;
        const voiceSettings = {
@@ -1201,6 +1253,7 @@ export async function textToSpeech(params: {
        audioBuffer = await openaiTTS({
          text: params.text,
          apiKey,
+          baseUrl: config.openai.baseUrl,
          model: openaiModelOverride ?? config.openai.model,
          voice: openaiVoiceOverride ?? config.openai.voice,
          responseFormat: output.openai,
@@ -1239,100 +1292,6 @@ export async function textToSpeech(params: {
  };
 }

-export async function textToSpeechTelephony(params: {
-  text: string;
-  cfg: ClawdbotConfig;
-  prefsPath?: string;
-}): Promise<TtsTelephonyResult> {
-  const config = resolveTtsConfig(params.cfg);
-  const prefsPath = params.prefsPath ?? resolveTtsPrefsPath(config);
-
-  if (params.text.length > config.maxTextLength) {
-    return {
-      success: false,
-      error: `Text too long (${params.text.length} chars, max ${config.maxTextLength})`,
-    };
-  }
-
-  const userProvider = getTtsProvider(config, prefsPath);
-  const providers = resolveTtsProviderOrder(userProvider);
-
-  let lastError: string | undefined;
-
-  for (const provider of providers) {
-    const providerStart = Date.now();
-    try {
-      if (provider === "edge") {
-        lastError = "edge: unsupported for telephony";
-        continue;
-      }
-
-      const apiKey = resolveTtsApiKey(config, provider);
-      if (!apiKey) {
-        lastError = `No API key for ${provider}`;
-        continue;
-      }
-
-      if (provider === "elevenlabs") {
-        const output = TELEPHONY_OUTPUT.elevenlabs;
-        const audioBuffer = await elevenLabsTTS({
-          text: params.text,
-          apiKey,
-          baseUrl: config.elevenlabs.baseUrl,
-          voiceId: config.elevenlabs.voiceId,
-          modelId: config.elevenlabs.modelId,
-          outputFormat: output.format,
-          seed: config.elevenlabs.seed,
-          applyTextNormalization: config.elevenlabs.applyTextNormalization,
-          languageCode: config.elevenlabs.languageCode,
-          voiceSettings: config.elevenlabs.voiceSettings,
-          timeoutMs: config.timeoutMs,
-        });
-
-        return {
-          success: true,
-          audioBuffer,
-          latencyMs: Date.now() - providerStart,
-          provider,
-          outputFormat: output.format,
-          sampleRate: output.sampleRate,
-        };
-      }
-
-      const output = TELEPHONY_OUTPUT.openai;
-      const audioBuffer = await openaiTTS({
-        text: params.text,
-        apiKey,
-        model: config.openai.model,
-        voice: config.openai.voice,
-        responseFormat: output.format,
-        timeoutMs: config.timeoutMs,
-      });
-
-      return {
-        success: true,
-        audioBuffer,
-        latencyMs: Date.now() - providerStart,
-        provider,
-        outputFormat: output.format,
-        sampleRate: output.sampleRate,
-      };
-    } catch (err) {
-      const error = err as Error;
-      if (error.name === "AbortError") {
-        lastError = `${provider}: request timed out`;
-      } else {
-        lastError = `${provider}: ${error.message}`;
-      }
-    }
-  }
-
-  return {
-    success: false,
-    error: `TTS conversion failed: ${lastError || "no providers available"}`,
-  };
-}
-
 export async function maybeApplyTtsToPayload(params: {
  payload: ReplyPayload;
  cfg: ClawdbotConfig;
@@ -1350,8 +1309,12 @@ export async function maybeApplyTtsToPayload(params: {
  });
  if (autoMode === "off") return params.payload;

+  const defaultProvider = getTtsProvider(config, prefsPath);
  const text = params.payload.text ?? "";
-  const directives = parseTtsDirectives(text, config.modelOverrides);
+  const directives = parseTtsDirectives(text, config.modelOverrides, {
+    defaultProvider,
+    openaiBaseUrl: config.openai.baseUrl,
+  });
  if (directives.warnings.length > 0) {
    logVerbose(`TTS: ignored directive overrides (${directives.warnings.join("; ")})`);
  }
Author	SHA1	Message	Date
Peter Steinberger	d649710627	fix: support custom TTS endpoints (#1701 ) (thanks @louzhixian)	2026-01-25 08:01:11 +00:00
zhixian	498c15555e	test: update tests for expanded OpenAI TTS model list - Accept tts-1 and tts-1-hd as valid models - Update OPENAI_TTS_MODELS length expectation to 3	2026-01-25 07:59:20 +00:00
zhixian	25345a909a	style: format code with oxfmt	2026-01-25 07:59:20 +00:00
zhixian	bfd8321b42	fix: strip trailing slashes from OPENAI_TTS_BASE_URL Address review feedback: normalize the base URL by removing trailing slashes to prevent double-slash paths like /v1//audio/speech which cause 404 errors on some OpenAI-compatible servers.	2026-01-25 07:59:20 +00:00
zhixian	fbbed439bd	feat(tts): support custom OpenAI-compatible TTS endpoints Add OPENAI_TTS_BASE_URL environment variable to allow using self-hosted or third-party OpenAI-compatible TTS services like Kokoro, LocalAI, or OpenedAI-Speech. Changes: - Add OPENAI_TTS_BASE_URL env var (defaults to OpenAI official API) - Relax model/voice validation when using custom endpoints - Add tts-1 and tts-1-hd to the model allowlist This enables users to: - Use local TTS for privacy and cost savings - Use models with better non-English language support (Chinese, Japanese) - Reduce latency with local inference Example usage: OPENAI_TTS_BASE_URL=http://localhost:8880/v1 Tested with Kokoro-FastAPI.	2026-01-25 07:59:20 +00:00