Compare commits

..
Author SHA1 Message Date
Peter Steinberger d649710627 fix: support custom TTS endpoints (#1701) (thanks @louzhixian) 2026-01-25 08:01:11 +00:00
zhixian 498c15555e test: update tests for expanded OpenAI TTS model list
- Accept tts-1 and tts-1-hd as valid models
- Update OPENAI_TTS_MODELS length expectation to 3
2026-01-25 07:59:20 +00:00
zhixian 25345a909a style: format code with oxfmt 2026-01-25 07:59:20 +00:00
zhixian bfd8321b42 fix: strip trailing slashes from OPENAI_TTS_BASE_URL
Address review feedback: normalize the base URL by removing trailing
slashes to prevent double-slash paths like /v1//audio/speech which
cause 404 errors on some OpenAI-compatible servers.
2026-01-25 07:59:20 +00:00
zhixian fbbed439bd feat(tts): support custom OpenAI-compatible TTS endpoints
Add OPENAI_TTS_BASE_URL environment variable to allow using self-hosted
or third-party OpenAI-compatible TTS services like Kokoro, LocalAI, or
OpenedAI-Speech.

Changes:
- Add OPENAI_TTS_BASE_URL env var (defaults to OpenAI official API)
- Relax model/voice validation when using custom endpoints
- Add tts-1 and tts-1-hd to the model allowlist

This enables users to:
- Use local TTS for privacy and cost savings
- Use models with better non-English language support (Chinese, Japanese)
- Reduce latency with local inference

Example usage:
  OPENAI_TTS_BASE_URL=http://localhost:8880/v1

Tested with Kokoro-FastAPI.
2026-01-25 07:59:20 +00:00
Peter Steinberger 653401774d fix(telegram): honor linkPreview on fallback (#1730)
* feat: add notice directive parsing

* fix: honor telegram linkPreview config (#1700) (thanks @zerone0x)
2026-01-25 07:55:39 +00:00
Peter Steinberger c6cdbb630c fix: harden exec spawn fallback 2026-01-25 06:37:39 +00:00
Peter Steinberger da2439f2cc docs: clarify Claude Pro Max auth 2026-01-25 06:05:11 +00:00
zerone0xandClaude 92ab3f22dc feat(telegram): add linkPreview config option
Add channels.telegram.linkPreview config to control whether link previews
are shown in outbound messages. When set to false, uses Telegram's
link_preview_options.is_disabled to suppress URL previews.

- Add linkPreview to TelegramAccountConfig type
- Add Zod schema validation for linkPreview
- Pass link_preview_options to sendMessage in send.ts and bot/delivery.ts
- Propagate linkPreview config through deliverReplies callers
- Add tests for link preview behavior

Fixes #1675

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-25 06:00:05 +00:00
Peter Steinberger 43a6c5b77f docs: clarify Gemini CLI OAuth 2026-01-25 05:53:25 +00:00
Peter Steinberger 495616d13e fix(ui): refine config save guardrails (#1707)
* fix: refine config save guardrails

* docs: add changelog for config save guardrails (#1707) (thanks @Glucksberg)
2026-01-25 05:52:32 +00:00
Peter Steinberger bac80f0886 fix: listen on ipv6 loopback for gateway 2026-01-25 05:49:48 +00:00
Peter Steinberger ef078fec70 docs: add Windows install troubleshooting 2026-01-25 05:48:24 +00:00
GlucksbergandClaude Opus 4.5 8e3ac01db6 fix(ui): improve config save UX (#1678)
Follow-up to #1609 fix:
- Remove formUnsafe check from canSave (was blocking save even with valid changes)
- Suppress disconnect message for code 1012 (service restart is expected during config save)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 05:46:55 +00:00
Peter Steinberger c3f90dd4e2 docs: add macOS VM link 2026-01-25 05:32:04 +00:00
Peter Steinberger 00c4556d7b docs: add Claude Max FAQ 2026-01-25 05:28:01 +00:00
Peter Steinberger a4bc69dbec docs: add first steps FAQ 2026-01-25 05:24:47 +00:00
Peter Steinberger 3f1457de2a docs: add new FAQ entries 2026-01-25 05:20:16 +00:00
Peter Steinberger 8507ea08bd docs: expand macOS VM guide (#1693) (thanks @f-trycua) 2026-01-25 05:16:41 +00:00
f-trycua 7ae2548fc6 docs: add macOS VM (Lume) platform guide
Add documentation for running Clawdbot in a sandboxed macOS VM
using Lume. This provides an alternative to buying dedicated
hardware or using cloud instances.

The guide covers:
- Installing Lume on Apple Silicon Macs
- Creating and configuring a macOS VM
- Installing Clawdbot inside the VM
- Running headlessly for 24/7 operation
- iMessage integration via BlueBubbles
- Saving golden images for easy reset
2026-01-25 05:14:13 +00:00
Peter Steinberger f06f83ddd0 docs: add CC comparison faq 2026-01-25 05:00:47 +00:00
Peter Steinberger c78297d80f docs: add account isolation faq 2026-01-25 04:56:41 +00:00
Peter Steinberger 69f6e1a20b docs: add multi-agent team faq 2026-01-25 04:55:22 +00:00
Peter Steinberger 5f6409a73d fix: configurable signal startup timeout 2026-01-25 04:51:35 +00:00
Rohan NagpalandPeter Steinberger 06a7e1e8ce Telegram: threaded conversation support (#1597)
* Telegram: isolate dm topic sessions

* Tests: cap vitest workers

* Tests: cap Vitest workers on CI macOS

* Tests: avoid timer-based pi-ai stream mock

* Tests: increase embedded runner timeout

* fix: harden telegram dm thread handling (#1597) (thanks @rohannagpal)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-01-25 04:48:51 +00:00
Peter Steinberger 9eaaadf8ee fix: clarify control ui auth hints (fixes #1690) 2026-01-25 04:46:42 +00:00
44 changed files with 1663 additions and 366 deletions

No files matched your search

+7
View File
@@ -9,15 +9,18 @@ Docs: https://docs.clawd.bot
- Venius (Venice AI): highlight provider guide + cross-links + expanded guidance. https://docs.clawd.bot/providers/venice
### Changes
- TTS: allow custom OpenAI-compatible TTS endpoints without API keys. (#1701) Thanks @louzhixian. https://docs.clawd.bot/tts
- TTS: add Edge TTS provider fallback, defaulting to keyless Edge with MP3 retry on format failures. (#1668) Thanks @steipete. https://docs.clawd.bot/tts
- Web search: add Brave freshness filter parameter for time-scoped results. (#1688) Thanks @JonUleis. https://docs.clawd.bot/tools/web
- TTS: add auto mode enum (off/always/inbound/tagged) with per-session `/tts` override. (#1667) Thanks @sebslight. https://docs.clawd.bot/tts
- Docs: expand FAQ (migration, scheduling, concurrency, model recommendations, OpenAI subscription auth, Pi sizing, hackable install, docs SSL workaround).
- Docs: add verbose installer troubleshooting guidance.
- Docs: add macOS VM guide with local/hosted options + VPS/nodes guidance. (#1693) Thanks @f-trycua.
- Docs: update Fly.io guide notes.
- Docs: add Bedrock EC2 instance role setup + IAM steps. (#1625) Thanks @sergical. https://docs.clawd.bot/bedrock
- Exec approvals: forward approval prompts to chat with `/approve` for all channels (including plugins). (#1621) Thanks @czekaj. https://docs.clawd.bot/tools/exec-approvals https://docs.clawd.bot/tools/slash-commands
- Gateway: expose config.patch in the gateway tool with safe partial updates + restart sentinel. (#1653) Thanks @Glucksberg.
- Telegram: add `channels.telegram.linkPreview` to toggle outbound link previews. (#1700) Thanks @zerone0x. https://docs.clawd.bot/channels/telegram
- Telegram: treat DM topics as separate sessions and keep DM history limits stable with thread suffixes. (#1597) Thanks @rohannagpal.
- Telegram: add verbose raw-update logging for inbound Telegram updates. (#1597) Thanks @rohannagpal.
@@ -25,17 +28,21 @@ Docs: https://docs.clawd.bot
- BlueBubbles: keep part-index GUIDs in reply tags when short IDs are missing.
- Web UI: hide internal `message_id` hints in chat bubbles.
- Web UI: show Stop button during active runs, swap back to New session when idle. (#1664) Thanks @ndbroadbent.
- Web UI: clear stale disconnect banners on reconnect; allow form saves with unsupported schema paths but block missing schema. (#1707) Thanks @Glucksberg.
- Heartbeat: normalize target identifiers for consistent routing.
- TUI: reload history after gateway reconnect to restore session state. (#1663)
- Telegram: use wrapped fetch for long-polling on Node to normalize AbortSignal handling. (#1639)
- Telegram: set fetch duplex="half" for uploads on Node 22 to avoid sendPhoto failures. (#1684) Thanks @commdata2338.
- Signal: repair reaction sends (group/UUID targets + CLI author flags). (#1651) Thanks @vilkasdev.
- Signal: add configurable signal-cli startup timeout + external daemon mode docs. (#1677) https://docs.clawd.bot/channels/signal
- Exec: keep approvals for elevated ask unless full mode. (#1616) Thanks @ivancasco.
- Agents: auto-compact on context overflow prompt errors before failing. (#1627) Thanks @rodrigouroz.
- Agents: use the active auth profile for auto-compaction recovery.
- Models: default missing custom provider fields so minimal configs are accepted.
- Gateway: skip Tailscale DNS probing when tailscale.mode is off. (#1671)
- Gateway: reduce log noise for late invokes + remote node probes; debounce skills refresh. (#1607) Thanks @petter-b.
- Gateway: clarify Control UI/WebChat auth error hints for missing tokens. (#1690)
- Gateway: listen on IPv6 loopback when bound to 127.0.0.1 so localhost webhooks work.
- macOS: default direct-transport `ws://` URLs to port 18789; document `gateway.remote.transport`. (#1603) Thanks @ngutman.
- Voice Call: return stream TwiML for outbound conversation calls on initial Twilio webhook. (#1634)
- Google Chat: tighten email allowlist matching, typing cleanup, media caps, and onboarding/docs/tests. (#1635) Thanks @iHildy.
+1 -1
View File
@@ -17,7 +17,7 @@ read_when:
- **Proxy:** optional `channels.telegram.proxy` uses `undici.ProxyAgent` through grammY’s `client.baseFetch`.
- **Webhook support:** `webhook-set.ts` wraps `setWebhook/deleteWebhook`; `webhook.ts` hosts the callback with health + graceful shutdown. Gateway enables webhook mode when `channels.telegram.webhookUrl` is set (otherwise it long-polls).
- **Sessions:** direct chats collapse into the agent main session (`agent:<agentId>:<mainKey>`); groups use `agent:<agentId>:telegram:group:<chatId>`; replies route back to the same channel.
- **Config knobs:** `channels.telegram.botToken`, `channels.telegram.dmPolicy`, `channels.telegram.groups` (allowlist + mention defaults), `channels.telegram.allowFrom`, `channels.telegram.groupAllowFrom`, `channels.telegram.groupPolicy`, `channels.telegram.mediaMaxMb`, `channels.telegram.proxy`, `channels.telegram.webhookSecret`, `channels.telegram.webhookUrl`.
- **Config knobs:** `channels.telegram.botToken`, `channels.telegram.dmPolicy`, `channels.telegram.groups` (allowlist + mention defaults), `channels.telegram.allowFrom`, `channels.telegram.groupAllowFrom`, `channels.telegram.groupPolicy`, `channels.telegram.mediaMaxMb`, `channels.telegram.linkPreview`, `channels.telegram.proxy`, `channels.telegram.webhookSecret`, `channels.telegram.webhookUrl`.
- **Draft streaming:** optional `channels.telegram.streamMode` uses `sendMessageDraft` in private topic chats (Bot API 9.3+). This is separate from channel block streaming.
- **Tests:** grammy mocks cover DM + group mention gating and outbound send; more media/webhook fixtures still welcome.
+17
View File
@@ -74,6 +74,22 @@ Example:
Multi-account support: use `channels.signal.accounts` with per-account config and optional `name`. See [`gateway/configuration`](/gateway/configuration#telegramaccounts--discordaccounts--slackaccounts--signalaccounts--imessageaccounts) for the shared pattern.
## External daemon mode (httpUrl)
If you want to manage `signal-cli` yourself (slow JVM cold starts, container init, or shared CPUs), run the daemon separately and point Clawdbot at it:
```json5
{
channels: {
signal: {
httpUrl: "http://127.0.0.1:8080",
autoStart: false
}
}
}
```
This skips auto-spawn and the startup wait inside Clawdbot. For slow starts when auto-spawning, set `channels.signal.startupTimeoutMs`.
## Access control (DMs + groups)
DMs:
- Default: `channels.signal.dmPolicy = "pairing"`.
@@ -142,6 +158,7 @@ Provider options:
- `channels.signal.httpUrl`: full daemon URL (overrides host/port).
- `channels.signal.httpHost`, `channels.signal.httpPort`: daemon bind (default 127.0.0.1:8080).
- `channels.signal.autoStart`: auto-spawn daemon (default true if `httpUrl` unset).
- `channels.signal.startupTimeoutMs`: startup wait timeout in ms (cap 120000).
- `channels.signal.receiveMode`: `on-start | manual`.
- `channels.signal.ignoreAttachments`: skip attachment downloads.
- `channels.signal.ignoreStories`: ignore stories from the daemon.
+1
View File
@@ -525,6 +525,7 @@ Provider options:
- `channels.telegram.replyToMode`: `off | first | all` (default: `first`).
- `channels.telegram.textChunkLimit`: outbound chunk size (chars).
- `channels.telegram.chunkMode`: `length` (default) or `newline` to split on newlines before length chunking.
- `channels.telegram.linkPreview`: toggle link previews for outbound messages (default: true).
- `channels.telegram.streamMode`: `off | partial | block` (draft streaming).
- `channels.telegram.mediaMaxMb`: inbound/outbound media cap (MB).
- `channels.telegram.retry`: retry policy for outbound Telegram API calls (attempts, minDelayMs, maxDelayMs, jitter).
+2
View File
@@ -89,6 +89,8 @@ Clawdbot ships with the pi‑ai catalog. These providers require **no**
- Gemini CLI OAuth is shipped as a bundled plugin (`google-gemini-cli-auth`, disabled by default).
- Enable: `clawdbot plugins enable google-gemini-cli-auth`
- Login: `clawdbot models auth login --provider google-gemini-cli --set-default`
- Note: you do **not** paste a client id or secret into `clawdbot.json`. The CLI login flow stores
tokens in auth profiles on the gateway host.
### Z.AI (GLM)
+1
View File
@@ -1048,6 +1048,7 @@
"pages": [
"platforms",
"platforms/macos",
"platforms/macos-vm",
"platforms/ios",
"platforms/android",
"platforms/windows",
+4
View File
@@ -1021,6 +1021,7 @@ Set `channels.telegram.configWrites: false` to block Telegram-initiated config w
],
historyLimit: 50, // include last N group messages as context (0 disables)
replyToMode: "first", // off | first | all
linkPreview: true, // toggle outbound link previews
streamMode: "partial", // off | partial | block (draft streaming; separate from block streaming)
draftChunk: { // optional; only for streamMode=block
minChars: 200,
@@ -1537,6 +1538,7 @@ voice notes; other channels send MP3 audio.
},
openai: {
apiKey: "openai_api_key",
baseUrl: "https://api.openai.com/v1",
model: "gpt-4o-mini-tts",
voice: "alloy"
}
@@ -1557,6 +1559,8 @@ Notes:
- `/tts limit` and `/tts summary` control per-user summarization settings.
- `apiKey` values fall back to `ELEVENLABS_API_KEY`/`XI_API_KEY` and `OPENAI_API_KEY`.
- `elevenlabs.baseUrl` overrides the ElevenLabs API base URL.
- `openai.baseUrl` overrides the OpenAI TTS base URL (defaults to `https://api.openai.com/v1`,
or `OPENAI_TTS_BASE_URL` when set). Custom endpoints do not require an API key.
- `elevenlabs.voiceSettings` supports `stability`/`similarityBoost`/`style` (0..1),
`useSpeakerBoost`, and `speed` (0.5..2.0).
+371 -176
View File
File diff suppressed because it is too large. Load diff
+6
View File
@@ -114,3 +114,9 @@ Git requirement:
If you choose `-InstallMethod git` and Git is missing, the installer will print the
Git for Windows link (`https://git-scm.com/download/win`) and exit.
Common Windows issues:
- **npm error spawn git / ENOENT**: install Git for Windows and reopen PowerShell, then rerun the installer.
- **"clawdbot" is not recognized**: your npm global bin folder is not on PATH. Most systems use
`%AppData%\\npm`. You can also run `npm config get prefix` and add `\\bin` to PATH, then reopen PowerShell.
+275
View File
@@ -0,0 +1,275 @@
---
summary: "Run Clawdbot in a sandboxed macOS VM (local or hosted) when you need isolation or iMessage"
read_when:
- You want Clawdbot isolated from your main macOS environment
- You want iMessage integration (BlueBubbles) in a sandbox
- You want a resettable macOS environment you can clone
- You want to compare local vs hosted macOS VM options
---
# Clawdbot on macOS VMs (Sandboxing)
## Recommended default (most users)
- **Small Linux VPS** for an always-on Gateway and low cost. See [VPS hosting](/vps).
- **Dedicated hardware** (Mac mini or Linux box) if you want full control and a **residential IP** for browser automation. Many sites block data center IPs, so local browsing often works better.
- **Hybrid:** keep the Gateway on a cheap VPS, and connect your Mac as a **node** when you need browser/UI automation. See [Nodes](/nodes) and [Gateway remote](/gateway/remote).
Use a macOS VM when you specifically need macOS-only capabilities (iMessage/BlueBubbles) or want strict isolation from your daily Mac.
## macOS VM options
### Local VM on your Apple Silicon Mac (Lume)
Run Clawdbot in a sandboxed macOS VM on your existing Apple Silicon Mac using [Lume](https://cua.ai/docs/lume).
This gives you:
- Full macOS environment in isolation (your host stays clean)
- iMessage support via BlueBubbles (impossible on Linux/Windows)
- Instant reset by cloning VMs
- No extra hardware or cloud costs
### Hosted Mac providers (cloud)
If you want macOS in the cloud, hosted Mac providers work too:
- [MacStadium](https://www.macstadium.com/) (hosted Macs)
- Other hosted Mac vendors also work; follow their VM + SSH docs
Once you have SSH access to a macOS VM, continue at step 6 below.
---
## Quick path (Lume, experienced users)
1. Install Lume
2. `lume create clawdbot --os macos --ipsw latest`
3. Complete Setup Assistant, enable Remote Login (SSH)
4. `lume run clawdbot --no-display`
5. SSH in, install Clawdbot, configure channels
6. Done
---
## What you need (Lume)
- Apple Silicon Mac (M1/M2/M3/M4)
- macOS Sequoia or later on the host
- ~60 GB free disk space per VM
- ~20 minutes
---
## 1) Install Lume
```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
```
If `~/.local/bin` isn't in your PATH:
```bash
echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.zshrc && source ~/.zshrc
```
Verify:
```bash
lume --version
```
Docs: [Lume Installation](https://cua.ai/docs/lume/guide/getting-started/installation)
---
## 2) Create the macOS VM
```bash
lume create clawdbot --os macos --ipsw latest
```
This downloads macOS and creates the VM. A VNC window opens automatically.
Note: The download can take a while depending on your connection.
---
## 3) Complete Setup Assistant
In the VNC window:
1. Select language and region
2. Skip Apple ID (or sign in if you want iMessage later)
3. Create a user account (remember the username and password)
4. Skip all optional features
After setup completes, enable SSH:
1. Open System Settings → General → Sharing
2. Enable "Remote Login"
---
## 4) Get the VM's IP address
```bash
lume get clawdbot
```
Look for the IP address (usually `192.168.64.x`).
---
## 5) SSH into the VM
```bash
ssh youruser@192.168.64.X
```
Replace `youruser` with the account you created, and the IP with your VM's IP.
---
## 6) Install Clawdbot
Inside the VM:
```bash
npm install -g clawdbot@latest
clawdbot onboard --install-daemon
```
Follow the onboarding prompts to set up your model provider (Anthropic, OpenAI, etc.).
---
## 7) Configure channels
Edit the config file:
```bash
nano ~/.clawdbot/clawdbot.json
```
Add your channels:
```json
{
"channels": {
"whatsapp": {
"dmPolicy": "allowlist",
"allowFrom": ["+15551234567"]
},
"telegram": {
"botToken": "YOUR_BOT_TOKEN"
}
}
}
```
Then login to WhatsApp (scan QR):
```bash
clawdbot channels login
```
---
## 8) Run the VM headlessly
Stop the VM and restart without display:
```bash
lume stop clawdbot
lume run clawdbot --no-display
```
The VM runs in the background. Clawdbot's daemon keeps the gateway running.
To check status:
```bash
ssh youruser@192.168.64.X "clawdbot status"
```
---
## Bonus: iMessage integration
This is the killer feature of running on macOS. Use [BlueBubbles](https://bluebubbles.app) to add iMessage to Clawdbot.
Inside the VM:
1. Download BlueBubbles from bluebubbles.app
2. Sign in with your Apple ID
3. Enable the Web API and set a password
4. Point BlueBubbles webhooks at your gateway (example: `https://your-gateway-host:3000/bluebubbles-webhook?password=<password>`)
Add to your Clawdbot config:
```json
{
"channels": {
"bluebubbles": {
"serverUrl": "http://localhost:1234",
"password": "your-api-password",
"webhookPath": "/bluebubbles-webhook"
}
}
}
```
Restart the gateway. Now your agent can send and receive iMessages.
Full setup details: [BlueBubbles channel](/channels/bluebubbles)
---
## Save a golden image
Before customizing further, snapshot your clean state:
```bash
lume stop clawdbot
lume clone clawdbot clawdbot-golden
```
Reset anytime:
```bash
lume stop clawdbot && lume delete clawdbot
lume clone clawdbot-golden clawdbot
lume run clawdbot --no-display
```
---
## Running 24/7
Keep the VM running by:
- Keeping your Mac plugged in
- Disabling sleep in System Settings → Energy Saver
- Using `caffeinate` if needed
For true always-on, consider a dedicated Mac mini or a small VPS. See [VPS hosting](/vps).
---
## Troubleshooting
| Problem | Solution |
|---------|----------|
| Can't SSH into VM | Check "Remote Login" is enabled in VM's System Settings |
| VM IP not showing | Wait for VM to fully boot, run `lume get clawdbot` again |
| Lume command not found | Add `~/.local/bin` to your PATH |
| WhatsApp QR not scanning | Ensure you're logged into the VM (not host) when running `clawdbot channels login` |
---
## Related docs
- [VPS hosting](/vps)
- [Nodes](/nodes)
- [Gateway remote](/gateway/remote)
- [BlueBubbles channel](/channels/bluebubbles)
- [Lume Quickstart](https://cua.ai/docs/lume/guide/getting-started/quickstart)
- [Lume CLI Reference](https://cua.ai/docs/lume/reference/cli-reference)
- [Unattended VM Setup](https://cua.ai/docs/lume/guide/fundamentals/unattended-setup) (advanced)
- [Docker Sandboxing](/install/docker) (alternative isolation approach)
+34 -1
View File
@@ -33,11 +33,18 @@ does not publish limits, so assume similar or lower limits. citeturn0searc
If you want OpenAI or ElevenLabs:
- `ELEVENLABS_API_KEY` (or `XI_API_KEY`)
- `OPENAI_API_KEY`
- `OPENAI_API_KEY` (optional for custom OpenAI-compatible endpoints)
Optional:
- `OPENAI_TTS_BASE_URL` (defaults to `https://api.openai.com/v1`)
Edge TTS does **not** require an API key. If no API keys are found, Clawdbot defaults
to Edge TTS (unless disabled via `messages.tts.edge.enabled=false`).
When `OPENAI_TTS_BASE_URL` points to a non-OpenAI endpoint, Clawdbot relaxes
OpenAI model/voice validation and does **not** require an API key. If your
endpoint needs authentication, set `messages.tts.openai.apiKey` or `OPENAI_API_KEY`.
If multiple providers are configured, the selected provider is used first and the others are fallback options.
Auto-summary uses the configured `summaryModel` (or `agents.defaults.model.primary`),
so that provider must also be authenticated if you enable summaries.
@@ -115,6 +122,32 @@ Full schema is in [Gateway configuration](/gateway/configuration).
}
```
### Custom OpenAI-compatible endpoint
Set an endpoint in the environment:
```bash
export OPENAI_TTS_BASE_URL="http://localhost:8880/v1"
```
Then configure TTS as usual (API key optional for custom endpoints):
```json5
{
messages: {
tts: {
auto: "always",
provider: "openai",
openai: {
baseUrl: "http://localhost:8880/v1",
model: "kokoro",
voice: "zm_yunxia"
}
}
}
}
```
### Edge TTS primary (no API key)
```json5
+73 -25
View File
@@ -1,5 +1,5 @@
import crypto from "node:crypto";
import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process";
import type { ChildProcessWithoutNullStreams } from "node:child_process";
import path from "node:path";
import type { AgentTool, AgentToolResult } from "@mariozechner/pi-agent-core";
import { Type } from "@sinclair/typebox";
@@ -27,6 +27,7 @@ import {
} from "../infra/shell-env.js";
import { enqueueSystemEvent } from "../infra/system-events.js";
import { logInfo, logWarn } from "../logger.js";
import { formatSpawnError, spawnWithFallback } from "../process/spawn-utils.js";
import {
type ProcessSession,
type SessionStdin,
@@ -362,23 +363,38 @@ async function runExecProcess(opts: {
let stdin: SessionStdin | undefined;
if (opts.sandbox) {
child = spawn(
"docker",
buildDockerExecArgs({
containerName: opts.sandbox.containerName,
command: opts.command,
workdir: opts.containerWorkdir ?? opts.sandbox.containerWorkdir,
env: opts.env,
tty: opts.usePty,
}),
{
const { child: spawned } = await spawnWithFallback({
argv: [
"docker",
...buildDockerExecArgs({
containerName: opts.sandbox.containerName,
command: opts.command,
workdir: opts.containerWorkdir ?? opts.sandbox.containerWorkdir,
env: opts.env,
tty: opts.usePty,
}),
],
options: {
cwd: opts.workdir,
env: process.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
},
) as ChildProcessWithoutNullStreams;
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (err, fallback) => {
const errText = formatSpawnError(err);
const warning = `Warning: spawn failed (${errText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${errText}); retrying with ${fallback.label}.`);
opts.warnings.push(warning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
stdin = child.stdin;
} else if (opts.usePty) {
const { shell, args: shellArgs } = getShellConfig();
@@ -422,24 +438,56 @@ async function runExecProcess(opts: {
const warning = `Warning: PTY spawn failed (${errText}); retrying without PTY for \`${opts.command}\`.`;
logWarn(`exec: PTY spawn failed (${errText}); retrying without PTY for "${opts.command}".`);
opts.warnings.push(warning);
child = spawn(shell, [...shellArgs, opts.command], {
const { child: spawned } = await spawnWithFallback({
argv: [shell, ...shellArgs, opts.command],
options: {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
},
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (fallbackErr, fallback) => {
const fallbackText = formatSpawnError(fallbackErr);
const fallbackWarning = `Warning: spawn failed (${fallbackText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${fallbackText}); retrying with ${fallback.label}.`);
opts.warnings.push(fallbackWarning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
} else {
const { shell, args: shellArgs } = getShellConfig();
const { child: spawned } = await spawnWithFallback({
argv: [shell, ...shellArgs, opts.command],
options: {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
}) as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
} else {
const { shell, args: shellArgs } = getShellConfig();
child = spawn(shell, [...shellArgs, opts.command], {
cwd: opts.workdir,
env: opts.env,
detached: process.platform !== "win32",
stdio: ["pipe", "pipe", "pipe"],
windowsHide: true,
}) as ChildProcessWithoutNullStreams;
},
fallbacks: [
{
label: "no-detach",
options: { detached: false },
},
],
onFallback: (err, fallback) => {
const errText = formatSpawnError(err);
const warning = `Warning: spawn failed (${errText}); retrying with ${fallback.label}.`;
logWarn(`exec: spawn failed (${errText}); retrying with ${fallback.label}.`);
opts.warnings.push(warning);
},
});
child = spawned as ChildProcessWithoutNullStreams;
stdin = child.stdin;
}
+19 -2
View File
@@ -1,7 +1,8 @@
import type { ReasoningLevel } from "../thinking.js";
import type { NoticeLevel, ReasoningLevel } from "../thinking.js";
import {
type ElevatedLevel,
normalizeElevatedLevel,
normalizeNoticeLevel,
normalizeReasoningLevel,
normalizeThinkLevel,
normalizeVerboseLevel,
@@ -112,6 +113,22 @@ export function extractVerboseDirective(body?: string): {
};
}
export function extractNoticeDirective(body?: string): {
cleaned: string;
noticeLevel?: NoticeLevel;
rawLevel?: string;
hasDirective: boolean;
} {
if (!body) return { cleaned: "", hasDirective: false };
const extracted = extractLevelDirective(body, ["notice", "notices"], normalizeNoticeLevel);
return {
cleaned: extracted.cleaned,
noticeLevel: extracted.level,
rawLevel: extracted.rawLevel,
hasDirective: extracted.hasDirective,
};
}
export function extractElevatedDirective(body?: string): {
cleaned: string;
elevatedLevel?: ElevatedLevel;
@@ -152,5 +169,5 @@ export function extractStatusDirective(body?: string): {
return extractSimpleDirective(body, ["status"]);
}
export type { ElevatedLevel, ReasoningLevel, ThinkLevel, VerboseLevel };
export type { ElevatedLevel, NoticeLevel, ReasoningLevel, ThinkLevel, VerboseLevel };
export { extractExecDirective } from "./exec/directive.js";
+11
View File
@@ -1,5 +1,6 @@
export type ThinkLevel = "off" | "minimal" | "low" | "medium" | "high" | "xhigh";
export type VerboseLevel = "off" | "on" | "full";
export type NoticeLevel = "off" | "on" | "full";
export type ElevatedLevel = "off" | "on" | "ask" | "full";
export type ElevatedMode = "off" | "ask" | "full";
export type ReasoningLevel = "off" | "on" | "stream";
@@ -93,6 +94,16 @@ export function normalizeVerboseLevel(raw?: string | null): VerboseLevel | undef
return undefined;
}
// Normalize system notice flags used to toggle system notifications.
export function normalizeNoticeLevel(raw?: string | null): NoticeLevel | undefined {
if (!raw) return undefined;
const key = raw.toLowerCase();
if (["off", "false", "no", "0"].includes(key)) return "off";
if (["full", "all", "everything"].includes(key)) return "full";
if (["on", "minimal", "true", "yes", "1"].includes(key)) return "on";
return undefined;
}
// Normalize response-usage display modes used to toggle per-response usage footers.
export function normalizeUsageDisplay(raw?: string | null): UsageDisplayLevel | undefined {
if (!raw) return undefined;
+2
View File
@@ -33,6 +33,8 @@ export type SignalAccountConfig = {
cliPath?: string;
/** Auto-start signal-cli daemon (default: true if httpUrl not set). */
autoStart?: boolean;
/** Max time to wait for signal-cli daemon startup (ms, cap 120000). */
startupTimeoutMs?: number;
receiveMode?: "on-start" | "manual";
ignoreAttachments?: boolean;
ignoreStories?: boolean;
+2
View File
@@ -118,6 +118,8 @@ export type TelegramAccountConfig = {
reactionLevel?: "off" | "ack" | "minimal" | "extensive";
/** Heartbeat visibility settings for this channel. */
heartbeat?: ChannelHeartbeatVisibilityConfig;
/** Controls whether link previews are shown in outbound messages. Default: true. */
linkPreview?: boolean;
};
export type TelegramTopicConfig = {
+1
View File
@@ -56,6 +56,7 @@ export type TtsConfig = {
/** OpenAI configuration. */
openai?: {
apiKey?: string;
baseUrl?: string;
model?: string;
voice?: string;
};
+1
View File
@@ -204,6 +204,7 @@ export const TtsConfigSchema = z
openai: z
.object({
apiKey: z.string().optional(),
baseUrl: z.string().optional(),
model: z.string().optional(),
voice: z.string().optional(),
})
+2
View File
@@ -125,6 +125,7 @@ export const TelegramAccountSchemaBase = z
reactionNotifications: z.enum(["off", "own", "all"]).optional(),
reactionLevel: z.enum(["off", "ack", "minimal", "extensive"]).optional(),
heartbeat: ChannelHeartbeatVisibilitySchema,
linkPreview: z.boolean().optional(),
})
.strict();
@@ -486,6 +487,7 @@ export const SignalAccountSchemaBase = z
httpPort: z.number().int().positive().optional(),
cliPath: ExecutableTokenSchema.optional(),
autoStart: z.boolean().optional(),
startupTimeoutMs: z.number().int().min(1000).max(120000).optional(),
receiveMode: z.union([z.literal("on-start"), z.literal("manual")]).optional(),
ignoreAttachments: z.boolean().optional(),
ignoreStories: z.boolean().optional(),
+17 -66
View File
@@ -1,77 +1,28 @@
import { beforeEach, describe, expect, test, vi } from "vitest";
import { describe, expect, it } from "vitest";
const testTailnetIPv4 = { value: undefined as string | undefined };
const testTailnetIPv6 = { value: undefined as string | undefined };
import { resolveGatewayListenHosts } from "./net.js";
vi.mock("../infra/tailnet.js", () => ({
pickPrimaryTailnetIPv4: () => testTailnetIPv4.value,
pickPrimaryTailnetIPv6: () => testTailnetIPv6.value,
}));
import { isLocalGatewayAddress, resolveGatewayClientIp } from "./net.js";
describe("gateway net", () => {
beforeEach(() => {
testTailnetIPv4.value = undefined;
testTailnetIPv6.value = undefined;
});
test("treats loopback as local", () => {
expect(isLocalGatewayAddress("127.0.0.1")).toBe(true);
expect(isLocalGatewayAddress("127.0.1.1")).toBe(true);
expect(isLocalGatewayAddress("::1")).toBe(true);
expect(isLocalGatewayAddress("::ffff:127.0.0.1")).toBe(true);
});
test("treats local tailnet IPv4 as local", () => {
testTailnetIPv4.value = "100.64.0.1";
expect(isLocalGatewayAddress("100.64.0.1")).toBe(true);
expect(isLocalGatewayAddress("::ffff:100.64.0.1")).toBe(true);
});
test("ignores non-matching tailnet IPv4", () => {
testTailnetIPv4.value = "100.64.0.1";
expect(isLocalGatewayAddress("100.64.0.2")).toBe(false);
});
test("treats local tailnet IPv6 as local", () => {
testTailnetIPv6.value = "fd7a:115c:a1e0::123";
expect(isLocalGatewayAddress("fd7a:115c:a1e0::123")).toBe(true);
});
test("uses forwarded-for when remote is a trusted proxy", () => {
const clientIp = resolveGatewayClientIp({
remoteAddr: "10.0.0.2",
forwardedFor: "203.0.113.9, 10.0.0.2",
trustedProxies: ["10.0.0.2"],
describe("resolveGatewayListenHosts", () => {
it("returns the input host when not loopback", async () => {
const hosts = await resolveGatewayListenHosts("0.0.0.0", {
canBindToHost: async () => {
throw new Error("should not be called");
},
});
expect(clientIp).toBe("203.0.113.9");
expect(hosts).toEqual(["0.0.0.0"]);
});
test("ignores forwarded-for from untrusted proxies", () => {
const clientIp = resolveGatewayClientIp({
remoteAddr: "10.0.0.3",
forwardedFor: "203.0.113.9",
trustedProxies: ["10.0.0.2"],
it("adds ::1 when IPv6 loopback is available", async () => {
const hosts = await resolveGatewayListenHosts("127.0.0.1", {
canBindToHost: async () => true,
});
expect(clientIp).toBe("10.0.0.3");
expect(hosts).toEqual(["127.0.0.1", "::1"]);
});
test("normalizes trusted proxy IPs and strips forwarded ports", () => {
const clientIp = resolveGatewayClientIp({
remoteAddr: "::ffff:10.0.0.2",
forwardedFor: "203.0.113.9:1234",
trustedProxies: ["10.0.0.2"],
it("keeps only IPv4 loopback when IPv6 is unavailable", async () => {
const hosts = await resolveGatewayListenHosts("127.0.0.1", {
canBindToHost: async () => false,
});
expect(clientIp).toBe("203.0.113.9");
});
test("falls back to x-real-ip when forwarded-for is missing", () => {
const clientIp = resolveGatewayClientIp({
remoteAddr: "10.0.0.2",
realIp: "203.0.113.10",
trustedProxies: ["10.0.0.2"],
});
expect(clientIp).toBe("203.0.113.10");
expect(hosts).toEqual(["127.0.0.1"]);
});
});
+16 -6
View File
@@ -97,14 +97,14 @@ export async function resolveGatewayBindHost(
if (mode === "loopback") {
// 127.0.0.1 rarely fails, but handle gracefully
if (await canBindTo("127.0.0.1")) return "127.0.0.1";
if (await canBindToHost("127.0.0.1")) return "127.0.0.1";
return "0.0.0.0"; // extreme fallback
}
if (mode === "tailnet") {
const tailnetIP = pickPrimaryTailnetIPv4();
if (tailnetIP && (await canBindTo(tailnetIP))) return tailnetIP;
if (await canBindTo("127.0.0.1")) return "127.0.0.1";
if (tailnetIP && (await canBindToHost(tailnetIP))) return tailnetIP;
if (await canBindToHost("127.0.0.1")) return "127.0.0.1";
return "0.0.0.0";
}
@@ -116,13 +116,13 @@ export async function resolveGatewayBindHost(
const host = customHost?.trim();
if (!host) return "0.0.0.0"; // invalid config → fall back to all
if (isValidIPv4(host) && (await canBindTo(host))) return host;
if (isValidIPv4(host) && (await canBindToHost(host))) return host;
// Custom IP failed → fall back to LAN
return "0.0.0.0";
}
if (mode === "auto") {
if (await canBindTo("127.0.0.1")) return "127.0.0.1";
if (await canBindToHost("127.0.0.1")) return "127.0.0.1";
return "0.0.0.0";
}
@@ -136,7 +136,7 @@ export async function resolveGatewayBindHost(
* @param host - The host address to test
* @returns True if we can successfully bind to this address
*/
async function canBindTo(host: string): Promise<boolean> {
export async function canBindToHost(host: string): Promise<boolean> {
return new Promise((resolve) => {
const testServer = net.createServer();
testServer.once("error", () => {
@@ -151,6 +151,16 @@ async function canBindTo(host: string): Promise<boolean> {
});
}
export async function resolveGatewayListenHosts(
bindHost: string,
opts?: { canBindToHost?: (host: string) => Promise<boolean> },
): Promise<string[]> {
if (bindHost !== "127.0.0.1") return [bindHost];
const canBind = opts?.canBindToHost ?? canBindToHost;
if (await canBind("::1")) return [bindHost, "::1"];
return [bindHost];
}
/**
* Validate if a string is a valid IPv4 address.
*
+15 -8
View File
@@ -28,6 +28,7 @@ export function createGatewayCloseHandler(params: {
browserControl: { stop: () => Promise<void> } | null;
wss: WebSocketServer;
httpServer: HttpServer;
httpServers?: HttpServer[];
}) {
return async (opts?: { reason?: string; restartExpectedMs?: number | null }) => {
const reasonRaw = typeof opts?.reason === "string" ? opts.reason.trim() : "";
@@ -108,14 +109,20 @@ export function createGatewayCloseHandler(params: {
await params.browserControl.stop().catch(() => {});
}
await new Promise<void>((resolve) => params.wss.close(() => resolve()));
const httpServer = params.httpServer as HttpServer & {
closeIdleConnections?: () => void;
};
if (typeof httpServer.closeIdleConnections === "function") {
httpServer.closeIdleConnections();
const servers =
params.httpServers && params.httpServers.length > 0
? params.httpServers
: [params.httpServer];
for (const server of servers) {
const httpServer = server as HttpServer & {
closeIdleConnections?: () => void;
};
if (typeof httpServer.closeIdleConnections === "function") {
httpServer.closeIdleConnections();
}
await new Promise<void>((resolve, reject) =>
httpServer.close((err) => (err ? reject(err) : resolve())),
);
}
await new Promise<void>((resolve, reject) =>
params.httpServer.close((err) => (err ? reject(err) : resolve())),
);
};
}
+1 -1
View File
@@ -131,7 +131,7 @@ export const ttsHandlers: GatewayRequestHandlers = {
{
id: "openai",
name: "OpenAI",
configured: Boolean(resolveTtsApiKey(config, "openai")),
configured: isTtsProviderConfigured(config, "openai"),
models: [...OPENAI_TTS_MODELS],
voices: [...OPENAI_TTS_VOICES],
},
+44 -19
View File
@@ -10,6 +10,7 @@ import type { ChatAbortControllerEntry } from "./chat-abort.js";
import type { HooksConfigResolved } from "./hooks.js";
import { createGatewayHooksRequestHandler } from "./server/hooks.js";
import { listenGatewayHttpServer } from "./server/http-listen.js";
import { resolveGatewayListenHosts } from "./net.js";
import { createGatewayPluginRequestHandler } from "./server/plugins-http.js";
import type { GatewayWsClient } from "./server/ws-types.js";
import { createGatewayBroadcaster } from "./server-broadcast.js";
@@ -38,11 +39,14 @@ export async function createGatewayRuntimeState(params: {
canvasHostEnabled: boolean;
allowCanvasHostInTests?: boolean;
logCanvas: { info: (msg: string) => void; warn: (msg: string) => void };
log: { info: (msg: string) => void; warn: (msg: string) => void };
logHooks: ReturnType<typeof createSubsystemLogger>;
logPlugins: ReturnType<typeof createSubsystemLogger>;
}): Promise<{
canvasHost: CanvasHostHandler | null;
httpServer: HttpServer;
httpServers: HttpServer[];
httpBindHosts: string[];
wss: WebSocketServer;
clients: Set<GatewayWsClient>;
broadcast: (
@@ -100,30 +104,49 @@ export async function createGatewayRuntimeState(params: {
log: params.logPlugins,
});
const httpServer = createGatewayHttpServer({
canvasHost,
controlUiEnabled: params.controlUiEnabled,
controlUiBasePath: params.controlUiBasePath,
openAiChatCompletionsEnabled: params.openAiChatCompletionsEnabled,
openResponsesEnabled: params.openResponsesEnabled,
openResponsesConfig: params.openResponsesConfig,
handleHooksRequest,
handlePluginRequest,
resolvedAuth: params.resolvedAuth,
tlsOptions: params.gatewayTls?.enabled ? params.gatewayTls.tlsOptions : undefined,
});
await listenGatewayHttpServer({
httpServer,
bindHost: params.bindHost,
port: params.port,
});
const bindHosts = await resolveGatewayListenHosts(params.bindHost);
const httpServers: HttpServer[] = [];
const httpBindHosts: string[] = [];
for (const host of bindHosts) {
const httpServer = createGatewayHttpServer({
canvasHost,
controlUiEnabled: params.controlUiEnabled,
controlUiBasePath: params.controlUiBasePath,
openAiChatCompletionsEnabled: params.openAiChatCompletionsEnabled,
openResponsesEnabled: params.openResponsesEnabled,
openResponsesConfig: params.openResponsesConfig,
handleHooksRequest,
handlePluginRequest,
resolvedAuth: params.resolvedAuth,
tlsOptions: params.gatewayTls?.enabled ? params.gatewayTls.tlsOptions : undefined,
});
try {
await listenGatewayHttpServer({
httpServer,
bindHost: host,
port: params.port,
});
httpServers.push(httpServer);
httpBindHosts.push(host);
} catch (err) {
if (host === bindHosts[0]) throw err;
params.log.warn(
`gateway: failed to bind loopback alias ${host}:${params.port} (${String(err)})`,
);
}
}
const httpServer = httpServers[0];
if (!httpServer) {
throw new Error("Gateway HTTP server failed to start");
}
const wss = new WebSocketServer({
noServer: true,
maxPayload: MAX_PAYLOAD_BYTES,
});
attachGatewayUpgradeHandler({ httpServer, wss, canvasHost });
for (const server of httpServers) {
attachGatewayUpgradeHandler({ httpServer: server, wss, canvasHost });
}
const clients = new Set<GatewayWsClient>();
const { broadcast } = createGatewayBroadcaster({ clients });
@@ -140,6 +163,8 @@ export async function createGatewayRuntimeState(params: {
return {
canvasHost,
httpServer,
httpServers,
httpBindHosts,
wss,
clients,
broadcast,
+9 -1
View File
@@ -7,6 +7,7 @@ import { getResolvedLoggerSettings } from "../logging.js";
export function logGatewayStartup(params: {
cfg: ReturnType<typeof loadConfig>;
bindHost: string;
bindHosts?: string[];
port: number;
tlsEnabled?: boolean;
log: { info: (msg: string, meta?: Record<string, unknown>) => void };
@@ -22,9 +23,16 @@ export function logGatewayStartup(params: {
consoleMessage: `agent model: ${chalk.whiteBright(modelRef)}`,
});
const scheme = params.tlsEnabled ? "wss" : "ws";
const formatHost = (host: string) => (host.includes(":") ? `[${host}]` : host);
const hosts =
params.bindHosts && params.bindHosts.length > 0 ? params.bindHosts : [params.bindHost];
const primaryHost = hosts[0] ?? params.bindHost;
params.log.info(
`listening on ${scheme}://${params.bindHost}:${params.port} (PID ${process.pid})`,
`listening on ${scheme}://${formatHost(primaryHost)}:${params.port} (PID ${process.pid})`,
);
for (const host of hosts.slice(1)) {
params.log.info(`listening on ${scheme}://${formatHost(host)}:${params.port}`);
}
params.log.info(`log file: ${getResolvedLoggerSettings().file}`);
if (params.isNixMode) {
params.log.info("gateway: running in Nix mode (config managed externally)");
+15
View File
@@ -230,6 +230,21 @@ describe("gateway server auth/connect", () => {
ws.close();
});
test("returns control ui hint when token is missing", async () => {
const ws = await openWs(port);
const res = await connectReq(ws, {
client: {
id: GATEWAY_CLIENT_NAMES.CONTROL_UI,
version: "1.0.0",
platform: "web",
mode: GATEWAY_CLIENT_MODES.WEBCHAT,
},
});
expect(res.ok).toBe(false);
expect(res.error?.message ?? "").toContain("Control UI settings");
ws.close();
});
test("rejects control ui without device identity by default", async () => {
const ws = await openWs(port);
const res = await connectReq(ws, {
+5
View File
@@ -263,6 +263,8 @@ export async function startGatewayServer(
const {
canvasHost,
httpServer,
httpServers,
httpBindHosts,
wss,
clients,
broadcast,
@@ -292,6 +294,7 @@ export async function startGatewayServer(
canvasHostEnabled,
allowCanvasHostInTests: opts.allowCanvasHostInTests,
logCanvas,
log,
logHooks,
logPlugins,
});
@@ -464,6 +467,7 @@ export async function startGatewayServer(
logGatewayStartup({
cfg: cfgAtStart,
bindHost,
bindHosts: httpBindHosts,
port,
tlsEnabled: gatewayTls.enabled,
log,
@@ -552,6 +556,7 @@ export async function startGatewayServer(
browserControl,
wss,
httpServer,
httpServers,
});
return {
@@ -66,19 +66,34 @@ function formatGatewayAuthFailureMessage(params: {
authMode: ResolvedGatewayAuth["mode"];
authProvided: AuthProvidedKind;
reason?: string;
client?: { id?: string | null; mode?: string | null };
}): string {
const { authMode, authProvided, reason } = params;
const { authMode, authProvided, reason, client } = params;
const isCli = isGatewayCliClient(client);
const isControlUi = client?.id === GATEWAY_CLIENT_IDS.CONTROL_UI;
const isWebchat = isWebchatClient(client);
const uiHint = "open a tokenized dashboard URL or paste token in Control UI settings";
const tokenHint = isCli
? "set gateway.remote.token to match gateway.auth.token"
: isControlUi || isWebchat
? uiHint
: "provide gateway auth token";
const passwordHint = isCli
? "set gateway.remote.password to match gateway.auth.password"
: isControlUi || isWebchat
? "enter the password in Control UI settings"
: "provide gateway auth password";
switch (reason) {
case "token_missing":
return "unauthorized: gateway token missing (set gateway.remote.token to match gateway.auth.token)";
return `unauthorized: gateway token missing (${tokenHint})`;
case "token_mismatch":
return "unauthorized: gateway token mismatch (set gateway.remote.token to match gateway.auth.token)";
return `unauthorized: gateway token mismatch (${tokenHint})`;
case "token_missing_config":
return "unauthorized: gateway token not configured on gateway (set gateway.auth.token)";
case "password_missing":
return "unauthorized: gateway password missing (set gateway.remote.password to match gateway.auth.password)";
return `unauthorized: gateway password missing (${passwordHint})`;
case "password_mismatch":
return "unauthorized: gateway password mismatch (set gateway.remote.password to match gateway.auth.password)";
return `unauthorized: gateway password mismatch (${passwordHint})`;
case "password_missing_config":
return "unauthorized: gateway password not configured on gateway (set gateway.auth.password)";
case "tailscale_user_missing":
@@ -90,10 +105,10 @@ function formatGatewayAuthFailureMessage(params: {
}
if (authMode === "token" && authProvided === "none") {
return "unauthorized: gateway token missing (set gateway.remote.token to match gateway.auth.token)";
return `unauthorized: gateway token missing (${tokenHint})`;
}
if (authMode === "password" && authProvided === "none") {
return "unauthorized: gateway password missing (set gateway.remote.password to match gateway.auth.password)";
return `unauthorized: gateway password missing (${passwordHint})`;
}
return "unauthorized";
}
@@ -532,6 +547,7 @@ export function attachGatewayWsMessageHandler(params: {
authMode: resolvedAuth.mode,
authProvided,
reason: authResult.reason,
client: connectParams.client,
});
setCloseCause("unauthorized", {
authMode: resolvedAuth.mode,
+11 -7
View File
@@ -4,6 +4,7 @@ import { promisify } from "node:util";
import { danger, shouldLogVerbose } from "../globals.js";
import { logDebug, logError } from "../logger.js";
import { resolveCommandStdio } from "./spawn-utils.js";
const execFileAsync = promisify(execFile);
@@ -78,19 +79,22 @@ export async function runCommandWithTimeout(
if (resolvedEnv.npm_config_fund == null) resolvedEnv.npm_config_fund = "false";
}
const stdio = resolveCommandStdio({ hasInput, preferInherit: true });
const child = spawn(argv[0], argv.slice(1), {
stdio,
cwd,
env: resolvedEnv,
windowsVerbatimArguments,
});
// Spawn with inherited stdin (TTY) so tools like `pi` stay interactive when needed.
return await new Promise((resolve, reject) => {
const child = spawn(argv[0], argv.slice(1), {
stdio: [hasInput ? "pipe" : "inherit", "pipe", "pipe"],
cwd,
env: resolvedEnv,
windowsVerbatimArguments,
});
let stdout = "";
let stderr = "";
let settled = false;
const timer = setTimeout(() => {
child.kill("SIGKILL");
if (typeof child.kill === "function") {
child.kill("SIGKILL");
}
}, timeoutMs);
if (hasInput && child.stdin) {
+64
View File
@@ -0,0 +1,64 @@
import { EventEmitter } from "node:events";
import { PassThrough } from "node:stream";
import type { ChildProcess } from "node:child_process";
import { describe, expect, it, vi } from "vitest";
import { spawnWithFallback } from "./spawn-utils.js";
function createStubChild() {
const child = new EventEmitter() as ChildProcess;
child.stdin = new PassThrough() as ChildProcess["stdin"];
child.stdout = new PassThrough() as ChildProcess["stdout"];
child.stderr = new PassThrough() as ChildProcess["stderr"];
child.pid = 1234;
child.killed = false;
child.kill = vi.fn(() => true) as ChildProcess["kill"];
queueMicrotask(() => {
child.emit("spawn");
});
return child;
}
describe("spawnWithFallback", () => {
it("retries on EBADF using fallback options", async () => {
const spawnMock = vi
.fn()
.mockImplementationOnce(() => {
const err = new Error("spawn EBADF");
(err as NodeJS.ErrnoException).code = "EBADF";
throw err;
})
.mockImplementationOnce(() => createStubChild());
const result = await spawnWithFallback({
argv: ["echo", "ok"],
options: { stdio: ["pipe", "pipe", "pipe"] },
fallbacks: [{ label: "safe-stdin", options: { stdio: ["ignore", "pipe", "pipe"] } }],
spawnImpl: spawnMock,
});
expect(result.usedFallback).toBe(true);
expect(result.fallbackLabel).toBe("safe-stdin");
expect(spawnMock).toHaveBeenCalledTimes(2);
expect(spawnMock.mock.calls[0]?.[2]?.stdio).toEqual(["pipe", "pipe", "pipe"]);
expect(spawnMock.mock.calls[1]?.[2]?.stdio).toEqual(["ignore", "pipe", "pipe"]);
});
it("does not retry on non-EBADF errors", async () => {
const spawnMock = vi.fn().mockImplementationOnce(() => {
const err = new Error("spawn ENOENT");
(err as NodeJS.ErrnoException).code = "ENOENT";
throw err;
});
await expect(
spawnWithFallback({
argv: ["missing"],
options: { stdio: ["pipe", "pipe", "pipe"] },
fallbacks: [{ label: "safe-stdin", options: { stdio: ["ignore", "pipe", "pipe"] } }],
spawnImpl: spawnMock,
}),
).rejects.toThrow(/ENOENT/);
expect(spawnMock).toHaveBeenCalledTimes(1);
});
});
+127
View File
@@ -0,0 +1,127 @@
import type { ChildProcess, SpawnOptions } from "node:child_process";
import { spawn } from "node:child_process";
export type SpawnFallback = {
label: string;
options: SpawnOptions;
};
export type SpawnWithFallbackResult = {
child: ChildProcess;
usedFallback: boolean;
fallbackLabel?: string;
};
type SpawnWithFallbackParams = {
argv: string[];
options: SpawnOptions;
fallbacks?: SpawnFallback[];
spawnImpl?: typeof spawn;
retryCodes?: string[];
onFallback?: (err: unknown, fallback: SpawnFallback) => void;
};
const DEFAULT_RETRY_CODES = ["EBADF"];
export function resolveCommandStdio(params: {
hasInput: boolean;
preferInherit: boolean;
}): ["pipe" | "inherit" | "ignore", "pipe", "pipe"] {
const stdin = params.hasInput ? "pipe" : params.preferInherit ? "inherit" : "pipe";
return [stdin, "pipe", "pipe"];
}
export function formatSpawnError(err: unknown): string {
if (!(err instanceof Error)) return String(err);
const details = err as NodeJS.ErrnoException;
const parts: string[] = [];
const message = err.message?.trim();
if (message) parts.push(message);
if (details.code && !message?.includes(details.code)) parts.push(details.code);
if (details.syscall) parts.push(`syscall=${details.syscall}`);
if (typeof details.errno === "number") parts.push(`errno=${details.errno}`);
return parts.join(" ");
}
function shouldRetry(err: unknown, codes: string[]): boolean {
const code =
err && typeof err === "object" && "code" in err ? String((err as { code?: unknown }).code) : "";
return code.length > 0 && codes.includes(code);
}
async function spawnAndWaitForSpawn(
spawnImpl: typeof spawn,
argv: string[],
options: SpawnOptions,
): Promise<ChildProcess> {
const child = spawnImpl(argv[0], argv.slice(1), options);
return await new Promise((resolve, reject) => {
let settled = false;
const cleanup = () => {
child.removeListener("error", onError);
child.removeListener("spawn", onSpawn);
};
const finishResolve = () => {
if (settled) return;
settled = true;
cleanup();
resolve(child);
};
const onError = (err: unknown) => {
if (settled) return;
settled = true;
cleanup();
reject(err);
};
const onSpawn = () => {
finishResolve();
};
child.once("error", onError);
child.once("spawn", onSpawn);
// Ensure mocked spawns that never emit "spawn" don't stall.
process.nextTick(() => {
if (typeof child.pid === "number") {
finishResolve();
}
});
});
}
export async function spawnWithFallback(
params: SpawnWithFallbackParams,
): Promise<SpawnWithFallbackResult> {
const spawnImpl = params.spawnImpl ?? spawn;
const retryCodes = params.retryCodes ?? DEFAULT_RETRY_CODES;
const baseOptions = { ...params.options };
const fallbacks = params.fallbacks ?? [];
const attempts: Array<{ label?: string; options: SpawnOptions }> = [
{ options: baseOptions },
...fallbacks.map((fallback) => ({
label: fallback.label,
options: { ...baseOptions, ...fallback.options },
})),
];
let lastError: unknown;
for (let index = 0; index < attempts.length; index += 1) {
const attempt = attempts[index];
try {
const child = await spawnAndWaitForSpawn(spawnImpl, params.argv, attempt.options);
return {
child,
usedFallback: index > 0,
fallbackLabel: attempt.label,
};
} catch (err) {
lastError = err;
const nextFallback = fallbacks[index];
if (!nextFallback || !shouldRetry(err, retryCodes)) {
throw err;
}
params.onFallback?.(err, nextFallback);
}
}
throw lastError;
}
@@ -126,6 +126,90 @@ describe("monitorSignalProvider tool results", () => {
}),
);
});
it("uses startupTimeoutMs override when provided", async () => {
const runtime = {
log: vi.fn(),
error: vi.fn(),
exit: ((code: number): never => {
throw new Error(`exit ${code}`);
}) as (code: number) => never,
};
config = {
...config,
channels: {
...config.channels,
signal: {
autoStart: true,
dmPolicy: "open",
allowFrom: ["*"],
startupTimeoutMs: 60_000,
},
},
};
const abortController = new AbortController();
streamMock.mockImplementation(async () => {
abortController.abort();
return;
});
await monitorSignalProvider({
autoStart: true,
baseUrl: "http://127.0.0.1:8080",
abortSignal: abortController.signal,
runtime,
startupTimeoutMs: 90_000,
});
expect(waitForTransportReadyMock).toHaveBeenCalledTimes(1);
expect(waitForTransportReadyMock).toHaveBeenCalledWith(
expect.objectContaining({
timeoutMs: 90_000,
}),
);
});
it("caps startupTimeoutMs at 2 minutes", async () => {
const runtime = {
log: vi.fn(),
error: vi.fn(),
exit: ((code: number): never => {
throw new Error(`exit ${code}`);
}) as (code: number) => never,
};
config = {
...config,
channels: {
...config.channels,
signal: {
autoStart: true,
dmPolicy: "open",
allowFrom: ["*"],
startupTimeoutMs: 180_000,
},
},
};
const abortController = new AbortController();
streamMock.mockImplementation(async () => {
abortController.abort();
return;
});
await monitorSignalProvider({
autoStart: true,
baseUrl: "http://127.0.0.1:8080",
abortSignal: abortController.signal,
runtime,
});
expect(waitForTransportReadyMock).toHaveBeenCalledTimes(1);
expect(waitForTransportReadyMock).toHaveBeenCalledWith(
expect.objectContaining({
timeoutMs: 120_000,
}),
);
});
it("sends tool summaries with responsePrefix", async () => {
const abortController = new AbortController();
replyMock.mockImplementation(async (_ctx, opts) => {
+6 -1
View File
@@ -43,6 +43,7 @@ export type MonitorSignalOpts = {
config?: ClawdbotConfig;
baseUrl?: string;
autoStart?: boolean;
startupTimeoutMs?: number;
cliPath?: string;
httpHost?: string;
httpPort?: number;
@@ -285,6 +286,10 @@ export async function monitorSignalProvider(opts: MonitorSignalOpts = {}): Promi
const sendReadReceipts = Boolean(opts.sendReadReceipts ?? accountInfo.config.sendReadReceipts);
const autoStart = opts.autoStart ?? accountInfo.config.autoStart ?? !accountInfo.config.httpUrl;
const startupTimeoutMs = Math.min(
120_000,
Math.max(1_000, opts.startupTimeoutMs ?? accountInfo.config.startupTimeoutMs ?? 30_000),
);
const readReceiptsViaDaemon = Boolean(autoStart && sendReadReceipts);
let daemonHandle: ReturnType<typeof spawnSignalDaemon> | null = null;
@@ -315,7 +320,7 @@ export async function monitorSignalProvider(opts: MonitorSignalOpts = {}): Promi
await waitForSignalDaemonReady({
baseUrl,
abortSignal: opts.abortSignal,
timeoutMs: 30_000,
timeoutMs: startupTimeoutMs,
logAfterMs: 10_000,
logIntervalMs: 10_000,
runtime,
+1
View File
@@ -151,6 +151,7 @@ export const dispatchTelegramMessage = async ({
tableMode,
chunkMode,
onVoiceRecording: sendRecordVoice,
linkPreview: telegramCfg.linkPreview,
});
},
onError: (err, info) => {
+1
View File
@@ -348,6 +348,7 @@ export const registerTelegramNativeCommands = ({
messageThreadId: resolvedThreadId,
tableMode,
chunkMode,
linkPreview: telegramCfg.linkPreview,
});
},
onError: (err, info) => {
+56
View File
@@ -108,4 +108,60 @@ describe("deliverReplies", () => {
}),
);
});
it("includes link_preview_options when linkPreview is false", async () => {
const runtime = { error: vi.fn(), log: vi.fn() };
const sendMessage = vi.fn().mockResolvedValue({
message_id: 3,
chat: { id: "123" },
});
const bot = { api: { sendMessage } } as unknown as Bot;
await deliverReplies({
replies: [{ text: "Check https://example.com" }],
chatId: "123",
token: "tok",
runtime,
bot,
replyToMode: "off",
textLimit: 4000,
linkPreview: false,
});
expect(sendMessage).toHaveBeenCalledWith(
"123",
expect.any(String),
expect.objectContaining({
link_preview_options: { is_disabled: true },
}),
);
});
it("does not include link_preview_options when linkPreview is true", async () => {
const runtime = { error: vi.fn(), log: vi.fn() };
const sendMessage = vi.fn().mockResolvedValue({
message_id: 4,
chat: { id: "123" },
});
const bot = { api: { sendMessage } } as unknown as Bot;
await deliverReplies({
replies: [{ text: "Check https://example.com" }],
chatId: "123",
token: "tok",
runtime,
bot,
replyToMode: "off",
textLimit: 4000,
linkPreview: true,
});
expect(sendMessage).toHaveBeenCalledWith(
"123",
expect.any(String),
expect.not.objectContaining({
link_preview_options: expect.anything(),
}),
);
});
});
+12 -1
View File
@@ -36,8 +36,11 @@ export async function deliverReplies(params: {
chunkMode?: ChunkMode;
/** Callback invoked before sending a voice message to switch typing indicator. */
onVoiceRecording?: () => Promise<void> | void;
/** Controls whether link previews are shown. Default: true (previews enabled). */
linkPreview?: boolean;
}) {
const { replies, chatId, runtime, bot, replyToMode, textLimit, messageThreadId } = params;
const { replies, chatId, runtime, bot, replyToMode, textLimit, messageThreadId, linkPreview } =
params;
const chunkMode = params.chunkMode ?? "length";
const threadParams = buildTelegramThreadParams(messageThreadId);
let hasReplied = false;
@@ -85,6 +88,7 @@ export async function deliverReplies(params: {
messageThreadId,
textMode: "html",
plainText: chunk.text,
linkPreview,
});
if (replyToId && !hasReplied) {
hasReplied = true;
@@ -180,6 +184,7 @@ export async function deliverReplies(params: {
messageThreadId,
textMode: "html",
plainText: chunk.text,
linkPreview,
});
if (replyToId && !hasReplied) {
hasReplied = true;
@@ -248,17 +253,22 @@ async function sendTelegramText(
messageThreadId?: number;
textMode?: "markdown" | "html";
plainText?: string;
linkPreview?: boolean;
},
): Promise<number | undefined> {
const baseParams = buildTelegramSendParams({
replyToMessageId: opts?.replyToMessageId,
messageThreadId: opts?.messageThreadId,
});
// Add link_preview_options when link preview is disabled.
const linkPreviewEnabled = opts?.linkPreview ?? true;
const linkPreviewOptions = linkPreviewEnabled ? undefined : { is_disabled: true };
const textMode = opts?.textMode ?? "markdown";
const htmlText = textMode === "html" ? text : markdownToTelegramHtml(text);
try {
const res = await bot.api.sendMessage(chatId, htmlText, {
parse_mode: "HTML",
...(linkPreviewOptions ? { link_preview_options: linkPreviewOptions } : {}),
...baseParams,
});
return res.message_id;
@@ -268,6 +278,7 @@ async function sendTelegramText(
runtime.log?.(`telegram HTML parse failed; retrying without formatting: ${errText}`);
const fallbackText = opts?.plainText ?? text;
const res = await bot.api.sendMessage(chatId, fallbackText, {
...(linkPreviewOptions ? { link_preview_options: linkPreviewOptions } : {}),
...baseParams,
});
return res.message_id;
@@ -152,6 +152,62 @@ describe("sendMessageTelegram", () => {
expect(res.messageId).toBe("42");
});
it("adds link_preview_options when previews are disabled in config", async () => {
const chatId = "123";
const sendMessage = vi.fn().mockResolvedValue({
message_id: 7,
chat: { id: chatId },
});
const api = { sendMessage } as unknown as {
sendMessage: typeof sendMessage;
};
loadConfig.mockReturnValue({
channels: { telegram: { linkPreview: false } },
});
await sendMessageTelegram(chatId, "hi", { token: "tok", api });
expect(sendMessage).toHaveBeenCalledWith(chatId, "hi", {
parse_mode: "HTML",
link_preview_options: { is_disabled: true },
});
});
it("keeps link_preview_options on plain-text fallback when disabled", async () => {
const chatId = "123";
const parseErr = new Error(
"400: Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 9",
);
const sendMessage = vi
.fn()
.mockRejectedValueOnce(parseErr)
.mockResolvedValueOnce({
message_id: 42,
chat: { id: chatId },
});
const api = { sendMessage } as unknown as {
sendMessage: typeof sendMessage;
};
loadConfig.mockReturnValue({
channels: { telegram: { linkPreview: false } },
});
await sendMessageTelegram(chatId, "_oops_", {
token: "tok",
api,
});
expect(sendMessage).toHaveBeenNthCalledWith(1, chatId, "<i>oops</i>", {
parse_mode: "HTML",
link_preview_options: { is_disabled: true },
});
expect(sendMessage).toHaveBeenNthCalledWith(2, chatId, "_oops_", {
link_preview_options: { is_disabled: true },
});
});
it("uses native fetch for BAN compatibility when api is omitted", async () => {
const originalFetch = globalThis.fetch;
const originalBun = (globalThis as { Bun?: unknown }).Bun;
+14 -9
View File
@@ -198,20 +198,25 @@ export async function sendMessageTelegram(
});
const renderHtmlText = (value: string) => renderTelegramHtmlText(value, { textMode, tableMode });
// Resolve link preview setting from config (default: enabled).
const linkPreviewEnabled = account.config.linkPreview ?? true;
const linkPreviewOptions = linkPreviewEnabled ? undefined : { is_disabled: true };
const sendTelegramText = async (
rawText: string,
params?: Record<string, unknown>,
fallbackText?: string,
) => {
const htmlText = renderHtmlText(rawText);
const sendParams = params
? {
parse_mode: "HTML" as const,
...params,
}
: {
parse_mode: "HTML" as const,
};
const baseParams = params ? { ...params } : {};
if (linkPreviewOptions) {
baseParams.link_preview_options = linkPreviewOptions;
}
const hasBaseParams = Object.keys(baseParams).length > 0;
const sendParams = {
parse_mode: "HTML" as const,
...baseParams,
};
const res = await request(() => api.sendMessage(chatId, htmlText, sendParams), "message").catch(
async (err) => {
// Telegram rejects malformed HTML (e.g., unsupported tags or entities).
@@ -222,7 +227,7 @@ export async function sendMessageTelegram(
console.warn(`telegram HTML parse failed, retrying as plain text: ${errText}`);
}
const fallback = fallbackText ?? rawText;
const plainParams = params && Object.keys(params).length > 0 ? { ...params } : undefined;
const plainParams = hasBaseParams ? baseParams : undefined;
return await request(
() =>
plainParams
+114 -6
View File
@@ -106,26 +106,36 @@ describe("tts", () => {
expect(isValidOpenAIVoice("alloy ")).toBe(false);
expect(isValidOpenAIVoice(" alloy")).toBe(false);
});
it("accepts custom voices for custom endpoints", () => {
expect(isValidOpenAIVoice("zm_yunxia", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
});
});
describe("isValidOpenAIModel", () => {
it("accepts gpt-4o-mini-tts model", () => {
it("accepts supported models", () => {
expect(isValidOpenAIModel("gpt-4o-mini-tts")).toBe(true);
expect(isValidOpenAIModel("tts-1")).toBe(true);
expect(isValidOpenAIModel("tts-1-hd")).toBe(true);
});
it("rejects other models", () => {
expect(isValidOpenAIModel("tts-1")).toBe(false);
expect(isValidOpenAIModel("tts-1-hd")).toBe(false);
expect(isValidOpenAIModel("invalid")).toBe(false);
expect(isValidOpenAIModel("")).toBe(false);
expect(isValidOpenAIModel("gpt-4")).toBe(false);
});
it("accepts custom models for custom endpoints", () => {
expect(isValidOpenAIModel("kokoro", { baseUrl: "http://localhost:8880/v1" })).toBe(true);
});
});
describe("OPENAI_TTS_MODELS", () => {
it("contains only gpt-4o-mini-tts", () => {
it("contains supported models", () => {
expect(OPENAI_TTS_MODELS).toContain("gpt-4o-mini-tts");
expect(OPENAI_TTS_MODELS).toHaveLength(1);
expect(OPENAI_TTS_MODELS).toContain("tts-1");
expect(OPENAI_TTS_MODELS).toContain("tts-1-hd");
expect(OPENAI_TTS_MODELS).toHaveLength(3);
});
it("is a non-empty array", () => {
@@ -200,6 +210,30 @@ describe("tts", () => {
expect(result.overrides.provider).toBe("edge");
});
it("routes custom OpenAI models when custom endpoints are used", () => {
const policy = resolveModelOverridePolicy({ enabled: true });
const input = "Hello [[tts:provider=openai model=kokoro voice=zm_yunxia]] world";
const result = parseTtsDirectives(input, policy, {
defaultProvider: "openai",
openaiBaseUrl: "http://localhost:8880/v1",
});
expect(result.overrides.openai?.model).toBe("kokoro");
expect(result.overrides.openai?.voice).toBe("zm_yunxia");
});
it("routes model overrides to ElevenLabs when provider is ElevenLabs", () => {
const policy = resolveModelOverridePolicy({ enabled: true });
const input = "Hello [[tts:provider=elevenlabs model=eleven_multilingual_v2]] world";
const result = parseTtsDirectives(input, policy, {
defaultProvider: "openai",
openaiBaseUrl: "http://localhost:8880/v1",
});
expect(result.overrides.elevenlabs?.modelId).toBe("eleven_multilingual_v2");
expect(result.overrides.openai?.model).toBeUndefined();
});
it("keeps text intact when overrides are disabled", () => {
const policy = resolveModelOverridePolicy({ enabled: false });
const input = "Hello [[tts:voice=alloy]] world";
@@ -357,7 +391,12 @@ describe("tts", () => {
};
const restoreEnv = (snapshot: Record<string, string | undefined>) => {
const keys = ["OPENAI_API_KEY", "ELEVENLABS_API_KEY", "XI_API_KEY"] as const;
const keys = [
"OPENAI_API_KEY",
"OPENAI_TTS_BASE_URL",
"ELEVENLABS_API_KEY",
"XI_API_KEY",
] as const;
for (const key of keys) {
const value = snapshot[key];
if (value === undefined) {
@@ -371,6 +410,7 @@ describe("tts", () => {
const withEnv = (env: Record<string, string | undefined>, run: () => void) => {
const snapshot = {
OPENAI_API_KEY: process.env.OPENAI_API_KEY,
OPENAI_TTS_BASE_URL: process.env.OPENAI_TTS_BASE_URL,
ELEVENLABS_API_KEY: process.env.ELEVENLABS_API_KEY,
XI_API_KEY: process.env.XI_API_KEY,
};
@@ -392,6 +432,7 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: "test-openai-key",
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
@@ -407,6 +448,7 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: "test-elevenlabs-key",
XI_API_KEY: undefined,
},
@@ -422,6 +464,7 @@ describe("tts", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: undefined,
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
@@ -432,6 +475,22 @@ describe("tts", () => {
},
);
});
it("prefers OpenAI when a custom endpoint is configured without API keys", () => {
withEnv(
{
OPENAI_API_KEY: undefined,
OPENAI_TTS_BASE_URL: "http://localhost:8880/v1",
ELEVENLABS_API_KEY: undefined,
XI_API_KEY: undefined,
},
() => {
const config = resolveTtsConfig(baseCfg);
const provider = getTtsProvider(config, "/tmp/tts-prefs-openai-custom.json");
expect(provider).toBe("openai");
},
);
});
});
describe("maybeApplyTtsToPayload", () => {
@@ -495,6 +554,55 @@ describe("tts", () => {
process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
});
it("allows custom OpenAI endpoints without API keys", async () => {
const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
const prevBaseUrl = process.env.OPENAI_TTS_BASE_URL;
const prevOpenAiKey = process.env.OPENAI_API_KEY;
process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
process.env.OPENAI_TTS_BASE_URL = "http://localhost:8880/v1";
delete process.env.OPENAI_API_KEY;
const originalFetch = globalThis.fetch;
const fetchMock = vi.fn(async () => ({
ok: true,
arrayBuffer: async () => new ArrayBuffer(1),
}));
globalThis.fetch = fetchMock as unknown as typeof fetch;
const cfg = {
agents: { defaults: { model: { primary: "openai/gpt-4o-mini" } } },
messages: {
tts: {
auto: "always",
provider: "openai",
openai: { model: "kokoro", voice: "zm_yunxia" },
},
},
};
const result = await maybeApplyTtsToPayload({
payload: { text: "Hello world" },
cfg,
kind: "final",
});
expect(result.mediaUrl).toBeDefined();
expect(fetchMock).toHaveBeenCalledTimes(1);
const [url, options] = fetchMock.mock.calls[0] ?? [];
expect(url).toBe("http://localhost:8880/v1/audio/speech");
expect(
(options as { headers?: Record<string, string> })?.headers?.Authorization,
).toBeUndefined();
globalThis.fetch = originalFetch;
process.env.OPENAI_TTS_BASE_URL = prevBaseUrl;
if (prevOpenAiKey === undefined) {
delete process.env.OPENAI_API_KEY;
} else {
process.env.OPENAI_API_KEY = prevOpenAiKey;
}
process.env.CLAWDBOT_TTS_PREFS = prevPrefs;
});
it("skips auto-TTS in tagged mode unless a tts tag is present", async () => {
const prevPrefs = process.env.CLAWDBOT_TTS_PREFS;
process.env.CLAWDBOT_TTS_PREFS = `/tmp/tts-test-${Date.now()}.json`;
+111 -25
View File
@@ -48,6 +48,7 @@ const DEFAULT_ELEVENLABS_VOICE_ID = "pMsXgVXv3BLzUgSXRplE";
const DEFAULT_ELEVENLABS_MODEL_ID = "eleven_multilingual_v2";
const DEFAULT_OPENAI_MODEL = "gpt-4o-mini-tts";
const DEFAULT_OPENAI_VOICE = "alloy";
const DEFAULT_OPENAI_TTS_BASE_URL = "https://api.openai.com/v1";
const DEFAULT_EDGE_VOICE = "en-US-MichelleNeural";
const DEFAULT_EDGE_LANG = "en-US";
const DEFAULT_EDGE_OUTPUT_FORMAT = "audio-24khz-48kbitrate-mono-mp3";
@@ -103,6 +104,7 @@ export type ResolvedTtsConfig = {
};
openai: {
apiKey?: string;
baseUrl: string;
model: string;
voice: string;
};
@@ -234,6 +236,9 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
const raw: TtsConfig = cfg.messages?.tts ?? {};
const providerSource = raw.provider ? "config" : "default";
const edgeOutputFormat = raw.edge?.outputFormat?.trim();
const openaiBaseUrl = normalizeOpenAiTtsBaseUrl(
raw.openai?.baseUrl?.trim() || process.env.OPENAI_TTS_BASE_URL,
);
const auto = normalizeTtsAutoMode(raw.auto) ?? (raw.enabled ? "always" : "off");
return {
auto,
@@ -265,6 +270,7 @@ export function resolveTtsConfig(cfg: ClawdbotConfig): ResolvedTtsConfig {
},
openai: {
apiKey: raw.openai?.apiKey,
baseUrl: openaiBaseUrl,
model: raw.openai?.model ?? DEFAULT_OPENAI_MODEL,
voice: raw.openai?.voice ?? DEFAULT_OPENAI_VOICE,
},
@@ -395,7 +401,9 @@ export function getTtsProvider(config: ResolvedTtsConfig, prefsPath: string): Tt
if (prefs.tts?.provider) return prefs.tts.provider;
if (config.providerSource === "config") return config.provider;
if (resolveTtsApiKey(config, "openai")) return "openai";
if (resolveTtsApiKey(config, "openai") || isCustomOpenAiTtsEndpoint(config.openai.baseUrl)) {
return "openai";
}
if (resolveTtsApiKey(config, "elevenlabs")) return "elevenlabs";
return "edge";
}
@@ -470,6 +478,12 @@ export function resolveTtsProviderOrder(primary: TtsProvider): TtsProvider[] {
export function isTtsProviderConfigured(config: ResolvedTtsConfig, provider: TtsProvider): boolean {
if (provider === "edge") return config.edge.enabled;
if (provider === "openai") {
return (
Boolean(resolveTtsApiKey(config, provider)) ||
isCustomOpenAiTtsEndpoint(config.openai.baseUrl)
);
}
return Boolean(resolveTtsApiKey(config, provider));
}
@@ -483,6 +497,16 @@ function normalizeElevenLabsBaseUrl(baseUrl: string): string {
return trimmed.replace(/\/+$/, "");
}
function normalizeOpenAiTtsBaseUrl(baseUrl?: string): string {
const trimmed = baseUrl?.trim();
if (!trimmed) return DEFAULT_OPENAI_TTS_BASE_URL;
return trimmed.replace(/\/+$/, "");
}
function isCustomOpenAiTtsEndpoint(baseUrl: string): boolean {
return normalizeOpenAiTtsBaseUrl(baseUrl) !== DEFAULT_OPENAI_TTS_BASE_URL;
}
function requireInRange(value: number, min: number, max: number, label: string): void {
if (!Number.isFinite(value) || value < min || value > max) {
throw new Error(`${label} must be between ${min} and ${max}`);
@@ -538,6 +562,7 @@ function parseNumberValue(value: string): number | undefined {
function parseTtsDirectives(
text: string,
policy: ResolvedTtsModelOverrides,
options: { defaultProvider?: TtsProvider; openaiBaseUrl?: string } = {},
): TtsDirectiveParseResult {
if (!policy.enabled) {
return { cleanedText: text, overrides: {}, warnings: [], hasDirective: false };
@@ -561,6 +586,30 @@ function parseTtsDirectives(
cleanedText = cleanedText.replace(directiveRegex, (_match, body: string) => {
hasDirective = true;
const tokens = body.split(/\s+/).filter(Boolean);
let providerOverride: TtsProvider | undefined;
for (const token of tokens) {
const eqIndex = token.indexOf("=");
if (eqIndex === -1) continue;
const rawKey = token.slice(0, eqIndex).trim();
const rawValue = token.slice(eqIndex + 1).trim();
if (!rawKey || !rawValue) continue;
if (rawKey.toLowerCase() !== "provider") continue;
if (!policy.allowProvider) break;
if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
providerOverride = rawValue;
} else {
warnings.push(`unsupported provider "${rawValue}"`);
}
break;
}
if (providerOverride) {
overrides.provider = providerOverride;
}
const providerHint = overrides.provider ?? options.defaultProvider;
const openAiContext = { baseUrl: options.openaiBaseUrl };
for (const token of tokens) {
const eqIndex = token.indexOf("=");
if (eqIndex === -1) continue;
@@ -571,18 +620,12 @@ function parseTtsDirectives(
try {
switch (key) {
case "provider":
if (!policy.allowProvider) break;
if (rawValue === "openai" || rawValue === "elevenlabs" || rawValue === "edge") {
overrides.provider = rawValue;
} else {
warnings.push(`unsupported provider "${rawValue}"`);
}
break;
case "voice":
case "openai_voice":
case "openaivoice":
if (!policy.allowVoice) break;
if (isValidOpenAIVoice(rawValue)) {
if (isValidOpenAIVoice(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, voice: rawValue };
} else {
warnings.push(`invalid OpenAI voice "${rawValue}"`);
@@ -602,17 +645,40 @@ function parseTtsDirectives(
case "model":
case "modelid":
case "model_id":
case "elevenlabs_model":
case "elevenlabsmodel":
case "openai_model":
case "openaimodel":
if (!policy.allowModelId) break;
if (isValidOpenAIModel(rawValue)) {
if (key === "openai_model" || key === "openaimodel") {
if (isValidOpenAIModel(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
warnings.push(`invalid OpenAI model "${rawValue}"`);
}
break;
}
if (providerHint === "openai") {
if (isValidOpenAIModel(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
warnings.push(`invalid OpenAI model "${rawValue}"`);
}
break;
}
if (providerHint === "elevenlabs") {
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
break;
}
if (isValidOpenAIModel(rawValue, openAiContext)) {
overrides.openai = { ...overrides.openai, model: rawValue };
} else {
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
}
break;
case "elevenlabs_model":
case "elevenlabsmodel":
if (!policy.allowModelId) break;
overrides.elevenlabs = { ...overrides.elevenlabs, modelId: rawValue };
break;
case "stability":
if (!policy.allowVoiceSettings) break;
{
@@ -736,7 +802,7 @@ function parseTtsDirectives(
};
}
export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts"] as const;
export const OPENAI_TTS_MODELS = ["gpt-4o-mini-tts", "tts-1", "tts-1-hd"] as const;
export const OPENAI_TTS_VOICES = [
"alloy",
"ash",
@@ -750,12 +816,15 @@ export const OPENAI_TTS_VOICES = [
] as const;
type OpenAiTtsVoice = (typeof OPENAI_TTS_VOICES)[number];
type OpenAiTtsValidationContext = { baseUrl?: string };
function isValidOpenAIModel(model: string): boolean {
function isValidOpenAIModel(model: string, context: OpenAiTtsValidationContext = {}): boolean {
if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
return OPENAI_TTS_MODELS.includes(model as (typeof OPENAI_TTS_MODELS)[number]);
}
function isValidOpenAIVoice(voice: string): voice is OpenAiTtsVoice {
function isValidOpenAIVoice(voice: string, context: OpenAiTtsValidationContext = {}): boolean {
if (context.baseUrl && isCustomOpenAiTtsEndpoint(context.baseUrl)) return true;
return OPENAI_TTS_VOICES.includes(voice as OpenAiTtsVoice);
}
@@ -963,18 +1032,21 @@ async function elevenLabsTTS(params: {
async function openaiTTS(params: {
text: string;
apiKey: string;
apiKey?: string;
baseUrl: string;
model: string;
voice: string;
responseFormat: "mp3" | "opus";
timeoutMs: number;
}): Promise<Buffer> {
const { text, apiKey, model, voice, responseFormat, timeoutMs } = params;
const { text, model, voice, responseFormat, timeoutMs } = params;
const apiKey = params.apiKey?.trim();
const baseUrl = normalizeOpenAiTtsBaseUrl(params.baseUrl);
if (!isValidOpenAIModel(model)) {
if (!isValidOpenAIModel(model, { baseUrl })) {
throw new Error(`Invalid model: ${model}`);
}
if (!isValidOpenAIVoice(voice)) {
if (!isValidOpenAIVoice(voice, { baseUrl })) {
throw new Error(`Invalid voice: ${voice}`);
}
@@ -982,12 +1054,15 @@ async function openaiTTS(params: {
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetch("https://api.openai.com/v1/audio/speech", {
const headers: Record<string, string> = {
"Content-Type": "application/json",
};
if (apiKey) {
headers.Authorization = `Bearer ${apiKey}`;
}
const response = await fetch(`${baseUrl}/audio/speech`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
headers,
body: JSON.stringify({
model,
input: text,
@@ -1137,13 +1212,19 @@ export async function textToSpeech(params: {
}
const apiKey = resolveTtsApiKey(config, provider);
if (!apiKey) {
const allowMissingKey =
provider === "openai" && isCustomOpenAiTtsEndpoint(config.openai.baseUrl);
if (!apiKey && !allowMissingKey) {
lastError = `No API key for ${provider}`;
continue;
}
let audioBuffer: Buffer;
if (provider === "elevenlabs") {
if (!apiKey) {
lastError = "No API key for elevenlabs";
continue;
}
const voiceIdOverride = params.overrides?.elevenlabs?.voiceId;
const modelIdOverride = params.overrides?.elevenlabs?.modelId;
const voiceSettings = {
@@ -1172,6 +1253,7 @@ export async function textToSpeech(params: {
audioBuffer = await openaiTTS({
text: params.text,
apiKey,
baseUrl: config.openai.baseUrl,
model: openaiModelOverride ?? config.openai.model,
voice: openaiVoiceOverride ?? config.openai.voice,
responseFormat: output.openai,
@@ -1227,8 +1309,12 @@ export async function maybeApplyTtsToPayload(params: {
});
if (autoMode === "off") return params.payload;
const defaultProvider = getTtsProvider(config, prefsPath);
const text = params.payload.text ?? "";
const directives = parseTtsDirectives(text, config.modelOverrides);
const directives = parseTtsDirectives(text, config.modelOverrides, {
defaultProvider,
openaiBaseUrl: config.openai.baseUrl,
});
if (directives.warnings.length > 0) {
logVerbose(`TTS: ignored directive overrides (${directives.warnings.join("; ")})`);
}
+5 -1
View File
@@ -124,6 +124,7 @@ export function connectGateway(host: GatewayHost) {
mode: "webchat",
onHello: (hello) => {
host.connected = true;
host.lastError = null;
host.hello = hello;
applySnapshot(host, hello);
void loadAssistantIdentity(host as unknown as ClawdbotApp);
@@ -134,7 +135,10 @@ export function connectGateway(host: GatewayHost) {
},
onClose: ({ code, reason }) => {
host.connected = false;
host.lastError = `disconnected (${code}): ${reason || "no reason"}`;
// Code 1012 = Service Restart (expected during config saves, don't show as error)
if (code !== 1012) {
host.lastError = `disconnected (${code}): ${reason || "no reason"}`;
}
},
onEvent: (evt) => handleGatewayEvent(host, evt),
onGap: ({ expected, received }) => {
+23 -1
View File
@@ -38,7 +38,7 @@ describe("config view", () => {
onSubsectionChange: vi.fn(),
});
it("disables save when form is unsafe", () => {
it("allows save when form is unsafe", () => {
const container = document.createElement("div");
render(
renderConfig({
@@ -59,6 +59,28 @@ describe("config view", () => {
container,
);
const saveButton = Array.from(
container.querySelectorAll("button"),
).find((btn) => btn.textContent?.trim() === "Save") as
| HTMLButtonElement
| undefined;
expect(saveButton).not.toBeUndefined();
expect(saveButton?.disabled).toBe(false);
});
it("disables save when schema is missing", () => {
const container = document.createElement("div");
render(
renderConfig({
...baseProps(),
schema: null,
formMode: "form",
formValue: { gateway: { mode: "local" } },
originalValue: {},
}),
container,
);
const saveButton = Array.from(
container.querySelectorAll("button"),
).find((btn) => btn.textContent?.trim() === "Save") as
+3 -2
View File
@@ -233,9 +233,10 @@ export function renderConfig(props: ConfigProps) {
const hasRawChanges = props.formMode === "raw" && props.raw !== props.originalRaw;
const hasChanges = props.formMode === "form" ? diff.length > 0 : hasRawChanges;
// Save/apply buttons require actual changes to be enabled
// Save/apply buttons require actual changes to be enabled.
// Note: formUnsafe warns about unsupported schema paths but shouldn't block saving.
const canSaveForm =
Boolean(props.formValue) && !props.loading && !formUnsafe;
Boolean(props.formValue) && !props.loading && Boolean(analysis.schema);
const canSave =
props.connected &&
!props.saving &&