Compare commits

...
Author SHA1 Message Date
Peter Steinberger fdf49a38b4 docs: add PR number to changelog 2026-01-03 03:49:22 +01:00
Peter Steinberger 2eff100434 style: format lint fixes 2026-01-03 03:49:08 +01:00
Peter Steinberger dd1cf54c54 test: cover gmail watcher restart 2026-01-03 03:48:23 +01:00
Peter Steinberger 0f95f32642 docs: thank @jverdi in changelog 2026-01-03 03:34:54 +01:00
Peter Steinberger f4b488e8bd docs: note gmail watcher auto-start 2026-01-03 03:30:36 +01:00
Peter Steinberger efd10c49db fix: harden gmail watcher restart 2026-01-03 03:30:27 +01:00
Jared Verdi aae54b4af2 Gmail watcher: start when gateway (re)starts 2026-01-01 23:09:44 -05:00
Peter Steinberger 35582cfe8a docs: fix broken clawd.md link in index 2026-01-02 02:45:01 +00:00
Peter Steinberger 76e24653e9 fix(media): preserve GIF animation, skip JPEG optimization
- Skip JPEG optimization for image/gif content type (both local and URL)
- Preserves animation in uploaded GIFs to Discord/other providers
- Added tests for GIF preservation from local files and URLs
- Updated changelog
2026-01-02 00:56:04 +00:00
Peter Steinberger 4c2812b429 fix: refine HEARTBEAT_OK handling 2026-01-02 01:42:27 +01:00
Peter Steinberger c31070db24 style: apply biome formatting 2026-01-02 01:29:05 +01:00
Peter Steinberger 336048441c docs: add imessage rpc and groups docs 2026-01-02 01:19:40 +01:00
Peter Steinberger cbac34347b feat: add imessage rpc adapter 2026-01-02 01:19:40 +01:00
Peter Steinberger 3ee27a00c7 docs(changelog): note log prefix cleanup 2026-01-02 00:15:03 +00:00
Peter Steinberger 4ec020a86d fix(logging): trim provider log prefixes 2026-01-02 00:15:01 +00:00
Peter Steinberger 464dabdc16 docs: default discord reactions to on 2026-01-02 01:11:04 +01:00
Peter Steinberger c0976ec099 fix(gateway): stream chat events for agent runs 2026-01-02 01:04:59 +01:00
Peter Steinberger 7f3113b8d4 feat: add discord reaction tool 2026-01-02 00:29:32 +01:00
Peter Steinberger 9180cbe821 fix: keep chat scrolled to bottom on session switch 2026-01-02 00:21:48 +01:00
62 changed files with 3208 additions and 241 deletions

No files matched your search

+10 -1
View File
@@ -2,6 +2,9 @@
## 2.0.0-beta5 — Unreleased
### Fixed
- Media: preserve GIF animation when uploading to Discord/other providers (skip JPEG optimization for image/gif).
### Breaking
- Skills config schema moved under `skills.*`:
- `skillsLoad.extraDirs` → `skills.load.extraDirs`
@@ -15,7 +18,9 @@
- Nix mode: opt-in declarative config + read-only settings UI when `CLAWDIS_NIX_MODE=1` (thanks @joshp123 for the persistence — earned my trust; I'll merge these going forward).
- Agent runtime: accept legacy `Z_AI_API_KEY` for Z.AI provider auth (maps to `ZAI_API_KEY`).
- Signal: add `signal-cli` JSON-RPC support for send/receive via the Signal provider.
- iMessage: add imsg JSON-RPC integration (stdio), chat_id routing, and group chat support.
- Chat UI: add recent-session dropdown switcher (main first) in macOS/iOS/Android + Control UI.
- Discord: allow agent-triggered reactions via `clawdis_discord` when enabled, and surface message ids in context.
- Tests: add a Z.AI live test gate for smoke validation when keys are present.
- macOS Debug: add app log verbosity and rolling file log toggle for swift-log-backed app logs.
- CLI: add onboarding wizard (gateway + workspace + skills) with daemon installers and Anthropic/Minimax setup paths.
@@ -30,8 +35,12 @@
- Browser tools: add remote CDP URL support, Linux launcher options (`executablePath`, `noSandbox`), and surface `cdpUrl` in status.
### Fixes
- Gmail hooks: auto-restart watcher on gateway restart, prevent double-start, redact gog tokens (#87) — thanks @jverdi.
- Chat UI: keep the chat scrolled to the latest message after switching sessions.
- WebChat: stream live updates for sessions even when runs start outside the chat UI.
- Gateway CLI: read `CLAWDIS_GATEWAY_PASSWORD` from environment in `callGateway()` — allows `doctor`/`health` commands to auth without explicit `--password` flag.
- Auto-reply: suppress stray `HEARTBEAT_OK` acks so they never get delivered as messages.
- Auto-reply: strip stray leading/trailing `HEARTBEAT_OK` from normal replies; drop short (≤ 30 chars) heartbeat acks.
- Logging: trim provider prefix duplication in Discord/Signal/Telegram runtime log lines.
- Discord: include recent guild context when replying to mentions and add `discord.historyLimit` to tune how many messages are captured.
- Skills: switch imsg installer to brew tap formula.
- Skills: gate macOS-only skills by OS and surface block reasons in the Skills UI.
+2 -2
View File
@@ -16,7 +16,7 @@
</p>
**Clawdis** is a *personal AI assistant* you run on your own devices.
It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.
It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, iMessage, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.
If you want a private, single-user assistant that feels local, fast, and always-on, this is it.
@@ -45,7 +45,7 @@ Your surfaces
## What Clawdis does
- **Personal assistant** — one user, one identity, one memory surface.
- **Multi-surface inbox** — WhatsApp, Telegram, Discord, WebChat, macOS, iOS. Signal support via `signal-cli` (see `docs/signal.md`).
- **Multi-surface inbox** — WhatsApp, Telegram, Discord, iMessage, WebChat, macOS, iOS. Signal support via `signal-cli` (see `docs/signal.md`). iMessage uses `imsg` (see `docs/imessage.md`).
- **Voice wake + push-to-talk** — local speech recognition on macOS/iOS.
- **Canvas** — a live visual workspace you can drive from the agent.
- **Automation-ready** — browser control, media handling, and tool streaming.
@@ -129,6 +129,9 @@ public struct ClawdisChatView: View {
self.scrollPosition = self.scrollerBottomID
self.hasPerformedInitialScroll = true
}
.onChange(of: self.viewModel.sessionKey) { _, _ in
self.hasPerformedInitialScroll = false
}
.onChange(of: self.viewModel.messages.count) { _, _ in
guard self.hasPerformedInitialScroll else { return }
withAnimation(.snappy(duration: 0.22)) {
+1 -1
View File
@@ -18,4 +18,4 @@ read_when:
- Output:
- Default: prints text (and `MEDIA:<url>` lines) to stdout.
- `--json`: prints structured payloads + meta.
- Optional: `--deliver` sends the reply back to the selected provider (requires `--to` for WhatsApp).
- Optional: `--deliver` sends the reply back to the selected provider (`whatsapp`, `telegram`, `discord`, `signal`, `imessage`).
+35 -4
View File
@@ -88,7 +88,7 @@ Allowlist of E.164 phone numbers that may trigger auto-replies.
### `routing.groupChat`
Group messages default to **require mention** (either metadata mention or regex patterns).
Group messages default to **require mention** (either metadata mention or regex patterns). Applies to WhatsApp, Telegram, Discord, and iMessage group chats.
```json5
{
@@ -114,6 +114,7 @@ Controls how inbound messages behave when an agent run is already active.
whatsapp: "interrupt",
telegram: "interrupt",
discord: "queue",
imessage: "interrupt",
webchat: "queue"
}
}
@@ -179,13 +180,38 @@ Configure the Discord bot by setting the bot token and optional gating:
},
requireMention: true, // require @bot mentions in guilds
mediaMaxMb: 8, // clamp inbound media size
historyLimit: 20 // include last N guild messages as context
historyLimit: 20, // include last N guild messages as context
enableReactions: true // allow agent-triggered reactions
}
}
```
Clawdis reads `DISCORD_BOT_TOKEN` or `discord.token` to start the provider (unless `discord.enabled` is `false`). Use `user:<id>` (DM) or `channel:<id>` (guild channel) when specifying delivery targets for cron/CLI commands.
### `imessage` (imsg CLI)
Clawdis spawns `imsg rpc` (JSON-RPC over stdio). No daemon or port required.
```json5
{
imessage: {
enabled: true,
cliPath: "imsg",
dbPath: "~/Library/Messages/chat.db",
allowFrom: ["+15555550123", "user@example.com", "chat_id:123"],
includeAttachments: false,
mediaMaxMb: 16,
service: "auto",
region: "US"
}
}
```
Notes:
- Requires Full Disk Access to the Messages DB.
- The first send will prompt for Messages automation permission.
- Prefer `chat_id:<id>` targets. Use `imsg chats --limit 20` to list chats.
### `agent.workspace`
Sets the **single global workspace directory** used by the agent for file operations.
@@ -283,7 +309,7 @@ Z.AI models are available as `zai/<model>` (e.g. `zai/glm-4.7`) and require
- `every`: duration string (`ms`, `s`, `m`, `h`); default unit minutes. Omit or set
`0m` to disable.
- `model`: optional override model for heartbeat runs (`provider/model`).
- `target`: optional delivery channel (`last`, `whatsapp`, `telegram`, `discord`, `none`). Default: `last`.
- `target`: optional delivery channel (`last`, `whatsapp`, `telegram`, `discord`, `imessage`, `none`). Default: `last`.
- `to`: optional recipient override (E.164 for WhatsApp, chat id for Telegram).
- `prompt`: optional override for the heartbeat body (default: `HEARTBEAT`).
@@ -618,6 +644,11 @@ Gmail helper config (used by `clawdis hooks gmail setup` / `run`):
}
```
Gateway auto-start:
- If `hooks.enabled=true` and `hooks.gmail.account` is set, the Gateway starts
`gog gmail watch serve` on boot and auto-renews the watch.
- Set `CLAWDIS_SKIP_GMAIL_WATCHER=1` to disable the auto-start (for manual runs).
Note: when `tailscale.mode` is on, Clawdis defaults `serve.path` to `/` so
Tailscale can proxy `/gmail-pubsub` correctly (it strips the set-path prefix).
@@ -717,7 +748,7 @@ Template placeholders are expanded in `routing.transcribeAudio.command` (and any
| `{{GroupMembers}}` | Group members preview (best effort) |
| `{{SenderName}}` | Sender display name (best effort) |
| `{{SenderE164}}` | Sender phone number (best effort) |
| `{{Surface}}` | Surface hint (whatsapp|telegram|discord|webchat|…) |
| `{{Surface}}` | Surface hint (whatsapp|telegram|discord|imessage|webchat|…) |
## Cron (Gateway scheduler)
+2 -2
View File
@@ -75,7 +75,7 @@ Each job is a JSON object with stable keys (unknown keys ignored for forward com
- For `sessionTarget:"main"`, `wakeMode` controls whether we trigger the heartbeat immediately or just enqueue and wait.
- `payload` (one of)
- `{"kind":"systemEvent","text":string}` (enqueue as `System:`)
- `{"kind":"agentTurn","message":string,"deliver"?:boolean,"channel"?: "last"|"whatsapp"|"telegram","to"?:string,"timeoutSeconds"?:number}`
- `{"kind":"agentTurn","message":string,"deliver"?:boolean,"channel"?: "last"|"whatsapp"|"telegram"|"discord"|"signal"|"imessage","to"?:string,"timeoutSeconds"?:number}`
- `isolation` (optional; only meaningful for isolated jobs)
- `{"postToMainPrefix"?: string}`
- `runtime` (optional)
@@ -264,7 +264,7 @@ Add a `cron` command group (all commands should also support `--json` where sens
- `--wake now|next-heartbeat`
- payload flags (choose one):
- `--system-event "<text>"`
- `--message "<agent message>" [--deliver] [--channel last|whatsapp|telegram|discord] [--to <dest>]`
- `--message "<agent message>" [--deliver] [--channel last|whatsapp|telegram|discord|signal|imessage] [--to <dest>]`
- `clawdis cron edit <id> ...` (patch-by-flags, non-interactive)
- `clawdis cron rm <id>`
+11 -1
View File
@@ -24,6 +24,7 @@ Status: ready for DM and guild text channels via the official Discord bot gatewa
7. Optional DM allowlist: reuse `discord.allowFrom` with user ids (`1234567890` or `discord:1234567890`). Use `"*"` to allow all DMs.
8. Optional guild allowlist: set `discord.guildAllowFrom` with `guilds` and/or `users` to gate who can invoke the bot in servers.
9. Optional guild context history: set `discord.historyLimit` (default 20) to include the last N guild messages as context when replying to a mention. Set `0` to disable.
10. Reactions (default on): set `discord.enableReactions = false` to disable agent-triggered reactions via the `clawdis_discord` tool.
Note: Discord does not provide a simple username → id lookup without extra guild context, so prefer ids or `<@id>` mentions for DM delivery targets.
@@ -47,7 +48,8 @@ Note: Discord does not provide a simple username → id lookup without extra gui
},
requireMention: true,
mediaMaxMb: 8,
historyLimit: 20
historyLimit: 20,
enableReactions: true
}
}
```
@@ -57,6 +59,14 @@ Note: Discord does not provide a simple username → id lookup without extra gui
- `requireMention`: when `true`, messages in guild channels must mention the bot.
- `mediaMaxMb`: clamp inbound media saved to disk.
- `historyLimit`: number of recent guild messages to include as context when replying to a mention (default 20, `0` disables).
- `enableReactions`: allow agent-triggered reactions via the `clawdis_discord` tool (default `true`).
## Reactions
When `discord.enableReactions = true`, the agent can call `clawdis_discord` with:
- `action: "react"`
- `channelId`, `messageId`, `emoji`
Discord message ids are surfaced in the injected context (`[discord message id: …]` and history lines) so the agent can target them.
## Safety & ops
- Treat the bot token like a password; prefer the `DISCORD_BOT_TOKEN` env var on supervised hosts or lock down the config file permissions.
+6 -1
View File
@@ -56,7 +56,12 @@ Want a custom endpoint? Use `--push-endpoint <url>` or `--tailscale off`.
Platform note: on macOS the wizard installs `gcloud`, `gogcli`, and `tailscale`
via Homebrew; on Linux install them manually first.
Run the daemon (starts `gog gmail watch serve` + auto-renew):
Gateway auto-start (recommended):
- When `hooks.enabled=true` and `hooks.gmail.account` is set, the Gateway starts
`gog gmail watch serve` on boot and auto-renews the watch.
- Set `CLAWDIS_SKIP_GMAIL_WATCHER=1` to opt out (useful if you run the daemon yourself).
Manual daemon (starts `gog gmail watch serve` + auto-renew):
```bash
clawdis hooks gmail run
+56
View File
@@ -0,0 +1,56 @@
---
summary: "Group chat behavior across surfaces (WhatsApp/Telegram/Discord/iMessage)"
read_when:
- Changing group chat behavior or mention gating
---
# Groups
Clawdis treats group chats consistently across surfaces: WhatsApp, Telegram, Discord, iMessage.
## Session keys
- Group sessions use `group:<id>` in `ctx.From`.
- Direct chats use the main session (or per-sender if configured).
- Heartbeats are skipped for group sessions.
## Mention gating (default)
Group messages require a mention unless overridden per group.
```json5
{
routing: {
groupChat: {
requireMention: true,
mentionPatterns: ["@clawd", "clawdbot", "\\+15555550123"],
historyLimit: 50
}
}
}
```
Notes:
- `mentionPatterns` are case-insensitive regexes.
- Surfaces that provide explicit mentions still pass; patterns are a fallback.
## Activation (owner-only)
Group owners can toggle per-group activation:
- `/activation mention`
- `/activation always`
Owner is determined by `routing.allowFrom` (or the bot’s default identity when unset).
## Context fields
Group inbound payloads set:
- `ChatType=group`
- `GroupSubject` (if known)
- `GroupMembers` (if known)
- `WasMentioned` (mention gating result)
The agent system prompt includes a group intro on the first turn of a new group session.
## iMessage specifics
- Prefer `chat_id:<id>` when routing or allowlisting.
- List chats: `imsg chats --limit 20`.
- Group replies always go back to the same `chat_id`.
## WhatsApp specifics
See `docs/group-messages.md` for WhatsApp-only behavior (history injection, mention handling details).
+10 -2
View File
@@ -10,10 +10,18 @@ surface anything that needs attention without spamming the user.
## Prompt contract
- Heartbeat body defaults to `HEARTBEAT` (configurable via `agent.heartbeat.prompt`).
- If nothing needs attention, the model must reply **exactly** `HEARTBEAT_OK`.
- Any response containing `HEARTBEAT_OK` is treated as an ack and discarded.
- If nothing needs attention, the model should reply **exactly** `HEARTBEAT_OK`.
- During heartbeat runs, Clawdis treats `HEARTBEAT_OK` as an ack when it appears at
the **start or end** of the reply. Clawdis strips the token and discards the
reply if the remaining content is **≤ 30 characters**.
- If `HEARTBEAT_OK` is in the **middle** of a reply, it is not treated specially.
- For alerts, do **not** include `HEARTBEAT_OK`; return only the alert text.
### Stray `HEARTBEAT_OK` outside heartbeats
If the model accidentally includes `HEARTBEAT_OK` at the start or end of a
normal (non-heartbeat) reply, Clawdis strips the token and logs a verbose
message. If the reply is only `HEARTBEAT_OK`, it is dropped.
## Config
```json5
+63
View File
@@ -0,0 +1,63 @@
---
summary: "iMessage support via imsg (JSON-RPC over stdio), setup, and chat_id routing"
read_when:
- Setting up iMessage support
- Debugging iMessage send/receive
---
# iMessage (imsg)
Status: external CLI integration. No daemon.
## Model
- Clawdis spawns `imsg rpc` as a child process.
- JSON-RPC runs over stdin/stdout (one JSON object per line).
- Gateway owns the process; no TCP port needed.
## Requirements
- macOS with Messages signed in.
- Full Disk Access for Clawdis + the `imsg` binary (Messages DB access).
- Automation permission for Messages when sending.
## Config
```json5
{
imessage: {
enabled: true,
cliPath: "imsg",
dbPath: "~/Library/Messages/chat.db",
allowFrom: ["+15555550123", "user@example.com", "chat_id:123"],
includeAttachments: false,
mediaMaxMb: 16,
service: "auto",
region: "US"
}
}
```
Notes:
- `allowFrom` accepts handles (phone/email) or `chat_id:<id>` entries.
- `service` defaults to `auto` (use `imessage` or `sms` to pin).
- `region` is only used for SMS targeting.
## Addressing / targets
Prefer `chat_id` for stable routing:
- `chat_id:123` (preferred)
- `chat_guid:...` (fallback)
- `chat_identifier:...` (fallback)
- direct handles: `imessage:+1555` / `sms:+1555` / `user@example.com`
List chats:
```
imsg chats --limit 20
```
## Group chat behavior
- Group messages set `ChatType=group`, `GroupSubject`, and `GroupMembers`.
- Group activation respects `routing.groupChat.requireMention` and `mentionPatterns`.
- Replies go back to the same `chat_id` (group or direct).
## Troubleshooting
- `clawdis gateway call providers.status --params '{"probe":true}'`
- Verify `imsg` is on PATH and has access to Messages DB.
+8 -4
View File
@@ -13,17 +13,17 @@ read_when:
</p>
<p align="center">
<strong>WhatsApp + Telegram + Discord gateway for AI agents (Pi).</strong><br>
<strong>WhatsApp + Telegram + Discord + iMessage gateway for AI agents (Pi).</strong><br>
Send a message, get an agent response — from your pocket.
</p>
<p align="center">
<a href="https://github.com/steipete/clawdis">GitHub</a> ·
<a href="https://github.com/steipete/clawdis/releases">Releases</a> ·
<a href="./clawd">Clawd setup</a>
<a href="./clawd.md">Clawd setup</a>
</p>
CLAWDIS bridges WhatsApp (via WhatsApp Web / Baileys), Telegram (Bot API / grammY), and Discord (Bot API / discord.js) to coding agents like [Pi](https://github.com/badlogic/pi-mono).
CLAWDIS bridges WhatsApp (via WhatsApp Web / Baileys), Telegram (Bot API / grammY), Discord (Bot API / discord.js), and iMessage (imsg CLI) to coding agents like [Pi](https://github.com/badlogic/pi-mono).
It’s built for [Clawd](https://clawd.me), a space lobster who needed a TARDIS.
## How it works
@@ -61,6 +61,7 @@ Most operations flow through the **Gateway** (`clawdis gateway`), a single long-
- 📱 **WhatsApp Integration** — Uses Baileys for WhatsApp Web protocol
- ✈️ **Telegram Bot** — DMs + groups via grammY
- 🎮 **Discord Bot** — DMs + guild channels via discord.js
- 💬 **iMessage** — Local imsg CLI integration (macOS)
- 🤖 **Agent bridge** — Pi (RPC mode) with tool streaming
- 💬 **Sessions** — Direct chats collapse into shared `main` (default); groups are isolated
- 👥 **Group Chat Support** — Mention-based by default; owner can toggle `/activation always|mention`
@@ -121,6 +122,7 @@ Example:
- [Skills](./skills.md)
- [Skills config](./skills-config.md)
- [Workspace templates](./templates/AGENTS.md)
- [RPC adapters](./rpc.md)
- [Gateway runbook](./gateway.md)
- [Nodes (iOS/Android)](./nodes.md)
- [Web surfaces (Control UI)](./web.md)
@@ -131,7 +133,9 @@ Example:
- [Control UI (browser)](./control-ui.md)
- [Telegram](./telegram.md)
- [Discord](./discord.md)
- [Group messages](./group-messages.md)
- [iMessage](./imessage.md)
- [Groups](./groups.md)
- [WhatsApp group messages](./group-messages.md)
- [Media: images](./images.md)
- [Media: audio](./audio.md)
- Ops and safety:
+35
View File
@@ -0,0 +1,35 @@
---
summary: "RPC adapters for external CLIs (signal-cli, imsg) and gateway patterns"
read_when:
- Adding or changing external CLI integrations
- Debugging RPC adapters (signal-cli, imsg)
---
# RPC adapters
Clawdis integrates external CLIs via JSON-RPC. Two patterns are used today.
## Pattern A: HTTP daemon (signal-cli)
- `signal-cli` runs as a daemon with JSON-RPC over HTTP.
- Event stream is SSE (`/api/v1/events`).
- Health probe: `/api/v1/check`.
- Clawdis owns lifecycle when `signal.autoStart=true`.
See `docs/signal.md` for setup and endpoints.
## Pattern B: stdio child process (imsg)
- Clawdis spawns `imsg rpc` as a child process.
- JSON-RPC is line-delimited over stdin/stdout (one JSON object per line).
- No TCP port, no daemon required.
Core methods used:
- `watch.subscribe` → notifications (`method: "message"`)
- `watch.unsubscribe`
- `send`
- `chats.list` (probe/diagnostics)
See `docs/imessage.md` for setup and addressing (`chat_id` preferred).
## Adapter guidelines
- Gateway owns the process (start/stop tied to provider lifecycle).
- Keep RPC clients resilient: timeouts, restart on exit.
- Prefer stable IDs (e.g., `chat_id`) over display strings.
+2 -2
View File
@@ -9,12 +9,12 @@ Updated: 2025-12-07
Goal: make replies deterministic per channel while keeping one shared context for direct chats.
- **Surfaces** (channel labels): `whatsapp`, `webchat`, `telegram`, `discord`, `voice`, etc. Add `Surface` to inbound `MsgContext` so templates/agents can log which channel a turn came from. Routing is fixed: replies go back to the origin surface; the model doesn’t choose.
- **Surfaces** (channel labels): `whatsapp`, `webchat`, `telegram`, `discord`, `imessage`, `voice`, etc. Add `Surface` to inbound `MsgContext` so templates/agents can log which channel a turn came from. Routing is fixed: replies go back to the origin surface; the model doesn’t choose.
- **Reply context:** inbound replies include `ReplyToId`, `ReplyToBody`, and `ReplyToSender`, and the quoted context is appended to `Body` as a `[Replying to ...]` block.
- **Canonical direct session:** All direct chats collapse into the single `main` session by default (no config needed). Groups stay `group:<jid>`, so they remain isolated.
- **Session store:** Keys are resolved via `resolveSessionKey(scope, ctx, mainKey)`; the agent JSONL path lives under `~/.clawdis/sessions/<SessionId>.jsonl`.
- **WebChat:** Always attaches to `main`, loads the full session transcript so desktop reflects cross-surface history, and writes new turns back to the same session.
- **Implementation hints:**
- Set `Surface` in each ingress (WhatsApp gateway, WebChat bridge, Telegram, Discord).
- Set `Surface` in each ingress (WhatsApp gateway, WebChat bridge, Telegram, Discord, iMessage).
- Keep routing deterministic: originate → same surface. Use the gateway WebSocket for sends; avoid side channels.
- Do not let the agent emit “send to X” decisions; keep that policy in the host code.
+3 -2
View File
@@ -65,6 +65,7 @@ It does **not** install or change anything on the remote host.
- Telegram: bot token.
- Discord: bot token.
- Signal: optional `signal-cli` install + account config.
- iMessage: local `imsg` CLI path + DB access.
6) **Daemon install**
- macOS: LaunchAgent
@@ -131,7 +132,7 @@ Typical fields in `~/.clawdis/clawdis.json`:
- `agent.workspace`
- `agent.model` / `models.providers` (if Minimax chosen)
- `gateway.*` (mode, bind, auth, tailscale)
- `telegram.botToken`, `discord.token`, `signal.*`
- `telegram.botToken`, `discord.token`, `signal.*`, `imessage.*`
- `skills.install.nodeManager`
- `wizard.lastRunAt`
- `wizard.lastRunVersion`
@@ -146,5 +147,5 @@ Sessions are stored under `~/.clawdis/sessions/`.
- macOS app onboarding: `docs/onboarding.md`
- Config reference: `docs/configuration.md`
- Providers: `docs/whatsapp.md`, `docs/telegram.md`, `docs/discord.md`, `docs/signal.md`
- Providers: `docs/whatsapp.md`, `docs/telegram.md`, `docs/discord.md`, `docs/signal.md`, `docs/imessage.md`
- Skills: `docs/skills.md`, `docs/skills-config.md`
+44
View File
@@ -41,6 +41,7 @@ import {
} from "../cli/nodes-screen.js";
import { parseDurationMs } from "../cli/parse-duration.js";
import { loadConfig } from "../config/config.js";
import { reactMessageDiscord } from "../discord/send.js";
import { callGateway } from "../gateway/call.js";
import { detectMime } from "../media/mime.js";
import { sanitizeToolResultImages } from "./tool-images.js";
@@ -1422,6 +1423,48 @@ const GatewayToolSchema = Type.Union([
}),
]);
const DiscordToolSchema = Type.Union([
Type.Object({
action: Type.Literal("react"),
channelId: Type.String(),
messageId: Type.String(),
emoji: Type.String(),
}),
]);
function createDiscordTool(): AnyAgentTool {
return {
label: "Clawdis Discord",
name: "clawdis_discord",
description:
"React to Discord messages. Controlled by discord.enableReactions (default: true).",
parameters: DiscordToolSchema,
execute: async (_toolCallId, args) => {
const params = args as Record<string, unknown>;
const action = readStringParam(params, "action", { required: true });
if (action !== "react") throw new Error(`Unknown action: ${action}`);
const cfg = loadConfig();
if (cfg.discord?.enableReactions === false) {
throw new Error(
"Discord reactions are disabled (set discord.enableReactions=true).",
);
}
const channelId = readStringParam(params, "channelId", {
required: true,
});
const messageId = readStringParam(params, "messageId", {
required: true,
});
const emoji = readStringParam(params, "emoji", { required: true });
await reactMessageDiscord(channelId, messageId, emoji);
return jsonResult({ ok: true });
},
};
}
function createGatewayTool(): AnyAgentTool {
return {
label: "Clawdis Gateway",
@@ -1470,6 +1513,7 @@ export function createClawdisTools(): AnyAgentTool[] {
createCanvasTool(),
createNodesTool(),
createCronTool(),
createDiscordTool(),
createGatewayTool(),
];
}
+1 -1
View File
@@ -95,7 +95,7 @@ export function buildAgentSystemPromptAppend(params: {
"## Heartbeats",
'If you receive a heartbeat poll (a user message containing just "HEARTBEAT"), and there is nothing that needs attention, reply exactly:',
"HEARTBEAT_OK",
'Any response containing "HEARTBEAT_OK" is treated as a heartbeat ack and will not be delivered.',
'Clawdis treats a leading/trailing "HEARTBEAT_OK" as a heartbeat ack (and may discard it).',
'If something needs attention, do NOT include "HEARTBEAT_OK"; reply with the alert text instead.',
"",
"## Runtime",
+65 -31
View File
@@ -5,55 +5,89 @@ import { HEARTBEAT_TOKEN } from "./tokens.js";
describe("stripHeartbeatToken", () => {
it("skips empty or token-only replies", () => {
expect(stripHeartbeatToken(undefined)).toEqual({
expect(stripHeartbeatToken(undefined, { mode: "heartbeat" })).toEqual({
shouldSkip: true,
text: "",
didStrip: false,
});
expect(stripHeartbeatToken(" ")).toEqual({
expect(stripHeartbeatToken(" ", { mode: "heartbeat" })).toEqual({
shouldSkip: true,
text: "",
didStrip: false,
});
expect(stripHeartbeatToken(HEARTBEAT_TOKEN)).toEqual({
expect(stripHeartbeatToken(HEARTBEAT_TOKEN, { mode: "heartbeat" })).toEqual(
{
shouldSkip: true,
text: "",
didStrip: true,
},
);
});
it("drops heartbeats with small junk in heartbeat mode", () => {
expect(
stripHeartbeatToken("HEARTBEAT_OK 🦞", { mode: "heartbeat" }),
).toEqual({
shouldSkip: true,
text: "",
didStrip: true,
});
expect(
stripHeartbeatToken(`🦞 ${HEARTBEAT_TOKEN}`, { mode: "heartbeat" }),
).toEqual({
shouldSkip: true,
text: "",
didStrip: true,
});
});
it("skips any reply that includes the heartbeat token", () => {
expect(stripHeartbeatToken(`ALERT ${HEARTBEAT_TOKEN}`)).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("HEARTBEAT_OK 🦞")).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("HEARTBEAT_OK_OK_OK")).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("HEARTBEAT_OK_OK")).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("HEARTBEAT_OK _OK")).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("HEARTBEAT_OK OK")).toEqual({
shouldSkip: true,
text: "",
});
expect(stripHeartbeatToken("ALERT HEARTBEAT_OK_OK")).toEqual({
it("drops short remainder in heartbeat mode", () => {
expect(
stripHeartbeatToken(`ALERT ${HEARTBEAT_TOKEN}`, { mode: "heartbeat" }),
).toEqual({
shouldSkip: true,
text: "",
didStrip: true,
});
});
it("keeps non-heartbeat content", () => {
expect(stripHeartbeatToken("hello")).toEqual({
it("keeps heartbeat replies when remaining content exceeds threshold", () => {
const long = "A".repeat(31);
expect(
stripHeartbeatToken(`${long} ${HEARTBEAT_TOKEN}`, { mode: "heartbeat" }),
).toEqual({
shouldSkip: false,
text: long,
didStrip: true,
});
});
it("strips token at edges for normal messages", () => {
expect(
stripHeartbeatToken(`${HEARTBEAT_TOKEN} hello`, { mode: "message" }),
).toEqual({
shouldSkip: false,
text: "hello",
didStrip: true,
});
expect(
stripHeartbeatToken(`hello ${HEARTBEAT_TOKEN}`, { mode: "message" }),
).toEqual({
shouldSkip: false,
text: "hello",
didStrip: true,
});
});
it("does not touch token in the middle", () => {
expect(
stripHeartbeatToken(`hello ${HEARTBEAT_TOKEN} there`, {
mode: "message",
}),
).toEqual({
shouldSkip: false,
text: `hello ${HEARTBEAT_TOKEN} there`,
didStrip: false,
});
});
});
+64 -7
View File
@@ -2,12 +2,69 @@ import { HEARTBEAT_TOKEN } from "./tokens.js";
export const HEARTBEAT_PROMPT = "HEARTBEAT";
export function stripHeartbeatToken(raw?: string) {
if (!raw) return { shouldSkip: true, text: "" };
const trimmed = raw.trim();
if (!trimmed) return { shouldSkip: true, text: "" };
if (trimmed.includes(HEARTBEAT_TOKEN)) {
return { shouldSkip: true, text: "" };
export type StripHeartbeatMode = "heartbeat" | "message";
function stripTokenAtEdges(raw: string): { text: string; didStrip: boolean } {
let text = raw.trim();
if (!text) return { text: "", didStrip: false };
const token = HEARTBEAT_TOKEN;
if (!text.includes(token)) return { text, didStrip: false };
let didStrip = false;
let changed = true;
while (changed) {
changed = false;
const next = text.trim();
if (next.startsWith(token)) {
const after = next.slice(token.length).trimStart();
text = after;
didStrip = true;
changed = true;
continue;
}
if (next.endsWith(token)) {
const before = next.slice(0, Math.max(0, next.length - token.length));
text = before.trimEnd();
didStrip = true;
changed = true;
}
}
return { shouldSkip: false, text: trimmed };
const collapsed = text.replace(/\s+/g, " ").trim();
return { text: collapsed, didStrip };
}
export function stripHeartbeatToken(
raw?: string,
opts: { mode?: StripHeartbeatMode; maxAckChars?: number } = {},
) {
if (!raw) return { shouldSkip: true, text: "", didStrip: false };
const trimmed = raw.trim();
if (!trimmed) return { shouldSkip: true, text: "", didStrip: false };
const mode: StripHeartbeatMode = opts.mode ?? "message";
const maxAckChars = Math.max(0, opts.maxAckChars ?? 30);
if (!trimmed.includes(HEARTBEAT_TOKEN)) {
return { shouldSkip: false, text: trimmed, didStrip: false };
}
const stripped = stripTokenAtEdges(trimmed);
if (!stripped.didStrip) {
return { shouldSkip: false, text: trimmed, didStrip: false };
}
if (!stripped.text) {
return { shouldSkip: true, text: "", didStrip: true };
}
if (mode === "heartbeat") {
const rest = stripped.text.trim();
if (rest.length <= maxAckChars) {
return { shouldSkip: true, text: "", didStrip: true };
}
}
return { shouldSkip: false, text: stripped.text, didStrip: true };
}
+25
View File
@@ -186,6 +186,31 @@ describe("trigger handling", () => {
});
});
it("strips HEARTBEAT_OK at edges outside heartbeat runs", async () => {
await withTempHome(async (home) => {
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: `${HEARTBEAT_TOKEN} hello` }],
meta: {
durationMs: 1,
agentMeta: { sessionId: "s", provider: "p", model: "m" },
},
});
const res = await getReplyFromConfig(
{
Body: "hello",
From: "+1002",
To: "+2000",
},
{},
makeCfg(home),
);
const text = Array.isArray(res) ? res[0]?.text : res?.text;
expect(text).toBe("hello");
});
});
it("updates group activation when the owner sends /activation", async () => {
await withTempHome(async (home) => {
const cfg = makeCfg(home);
+62 -46
View File
@@ -37,6 +37,7 @@ import {
saveSessionStore,
} from "../config/sessions.js";
import { logVerbose } from "../globals.js";
import { registerAgentRunContext } from "../infra/agent-events.js";
import { buildProviderSummary } from "../infra/provider-summary.js";
import { triggerClawdisRestart } from "../infra/restart.js";
import {
@@ -52,6 +53,7 @@ import {
normalizeGroupActivation,
parseActivationCommand,
} from "./group-activation.js";
import { stripHeartbeatToken } from "./heartbeat.js";
import { extractModelDirective } from "./model.js";
import { buildStatusMessage } from "./status.js";
import type { MsgContext, TemplateContext } from "./templating.js";
@@ -61,7 +63,7 @@ import {
type ThinkLevel,
type VerboseLevel,
} from "./thinking.js";
import { HEARTBEAT_TOKEN, SILENT_REPLY_TOKEN } from "./tokens.js";
import { SILENT_REPLY_TOKEN } from "./tokens.js";
import { isAudio, transcribeInboundAudio } from "./transcription.js";
import type { GetReplyOptions, ReplyPayload } from "./types.js";
@@ -1190,12 +1192,15 @@ export async function getReplyFromConfig(
return undefined;
}
let suppressedByHeartbeatAck = false;
let didLogHeartbeatStrip = false;
try {
if (shouldEagerType) {
await startTypingLoop();
}
const runId = crypto.randomUUID();
if (sessionKey) {
registerAgentRunContext(runId, { sessionKey });
}
let runResult: Awaited<ReturnType<typeof runEmbeddedPiAgent>>;
try {
runResult = await runEmbeddedPiAgent({
@@ -1217,20 +1222,24 @@ export async function getReplyFromConfig(
runId,
onPartialReply: opts?.onPartialReply
? async (payload) => {
if (
!opts?.isHeartbeat &&
payload.text?.includes(HEARTBEAT_TOKEN)
) {
suppressedByHeartbeatAck = true;
logVerbose(
"Suppressing partial reply: detected HEARTBEAT_OK token",
);
return;
let text = payload.text;
if (!opts?.isHeartbeat && text?.includes("HEARTBEAT_OK")) {
const stripped = stripHeartbeatToken(text, { mode: "message" });
if (stripped.didStrip && !didLogHeartbeatStrip) {
didLogHeartbeatStrip = true;
logVerbose("Stripped stray HEARTBEAT_OK token from reply");
}
if (
stripped.shouldSkip &&
(payload.mediaUrls?.length ?? 0) === 0
) {
return;
}
text = stripped.text;
}
if (suppressedByHeartbeatAck) return;
await startTypingOnText(payload.text);
await startTypingOnText(text);
await opts.onPartialReply?.({
text: payload.text,
text,
mediaUrls: payload.mediaUrls,
});
}
@@ -1238,22 +1247,23 @@ export async function getReplyFromConfig(
shouldEmitToolResult,
onToolResult: opts?.onToolResult
? async (payload) => {
if (
!opts?.isHeartbeat &&
payload.text?.includes(HEARTBEAT_TOKEN)
) {
suppressedByHeartbeatAck = true;
logVerbose(
"Suppressing tool result: detected HEARTBEAT_OK token",
);
return;
let text = payload.text;
if (!opts?.isHeartbeat && text?.includes("HEARTBEAT_OK")) {
const stripped = stripHeartbeatToken(text, { mode: "message" });
if (stripped.didStrip && !didLogHeartbeatStrip) {
didLogHeartbeatStrip = true;
logVerbose("Stripped stray HEARTBEAT_OK token from reply");
}
if (
stripped.shouldSkip &&
(payload.mediaUrls?.length ?? 0) === 0
) {
return;
}
text = stripped.text;
}
if (suppressedByHeartbeatAck) return;
await startTypingOnText(payload.text);
await opts.onToolResult?.({
text: payload.text,
mediaUrls: payload.mediaUrls,
});
await startTypingOnText(text);
await opts.onToolResult?.({ text, mediaUrls: payload.mediaUrls });
}
: undefined,
});
@@ -1284,22 +1294,28 @@ export async function getReplyFromConfig(
const payloadArray = runResult.payloads ?? [];
if (payloadArray.length === 0) return undefined;
if (
suppressedByHeartbeatAck ||
(!opts?.isHeartbeat &&
payloadArray.some((payload) => payload.text?.includes(HEARTBEAT_TOKEN)))
) {
logVerbose("Suppressing reply: detected HEARTBEAT_OK token");
return undefined;
}
const shouldSignalTyping = payloadArray.some((payload) => {
const sanitizedPayloads = opts?.isHeartbeat
? payloadArray
: payloadArray.flatMap((payload) => {
const text = payload.text;
if (!text || !text.includes("HEARTBEAT_OK")) return [payload];
const stripped = stripHeartbeatToken(text, { mode: "message" });
if (stripped.didStrip && !didLogHeartbeatStrip) {
didLogHeartbeatStrip = true;
logVerbose("Stripped stray HEARTBEAT_OK token from reply");
}
const hasMedia =
Boolean(payload.mediaUrl) || (payload.mediaUrls?.length ?? 0) > 0;
if (stripped.shouldSkip && !hasMedia) return [];
return [{ ...payload, text: stripped.text }];
});
if (sanitizedPayloads.length === 0) return undefined;
const shouldSignalTyping = sanitizedPayloads.some((payload) => {
const trimmed = payload.text?.trim();
if (
trimmed &&
trimmed !== SILENT_REPLY_TOKEN &&
!trimmed.includes(HEARTBEAT_TOKEN)
)
return true;
if (trimmed && trimmed !== SILENT_REPLY_TOKEN) return true;
if (payload.mediaUrl) return true;
if (payload.mediaUrls && payload.mediaUrls.length > 0) return true;
return false;
@@ -1352,11 +1368,11 @@ export async function getReplyFromConfig(
}
// If verbose is enabled and this is a new session, prepend a session hint.
let finalPayloads = payloadArray;
let finalPayloads = sanitizedPayloads;
if (resolvedVerboseLevel === "on" && isNewSession) {
finalPayloads = [
{ text: `🧭 New session: ${sessionIdFinal}` },
...payloadArray,
...finalPayloads,
];
}
+2 -2
View File
@@ -155,7 +155,7 @@ export function registerCronCli(program: Command) {
.option("--deliver", "Deliver agent output", false)
.option(
"--channel <channel>",
"Delivery channel (last|whatsapp|telegram|discord|signal)",
"Delivery channel (last|whatsapp|telegram|discord|signal|imessage)",
"last",
)
.option(
@@ -414,7 +414,7 @@ export function registerCronCli(program: Command) {
.option("--deliver", "Deliver agent output", false)
.option(
"--channel <channel>",
"Delivery channel (last|whatsapp|telegram|discord|signal)",
"Delivery channel (last|whatsapp|telegram|discord|signal|imessage)",
)
.option(
"--to <dest>",
+3
View File
@@ -1,4 +1,5 @@
import { sendMessageDiscord } from "../discord/send.js";
import { sendMessageIMessage } from "../imessage/send.js";
import { logWebSelfId, sendMessageWhatsApp } from "../providers/web/index.js";
import { sendMessageSignal } from "../signal/send.js";
import { sendMessageTelegram } from "../telegram/send.js";
@@ -8,6 +9,7 @@ export type CliDeps = {
sendMessageTelegram: typeof sendMessageTelegram;
sendMessageDiscord: typeof sendMessageDiscord;
sendMessageSignal: typeof sendMessageSignal;
sendMessageIMessage: typeof sendMessageIMessage;
};
export function createDefaultDeps(): CliDeps {
@@ -16,6 +18,7 @@ export function createDefaultDeps(): CliDeps {
sendMessageTelegram,
sendMessageDiscord,
sendMessageSignal,
sendMessageIMessage,
};
}
+6 -4
View File
@@ -282,10 +282,12 @@ export function buildProgram() {
program
.command("send")
.description("Send a message (WhatsApp Web, Telegram bot, Discord, Signal)")
.description(
"Send a message (WhatsApp Web, Telegram bot, Discord, Signal, iMessage)",
)
.requiredOption(
"-t, --to <number>",
"Recipient: E.164 for WhatsApp/Signal, Telegram chat id/@username, or Discord channel/user",
"Recipient: E.164 for WhatsApp/Signal, Telegram chat id/@username, Discord channel/user, or iMessage handle/chat_id",
)
.requiredOption("-m, --message <text>", "Message body")
.option(
@@ -294,7 +296,7 @@ export function buildProgram() {
)
.option(
"--provider <provider>",
"Delivery provider: whatsapp|telegram|discord|signal (default: whatsapp)",
"Delivery provider: whatsapp|telegram|discord|signal|imessage (default: whatsapp)",
)
.option("--dry-run", "Print payload and skip sending", false)
.option("--json", "Output result as JSON", false)
@@ -337,7 +339,7 @@ Examples:
.option("--verbose <on|off>", "Persist agent verbose level for the session")
.option(
"--provider <provider>",
"Delivery provider: whatsapp|telegram|discord|signal (default: whatsapp)",
"Delivery provider: whatsapp|telegram|discord|signal|imessage (default: whatsapp)",
)
.option(
"--deliver",
+1
View File
@@ -212,6 +212,7 @@ describe("agentCommand", () => {
.mockResolvedValue({ messageId: "t1", chatId: "123" }),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
const prevTelegramToken = process.env.TELEGRAM_BOT_TOKEN;
+56 -3
View File
@@ -36,7 +36,10 @@ import {
type SessionEntry,
saveSessionStore,
} from "../config/sessions.js";
import { emitAgentEvent } from "../infra/agent-events.js";
import {
emitAgentEvent,
registerAgentRunContext,
} from "../infra/agent-events.js";
import { defaultRuntime, type RuntimeEnv } from "../runtime.js";
import { resolveTelegramToken } from "../telegram/token.js";
import { normalizeE164 } from "../utils.js";
@@ -204,6 +207,10 @@ export async function agentCommand(
} = sessionResolution;
let sessionEntry = resolvedSessionEntry;
if (sessionKey) {
registerAgentRunContext(sessionId, { sessionKey });
}
const resolvedThinkLevel =
thinkOnce ??
thinkOverride ??
@@ -413,12 +420,15 @@ export async function agentCommand(
const payloads = result.payloads ?? [];
const deliver = opts.deliver === true;
const bestEffortDeliver = opts.bestEffortDeliver === true;
const deliveryProvider = (opts.provider ?? "whatsapp").toLowerCase();
const deliveryProviderRaw = (opts.provider ?? "whatsapp").toLowerCase();
const deliveryProvider =
deliveryProviderRaw === "imsg" ? "imessage" : deliveryProviderRaw;
const whatsappTarget = opts.to ? normalizeE164(opts.to) : allowFrom[0];
const telegramTarget = opts.to?.trim() || undefined;
const discordTarget = opts.to?.trim() || undefined;
const signalTarget = opts.to?.trim() || undefined;
const imessageTarget = opts.to?.trim() || undefined;
const logDeliveryError = (err: unknown) => {
const deliveryTarget =
@@ -430,7 +440,9 @@ export async function agentCommand(
? discordTarget
: deliveryProvider === "signal"
? signalTarget
: undefined;
: deliveryProvider === "imessage"
? imessageTarget
: undefined;
const message = `Delivery failed (${deliveryProvider}${deliveryTarget ? ` to ${deliveryTarget}` : ""}): ${String(err)}`;
runtime.error?.(message);
if (!runtime.error) runtime.log(message);
@@ -463,6 +475,13 @@ export async function agentCommand(
if (!bestEffortDeliver) throw err;
logDeliveryError(err);
}
if (deliveryProvider === "imessage" && !imessageTarget) {
const err = new Error(
"Delivering to iMessage requires --to <handle|chat_id:ID>",
);
if (!bestEffortDeliver) throw err;
logDeliveryError(err);
}
if (deliveryProvider === "webchat") {
const err = new Error(
"Delivering to WebChat is not supported via `clawdis agent`; use WhatsApp/Telegram or run with --deliver=false.",
@@ -475,6 +494,7 @@ export async function agentCommand(
deliveryProvider !== "telegram" &&
deliveryProvider !== "discord" &&
deliveryProvider !== "signal" &&
deliveryProvider !== "imessage" &&
deliveryProvider !== "webchat"
) {
const err = new Error(`Unknown provider: ${deliveryProvider}`);
@@ -621,5 +641,38 @@ export async function agentCommand(
logDeliveryError(err);
}
}
if (deliveryProvider === "imessage" && imessageTarget) {
try {
if (media.length === 0) {
for (const chunk of chunkText(text, 4000)) {
await deps.sendMessageIMessage(imessageTarget, chunk, {
maxBytes: cfg.imessage?.mediaMaxMb
? cfg.imessage.mediaMaxMb * 1024 * 1024
: cfg.agent?.mediaMaxMb
? cfg.agent.mediaMaxMb * 1024 * 1024
: undefined,
});
}
} else {
let first = true;
for (const url of media) {
const caption = first ? text : "";
first = false;
await deps.sendMessageIMessage(imessageTarget, caption, {
mediaUrl: url,
maxBytes: cfg.imessage?.mediaMaxMb
? cfg.imessage.mediaMaxMb * 1024 * 1024
: cfg.agent?.mediaMaxMb
? cfg.agent.mediaMaxMb * 1024 * 1024
: undefined,
});
}
}
} catch (err) {
if (!bestEffortDeliver) throw err;
logDeliveryError(err);
}
}
}
}
+16
View File
@@ -20,6 +20,7 @@ import {
import { GATEWAY_LAUNCH_AGENT_LABEL } from "../daemon/constants.js";
import { resolveGatewayProgramArguments } from "../daemon/program-args.js";
import { resolveGatewayService } from "../daemon/service.js";
import { pickPrimaryTailnetIPv4 } from "../infra/tailnet.js";
import type { RuntimeEnv } from "../runtime.js";
import { defaultRuntime } from "../runtime.js";
import { resolveUserPath, sleep } from "../utils.js";
@@ -478,5 +479,20 @@ export async function runInteractiveOnboarding(
"Optional apps",
);
note(
(() => {
const tailnetIPv4 = pickPrimaryTailnetIPv4();
const host =
bind === "tailnet" || (bind === "auto" && tailnetIPv4)
? (tailnetIPv4 ?? "127.0.0.1")
: "127.0.0.1";
return [
`Control UI: http://${host}:${port}/`,
`Gateway WS: ws://${host}:${port}`,
].join("\n");
})(),
"Open the Control UI",
);
outro("Onboarding complete.");
}
+140 -1
View File
@@ -1,12 +1,13 @@
import fs from "node:fs/promises";
import path from "node:path";
import { confirm, multiselect, note, text } from "@clack/prompts";
import { confirm, multiselect, note, select, text } from "@clack/prompts";
import chalk from "chalk";
import type { ClawdisConfig } from "../config/config.js";
import { loginWeb } from "../provider-web.js";
import type { RuntimeEnv } from "../runtime.js";
import { normalizeE164 } from "../utils.js";
import { resolveWebAuthDir } from "../web/session.js";
import { detectBinary, guardCancel } from "./onboard-helpers.js";
import type { ProviderChoice } from "./onboard-types.js";
@@ -33,6 +34,7 @@ function noteProviderPrimer(): void {
"Telegram: Bot API (token from @BotFather), replies via your bot.",
"Discord: Bot token from Discord Developer Portal; invite bot to your server.",
"Signal: signal-cli as a linked device (recommended: separate bot number).",
"iMessage: local imsg CLI (JSON-RPC over stdio) reading Messages DB.",
].join("\n"),
"How providers work",
);
@@ -79,6 +81,11 @@ export async function setupProviders(
);
const signalCliPath = cfg.signal?.cliPath ?? "signal-cli";
const signalCliDetected = await detectBinary(signalCliPath);
const imessageConfigured = Boolean(
cfg.imessage?.cliPath || cfg.imessage?.dbPath || cfg.imessage?.allowFrom,
);
const imessageCliPath = cfg.imessage?.cliPath ?? "imsg";
const imessageCliDetected = await detectBinary(imessageCliPath);
note(
[
@@ -100,9 +107,17 @@ export async function setupProviders(
? chalk.green("configured")
: chalk.yellow("needs setup")
}`,
`iMessage: ${
imessageConfigured
? chalk.green("configured")
: chalk.yellow("needs setup")
}`,
`signal-cli: ${
signalCliDetected ? chalk.green("found") : chalk.red("missing")
} (${signalCliPath})`,
`imsg: ${
imessageCliDetected ? chalk.green("found") : chalk.red("missing")
} (${imessageCliPath})`,
].join("\n"),
"Provider status",
);
@@ -142,6 +157,11 @@ export async function setupProviders(
label: "Signal (signal-cli)",
hint: signalCliDetected ? "signal-cli found" : "signal-cli missing",
},
{
value: "imessage",
label: "iMessage (imsg)",
hint: imessageCliDetected ? "imsg found" : "imsg missing",
},
],
}),
runtime,
@@ -177,6 +197,71 @@ export async function setupProviders(
} else if (!whatsappLinked) {
note("Run `clawdis login` later to link WhatsApp.", "WhatsApp");
}
const existingAllowFrom = cfg.routing?.allowFrom ?? [];
if (existingAllowFrom.length === 0) {
note(
[
"WhatsApp direct chats are gated by `routing.allowFrom`.",
'Default (unset) = self-chat only; use "*" to allow anyone.',
].join("\n"),
"Allowlist (recommended)",
);
const mode = guardCancel(
await select({
message: "Who can trigger the bot via WhatsApp?",
options: [
{ value: "self", label: "Self-chat only (default)" },
{ value: "list", label: "Specific numbers (recommended)" },
{ value: "any", label: "Anyone (*)" },
],
}),
runtime,
) as "self" | "list" | "any";
if (mode === "any") {
next = {
...next,
routing: { ...next.routing, allowFrom: ["*"] },
};
} else if (mode === "list") {
const allowRaw = guardCancel(
await text({
message: "Allowed sender numbers (comma-separated, E.164)",
placeholder: "+15555550123, +447700900123",
validate: (value) => {
const raw = String(value ?? "").trim();
if (!raw) return "Required";
const parts = raw
.split(/[\n,;]+/g)
.map((p) => p.trim())
.filter(Boolean);
if (parts.length === 0) return "Required";
for (const part of parts) {
if (part === "*") continue;
const normalized = normalizeE164(part);
if (!normalized) return `Invalid number: ${part}`;
}
return undefined;
},
}),
runtime,
);
const parts = String(allowRaw)
.split(/[\n,;]+/g)
.map((p) => p.trim())
.filter(Boolean);
const normalized = parts.map((part) =>
part === "*" ? "*" : normalizeE164(part),
);
const unique = [...new Set(normalized.filter(Boolean))];
next = {
...next,
routing: { ...next.routing, allowFrom: unique },
};
}
}
}
if (selection.includes("telegram")) {
@@ -395,6 +480,44 @@ export async function setupProviders(
);
}
if (selection.includes("imessage")) {
let resolvedCliPath = imessageCliPath;
if (!imessageCliDetected) {
const entered = guardCancel(
await text({
message: "imsg CLI path",
initialValue: resolvedCliPath,
validate: (value) => (value?.trim() ? undefined : "Required"),
}),
runtime,
);
resolvedCliPath = String(entered).trim();
if (!resolvedCliPath) {
note("imsg CLI path required to enable iMessage.", "iMessage");
}
}
if (resolvedCliPath) {
next = {
...next,
imessage: {
...next.imessage,
enabled: true,
cliPath: resolvedCliPath,
},
};
}
note(
[
"Ensure Clawdis has Full Disk Access to Messages DB.",
"Grant Automation permission for Messages when prompted.",
"List chats with: imsg chats --limit 20",
].join("\n"),
"iMessage next steps",
);
}
if (options?.allowDisable) {
if (!selection.includes("telegram") && telegramConfigured) {
const disable = guardCancel(
@@ -443,6 +566,22 @@ export async function setupProviders(
};
}
}
if (!selection.includes("imessage") && imessageConfigured) {
const disable = guardCancel(
await confirm({
message: "Disable iMessage provider?",
initialValue: false,
}),
runtime,
);
if (disable) {
next = {
...next,
imessage: { ...next.imessage, enabled: false },
};
}
}
}
return next;
+6 -1
View File
@@ -5,7 +5,12 @@ export type ResetScope = "config" | "config+creds+sessions" | "full";
export type GatewayBind = "loopback" | "lan" | "tailnet" | "auto";
export type TailscaleMode = "off" | "serve" | "funnel";
export type NodeManagerChoice = "npm" | "pnpm" | "bun";
export type ProviderChoice = "whatsapp" | "telegram" | "discord" | "signal";
export type ProviderChoice =
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
export type OnboardOptions = {
mode?: OnboardMode;
+18
View File
@@ -42,6 +42,7 @@ const makeDeps = (overrides: Partial<CliDeps> = {}): CliDeps => ({
sendMessageTelegram: vi.fn(),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
...overrides,
});
@@ -151,6 +152,23 @@ describe("sendCommand", () => {
expect(deps.sendMessageWhatsApp).not.toHaveBeenCalled();
});
it("routes to imessage provider", async () => {
const deps = makeDeps({
sendMessageIMessage: vi.fn().mockResolvedValue({ messageId: "i1" }),
});
await sendCommand(
{ to: "chat_id:42", message: "hi", provider: "imessage" },
deps,
runtime,
);
expect(deps.sendMessageIMessage).toHaveBeenCalledWith(
"chat_id:42",
"hi",
expect.objectContaining({ mediaUrl: undefined }),
);
expect(deps.sendMessageWhatsApp).not.toHaveBeenCalled();
});
it("emits json output", async () => {
callGatewayMock.mockResolvedValueOnce({ messageId: "direct2" });
const deps = makeDeps();
+25
View File
@@ -108,6 +108,31 @@ export async function sendCommand(
return;
}
if (provider === "imessage" || provider === "imsg") {
const result = await deps.sendMessageIMessage(opts.to, opts.message, {
mediaUrl: opts.media,
});
runtime.log(
success(`✅ Sent via iMessage. Message ID: ${result.messageId}`),
);
if (opts.json) {
runtime.log(
JSON.stringify(
{
provider: "imessage",
via: "direct",
to: opts.to,
messageId: result.messageId,
mediaUrl: opts.media ?? null,
},
null,
2,
),
);
}
return;
}
// Always send via gateway over WS to avoid multi-session corruption.
const sendViaGateway = async () =>
callGateway<{
+57 -3
View File
@@ -102,7 +102,13 @@ export type HookMappingConfig = {
messageTemplate?: string;
textTemplate?: string;
deliver?: boolean;
channel?: "last" | "whatsapp" | "telegram" | "discord";
channel?:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
to?: string;
thinking?: string;
timeoutSeconds?: number;
@@ -171,6 +177,8 @@ export type DiscordConfig = {
mediaMaxMb?: number;
/** Number of recent guild messages to include for context (default: 20). */
historyLimit?: number;
/** Allow agent-triggered Discord reactions (default: true). */
enableReactions?: boolean;
};
export type SignalConfig = {
@@ -196,6 +204,25 @@ export type SignalConfig = {
mediaMaxMb?: number;
};
export type IMessageConfig = {
/** If false, do not start the iMessage provider. Default: true. */
enabled?: boolean;
/** imsg CLI binary path (default: imsg). */
cliPath?: string;
/** Optional Messages db path override. */
dbPath?: string;
/** Optional default send service (imessage|sms|auto). */
service?: "imessage" | "sms" | "auto";
/** Optional default region (used when sending SMS). */
region?: string;
/** Optional allowlist for inbound handles or chat_id targets. */
allowFrom?: Array<string | number>;
/** Include attachments + reactions in watch payloads. */
includeAttachments?: boolean;
/** Max outbound media size in MB. */
mediaMaxMb?: number;
};
export type QueueMode = "queue" | "interrupt";
export type QueueModeBySurface = {
@@ -203,6 +230,7 @@ export type QueueModeBySurface = {
telegram?: QueueMode;
discord?: QueueMode;
signal?: QueueMode;
imessage?: QueueMode;
webchat?: QueueMode;
};
@@ -448,8 +476,15 @@ export type ClawdisConfig = {
every?: string;
/** Heartbeat model override (provider/model). */
model?: string;
/** Delivery target (last|whatsapp|telegram|discord|signal|none). */
target?: "last" | "whatsapp" | "telegram" | "discord" | "signal" | "none";
/** Delivery target (last|whatsapp|telegram|discord|signal|imessage|none). */
target?:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage"
| "none";
/** Optional delivery override (E.164 for WhatsApp, chat id for Telegram). */
to?: string;
/** Override the heartbeat prompt body (default: "HEARTBEAT"). */
@@ -474,6 +509,7 @@ export type ClawdisConfig = {
telegram?: TelegramConfig;
discord?: DiscordConfig;
signal?: SignalConfig;
imessage?: IMessageConfig;
cron?: CronConfig;
hooks?: HooksConfig;
bridge?: BridgeConfig;
@@ -568,6 +604,7 @@ const QueueModeBySurfaceSchema = z
telegram: QueueModeSchema.optional(),
discord: QueueModeSchema.optional(),
signal: QueueModeSchema.optional(),
imessage: QueueModeSchema.optional(),
webchat: QueueModeSchema.optional(),
})
.optional();
@@ -614,6 +651,7 @@ const HeartbeatSchema = z
z.literal("telegram"),
z.literal("discord"),
z.literal("signal"),
z.literal("imessage"),
z.literal("none"),
])
.optional(),
@@ -673,6 +711,7 @@ const HookMappingSchema = z
z.literal("telegram"),
z.literal("discord"),
z.literal("signal"),
z.literal("imessage"),
])
.optional(),
to: z.string().optional(),
@@ -879,6 +918,7 @@ const ClawdisSchema = z.object({
requireMention: z.boolean().optional(),
mediaMaxMb: z.number().positive().optional(),
historyLimit: z.number().int().min(0).optional(),
enableReactions: z.boolean().optional(),
})
.optional(),
signal: z
@@ -900,6 +940,20 @@ const ClawdisSchema = z.object({
mediaMaxMb: z.number().positive().optional(),
})
.optional(),
imessage: z
.object({
enabled: z.boolean().optional(),
cliPath: z.string().optional(),
dbPath: z.string().optional(),
service: z
.union([z.literal("imessage"), z.literal("sms"), z.literal("auto")])
.optional(),
region: z.string().optional(),
allowFrom: z.array(z.union([z.string(), z.number()])).optional(),
includeAttachments: z.boolean().optional(),
mediaMaxMb: z.number().positive().optional(),
})
.optional(),
bridge: z
.object({
enabled: z.boolean().optional(),
+7 -1
View File
@@ -27,7 +27,13 @@ export type SessionEntry = {
totalTokens?: number;
model?: string;
contextTokens?: number;
lastChannel?: "whatsapp" | "telegram" | "discord" | "signal" | "webchat";
lastChannel?:
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage"
| "webchat";
lastTo?: string;
skillsSnapshot?: SessionSkillSnapshot;
};
+6
View File
@@ -97,6 +97,7 @@ describe("runCronIsolatedAgentTurn", () => {
sendMessageTelegram: vi.fn(),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: "first" }, { text: " " }, { text: " last " }],
@@ -128,6 +129,7 @@ describe("runCronIsolatedAgentTurn", () => {
sendMessageTelegram: vi.fn(),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
const long = "a".repeat(2001);
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
@@ -160,6 +162,7 @@ describe("runCronIsolatedAgentTurn", () => {
sendMessageTelegram: vi.fn(),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: "hello" }],
@@ -199,6 +202,7 @@ describe("runCronIsolatedAgentTurn", () => {
sendMessageTelegram: vi.fn(),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: "hello" }],
@@ -240,6 +244,7 @@ describe("runCronIsolatedAgentTurn", () => {
}),
sendMessageDiscord: vi.fn(),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: "hello from cron" }],
@@ -294,6 +299,7 @@ describe("runCronIsolatedAgentTurn", () => {
channelId: "chan",
}),
sendMessageSignal: vi.fn(),
sendMessageIMessage: vi.fn(),
};
vi.mocked(runEmbeddedPiAgent).mockResolvedValue({
payloads: [{ text: "hello from cron" }],
+51 -2
View File
@@ -24,6 +24,7 @@ import {
type SessionEntry,
saveSessionStore,
} from "../config/sessions.js";
import { registerAgentRunContext } from "../infra/agent-events.js";
import { resolveTelegramToken } from "../telegram/token.js";
import { normalizeE164 } from "../utils.js";
import type { CronJob } from "./types.js";
@@ -54,7 +55,13 @@ function pickSummaryFromPayloads(
function resolveDeliveryTarget(
cfg: ClawdisConfig,
jobPayload: {
channel?: "last" | "whatsapp" | "telegram" | "discord" | "signal";
channel?:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
to?: string;
},
) {
@@ -81,7 +88,8 @@ function resolveDeliveryTarget(
requestedChannel === "whatsapp" ||
requestedChannel === "telegram" ||
requestedChannel === "discord" ||
requestedChannel === "signal"
requestedChannel === "signal" ||
requestedChannel === "imessage"
) {
return requestedChannel;
}
@@ -244,6 +252,9 @@ export async function runCronIsolatedAgentTurn(params: {
const sessionFile = resolveSessionTranscriptPath(
cronSession.sessionEntry.sessionId,
);
registerAgentRunContext(cronSession.sessionEntry.sessionId, {
sessionKey: params.sessionKey,
});
runResult = await runEmbeddedPiAgent({
sessionId: cronSession.sessionEntry.sessionId,
sessionKey: params.sessionKey,
@@ -457,6 +468,44 @@ export async function runCronIsolatedAgentTurn(params: {
return { status: "error", summary, error: String(err) };
return { status: "ok", summary };
}
} else if (resolvedDelivery.channel === "imessage") {
if (!resolvedDelivery.to) {
if (!bestEffortDeliver)
return {
status: "error",
summary,
error: "Cron delivery to iMessage requires a recipient.",
};
return {
status: "skipped",
summary: "Delivery skipped (no iMessage recipient).",
};
}
const to = resolvedDelivery.to;
try {
for (const payload of payloads) {
const mediaList =
payload.mediaUrls ?? (payload.mediaUrl ? [payload.mediaUrl] : []);
if (mediaList.length === 0) {
for (const chunk of chunkText(payload.text ?? "", 4000)) {
await params.deps.sendMessageIMessage(to, chunk);
}
} else {
let first = true;
for (const url of mediaList) {
const caption = first ? (payload.text ?? "") : "";
first = false;
await params.deps.sendMessageIMessage(to, caption, {
mediaUrl: url,
});
}
}
}
} catch (err) {
if (!bestEffortDeliver)
return { status: "error", summary, error: String(err) };
return { status: "ok", summary };
}
}
}
+7 -1
View File
@@ -14,7 +14,13 @@ export type CronPayload =
thinking?: string;
timeoutSeconds?: number;
deliver?: boolean;
channel?: "last" | "whatsapp" | "telegram" | "discord" | "signal";
channel?:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
to?: string;
bestEffortDeliver?: boolean;
};
+9 -6
View File
@@ -44,6 +44,7 @@ type DiscordHistoryEntry = {
sender: string;
body: string;
timestamp?: number;
messageId?: string;
};
export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
@@ -93,11 +94,11 @@ export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
const guildHistories = new Map<string, DiscordHistoryEntry[]>();
client.once(Events.ClientReady, () => {
runtime.log?.(`discord: logged in as ${client.user?.tag ?? "unknown"}`);
runtime.log?.(`logged in as ${client.user?.tag ?? "unknown"}`);
});
client.on(Events.Error, (err) => {
runtime.error?.(danger(`discord client error: ${String(err)}`));
runtime.error?.(danger(`client error: ${String(err)}`));
});
client.on(Events.MessageCreate, async (message) => {
@@ -122,6 +123,7 @@ export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
sender: message.member?.displayName ?? message.author.tag,
body: baseText,
timestamp: message.createdTimestamp,
messageId: message.id,
});
while (history.length > historyLimit) history.shift();
guildHistories.set(message.channelId, history);
@@ -196,11 +198,12 @@ export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
const fromLabel = isDirectMessage
? buildDirectLabel(message)
: buildGuildLabel(message);
const textWithId = `${text}\n[discord message id: ${message.id} channel: ${message.channelId}]`;
let combinedBody = formatAgentEnvelope({
surface: "Discord",
from: fromLabel,
timestamp: message.createdTimestamp,
body: text,
body: textWithId,
});
let shouldClearHistory = false;
if (!isDirectMessage) {
@@ -215,7 +218,7 @@ export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
surface: "Discord",
from: fromLabel,
timestamp: entry.timestamp,
body: `${entry.sender}: ${entry.body}`,
body: `${entry.sender}: ${entry.body} [id:${entry.messageId ?? "unknown"} channel:${message.channelId}]`,
}),
)
.join("\n");
@@ -291,7 +294,7 @@ export async function monitorDiscordProvider(opts: MonitorDiscordOpts = {}) {
guildHistories.set(message.channelId, []);
}
} catch (err) {
runtime.error?.(danger(`Discord handler failed: ${String(err)}`));
runtime.error?.(danger(`handler failed: ${String(err)}`));
}
});
@@ -424,6 +427,6 @@ async function deliverReplies({
});
}
}
runtime.log?.(`discord: delivered reply to ${target}`);
runtime.log?.(`delivered reply to ${target}`);
}
}
+30
View File
@@ -29,6 +29,11 @@ export type DiscordSendResult = {
channelId: string;
};
export type DiscordReactOpts = {
token?: string;
rest?: REST;
};
function resolveToken(explicit?: string) {
const cfgToken = loadConfig().discord?.token;
const token = normalizeDiscordToken(
@@ -42,6 +47,18 @@ function resolveToken(explicit?: string) {
return token;
}
function normalizeReactionEmoji(raw: string) {
const trimmed = raw.trim();
if (!trimmed) {
throw new Error("emoji required");
}
const customMatch = trimmed.match(/^<a?:([^:>]+):(\d+)>$/);
const identifier = customMatch
? `${customMatch[1]}:${customMatch[2]}`
: trimmed;
return encodeURIComponent(identifier);
}
function parseRecipient(raw: string): DiscordRecipient {
const trimmed = raw.trim();
if (!trimmed) {
@@ -164,3 +181,16 @@ export async function sendMessageDiscord(
channelId: String(result.channel_id ?? channelId),
};
}
export async function reactMessageDiscord(
channelId: string,
messageId: string,
emoji: string,
opts: DiscordReactOpts = {},
) {
const token = resolveToken(opts.token);
const rest = opts.rest ?? new REST({ version: "10" }).setToken(token);
const encoded = normalizeReactionEmoji(emoji);
await rest.put(Routes.channelMessageReaction(channelId, messageId, encoded));
return { ok: true };
}
+75 -1
View File
@@ -12,7 +12,11 @@ import {
STATE_DIR_CLAWDIS,
writeConfigFile,
} from "../config/config.js";
import { emitAgentEvent } from "../infra/agent-events.js";
import {
emitAgentEvent,
registerAgentRunContext,
resetAgentRunContextForTest,
} from "../infra/agent-events.js";
import { GatewayLockError } from "../infra/gateway-lock.js";
import { emitHeartbeatEvent } from "../infra/heartbeat-events.js";
import { drainSystemEvents, peekSystemEvents } from "../infra/system-events.js";
@@ -277,6 +281,7 @@ beforeEach(async () => {
testCanvasHostPort = undefined;
cronIsolatedRun.mockClear();
drainSystemEvents();
resetAgentRunContextForTest();
__resetModelCatalogCacheForTest();
piSdkMock.enabled = false;
piSdkMock.discoverCalls = 0;
@@ -3621,6 +3626,75 @@ describe("gateway server", () => {
await server.close();
});
test("agent events stream to webchat clients when run context is registered", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdis-gw-"));
testSessionStorePath = path.join(dir, "sessions.json");
await fs.writeFile(
testSessionStorePath,
JSON.stringify(
{
main: {
sessionId: "sess-main",
updatedAt: Date.now(),
},
},
null,
2,
),
"utf-8",
);
const { server, ws } = await startServerWithClient();
await connectOk(ws, {
client: {
name: "webchat",
version: "1.0.0",
platform: "test",
mode: "webchat",
},
});
registerAgentRunContext("run-auto-1", { sessionKey: "main" });
const finalChatP = onceMessage<{
type: "event";
event: string;
payload?: unknown;
}>(
ws,
(o) => {
if (o.type !== "event" || o.event !== "chat") return false;
const payload = o.payload as
| { state?: unknown; runId?: unknown }
| undefined;
return payload?.state === "final" && payload.runId === "run-auto-1";
},
8000,
);
emitAgentEvent({
runId: "run-auto-1",
stream: "assistant",
data: { text: "hi from agent" },
});
emitAgentEvent({
runId: "run-auto-1",
stream: "job",
data: { state: "done" },
});
const evt = await finalChatP;
const payload =
evt.payload && typeof evt.payload === "object"
? (evt.payload as Record<string, unknown>)
: {};
expect(payload.sessionKey).toBe("main");
expect(payload.runId).toBe("run-auto-1");
ws.close();
await server.close();
});
test("bridge chat.abort cancels while saving the session store", async () => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdis-gw-"));
testSessionStorePath = path.join(dir, "sessions.json");
+367 -83
View File
@@ -75,7 +75,18 @@ import {
} from "../discord/index.js";
import { type DiscordProbe, probeDiscord } from "../discord/probe.js";
import { isVerbose } from "../globals.js";
import { onAgentEvent } from "../infra/agent-events.js";
import { startGmailWatcher, stopGmailWatcher } from "../hooks/gmail-watcher.js";
import {
monitorIMessageProvider,
sendMessageIMessage,
} from "../imessage/index.js";
import { type IMessageProbe, probeIMessage } from "../imessage/probe.js";
import {
clearAgentRunContext,
getAgentRunContext,
onAgentEvent,
registerAgentRunContext,
} from "../infra/agent-events.js";
import { startGatewayBonjourAdvertiser } from "../infra/bonjour.js";
import { startNodeBridgeServer } from "../infra/bridge/server.js";
import { resolveCanvasHostUrl } from "../infra/canvas-host-url.js";
@@ -287,11 +298,13 @@ const logWhatsApp = logProviders.child("whatsapp");
const logTelegram = logProviders.child("telegram");
const logDiscord = logProviders.child("discord");
const logSignal = logProviders.child("signal");
const logIMessage = logProviders.child("imessage");
const canvasRuntime = runtimeForLogger(logCanvas);
const whatsappRuntimeEnv = runtimeForLogger(logWhatsApp);
const telegramRuntimeEnv = runtimeForLogger(logTelegram);
const discordRuntimeEnv = runtimeForLogger(logDiscord);
const signalRuntimeEnv = runtimeForLogger(logSignal);
const imessageRuntimeEnv = runtimeForLogger(logIMessage);
function resolveBonjourCliPath(): string | undefined {
const envPath = process.env.CLAWDIS_CLI_PATH?.trim();
@@ -1345,7 +1358,13 @@ export async function startGatewayServer(
wakeMode: "now" | "next-heartbeat";
sessionKey: string;
deliver: boolean;
channel: "last" | "whatsapp" | "telegram" | "discord" | "signal";
channel:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
to?: string;
thinking?: string;
timeoutSeconds?: number;
@@ -1371,15 +1390,18 @@ export async function startGatewayServer(
channelRaw === "telegram" ||
channelRaw === "discord" ||
channelRaw === "signal" ||
channelRaw === "imessage" ||
channelRaw === "last"
? channelRaw
: channelRaw === undefined
? "last"
: null;
: channelRaw === "imsg"
? "imessage"
: channelRaw === undefined
? "last"
: null;
if (channel === null) {
return {
ok: false,
error: "channel must be last|whatsapp|telegram|discord|signal",
error: "channel must be last|whatsapp|telegram|discord|signal|imessage",
};
}
const toRaw = payload.to;
@@ -1430,7 +1452,13 @@ export async function startGatewayServer(
wakeMode: "now" | "next-heartbeat";
sessionKey: string;
deliver: boolean;
channel: "last" | "whatsapp" | "telegram" | "discord" | "signal";
channel:
| "last"
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage";
to?: string;
thinking?: string;
timeoutSeconds?: number;
@@ -1714,10 +1742,12 @@ export async function startGatewayServer(
let telegramAbort: AbortController | null = null;
let discordAbort: AbortController | null = null;
let signalAbort: AbortController | null = null;
let imessageAbort: AbortController | null = null;
let whatsappTask: Promise<unknown> | null = null;
let telegramTask: Promise<unknown> | null = null;
let discordTask: Promise<unknown> | null = null;
let signalTask: Promise<unknown> | null = null;
let imessageTask: Promise<unknown> | null = null;
let whatsappRuntime: WebProviderStatus = {
running: false,
connected: false,
@@ -1765,12 +1795,27 @@ export async function startGatewayServer(
lastError: null,
baseUrl: null,
};
let imessageRuntime: {
running: boolean;
lastStartAt?: number | null;
lastStopAt?: number | null;
lastError?: string | null;
cliPath?: string | null;
dbPath?: string | null;
} = {
running: false,
lastStartAt: null,
lastStopAt: null,
lastError: null,
cliPath: null,
dbPath: null,
};
const clients = new Set<Client>();
let seq = 0;
// Track per-run sequence to detect out-of-order/lost agent events.
const agentRunSeq = new Map<string, number>();
const dedupe = new Map<string, DedupeEntry>();
// Map agent sessionId -> pending chat runs for WebChat clients.
// Map agent runId -> pending chat runs for WebChat clients.
const chatRunSessions = new Map<
string,
Array<{ sessionKey: string; clientRunId: string }>
@@ -1812,6 +1857,21 @@ export async function startGatewayServer(
if (!queue.length) chatRunSessions.delete(sessionId);
return entry;
};
const resolveSessionKeyForRun = (runId: string) => {
const cached = getAgentRunContext(runId)?.sessionKey;
if (cached) return cached;
const cfg = loadConfig();
const storePath = resolveStorePath(cfg.session?.store);
const store = loadSessionStore(storePath);
const found = Object.entries(store).find(
([, entry]) => entry?.sessionId === runId,
);
const sessionKey = found?.[0];
if (sessionKey) {
registerAgentRunContext(runId, { sessionKey });
}
return sessionKey;
};
const chatRunBuffers = new Map<string, string>();
const chatDeltaSentAt = new Map<string, number>();
const chatAbortControllers = new Map<
@@ -2221,11 +2281,92 @@ export async function startGatewayServer(
};
};
const startIMessageProvider = async () => {
if (imessageTask) return;
const cfg = loadConfig();
if (!cfg.imessage) {
imessageRuntime = {
...imessageRuntime,
running: false,
lastError: "not configured",
};
logIMessage.info("skipping provider start (imessage not configured)");
return;
}
if (cfg.imessage?.enabled === false) {
imessageRuntime = {
...imessageRuntime,
running: false,
lastError: "disabled",
};
logIMessage.info("skipping provider start (imessage.enabled=false)");
return;
}
const cliPath = cfg.imessage?.cliPath?.trim() || "imsg";
const dbPath = cfg.imessage?.dbPath?.trim();
logIMessage.info(
`starting provider (${cliPath}${dbPath ? ` db=${dbPath}` : ""})`,
);
imessageAbort = new AbortController();
imessageRuntime = {
...imessageRuntime,
running: true,
lastStartAt: Date.now(),
lastError: null,
cliPath,
dbPath: dbPath ?? null,
};
const task = monitorIMessageProvider({
cliPath,
dbPath,
allowFrom: cfg.imessage?.allowFrom,
includeAttachments: cfg.imessage?.includeAttachments,
mediaMaxMb: cfg.imessage?.mediaMaxMb,
runtime: imessageRuntimeEnv,
abortSignal: imessageAbort.signal,
})
.catch((err) => {
imessageRuntime = {
...imessageRuntime,
lastError: formatError(err),
};
logIMessage.error(`provider exited: ${formatError(err)}`);
})
.finally(() => {
imessageAbort = null;
imessageTask = null;
imessageRuntime = {
...imessageRuntime,
running: false,
lastStopAt: Date.now(),
};
});
imessageTask = task;
};
const stopIMessageProvider = async () => {
if (!imessageAbort && !imessageTask) return;
imessageAbort?.abort();
try {
await imessageTask;
} catch {
// ignore
}
imessageAbort = null;
imessageTask = null;
imessageRuntime = {
...imessageRuntime,
running: false,
lastStopAt: Date.now(),
};
};
const startProviders = async () => {
await startWhatsAppProvider();
await startDiscordProvider();
await startTelegramProvider();
await startSignalProvider();
await startIMessageProvider();
};
const broadcast = (
@@ -3269,7 +3410,8 @@ export async function startGatewayServer(
const provider =
channel === "whatsapp" ||
channel === "telegram" ||
channel === "signal"
channel === "signal" ||
channel === "imessage"
? channel
: undefined;
const to =
@@ -3586,81 +3728,152 @@ export async function startGatewayServer(
broadcast("agent", evt);
const chatLink = peekChatRun(evt.runId);
if (chatLink) {
// Map agent bus events to chat events for WS WebChat clients.
// Use clientRunId so the webchat can correlate with its pending promise.
const { sessionKey, clientRunId } = chatLink;
bridgeSendToSession(sessionKey, "agent", evt);
if (evt.stream === "assistant" && typeof evt.data?.text === "string") {
const base = {
runId: clientRunId,
sessionKey,
seq: evt.seq,
};
chatRunBuffers.set(clientRunId, evt.data.text);
const now = Date.now();
const last = chatDeltaSentAt.get(clientRunId) ?? 0;
// Throttle UI delta events so slow clients don't accumulate unbounded buffers.
if (now - last >= 150) {
chatDeltaSentAt.set(clientRunId, now);
const payload = {
...base,
state: "delta" as const,
message: {
role: "assistant",
content: [{ type: "text", text: evt.data.text }],
timestamp: now,
},
const sessionKey =
chatLink?.sessionKey ?? resolveSessionKeyForRun(evt.runId);
const jobState =
evt.stream === "job" && typeof evt.data?.state === "string"
? evt.data.state
: null;
if (sessionKey) {
if (chatLink) {
// Map agent bus events to chat events for WS WebChat clients.
// Use clientRunId so the webchat can correlate with its pending promise.
const { clientRunId } = chatLink;
bridgeSendToSession(sessionKey, "agent", evt);
if (evt.stream === "assistant" && typeof evt.data?.text === "string") {
const base = {
runId: clientRunId,
sessionKey,
seq: evt.seq,
};
broadcast("chat", payload, { dropIfSlow: true });
bridgeSendToSession(sessionKey, "chat", payload);
chatRunBuffers.set(clientRunId, evt.data.text);
const now = Date.now();
const last = chatDeltaSentAt.get(clientRunId) ?? 0;
// Throttle UI delta events so slow clients don't accumulate unbounded buffers.
if (now - last >= 150) {
chatDeltaSentAt.set(clientRunId, now);
const payload = {
...base,
state: "delta" as const,
message: {
role: "assistant",
content: [{ type: "text", text: evt.data.text }],
timestamp: now,
},
};
broadcast("chat", payload, { dropIfSlow: true });
bridgeSendToSession(sessionKey, "chat", payload);
}
} else if (jobState === "done" || jobState === "error") {
const finished = shiftChatRun(evt.runId);
if (!finished) {
if (jobState) clearAgentRunContext(evt.runId);
return;
}
const { sessionKey: finishedSessionKey, clientRunId: finishedRunId } =
finished;
const base = {
runId: finishedRunId,
sessionKey: finishedSessionKey,
seq: evt.seq,
};
const text = chatRunBuffers.get(finishedRunId)?.trim() ?? "";
chatRunBuffers.delete(finishedRunId);
chatDeltaSentAt.delete(finishedRunId);
if (jobState === "done") {
const payload = {
...base,
state: "final",
message: text
? {
role: "assistant",
content: [{ type: "text", text }],
timestamp: Date.now(),
}
: undefined,
};
broadcast("chat", payload);
bridgeSendToSession(finishedSessionKey, "chat", payload);
} else {
const payload = {
...base,
state: "error",
errorMessage: evt.data.error
? formatForLog(evt.data.error)
: undefined,
};
broadcast("chat", payload);
bridgeSendToSession(finishedSessionKey, "chat", payload);
}
}
} else if (
evt.stream === "job" &&
typeof evt.data?.state === "string" &&
(evt.data.state === "done" || evt.data.state === "error")
) {
const finished = shiftChatRun(evt.runId);
if (!finished) {
return;
}
const { sessionKey: finishedSessionKey, clientRunId: finishedRunId } =
finished;
const base = {
runId: finishedRunId,
sessionKey: finishedSessionKey,
seq: evt.seq,
};
const text = chatRunBuffers.get(finishedRunId)?.trim() ?? "";
chatRunBuffers.delete(finishedRunId);
chatDeltaSentAt.delete(finishedRunId);
if (evt.data.state === "done") {
const payload = {
...base,
state: "final",
message: text
? {
role: "assistant",
content: [{ type: "text", text }],
timestamp: Date.now(),
}
: undefined,
} else {
const clientRunId = evt.runId;
bridgeSendToSession(sessionKey, "agent", evt);
if (evt.stream === "assistant" && typeof evt.data?.text === "string") {
const base = {
runId: clientRunId,
sessionKey,
seq: evt.seq,
};
broadcast("chat", payload);
bridgeSendToSession(finishedSessionKey, "chat", payload);
} else {
const payload = {
...base,
state: "error",
errorMessage: evt.data.error
? formatForLog(evt.data.error)
: undefined,
chatRunBuffers.set(clientRunId, evt.data.text);
const now = Date.now();
const last = chatDeltaSentAt.get(clientRunId) ?? 0;
if (now - last >= 150) {
chatDeltaSentAt.set(clientRunId, now);
const payload = {
...base,
state: "delta" as const,
message: {
role: "assistant",
content: [{ type: "text", text: evt.data.text }],
timestamp: now,
},
};
broadcast("chat", payload, { dropIfSlow: true });
bridgeSendToSession(sessionKey, "chat", payload);
}
} else if (jobState === "done" || jobState === "error") {
const base = {
runId: clientRunId,
sessionKey,
seq: evt.seq,
};
broadcast("chat", payload);
bridgeSendToSession(finishedSessionKey, "chat", payload);
const text = chatRunBuffers.get(clientRunId)?.trim() ?? "";
chatRunBuffers.delete(clientRunId);
chatDeltaSentAt.delete(clientRunId);
if (jobState === "done") {
const payload = {
...base,
state: "final",
message: text
? {
role: "assistant",
content: [{ type: "text", text }],
timestamp: Date.now(),
}
: undefined,
};
broadcast("chat", payload);
bridgeSendToSession(sessionKey, "chat", payload);
} else {
const payload = {
...base,
state: "error",
errorMessage: evt.data.error
? formatForLog(evt.data.error)
: undefined,
};
broadcast("chat", payload);
bridgeSendToSession(sessionKey, "chat", payload);
}
}
}
}
if (jobState === "done" || jobState === "error") {
clearAgentRunContext(evt.runId);
}
});
const heartbeatUnsub = onHeartbeatEvent((evt) => {
@@ -4116,6 +4329,17 @@ export async function startGatewayServer(
signalLastProbeAt = Date.now();
}
const imessageCfg = cfg.imessage;
const imessageEnabled = imessageCfg?.enabled !== false;
const imessageConfigured =
Boolean(imessageCfg) && imessageEnabled;
let imessageProbe: IMessageProbe | undefined;
let imessageLastProbeAt: number | null = null;
if (probe && imessageConfigured) {
imessageProbe = await probeIMessage(timeoutMs);
imessageLastProbeAt = Date.now();
}
const linked = await webAuthExists();
const authAgeMs = getWebAuthAgeMs();
const self = readWebSelfId();
@@ -4169,6 +4393,17 @@ export async function startGatewayServer(
probe: signalProbe,
lastProbeAt: signalLastProbeAt,
},
imessage: {
configured: imessageConfigured,
running: imessageRuntime.running,
lastStartAt: imessageRuntime.lastStartAt ?? null,
lastStopAt: imessageRuntime.lastStopAt ?? null,
lastError: imessageRuntime.lastError ?? null,
cliPath: imessageRuntime.cliPath ?? null,
dbPath: imessageRuntime.dbPath ?? null,
probe: imessageProbe,
lastProbeAt: imessageLastProbeAt,
},
},
undefined,
);
@@ -6022,7 +6257,9 @@ export async function startGatewayServer(
}
const to = params.to.trim();
const message = params.message.trim();
const provider = (params.provider ?? "whatsapp").toLowerCase();
const providerRaw = (params.provider ?? "whatsapp").toLowerCase();
const provider =
providerRaw === "imsg" ? "imessage" : providerRaw;
try {
if (provider === "telegram") {
const cfg = loadConfig();
@@ -6083,6 +6320,27 @@ export async function startGatewayServer(
payload,
});
respond(true, payload, undefined, { provider });
} else if (provider === "imessage") {
const cfg = loadConfig();
const result = await sendMessageIMessage(to, message, {
mediaUrl: params.mediaUrl,
cliPath: cfg.imessage?.cliPath,
dbPath: cfg.imessage?.dbPath,
maxBytes: cfg.imessage?.mediaMaxMb
? cfg.imessage.mediaMaxMb * 1024 * 1024
: undefined,
});
const payload = {
runId: idem,
messageId: result.messageId,
provider,
};
dedupe.set(`send:${idem}`, {
ts: Date.now(),
ok: true,
payload,
});
respond(true, payload, undefined, { provider });
} else {
const result = await sendMessageWhatsApp(to, message, {
mediaUrl: params.mediaUrl,
@@ -6197,9 +6455,13 @@ export async function startGatewayServer(
const requestedChannelRaw =
typeof params.channel === "string" ? params.channel.trim() : "";
const requestedChannel = requestedChannelRaw
const requestedChannelNormalized = requestedChannelRaw
? requestedChannelRaw.toLowerCase()
: "last";
const requestedChannel =
requestedChannelNormalized === "imsg"
? "imessage"
: requestedChannelNormalized;
const lastChannel = sessionEntry?.lastChannel;
const lastTo =
@@ -6220,6 +6482,7 @@ export async function startGatewayServer(
requestedChannel === "telegram" ||
requestedChannel === "discord" ||
requestedChannel === "signal" ||
requestedChannel === "imessage" ||
requestedChannel === "webchat"
) {
return requestedChannel;
@@ -6239,7 +6502,8 @@ export async function startGatewayServer(
resolvedChannel === "whatsapp" ||
resolvedChannel === "telegram" ||
resolvedChannel === "discord" ||
resolvedChannel === "signal"
resolvedChannel === "signal" ||
resolvedChannel === "imessage"
) {
return lastTo || undefined;
}
@@ -6428,6 +6692,24 @@ export async function startGatewayServer(
logBrowser.error(`server failed to start: ${String(err)}`);
}
// Start Gmail watcher if configured (hooks.gmail.account).
if (process.env.CLAWDIS_SKIP_GMAIL_WATCHER !== "1") {
try {
const gmailResult = await startGmailWatcher(cfgAtStart);
if (gmailResult.started) {
logHooks.info("gmail watcher started");
} else if (
gmailResult.reason &&
gmailResult.reason !== "hooks not enabled" &&
gmailResult.reason !== "no gmail account configured"
) {
logHooks.warn(`gmail watcher not started: ${gmailResult.reason}`);
}
} catch (err) {
logHooks.error(`gmail watcher failed to start: ${String(err)}`);
}
}
// Launch configured providers (WhatsApp Web, Discord, Telegram) so gateway replies via the
// surface the message came from. Tests can opt out via CLAWDIS_SKIP_PROVIDERS.
if (process.env.CLAWDIS_SKIP_PROVIDERS !== "1") {
@@ -6485,6 +6767,8 @@ export async function startGatewayServer(
await stopTelegramProvider();
await stopDiscordProvider();
await stopSignalProvider();
await stopIMessageProvider();
await stopGmailWatcher();
cron.stop();
heartbeatRunner.stop();
broadcast("shutdown", {
@@ -6522,9 +6806,9 @@ export async function startGatewayServer(
await stopBrowserControlServerIfStarted().catch(() => {});
}
await Promise.allSettled(
[whatsappTask, telegramTask, signalTask].filter(Boolean) as Array<
Promise<unknown>
>,
[whatsappTask, telegramTask, signalTask, imessageTask].filter(
Boolean,
) as Array<Promise<unknown>>,
);
await new Promise<void>((resolve) => wss.close(() => resolve()));
await new Promise<void>((resolve, reject) =>
+124
View File
@@ -0,0 +1,124 @@
import type { ChildProcess } from "node:child_process";
import { EventEmitter } from "node:events";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const runtimeConfig = {
account: "clawdbot@gmail.com",
label: "INBOX",
topic: "projects/test/topics/gog-gmail-watch",
subscription: "gog-gmail-watch-push",
pushToken: "push-token",
hookToken: "hook-token",
hookUrl: "http://127.0.0.1:18789/hooks/gmail",
includeBody: false,
maxBytes: 0,
renewEveryMinutes: 720,
serve: { bind: "127.0.0.1", port: 8788, path: "/" },
tailscale: { mode: "off", path: "/gmail-pubsub" },
};
class MockChild extends EventEmitter {
stdout = new EventEmitter();
stderr = new EventEmitter();
exitCode: number | null = null;
kill = vi.fn();
}
const spawnMock = vi.fn(() => new MockChild() as unknown as ChildProcess);
vi.mock("node:child_process", async () => {
const actual =
await vi.importActual<typeof import("node:child_process")>(
"node:child_process",
);
return {
...actual,
spawn: spawnMock,
};
});
const runCommandWithTimeoutMock = vi.fn(async () => ({
code: 0,
stdout: "",
stderr: "",
}));
vi.mock("../agents/skills.js", () => ({
hasBinary: () => true,
}));
vi.mock("../process/exec.js", () => ({
runCommandWithTimeout: runCommandWithTimeoutMock,
}));
vi.mock("./gmail-setup-utils.js", () => ({
ensureTailscaleEndpoint: vi.fn(async () => {}),
}));
vi.mock("./gmail.js", () => ({
buildGogWatchServeArgs: () => [
"gmail",
"watch",
"serve",
"--token",
runtimeConfig.pushToken,
"--hook-url",
runtimeConfig.hookUrl,
"--hook-token",
runtimeConfig.hookToken,
],
buildGogWatchStartArgs: () => ["gmail", "watch", "start"],
resolveGmailHookRuntimeConfig: () => ({ ok: true, value: runtimeConfig }),
}));
const { startGmailWatcher, stopGmailWatcher } = await import(
"./gmail-watcher.js"
);
const cfg = {
hooks: {
enabled: true,
gmail: { account: runtimeConfig.account },
},
};
beforeEach(() => {
spawnMock.mockClear();
runCommandWithTimeoutMock.mockClear();
});
afterEach(async () => {
await stopGmailWatcher();
vi.useRealTimers();
});
describe("gmail watcher", () => {
it("does not start twice when already running", async () => {
const first = await startGmailWatcher(cfg);
const second = await startGmailWatcher(cfg);
expect(first.started).toBe(true);
expect(second.started).toBe(true);
expect(spawnMock).toHaveBeenCalledTimes(1);
});
it("restarts the gog watcher after a process error", async () => {
vi.useFakeTimers();
await startGmailWatcher(cfg);
const child = spawnMock.mock.results[0]?.value as MockChild | undefined;
expect(child).toBeDefined();
child?.emit("error", new Error("boom"));
expect(spawnMock).toHaveBeenCalledTimes(1);
await vi.advanceTimersByTimeAsync(5000);
expect(spawnMock).toHaveBeenCalledTimes(2);
const restartChild = spawnMock.mock.results[1]?.value as
| MockChild
| undefined;
const stopPromise = stopGmailWatcher();
restartChild?.emit("exit", 0, null);
await stopPromise;
});
});
+273
View File
@@ -0,0 +1,273 @@
/**
* Gmail Watcher Service
*
* Automatically starts `gog gmail watch serve` when the gateway starts,
* if hooks.gmail is configured with an account.
*/
import { type ChildProcess, spawn } from "node:child_process";
import { hasBinary } from "../agents/skills.js";
import type { ClawdisConfig } from "../config/config.js";
import { createSubsystemLogger } from "../logging.js";
import { runCommandWithTimeout } from "../process/exec.js";
import {
buildGogWatchServeArgs,
buildGogWatchStartArgs,
type GmailHookRuntimeConfig,
resolveGmailHookRuntimeConfig,
} from "./gmail.js";
import { ensureTailscaleEndpoint } from "./gmail-setup-utils.js";
const log = createSubsystemLogger("gmail-watcher");
let watcherProcess: ChildProcess | null = null;
let renewInterval: ReturnType<typeof setInterval> | null = null;
let restartTimer: ReturnType<typeof setTimeout> | null = null;
let shuttingDown = false;
let currentConfig: GmailHookRuntimeConfig | null = null;
const redactedValue = "<redacted>";
const redactedFlags = new Set(["--token", "--hook-token", "--hook-url"]);
function redactArgs(args: string[]): string[] {
const redacted = [...args];
for (let i = 0; i < redacted.length; i += 1) {
if (redactedFlags.has(redacted[i] ?? "") && redacted[i + 1]) {
redacted[i + 1] = redactedValue;
i += 1;
}
}
return redacted;
}
function scheduleRestart(reason: string) {
if (shuttingDown || !currentConfig || restartTimer) return;
log.warn(`${reason}; restarting in 5s`);
restartTimer = setTimeout(() => {
restartTimer = null;
if (shuttingDown || !currentConfig) return;
watcherProcess = spawnGogServe(currentConfig);
}, 5000);
}
/**
* Check if gog binary is available
*/
function isGogAvailable(): boolean {
return hasBinary("gog");
}
/**
* Start the Gmail watch (registers with Gmail API)
*/
async function startGmailWatch(
cfg: Pick<GmailHookRuntimeConfig, "account" | "label" | "topic">,
): Promise<boolean> {
const args = ["gog", ...buildGogWatchStartArgs(cfg)];
try {
const result = await runCommandWithTimeout(args, { timeoutMs: 120_000 });
if (result.code !== 0) {
const message =
result.stderr || result.stdout || "gog watch start failed";
log.error(`watch start failed: ${message}`);
return false;
}
log.info(`watch started for ${cfg.account}`);
return true;
} catch (err) {
log.error(`watch start error: ${String(err)}`);
return false;
}
}
/**
* Spawn the gog gmail watch serve process
*/
function spawnGogServe(cfg: GmailHookRuntimeConfig): ChildProcess {
const args = buildGogWatchServeArgs(cfg);
if (restartTimer) {
clearTimeout(restartTimer);
restartTimer = null;
}
log.info(`starting gog ${redactArgs(args).join(" ")}`);
const child = spawn("gog", args, {
stdio: ["ignore", "pipe", "pipe"],
detached: false,
});
child.stdout?.on("data", (data: Buffer) => {
const line = data.toString().trim();
if (line) log.info(`[gog] ${line}`);
});
child.stderr?.on("data", (data: Buffer) => {
const line = data.toString().trim();
if (line) log.warn(`[gog] ${line}`);
});
child.on("error", (err) => {
log.error(`gog process error: ${String(err)}`);
if (watcherProcess === child) {
watcherProcess = null;
}
scheduleRestart("gog process error");
});
child.on("exit", (code, signal) => {
if (shuttingDown) return;
if (watcherProcess === child) {
watcherProcess = null;
}
scheduleRestart(`gog exited (code=${code}, signal=${signal})`);
});
return child;
}
export type GmailWatcherStartResult = {
started: boolean;
reason?: string;
};
/**
* Start the Gmail watcher service.
* Called automatically by the gateway if hooks.gmail is configured.
*/
export async function startGmailWatcher(
cfg: ClawdisConfig,
): Promise<GmailWatcherStartResult> {
// Check if gmail hooks are configured
if (!cfg.hooks?.enabled) {
return { started: false, reason: "hooks not enabled" };
}
if (!cfg.hooks?.gmail?.account) {
return { started: false, reason: "no gmail account configured" };
}
// Check if gog is available
const gogAvailable = isGogAvailable();
if (!gogAvailable) {
return { started: false, reason: "gog binary not found" };
}
// Resolve the full runtime config
const resolved = resolveGmailHookRuntimeConfig(cfg, {});
if (!resolved.ok) {
return { started: false, reason: resolved.error };
}
const runtimeConfig = resolved.value;
if (isGmailWatcherRunning()) {
log.info("gmail watcher already running; skipping start");
return { started: true };
}
if (renewInterval) {
clearInterval(renewInterval);
renewInterval = null;
}
if (restartTimer) {
clearTimeout(restartTimer);
restartTimer = null;
}
if (watcherProcess) {
watcherProcess = null;
}
currentConfig = runtimeConfig;
// Set up Tailscale endpoint if needed
if (runtimeConfig.tailscale.mode !== "off") {
try {
await ensureTailscaleEndpoint({
mode: runtimeConfig.tailscale.mode,
path: runtimeConfig.tailscale.path,
port: runtimeConfig.serve.port,
});
log.info(
`tailscale ${runtimeConfig.tailscale.mode} configured for port ${runtimeConfig.serve.port}`,
);
} catch (err) {
log.error(`tailscale setup failed: ${String(err)}`);
return {
started: false,
reason: `tailscale setup failed: ${String(err)}`,
};
}
}
// Start the Gmail watch (register with Gmail API)
const watchStarted = await startGmailWatch(runtimeConfig);
if (!watchStarted) {
log.warn("gmail watch start failed, but continuing with serve");
}
// Spawn the gog serve process
shuttingDown = false;
watcherProcess = spawnGogServe(runtimeConfig);
// Set up renewal interval
const renewMs = runtimeConfig.renewEveryMinutes * 60_000;
renewInterval = setInterval(() => {
if (shuttingDown) return;
void startGmailWatch(runtimeConfig);
}, renewMs);
log.info(
`gmail watcher started for ${runtimeConfig.account} (renew every ${runtimeConfig.renewEveryMinutes}m)`,
);
return { started: true };
}
/**
* Stop the Gmail watcher service.
*/
export async function stopGmailWatcher(): Promise<void> {
shuttingDown = true;
if (restartTimer) {
clearTimeout(restartTimer);
restartTimer = null;
}
if (renewInterval) {
clearInterval(renewInterval);
renewInterval = null;
}
if (watcherProcess) {
log.info("stopping gmail watcher");
watcherProcess.kill("SIGTERM");
// Wait a bit for graceful shutdown
await new Promise<void>((resolve) => {
const timeout = setTimeout(() => {
if (watcherProcess) {
watcherProcess.kill("SIGKILL");
}
resolve();
}, 3000);
watcherProcess?.on("exit", () => {
clearTimeout(timeout);
resolve();
});
});
watcherProcess = null;
}
currentConfig = null;
log.info("gmail watcher stopped");
}
/**
* Check if the Gmail watcher is running.
*/
export function isGmailWatcherRunning(): boolean {
return (
watcherProcess !== null && !shuttingDown && watcherProcess.exitCode === null
);
}
+215
View File
@@ -0,0 +1,215 @@
import { type ChildProcessWithoutNullStreams, spawn } from "node:child_process";
import { createInterface, type Interface } from "node:readline";
import type { RuntimeEnv } from "../runtime.js";
import { resolveUserPath } from "../utils.js";
export type IMessageRpcError = {
code?: number;
message?: string;
data?: unknown;
};
export type IMessageRpcResponse<T> = {
jsonrpc?: string;
id?: string | number | null;
result?: T;
error?: IMessageRpcError;
method?: string;
params?: unknown;
};
export type IMessageRpcNotification = {
method: string;
params?: unknown;
};
export type IMessageRpcClientOptions = {
cliPath?: string;
dbPath?: string;
runtime?: RuntimeEnv;
onNotification?: (msg: IMessageRpcNotification) => void;
};
type PendingRequest = {
resolve: (value: unknown) => void;
reject: (error: Error) => void;
timer?: NodeJS.Timeout;
};
export class IMessageRpcClient {
private readonly cliPath: string;
private readonly dbPath?: string;
private readonly runtime?: RuntimeEnv;
private readonly onNotification?: (msg: IMessageRpcNotification) => void;
private readonly pending = new Map<string, PendingRequest>();
private readonly closed: Promise<void>;
private closedResolve: (() => void) | null = null;
private child: ChildProcessWithoutNullStreams | null = null;
private reader: Interface | null = null;
private nextId = 1;
constructor(opts: IMessageRpcClientOptions = {}) {
this.cliPath = opts.cliPath?.trim() || "imsg";
this.dbPath = opts.dbPath?.trim()
? resolveUserPath(opts.dbPath)
: undefined;
this.runtime = opts.runtime;
this.onNotification = opts.onNotification;
this.closed = new Promise((resolve) => {
this.closedResolve = resolve;
});
}
async start(): Promise<void> {
if (this.child) return;
const args = ["rpc"];
if (this.dbPath) {
args.push("--db", this.dbPath);
}
const child = spawn(this.cliPath, args, {
stdio: ["pipe", "pipe", "pipe"],
});
this.child = child;
this.reader = createInterface({ input: child.stdout });
this.reader.on("line", (line) => {
const trimmed = line.trim();
if (!trimmed) return;
this.handleLine(trimmed);
});
child.stderr?.on("data", (chunk) => {
const lines = chunk.toString().split(/\r?\n/);
for (const line of lines) {
if (!line.trim()) continue;
this.runtime?.error?.(`imsg rpc: ${line.trim()}`);
}
});
child.on("error", (err) => {
this.failAll(err instanceof Error ? err : new Error(String(err)));
this.closedResolve?.();
});
child.on("close", (code, signal) => {
if (code !== 0 && code !== null) {
const reason = signal ? `signal ${signal}` : `code ${code}`;
this.failAll(new Error(`imsg rpc exited (${reason})`));
} else {
this.failAll(new Error("imsg rpc closed"));
}
this.closedResolve?.();
});
}
async stop(): Promise<void> {
if (!this.child) return;
this.reader?.close();
this.reader = null;
this.child.stdin?.end();
const child = this.child;
this.child = null;
await Promise.race([
this.closed,
new Promise<void>((resolve) => {
setTimeout(() => {
if (!child.killed) child.kill("SIGTERM");
resolve();
}, 500);
}),
]);
}
async waitForClose(): Promise<void> {
await this.closed;
}
async request<T = unknown>(
method: string,
params?: Record<string, unknown>,
opts?: { timeoutMs?: number },
): Promise<T> {
if (!this.child || !this.child.stdin) {
throw new Error("imsg rpc not running");
}
const id = this.nextId++;
const payload = {
jsonrpc: "2.0",
id,
method,
params: params ?? {},
};
const line = `${JSON.stringify(payload)}\n`;
const timeoutMs = opts?.timeoutMs ?? 10_000;
const response = new Promise<T>((resolve, reject) => {
const key = String(id);
const timer =
timeoutMs > 0
? setTimeout(() => {
this.pending.delete(key);
reject(new Error(`imsg rpc timeout (${method})`));
}, timeoutMs)
: undefined;
this.pending.set(key, {
resolve: (value) => resolve(value as T),
reject,
timer,
});
});
this.child.stdin.write(line);
return await response;
}
private handleLine(line: string) {
let parsed: IMessageRpcResponse<unknown>;
try {
parsed = JSON.parse(line) as IMessageRpcResponse<unknown>;
} catch (_err) {
this.runtime?.error?.(`imsg rpc: failed to parse ${line}`);
return;
}
if (parsed.id !== undefined && parsed.id !== null) {
const key = String(parsed.id);
const pending = this.pending.get(key);
if (!pending) return;
if (pending.timer) clearTimeout(pending.timer);
this.pending.delete(key);
if (parsed.error) {
const msg = parsed.error.message ?? "imsg rpc error";
pending.reject(new Error(msg));
return;
}
pending.resolve(parsed.result);
return;
}
if (parsed.method) {
this.onNotification?.({
method: parsed.method,
params: parsed.params,
});
}
}
private failAll(err: Error) {
for (const [key, pending] of this.pending.entries()) {
if (pending.timer) clearTimeout(pending.timer);
pending.reject(err);
this.pending.delete(key);
}
}
}
export async function createIMessageRpcClient(
opts: IMessageRpcClientOptions = {},
): Promise<IMessageRpcClient> {
const client = new IMessageRpcClient(opts);
await client.start();
return client;
}
+3
View File
@@ -0,0 +1,3 @@
export { monitorIMessageProvider } from "./monitor.js";
export { probeIMessage } from "./probe.js";
export { sendMessageIMessage } from "./send.js";
+200
View File
@@ -0,0 +1,200 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { monitorIMessageProvider } from "./monitor.js";
const requestMock = vi.fn();
const stopMock = vi.fn();
const sendMock = vi.fn();
const replyMock = vi.fn();
const updateLastRouteMock = vi.fn();
let config: Record<string, unknown> = {};
let notificationHandler:
| ((msg: { method: string; params?: unknown }) => void)
| undefined;
let closeResolve: (() => void) | undefined;
vi.mock("../config/config.js", () => ({
loadConfig: () => config,
}));
vi.mock("../auto-reply/reply.js", () => ({
getReplyFromConfig: (...args: unknown[]) => replyMock(...args),
}));
vi.mock("./send.js", () => ({
sendMessageIMessage: (...args: unknown[]) => sendMock(...args),
}));
vi.mock("../config/sessions.js", () => ({
resolveStorePath: vi.fn(() => "/tmp/clawdis-sessions.json"),
updateLastRoute: (...args: unknown[]) => updateLastRouteMock(...args),
}));
vi.mock("./client.js", () => ({
createIMessageRpcClient: vi.fn(
async (opts: { onNotification?: typeof notificationHandler }) => {
notificationHandler = opts.onNotification;
return {
request: (...args: unknown[]) => requestMock(...args),
waitForClose: () =>
new Promise<void>((resolve) => {
closeResolve = resolve;
}),
stop: (...args: unknown[]) => stopMock(...args),
};
},
),
}));
const flush = () => new Promise((resolve) => setTimeout(resolve, 0));
async function waitForSubscribe() {
for (let i = 0; i < 5; i += 1) {
if (requestMock.mock.calls.some((call) => call[0] === "watch.subscribe"))
return;
await flush();
}
}
beforeEach(() => {
config = {
imessage: {},
session: { mainKey: "main" },
routing: {
groupChat: { mentionPatterns: ["@clawd"], requireMention: true },
allowFrom: [],
},
};
requestMock.mockReset().mockImplementation((method: string) => {
if (method === "watch.subscribe")
return Promise.resolve({ subscription: 1 });
return Promise.resolve({});
});
stopMock.mockReset().mockResolvedValue(undefined);
sendMock.mockReset().mockResolvedValue({ messageId: "ok" });
replyMock.mockReset().mockResolvedValue({ text: "ok" });
updateLastRouteMock.mockReset();
notificationHandler = undefined;
closeResolve = undefined;
});
describe("monitorIMessageProvider", () => {
it("skips group messages without a mention by default", async () => {
const run = monitorIMessageProvider();
await waitForSubscribe();
notificationHandler?.({
method: "message",
params: {
message: {
id: 1,
chat_id: 99,
sender: "+15550001111",
is_from_me: false,
text: "hello group",
is_group: true,
},
},
});
await flush();
closeResolve?.();
await run;
expect(replyMock).not.toHaveBeenCalled();
expect(sendMock).not.toHaveBeenCalled();
});
it("delivers group replies when mentioned", async () => {
replyMock.mockResolvedValueOnce({ text: "yo" });
const run = monitorIMessageProvider();
await waitForSubscribe();
notificationHandler?.({
method: "message",
params: {
message: {
id: 2,
chat_id: 42,
sender: "+15550002222",
is_from_me: false,
text: "@clawd ping",
is_group: true,
chat_name: "Lobster Squad",
participants: ["+1555", "+1556"],
},
},
});
await flush();
closeResolve?.();
await run;
expect(sendMock).toHaveBeenCalledWith(
"chat_id:42",
"yo",
expect.objectContaining({ client: expect.any(Object) }),
);
});
it("honors allowFrom entries", async () => {
config = {
...config,
imessage: { allowFrom: ["chat_id:101"] },
};
const run = monitorIMessageProvider();
await waitForSubscribe();
notificationHandler?.({
method: "message",
params: {
message: {
id: 3,
chat_id: 202,
sender: "+15550003333",
is_from_me: false,
text: "@clawd hi",
is_group: true,
},
},
});
await flush();
closeResolve?.();
await run;
expect(replyMock).not.toHaveBeenCalled();
});
it("updates last route with chat_id for direct messages", async () => {
replyMock.mockResolvedValueOnce({ text: "ok" });
const run = monitorIMessageProvider();
await waitForSubscribe();
notificationHandler?.({
method: "message",
params: {
message: {
id: 4,
chat_id: 7,
sender: "+15550004444",
is_from_me: false,
text: "hey",
is_group: false,
},
},
});
await flush();
closeResolve?.();
await run;
expect(updateLastRouteMock).toHaveBeenCalledWith(
expect.objectContaining({
channel: "imessage",
to: "chat_id:7",
}),
);
});
});
+308
View File
@@ -0,0 +1,308 @@
import { chunkText } from "../auto-reply/chunk.js";
import { formatAgentEnvelope } from "../auto-reply/envelope.js";
import { getReplyFromConfig } from "../auto-reply/reply.js";
import type { ReplyPayload } from "../auto-reply/types.js";
import { loadConfig } from "../config/config.js";
import { resolveStorePath, updateLastRoute } from "../config/sessions.js";
import { danger, isVerbose, logVerbose } from "../globals.js";
import { mediaKindFromMime } from "../media/constants.js";
import type { RuntimeEnv } from "../runtime.js";
import { createIMessageRpcClient } from "./client.js";
import { sendMessageIMessage } from "./send.js";
import {
formatIMessageChatTarget,
isAllowedIMessageSender,
normalizeIMessageHandle,
} from "./targets.js";
type IMessageAttachment = {
original_path?: string | null;
mime_type?: string | null;
missing?: boolean | null;
};
type IMessagePayload = {
id?: number | null;
chat_id?: number | null;
sender?: string | null;
is_from_me?: boolean | null;
text?: string | null;
created_at?: string | null;
attachments?: IMessageAttachment[] | null;
chat_identifier?: string | null;
chat_guid?: string | null;
chat_name?: string | null;
participants?: string[] | null;
is_group?: boolean | null;
};
export type MonitorIMessageOpts = {
runtime?: RuntimeEnv;
abortSignal?: AbortSignal;
cliPath?: string;
dbPath?: string;
allowFrom?: Array<string | number>;
includeAttachments?: boolean;
mediaMaxMb?: number;
requireMention?: boolean;
};
function resolveRuntime(opts: MonitorIMessageOpts): RuntimeEnv {
return (
opts.runtime ?? {
log: console.log,
error: console.error,
exit: (code: number): never => {
throw new Error(`exit ${code}`);
},
}
);
}
function resolveAllowFrom(opts: MonitorIMessageOpts): string[] {
const cfg = loadConfig();
const raw =
opts.allowFrom ?? cfg.imessage?.allowFrom ?? cfg.routing?.allowFrom ?? [];
return raw.map((entry) => String(entry).trim()).filter(Boolean);
}
function resolveMentionRegexes(cfg: ReturnType<typeof loadConfig>): RegExp[] {
return (
cfg.routing?.groupChat?.mentionPatterns
?.map((pattern) => {
try {
return new RegExp(pattern, "i");
} catch {
return null;
}
})
.filter((val): val is RegExp => Boolean(val)) ?? []
);
}
function resolveRequireMention(opts: MonitorIMessageOpts): boolean {
const cfg = loadConfig();
if (typeof opts.requireMention === "boolean") return opts.requireMention;
return cfg.routing?.groupChat?.requireMention ?? true;
}
function isMentioned(text: string, regexes: RegExp[]): boolean {
if (!text) return false;
const cleaned = text
.replace(/[\u200b-\u200f\u202a-\u202e\u2060-\u206f]/g, "")
.toLowerCase();
return regexes.some((re) => re.test(cleaned));
}
async function deliverReplies(params: {
replies: ReplyPayload[];
target: string;
client: Awaited<ReturnType<typeof createIMessageRpcClient>>;
runtime: RuntimeEnv;
maxBytes: number;
}) {
const { replies, target, client, runtime, maxBytes } = params;
for (const payload of replies) {
const mediaList =
payload.mediaUrls ?? (payload.mediaUrl ? [payload.mediaUrl] : []);
const text = payload.text ?? "";
if (!text && mediaList.length === 0) continue;
if (mediaList.length === 0) {
for (const chunk of chunkText(text, 4000)) {
await sendMessageIMessage(target, chunk, { maxBytes, client });
}
} else {
let first = true;
for (const url of mediaList) {
const caption = first ? text : "";
first = false;
await sendMessageIMessage(target, caption, {
mediaUrl: url,
maxBytes,
client,
});
}
}
runtime.log?.(`imessage: delivered reply to ${target}`);
}
}
export async function monitorIMessageProvider(
opts: MonitorIMessageOpts = {},
): Promise<void> {
const runtime = resolveRuntime(opts);
const cfg = loadConfig();
const allowFrom = resolveAllowFrom(opts);
const mentionRegexes = resolveMentionRegexes(cfg);
const requireMention = resolveRequireMention(opts);
const includeAttachments =
opts.includeAttachments ?? cfg.imessage?.includeAttachments ?? false;
const mediaMaxBytes =
(opts.mediaMaxMb ?? cfg.imessage?.mediaMaxMb ?? 16) * 1024 * 1024;
const handleMessage = async (raw: unknown) => {
const params = raw as { message?: IMessagePayload | null };
const message = params?.message ?? null;
if (!message) return;
const senderRaw = message.sender ?? "";
const sender = senderRaw.trim();
if (!sender) return;
if (message.is_from_me) return;
const chatId = message.chat_id ?? undefined;
const chatGuid = message.chat_guid ?? undefined;
const chatIdentifier = message.chat_identifier ?? undefined;
const isGroup = Boolean(message.is_group);
if (isGroup && !chatId) return;
if (
!isAllowedIMessageSender({
allowFrom,
sender,
chatId: chatId ?? undefined,
chatGuid,
chatIdentifier,
})
) {
logVerbose(`Blocked iMessage sender ${sender} (not in allowFrom)`);
return;
}
const messageText = (message.text ?? "").trim();
const mentioned = isGroup ? isMentioned(messageText, mentionRegexes) : true;
if (isGroup && requireMention && !mentioned) {
logVerbose(`imessage: skipping group message (no mention)`);
return;
}
const attachments = includeAttachments ? (message.attachments ?? []) : [];
const firstAttachment = attachments?.find(
(entry) => entry?.original_path && !entry?.missing,
);
const mediaPath = firstAttachment?.original_path ?? undefined;
const mediaType = firstAttachment?.mime_type ?? undefined;
const kind = mediaKindFromMime(mediaType ?? undefined);
const placeholder = kind
? `<media:${kind}>`
: attachments?.length
? "<media:attachment>"
: "";
const bodyText = messageText || placeholder;
if (!bodyText) return;
const chatTarget = formatIMessageChatTarget(chatId);
const fromLabel = isGroup
? `${message.chat_name || "iMessage Group"} id:${chatId ?? "unknown"}`
: `${normalizeIMessageHandle(sender)} id:${sender}`;
const createdAt = message.created_at
? Date.parse(message.created_at)
: undefined;
const body = formatAgentEnvelope({
surface: "iMessage",
from: fromLabel,
timestamp: createdAt,
body: bodyText,
});
const ctxPayload = {
Body: body,
From: isGroup ? `group:${chatId}` : `imessage:${sender}`,
To: chatTarget || `imessage:${sender}`,
ChatType: isGroup ? "group" : "direct",
GroupSubject: isGroup ? (message.chat_name ?? undefined) : undefined,
GroupMembers: isGroup
? (message.participants ?? []).filter(Boolean).join(", ")
: undefined,
SenderName: sender,
Surface: "imessage",
MessageSid: message.id ? String(message.id) : undefined,
Timestamp: createdAt,
MediaPath: mediaPath,
MediaType: mediaType,
MediaUrl: mediaPath,
WasMentioned: mentioned,
};
if (!isGroup) {
const sessionCfg = cfg.session;
const mainKey = (sessionCfg?.mainKey ?? "main").trim() || "main";
const storePath = resolveStorePath(sessionCfg?.store);
const to = chatTarget || sender;
if (to) {
await updateLastRoute({
storePath,
sessionKey: mainKey,
channel: "imessage",
to,
});
}
}
if (isVerbose()) {
const preview = body.slice(0, 200).replace(/\n/g, "\\n");
logVerbose(
`imessage inbound: chatId=${chatId ?? "unknown"} from=${ctxPayload.From} len=${body.length} preview="${preview}"`,
);
}
const replyResult = await getReplyFromConfig(ctxPayload, undefined, cfg);
const replies = replyResult
? Array.isArray(replyResult)
? replyResult
: [replyResult]
: [];
if (replies.length === 0) return;
await deliverReplies({
replies,
target: ctxPayload.To,
client,
runtime,
maxBytes: mediaMaxBytes,
});
};
const client = await createIMessageRpcClient({
cliPath: opts.cliPath ?? cfg.imessage?.cliPath,
dbPath: opts.dbPath ?? cfg.imessage?.dbPath,
runtime,
onNotification: (msg) => {
if (msg.method === "message") {
void handleMessage(msg.params).catch((err) => {
runtime.error?.(`imessage: handler failed: ${String(err)}`);
});
} else if (msg.method === "error") {
runtime.error?.(`imessage: watch error ${JSON.stringify(msg.params)}`);
}
},
});
let subscriptionId: number | null = null;
const abort = opts.abortSignal;
const onAbort = () => {
if (subscriptionId) {
void client.request("watch.unsubscribe", {
subscription: subscriptionId,
});
}
void client.stop();
};
abort?.addEventListener("abort", onAbort, { once: true });
try {
const result = await client.request<{ subscription?: number }>(
"watch.subscribe",
{ attachments: includeAttachments },
);
subscriptionId = result?.subscription ?? null;
await client.waitForClose();
} catch (err) {
if (abort?.aborted) return;
runtime.error?.(danger(`imessage: monitor failed: ${String(err)}`));
throw err;
} finally {
abort?.removeEventListener("abort", onAbort);
await client.stop();
}
}
+31
View File
@@ -0,0 +1,31 @@
import { detectBinary } from "../commands/onboard-helpers.js";
import { loadConfig } from "../config/config.js";
import { createIMessageRpcClient } from "./client.js";
export type IMessageProbe = {
ok: boolean;
error?: string | null;
};
export async function probeIMessage(timeoutMs = 2000): Promise<IMessageProbe> {
const cfg = loadConfig();
const cliPath = cfg.imessage?.cliPath?.trim() || "imsg";
const dbPath = cfg.imessage?.dbPath?.trim();
const detected = await detectBinary(cliPath);
if (!detected) {
return { ok: false, error: `imsg not found (${cliPath})` };
}
const client = await createIMessageRpcClient({
cliPath,
dbPath,
});
try {
await client.request("chats.list", { limit: 1 }, { timeoutMs });
return { ok: true };
} catch (err) {
return { ok: false, error: String(err) };
} finally {
await client.stop();
}
}
+64
View File
@@ -0,0 +1,64 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { sendMessageIMessage } from "./send.js";
const requestMock = vi.fn();
const stopMock = vi.fn();
vi.mock("../config/config.js", () => ({
loadConfig: () => ({}),
}));
vi.mock("./client.js", () => ({
createIMessageRpcClient: vi.fn().mockResolvedValue({
request: (...args: unknown[]) => requestMock(...args),
stop: (...args: unknown[]) => stopMock(...args),
}),
}));
vi.mock("../web/media.js", () => ({
loadWebMedia: vi.fn().mockResolvedValue({
buffer: Buffer.from("data"),
contentType: "image/jpeg",
}),
}));
vi.mock("../media/store.js", () => ({
saveMediaBuffer: vi.fn().mockResolvedValue({
path: "/tmp/imessage-media.jpg",
contentType: "image/jpeg",
}),
}));
describe("sendMessageIMessage", () => {
beforeEach(() => {
requestMock.mockReset().mockResolvedValue({ ok: true });
stopMock.mockReset().mockResolvedValue(undefined);
});
it("sends to chat_id targets", async () => {
await sendMessageIMessage("chat_id:123", "hi");
const params = requestMock.mock.calls[0]?.[1] as Record<string, unknown>;
expect(requestMock).toHaveBeenCalledWith(
"send",
expect.any(Object),
expect.any(Object),
);
expect(params.chat_id).toBe(123);
expect(params.text).toBe("hi");
});
it("applies sms service prefix", async () => {
await sendMessageIMessage("sms:+1555", "hello");
const params = requestMock.mock.calls[0]?.[1] as Record<string, unknown>;
expect(params.service).toBe("sms");
expect(params.to).toBe("+1555");
});
it("adds file attachment with placeholder text", async () => {
await sendMessageIMessage("chat_id:7", "", { mediaUrl: "http://x/y.jpg" });
const params = requestMock.mock.calls[0]?.[1] as Record<string, unknown>;
expect(params.file).toBe("/tmp/imessage-media.jpg");
expect(params.text).toBe("<media:image>");
});
});
+127
View File
@@ -0,0 +1,127 @@
import { loadConfig } from "../config/config.js";
import { mediaKindFromMime } from "../media/constants.js";
import { saveMediaBuffer } from "../media/store.js";
import { loadWebMedia } from "../web/media.js";
import { createIMessageRpcClient, type IMessageRpcClient } from "./client.js";
import {
formatIMessageChatTarget,
type IMessageService,
parseIMessageTarget,
} from "./targets.js";
export type IMessageSendOpts = {
cliPath?: string;
dbPath?: string;
service?: IMessageService;
region?: string;
mediaUrl?: string;
maxBytes?: number;
timeoutMs?: number;
chatId?: number;
client?: IMessageRpcClient;
};
export type IMessageSendResult = {
messageId: string;
};
function resolveCliPath(explicit?: string): string {
const cfg = loadConfig();
return explicit?.trim() || cfg.imessage?.cliPath?.trim() || "imsg";
}
function resolveDbPath(explicit?: string): string | undefined {
const cfg = loadConfig();
return explicit?.trim() || cfg.imessage?.dbPath?.trim() || undefined;
}
function resolveService(explicit?: IMessageService): IMessageService {
const cfg = loadConfig();
return (
explicit || (cfg.imessage?.service as IMessageService | undefined) || "auto"
);
}
function resolveRegion(explicit?: string): string {
const cfg = loadConfig();
return explicit?.trim() || cfg.imessage?.region?.trim() || "US";
}
async function resolveAttachment(
mediaUrl: string,
maxBytes: number,
): Promise<{ path: string; contentType?: string }> {
const media = await loadWebMedia(mediaUrl, maxBytes);
const saved = await saveMediaBuffer(
media.buffer,
media.contentType ?? undefined,
"outbound",
maxBytes,
);
return { path: saved.path, contentType: saved.contentType };
}
export async function sendMessageIMessage(
to: string,
text: string,
opts: IMessageSendOpts = {},
): Promise<IMessageSendResult> {
const cliPath = resolveCliPath(opts.cliPath);
const dbPath = resolveDbPath(opts.dbPath);
const target = parseIMessageTarget(
opts.chatId ? formatIMessageChatTarget(opts.chatId) : to,
);
const service =
opts.service ?? (target.kind === "handle" ? target.service : undefined);
const region = resolveRegion(opts.region);
const maxBytes = opts.maxBytes ?? 16 * 1024 * 1024;
let message = text ?? "";
let filePath: string | undefined;
if (opts.mediaUrl?.trim()) {
const resolved = await resolveAttachment(opts.mediaUrl.trim(), maxBytes);
filePath = resolved.path;
if (!message.trim()) {
const kind = mediaKindFromMime(resolved.contentType ?? undefined);
if (kind)
message = kind === "image" ? "<media:image>" : `<media:${kind}>`;
}
}
if (!message.trim() && !filePath) {
throw new Error("iMessage send requires text or media");
}
const params: Record<string, unknown> = {
text: message,
service: resolveService(service),
region,
};
if (filePath) params.file = filePath;
if (target.kind === "chat_id") {
params.chat_id = target.chatId;
} else if (target.kind === "chat_guid") {
params.chat_guid = target.chatGuid;
} else if (target.kind === "chat_identifier") {
params.chat_identifier = target.chatIdentifier;
} else {
params.to = target.to;
}
const client =
opts.client ?? (await createIMessageRpcClient({ cliPath, dbPath }));
const shouldClose = !opts.client;
try {
const result = await client.request<{ ok?: boolean }>("send", params, {
timeoutMs: opts.timeoutMs,
});
return {
messageId: result?.ok ? "ok" : "unknown",
};
} finally {
if (shouldClose) {
await client.stop();
}
}
}
+54
View File
@@ -0,0 +1,54 @@
import { describe, expect, it } from "vitest";
import {
formatIMessageChatTarget,
isAllowedIMessageSender,
normalizeIMessageHandle,
parseIMessageTarget,
} from "./targets.js";
describe("imessage targets", () => {
it("parses chat_id targets", () => {
const target = parseIMessageTarget("chat_id:123");
expect(target).toEqual({ kind: "chat_id", chatId: 123 });
});
it("parses group chat targets", () => {
const target = parseIMessageTarget("group:456");
expect(target).toEqual({ kind: "chat_id", chatId: 456 });
});
it("parses sms handles with service", () => {
const target = parseIMessageTarget("sms:+1555");
expect(target).toEqual({ kind: "handle", to: "+1555", service: "sms" });
});
it("normalizes handles", () => {
expect(normalizeIMessageHandle("Name@Example.com")).toBe(
"name@example.com",
);
expect(normalizeIMessageHandle(" +1 (555) 222-3333 ")).toBe("+15552223333");
});
it("checks allowFrom against chat_id", () => {
const ok = isAllowedIMessageSender({
allowFrom: ["chat_id:9"],
sender: "+1555",
chatId: 9,
});
expect(ok).toBe(true);
});
it("checks allowFrom against handle", () => {
const ok = isAllowedIMessageSender({
allowFrom: ["user@example.com"],
sender: "User@Example.com",
});
expect(ok).toBe(true);
});
it("formats chat targets", () => {
expect(formatIMessageChatTarget(42)).toBe("chat_id:42");
expect(formatIMessageChatTarget(undefined)).toBe("");
});
});
+175
View File
@@ -0,0 +1,175 @@
import { normalizeE164 } from "../utils.js";
export type IMessageService = "imessage" | "sms" | "auto";
export type IMessageTarget =
| { kind: "chat_id"; chatId: number }
| { kind: "chat_guid"; chatGuid: string }
| { kind: "chat_identifier"; chatIdentifier: string }
| { kind: "handle"; to: string; service: IMessageService };
export type IMessageAllowTarget =
| { kind: "chat_id"; chatId: number }
| { kind: "chat_guid"; chatGuid: string }
| { kind: "chat_identifier"; chatIdentifier: string }
| { kind: "handle"; handle: string };
const CHAT_ID_PREFIXES = ["chat_id:", "chatid:", "chat:"];
const CHAT_GUID_PREFIXES = ["chat_guid:", "chatguid:", "guid:"];
const CHAT_IDENTIFIER_PREFIXES = [
"chat_identifier:",
"chatidentifier:",
"chatident:",
];
const SERVICE_PREFIXES: Array<{ prefix: string; service: IMessageService }> = [
{ prefix: "imessage:", service: "imessage" },
{ prefix: "sms:", service: "sms" },
{ prefix: "auto:", service: "auto" },
];
function stripPrefix(value: string, prefix: string): string {
return value.slice(prefix.length).trim();
}
export function normalizeIMessageHandle(raw: string): string {
const trimmed = raw.trim();
if (!trimmed) return "";
const lowered = trimmed.toLowerCase();
if (lowered.startsWith("imessage:"))
return normalizeIMessageHandle(trimmed.slice(9));
if (lowered.startsWith("sms:"))
return normalizeIMessageHandle(trimmed.slice(4));
if (lowered.startsWith("auto:"))
return normalizeIMessageHandle(trimmed.slice(5));
if (trimmed.includes("@")) return trimmed.toLowerCase();
const normalized = normalizeE164(trimmed);
if (normalized) return normalized;
return trimmed.replace(/\s+/g, "");
}
export function parseIMessageTarget(raw: string): IMessageTarget {
const trimmed = raw.trim();
if (!trimmed) throw new Error("iMessage target is required");
const lower = trimmed.toLowerCase();
for (const prefix of CHAT_ID_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
const chatId = Number.parseInt(value, 10);
if (!Number.isFinite(chatId)) {
throw new Error(`Invalid chat_id: ${value}`);
}
return { kind: "chat_id", chatId };
}
}
for (const prefix of CHAT_GUID_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
if (!value) throw new Error("chat_guid is required");
return { kind: "chat_guid", chatGuid: value };
}
}
for (const prefix of CHAT_IDENTIFIER_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
if (!value) throw new Error("chat_identifier is required");
return { kind: "chat_identifier", chatIdentifier: value };
}
}
if (lower.startsWith("group:")) {
const value = stripPrefix(trimmed, "group:");
const chatId = Number.parseInt(value, 10);
if (Number.isFinite(chatId)) {
return { kind: "chat_id", chatId };
}
if (!value) throw new Error("group target is required");
return { kind: "chat_guid", chatGuid: value };
}
for (const { prefix, service } of SERVICE_PREFIXES) {
if (lower.startsWith(prefix)) {
const to = stripPrefix(trimmed, prefix);
if (!to) throw new Error(`${prefix} target is required`);
return { kind: "handle", to, service };
}
}
return { kind: "handle", to: trimmed, service: "auto" };
}
export function parseIMessageAllowTarget(raw: string): IMessageAllowTarget {
const trimmed = raw.trim();
if (!trimmed) return { kind: "handle", handle: "" };
const lower = trimmed.toLowerCase();
for (const prefix of CHAT_ID_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
const chatId = Number.parseInt(value, 10);
if (Number.isFinite(chatId)) return { kind: "chat_id", chatId };
}
}
for (const prefix of CHAT_GUID_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
if (value) return { kind: "chat_guid", chatGuid: value };
}
}
for (const prefix of CHAT_IDENTIFIER_PREFIXES) {
if (lower.startsWith(prefix)) {
const value = stripPrefix(trimmed, prefix);
if (value) return { kind: "chat_identifier", chatIdentifier: value };
}
}
if (lower.startsWith("group:")) {
const value = stripPrefix(trimmed, "group:");
const chatId = Number.parseInt(value, 10);
if (Number.isFinite(chatId)) return { kind: "chat_id", chatId };
if (value) return { kind: "chat_guid", chatGuid: value };
}
return { kind: "handle", handle: normalizeIMessageHandle(trimmed) };
}
export function isAllowedIMessageSender(params: {
allowFrom: Array<string | number>;
sender: string;
chatId?: number | null;
chatGuid?: string | null;
chatIdentifier?: string | null;
}): boolean {
const allowFrom = params.allowFrom.map((entry) => String(entry).trim());
if (allowFrom.length === 0) return true;
if (allowFrom.includes("*")) return true;
const senderNormalized = normalizeIMessageHandle(params.sender);
const chatId = params.chatId ?? undefined;
const chatGuid = params.chatGuid?.trim();
const chatIdentifier = params.chatIdentifier?.trim();
for (const entry of allowFrom) {
if (!entry) continue;
const parsed = parseIMessageAllowTarget(entry);
if (parsed.kind === "chat_id" && chatId !== undefined) {
if (parsed.chatId === chatId) return true;
} else if (parsed.kind === "chat_guid" && chatGuid) {
if (parsed.chatGuid === chatGuid) return true;
} else if (parsed.kind === "chat_identifier" && chatIdentifier) {
if (parsed.chatIdentifier === chatIdentifier) return true;
} else if (parsed.kind === "handle" && senderNormalized) {
if (parsed.handle === senderNormalized) return true;
}
}
return false;
}
export function formatIMessageChatTarget(chatId?: number | null): string {
if (!chatId || !Number.isFinite(chatId)) return "";
return `chat_id:${chatId}`;
}
+16 -1
View File
@@ -1,7 +1,22 @@
import { describe, expect, test } from "vitest";
import { emitAgentEvent, onAgentEvent } from "./agent-events.js";
import {
clearAgentRunContext,
emitAgentEvent,
getAgentRunContext,
onAgentEvent,
registerAgentRunContext,
resetAgentRunContextForTest,
} from "./agent-events.js";
describe("agent-events sequencing", () => {
test("stores and clears run context", async () => {
resetAgentRunContextForTest();
registerAgentRunContext("run-1", { sessionKey: "main" });
expect(getAgentRunContext("run-1")?.sessionKey).toBe("main");
clearAgentRunContext("run-1");
expect(getAgentRunContext("run-1")).toBeUndefined();
});
test("maintains monotonic seq per runId", async () => {
const seen: Record<string, number[]> = {};
const stop = onAgentEvent((evt) => {
+32
View File
@@ -13,9 +13,41 @@ export type AgentEventPayload = {
data: Record<string, unknown>;
};
export type AgentRunContext = {
sessionKey?: string;
};
// Keep per-run counters so streams stay strictly monotonic per runId.
const seqByRun = new Map<string, number>();
const listeners = new Set<(evt: AgentEventPayload) => void>();
const runContextById = new Map<string, AgentRunContext>();
export function registerAgentRunContext(
runId: string,
context: AgentRunContext,
) {
if (!runId) return;
const existing = runContextById.get(runId);
if (!existing) {
runContextById.set(runId, { ...context });
return;
}
if (context.sessionKey && existing.sessionKey !== context.sessionKey) {
existing.sessionKey = context.sessionKey;
}
}
export function getAgentRunContext(runId: string) {
return runContextById.get(runId);
}
export function clearAgentRunContext(runId: string) {
runContextById.delete(runId);
}
export function resetAgentRunContextForTest() {
runContextById.clear();
}
export function emitAgentEvent(event: Omit<AgentEventPayload, "seq" | "ts">) {
const nextSeq = (seqByRun.get(event.runId) ?? 0) + 1;
+41 -6
View File
@@ -15,6 +15,7 @@ import {
saveSessionStore,
} from "../config/sessions.js";
import { sendMessageDiscord } from "../discord/send.js";
import { sendMessageIMessage } from "../imessage/send.js";
import { formatErrorMessage } from "../infra/errors.js";
import { createSubsystemLogger } from "../logging.js";
import { getQueueSize } from "../process/command-queue.js";
@@ -38,10 +39,11 @@ export type HeartbeatTarget =
| "telegram"
| "discord"
| "signal"
| "imessage"
| "none";
export type HeartbeatDeliveryTarget = {
channel: "whatsapp" | "telegram" | "discord" | "signal" | "none";
channel: "whatsapp" | "telegram" | "discord" | "signal" | "imessage" | "none";
to?: string;
reason?: string;
};
@@ -52,6 +54,7 @@ type HeartbeatDeps = {
sendTelegram?: typeof sendMessageTelegram;
sendDiscord?: typeof sendMessageDiscord;
sendSignal?: typeof sendMessageSignal;
sendIMessage?: typeof sendMessageIMessage;
getQueueSize?: (lane?: string) => number;
nowMs?: () => number;
webAuthExists?: () => Promise<boolean>;
@@ -181,6 +184,7 @@ export function resolveHeartbeatDeliveryTarget(params: {
rawTarget === "telegram" ||
rawTarget === "discord" ||
rawTarget === "signal" ||
rawTarget === "imessage" ||
rawTarget === "none" ||
rawTarget === "last"
? rawTarget
@@ -201,13 +205,20 @@ export function resolveHeartbeatDeliveryTarget(params: {
: undefined;
const lastTo = typeof entry?.lastTo === "string" ? entry.lastTo.trim() : "";
const channel: "whatsapp" | "telegram" | "discord" | "signal" | undefined =
const channel:
| "whatsapp"
| "telegram"
| "discord"
| "signal"
| "imessage"
| undefined =
target === "last"
? lastChannel
: target === "whatsapp" ||
target === "telegram" ||
target === "discord" ||
target === "signal"
target === "signal" ||
target === "imessage"
? target
: undefined;
@@ -255,7 +266,10 @@ function normalizeHeartbeatReply(
payload: ReplyPayload,
responsePrefix?: string,
) {
const stripped = stripHeartbeatToken(payload.text);
const stripped = stripHeartbeatToken(payload.text, {
mode: "heartbeat",
maxAckChars: 30,
});
const hasMedia = Boolean(
payload.mediaUrl || (payload.mediaUrls?.length ?? 0) > 0,
);
@@ -274,14 +288,18 @@ function normalizeHeartbeatReply(
}
async function deliverHeartbeatReply(params: {
channel: "whatsapp" | "telegram" | "discord" | "signal";
channel: "whatsapp" | "telegram" | "discord" | "signal" | "imessage";
to: string;
text: string;
mediaUrls: string[];
deps: Required<
Pick<
HeartbeatDeps,
"sendWhatsApp" | "sendTelegram" | "sendDiscord" | "sendSignal"
| "sendWhatsApp"
| "sendTelegram"
| "sendDiscord"
| "sendSignal"
| "sendIMessage"
>
>;
}) {
@@ -318,6 +336,22 @@ async function deliverHeartbeatReply(params: {
return;
}
if (channel === "imessage") {
if (mediaUrls.length === 0) {
for (const chunk of chunkText(text, 4000)) {
await deps.sendIMessage(to, chunk);
}
return;
}
let first = true;
for (const url of mediaUrls) {
const caption = first ? text : "";
first = false;
await deps.sendIMessage(to, caption, { mediaUrl: url });
}
return;
}
if (channel === "telegram") {
if (mediaUrls.length === 0) {
for (const chunk of chunkText(text, 4000)) {
@@ -464,6 +498,7 @@ export async function runHeartbeatOnce(opts: {
sendTelegram: opts.deps?.sendTelegram ?? sendMessageTelegram,
sendDiscord: opts.deps?.sendDiscord ?? sendMessageDiscord,
sendSignal: opts.deps?.sendSignal ?? sendMessageSignal,
sendIMessage: opts.deps?.sendIMessage ?? sendMessageIMessage,
};
await deliverHeartbeatReply({
channel: delivery.channel,
+12
View File
@@ -60,6 +60,18 @@ export async function buildProviderSummary(
);
}
const imessageEnabled = effective.imessage?.enabled !== false;
if (!imessageEnabled) {
lines.push(chalk.cyan("iMessage: disabled"));
} else {
const imessageConfigured = Boolean(effective.imessage);
lines.push(
imessageConfigured
? chalk.green("iMessage: configured")
: chalk.cyan("iMessage: not configured"),
);
}
const allowFrom = effective.routing?.allowFrom?.length
? effective.routing.allowFrom.map(normalizeE164).filter(Boolean)
: [];
+6 -10
View File
@@ -127,7 +127,7 @@ async function waitForSignalDaemonReady(params: {
params.runtime.error?.(
danger(
`signal: daemon not ready after ${params.timeoutMs}ms (${lastError ?? "unknown error"})`,
`daemon not ready after ${params.timeoutMs}ms (${lastError ?? "unknown error"})`,
),
);
throw new Error(`signal daemon not ready (${lastError ?? "unknown error"})`);
@@ -207,7 +207,7 @@ async function deliverReplies(params: {
});
}
}
runtime.log?.(`signal: delivered reply to ${target}`);
runtime.log?.(`delivered reply to ${target}`);
}
}
@@ -267,13 +267,11 @@ export async function monitorSignalProvider(
try {
payload = JSON.parse(event.data) as SignalReceivePayload;
} catch (err) {
runtime.error?.(`signal: failed to parse event: ${String(err)}`);
runtime.error?.(`failed to parse event: ${String(err)}`);
return;
}
if (payload?.exception?.message) {
runtime.error?.(
`signal: receive exception: ${payload.exception.message}`,
);
runtime.error?.(`receive exception: ${payload.exception.message}`);
}
const envelope = payload?.envelope;
if (!envelope) return;
@@ -317,9 +315,7 @@ export async function monitorSignalProvider(
fetched.contentType ?? firstAttachment.contentType ?? undefined;
}
} catch (err) {
runtime.error?.(
danger(`signal: attachment fetch failed: ${String(err)}`),
);
runtime.error?.(danger(`attachment fetch failed: ${String(err)}`));
}
}
@@ -402,7 +398,7 @@ export async function monitorSignalProvider(
abortSignal: opts.abortSignal,
onEvent: (event) => {
void handleEvent(event).catch((err) => {
runtime.error?.(`signal: event handler failed: ${String(err)}`);
runtime.error?.(`event handler failed: ${String(err)}`);
});
},
});
+1 -1
View File
@@ -199,7 +199,7 @@ describe("telegram inbound media", () => {
expect(replySpy).not.toHaveBeenCalled();
expect(runtimeError).toHaveBeenCalledTimes(1);
const msg = String(runtimeError.mock.calls[0]?.[0] ?? "");
expect(msg).toContain("Telegram handler failed:");
expect(msg).toContain("handler failed:");
expect(msg).toContain("file_path");
fetchSpy.mockRestore();
+2 -2
View File
@@ -201,7 +201,7 @@ export function createTelegramBot(opts: TelegramBotOptions) {
bot,
});
} catch (err) {
runtime.error?.(danger(`Telegram handler failed: ${String(err)}`));
runtime.error?.(danger(`handler failed: ${String(err)}`));
}
});
@@ -225,7 +225,7 @@ async function deliverReplies(params: {
const { replies, chatId, runtime, bot } = params;
for (const reply of replies) {
if (!reply?.text && !reply?.mediaUrl && !(reply?.mediaUrls?.length ?? 0)) {
runtime.error?.(danger("Telegram reply missing text/media"));
runtime.error?.(danger("reply missing text/media"));
continue;
}
const mediaList = reply.mediaUrls?.length
+2 -4
View File
@@ -46,9 +46,7 @@ export async function startTelegramWebhook(opts: {
const handled = handler(req, res);
if (handled && typeof (handled as Promise<void>).catch === "function") {
void (handled as Promise<void>).catch((err) => {
runtime.log?.(
`Telegram webhook handler failed: ${formatErrorMessage(err)}`,
);
runtime.log?.(`webhook handler failed: ${formatErrorMessage(err)}`);
if (!res.headersSent) res.writeHead(500);
res.end();
});
@@ -64,7 +62,7 @@ export async function startTelegramWebhook(opts: {
});
await new Promise<void>((resolve) => server.listen(port, host, resolve));
runtime.log?.(`Telegram webhook listening on ${publicUrl}`);
runtime.log?.(`webhook listening on ${publicUrl}`);
const shutdown = () => {
server.close();
+33 -2
View File
@@ -185,7 +185,6 @@ export { stripHeartbeatToken };
function isSilentReply(payload?: ReplyPayload): boolean {
if (!payload) return false;
const text = payload.text?.trim();
if (text?.includes(HEARTBEAT_TOKEN)) return true;
if (!text || text !== SILENT_REPLY_TOKEN) return false;
if (payload.mediaUrl || payload.mediaUrls?.length) return false;
return true;
@@ -337,7 +336,10 @@ export async function runWebHeartbeatOnce(opts: {
const hasMedia = Boolean(
replyPayload.mediaUrl || (replyPayload.mediaUrls?.length ?? 0) > 0,
);
const stripped = stripHeartbeatToken(replyPayload.text);
const stripped = stripHeartbeatToken(replyPayload.text, {
mode: "heartbeat",
maxAckChars: 30,
});
if (stripped.shouldSkip && !hasMedia) {
// Don't let heartbeats keep sessions alive: restore previous updatedAt so idle expiry still works.
const storePath = resolveStorePath(cfg.session?.store);
@@ -1034,6 +1036,7 @@ export async function monitorWebProvider(
}
const responsePrefix = cfg.messages?.responsePrefix;
let didLogHeartbeatStrip = false;
let didSendReply = false;
let toolSendChain: Promise<void> = Promise.resolve();
const sendToolResult = (payload: ReplyPayload) => {
@@ -1046,6 +1049,20 @@ export async function monitorWebProvider(
}
if (isSilentReply(payload)) return;
const toolPayload: ReplyPayload = { ...payload };
if (toolPayload.text?.includes(HEARTBEAT_TOKEN)) {
const stripped = stripHeartbeatToken(toolPayload.text, {
mode: "message",
});
if (stripped.didStrip && !didLogHeartbeatStrip) {
didLogHeartbeatStrip = true;
logVerbose("Stripped stray HEARTBEAT_OK token from web reply");
}
const hasMedia = Boolean(
toolPayload.mediaUrl || (toolPayload.mediaUrls?.length ?? 0) > 0,
);
if (stripped.shouldSkip && !hasMedia) return;
toolPayload.text = stripped.text;
}
if (
responsePrefix &&
toolPayload.text &&
@@ -1134,6 +1151,20 @@ export async function monitorWebProvider(
await toolSendChain;
for (const replyPayload of sendableReplies) {
if (replyPayload.text?.includes(HEARTBEAT_TOKEN)) {
const stripped = stripHeartbeatToken(replyPayload.text, {
mode: "message",
});
if (stripped.didStrip && !didLogHeartbeatStrip) {
didLogHeartbeatStrip = true;
logVerbose("Stripped stray HEARTBEAT_OK token from web reply");
}
const hasMedia = Boolean(
replyPayload.mediaUrl || (replyPayload.mediaUrls?.length ?? 0) > 0,
);
if (stripped.shouldSkip && !hasMedia) continue;
replyPayload.text = stripped.text;
}
if (
responsePrefix &&
replyPayload.text &&
+77
View File
@@ -75,4 +75,81 @@ describe("web media loading", () => {
fetchMock.mockRestore();
});
it("preserves GIF animation by skipping JPEG optimization", async () => {
// Create a minimal valid GIF (1x1 pixel)
// GIF89a header + minimal image data
const gifBuffer = Buffer.from([
0x47,
0x49,
0x46,
0x38,
0x39,
0x61, // GIF89a
0x01,
0x00,
0x01,
0x00, // 1x1 dimensions
0x00,
0x00,
0x00, // no global color table
0x2c,
0x00,
0x00,
0x00,
0x00, // image descriptor
0x01,
0x00,
0x01,
0x00,
0x00, // 1x1 image
0x02,
0x01,
0x44,
0x00,
0x3b, // minimal LZW data + trailer
]);
const file = path.join(os.tmpdir(), `clawdis-media-${Date.now()}.gif`);
tmpFiles.push(file);
await fs.writeFile(file, gifBuffer);
const result = await loadWebMedia(file, 1024 * 1024);
expect(result.kind).toBe("image");
expect(result.contentType).toBe("image/gif");
// GIF should NOT be converted to JPEG
expect(result.buffer.slice(0, 3).toString()).toBe("GIF");
});
it("preserves GIF from URL without JPEG conversion", async () => {
const gifBytes = new Uint8Array([
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00,
0x00, 0x2c, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x00, 0x00, 0x02,
0x01, 0x44, 0x00, 0x3b,
]);
const fetchMock = vi.spyOn(globalThis, "fetch").mockResolvedValueOnce({
ok: true,
body: true,
arrayBuffer: async () =>
gifBytes.buffer.slice(
gifBytes.byteOffset,
gifBytes.byteOffset + gifBytes.byteLength,
),
headers: { get: () => "image/gif" },
status: 200,
} as Response);
const result = await loadWebMedia(
"https://example.com/animation.gif",
1024 * 1024,
);
expect(result.kind).toBe("image");
expect(result.contentType).toBe("image/gif");
expect(result.buffer.slice(0, 3).toString()).toBe("GIF");
fetchMock.mockRestore();
});
});
+22
View File
@@ -74,6 +74,17 @@ export async function loadWebMedia(
maxBytesForKind(kind),
);
if (kind === "image") {
// Skip optimization for GIFs to preserve animation
if (contentType === "image/gif") {
if (array.length > cap) {
throw new Error(
`GIF exceeds ${(cap / (1024 * 1024)).toFixed(0)}MB limit (got ${(
array.length / (1024 * 1024)
).toFixed(2)}MB)`,
);
}
return { buffer: array, contentType, kind, fileName };
}
return { ...(await optimizeAndClampImage(array, cap)), fileName };
}
if (array.length > cap) {
@@ -105,6 +116,17 @@ export async function loadWebMedia(
maxBytesForKind(kind),
);
if (kind === "image") {
// Skip optimization for GIFs to preserve animation
if (mime === "image/gif") {
if (data.length > cap) {
throw new Error(
`GIF exceeds ${(cap / (1024 * 1024)).toFixed(0)}MB limit (got ${(
data.length / (1024 * 1024)
).toFixed(2)}MB)`,
);
}
return { buffer: data, contentType: mime, kind, fileName };
}
return { ...(await optimizeAndClampImage(data, cap)), fileName };
}
if (data.length > cap) {