Compare commits

..
Author SHA1 Message Date
Peter Steinberger 3ab4c3a3c4 fix: add object capabilities coverage (#1071) (thanks @danielz1z) 2026-01-17 07:31:19 +00:00
danielz1z be6536a635 fix: handle object-format capabilities in normalizeCapabilities
When capabilities is configured as an object (e.g., { inlineButtons: "dm" })
instead of a string array, normalizeCapabilities() would crash with
"capabilities.map is not a function".

This can occur when using the new Telegram inline buttons scoping feature:
  channels.telegram.capabilities.inlineButtons = "dm"

The fix adds an Array.isArray() guard to return undefined for non-array
capabilities, allowing channel-specific handlers (like
resolveTelegramInlineButtonsScope) to process the object format separately.

Fixes crash when using object-format TelegramCapabilitiesConfig.
2026-01-17 07:23:16 +00:00
Peter SteinbergerandAdam Holt c2e10710f4 refactor: share sessions list row type
Co-authored-by: Adam Holt <mail@adamholt.co.nz>
2026-01-17 07:23:08 +00:00
70 changed files with 952 additions and 1777 deletions

No files matched your search

+3 -3
View File
@@ -7,7 +7,7 @@
- Plugins: add Zalo Personal plugin (`@clawdbot/zalouser`) and unify channel directory for plugins. (#1032) — thanks @suminhthanh.
- Models: add Vercel AI Gateway auth choice + onboarding updates. (#1016) — thanks @timolins.
- Sessions: add `session.identityLinks` for cross-platform DM session linking. (#1033) — thanks @thewilloftheshadow.
- Hooks: add hooks system with bundled hooks, CLI tooling, and docs. (#1028) — thanks @ThomsenDrake.
- Hooks: add internal hooks system with bundled hooks, CLI tooling, and docs. (#1028) — thanks @ThomsenDrake.
### Breaking
- **BREAKING:** Channel auth now prefers config over env for Discord/Telegram/Matrix (env is fallback only). (#1040) — thanks @thewilloftheshadow.
@@ -15,7 +15,7 @@
- **BREAKING:** Drop legacy `chatType: "room"` support; use `chatType: "channel"`.
- **BREAKING:** remove legacy provider-specific target resolution fallbacks; target resolution is centralized with plugin hints + directory lookups.
- **BREAKING:** drop legacy target normalization helpers; use outbound target normalization and resolver flows.
- **BREAKING:** `clawdbot hooks` is now `clawdbot webhooks`; hooks live under `clawdbot hooks`.
- **BREAKING:** `clawdbot hooks` is now `clawdbot webhooks`; internal hooks live under `clawdbot hooks`.
- **BREAKING:** `clawdbot plugins install <path>` now copies into `~/.clawdbot/extensions` (use `--link` to keep path-based loading).
### Changes
@@ -44,11 +44,11 @@
### Fixes
- Sub-agents: normalize announce delivery origin + queue bucketing by accountId to keep multi-account routing stable. (#1061, #1058) — thanks @adam91holt.
- Config: handle object-format Telegram capabilities in channel capability resolution. (#1071) — thanks @danielz1z.
- Sessions: include deliveryContext in sessions.list and reuse normalized delivery routing for announce/restart fallbacks. (#1058)
- Sessions: propagate deliveryContext into last-route updates to keep account/channel routing stable. (#1058)
- Gateway: honor explicit delivery targets without implicit accountId fallback; preserve lastAccountId for implicit routing.
- Gateway: avoid reusing last-to/accountId when the requested channel differs; sync deliveryContext with last route fields.
- Skills: narrow skills watcher ignores to `.git` so hidden workspaces still refresh. (#1074) — thanks @roshanasingh4.
- Repo: fix oxlint config filename and move ignore pattern into config. (#1064) — thanks @connorshea.
- Messages: `/stop` now hard-aborts queued followups and sub-agent runs; suppress zero-count stop notes.
- Messages: include sender labels for live group messages across channels, matching queued/history formatting. (#1059)
+11 -11
View File
@@ -1,16 +1,16 @@
---
summary: "CLI reference for `clawdbot hooks` (agent hooks)"
summary: "CLI reference for `clawdbot hooks` (internal hooks)"
read_when:
- You want to manage agent hooks
- You want to install or update hooks
- You want to manage internal agent hooks
- You want to install or update internal hooks
---
# `clawdbot hooks`
Manage agent hooks (event-driven automations for commands like `/new`, `/reset`, etc.).
Manage internal agent hooks (event-driven automations for commands like `/new`, `/reset`, etc.).
Related:
- Hooks: [Hooks](/hooks)
- Internal Hooks: [Internal Agent Hooks](/internal-hooks)
## List All Hooks
@@ -18,7 +18,7 @@ Related:
clawdbot hooks list
```
List all discovered hooks from workspace, managed, and bundled directories.
List all discovered internal hooks from workspace, managed, and bundled directories.
**Options:**
- `--eligible`: Show only eligible hooks (requirements met)
@@ -28,7 +28,7 @@ List all discovered hooks from workspace, managed, and bundled directories.
**Example output:**
```
Hooks (2/2 ready)
Internal Hooks (2/2 ready)
Ready:
📝 command-logger ✓ - Log all command events to a centralized audit file
@@ -82,7 +82,7 @@ Details:
Source: clawdbot-bundled
Path: /path/to/clawdbot/hooks/bundled/session-memory/HOOK.md
Handler: /path/to/clawdbot/hooks/bundled/session-memory/handler.ts
Homepage: https://docs.clawd.bot/hooks#session-memory
Homepage: https://docs.clawd.bot/internal-hooks#session-memory
Events: command:new
Requirements:
@@ -103,7 +103,7 @@ Show summary of hook eligibility status (how many are ready vs. not ready).
**Example output:**
```
Hooks Status
Internal Hooks Status
Total hooks: 2
Ready: 2
@@ -228,7 +228,7 @@ clawdbot hooks enable session-memory
**Output:** `~/clawd/memory/YYYY-MM-DD-slug.md`
**See:** [session-memory documentation](/hooks#session-memory)
**See:** [session-memory documentation](/internal-hooks#session-memory)
### command-logger
@@ -255,4 +255,4 @@ cat ~/.clawdbot/logs/commands.log | jq .
grep '"action":"new"' ~/.clawdbot/logs/commands.log | jq .
```
**See:** [command-logger documentation](/hooks#command-logger)
**See:** [command-logger documentation](/internal-hooks#command-logger)
+21 -21
View File
@@ -1,19 +1,19 @@
---
summary: "Hooks: event-driven automation for commands and lifecycle events"
summary: "Internal agent hooks: event-driven automation for commands and lifecycle events"
read_when:
- You want event-driven automation for /new, /reset, /stop, and agent lifecycle events
- You want to build, install, or debug hooks
- You want to build, install, or debug internal hooks
---
# Hooks
# Internal Agent Hooks
Hooks provide an extensible event-driven system for automating actions in response to agent commands and events. Hooks are automatically discovered from directories and can be managed via CLI commands, similar to how skills work in Clawdbot.
Internal hooks provide an extensible event-driven system for automating actions in response to agent commands and events. Hooks are automatically discovered from directories and can be managed via CLI commands, similar to how skills work in Clawdbot.
## Getting Oriented
Hooks are small scripts that run when something happens. There are two kinds:
- **Hooks** (this page): run inside the Gateway when agent events fire, like `/new`, `/reset`, `/stop`, or lifecycle events.
- **Webhooks**: external HTTP webhooks that let other systems trigger work in Clawdbot. See [Webhook Hooks](/automation/webhook) or use `clawdbot webhooks` for Gmail helper commands.
- **Internal hooks** (this page): run inside the Gateway when agent events fire, like `/new`, `/reset`, `/stop`, or lifecycle events.
- **Web-based hooks**: external HTTP webhooks that let other systems trigger work in Clawdbot. See [Webhook Hooks](/automation/webhook) or use `clawdbot webhooks` for Gmail helper commands.
Common uses:
- Save a memory snapshot when you reset a session
@@ -21,11 +21,11 @@ Common uses:
- Trigger follow-up automation when a session starts or ends
- Write files into the agent workspace or call external APIs when events fire
If you can write a small TypeScript function, you can write a hook. Hooks are discovered automatically, and you enable or disable them via the CLI.
If you can write a small TypeScript function, you can write an internal hook. Hooks are discovered automatically, and you enable or disable them via the CLI.
## Overview
The hooks system allows you to:
The internal hooks system allows you to:
- Save session context to memory when `/new` is issued
- Log all commands for auditing
- Trigger custom automations on agent lifecycle events
@@ -120,7 +120,7 @@ The `HOOK.md` file contains metadata in YAML frontmatter plus Markdown documenta
---
name: my-hook
description: "Short description of what this hook does"
homepage: https://docs.clawd.bot/hooks#my-hook
homepage: https://docs.clawd.bot/internal-hooks#my-hook
metadata: {"clawdbot":{"emoji":"🔗","events":["command:new"],"requires":{"bins":["node"]}}}
---
@@ -162,12 +162,12 @@ The `metadata.clawdbot` object supports:
### Handler Implementation
The `handler.ts` file exports a `HookHandler` function:
The `handler.ts` file exports an `InternalHookHandler` function:
```typescript
import type { HookHandler } from '../../src/hooks/hooks.js';
import type { InternalHookHandler } from '../../src/hooks/internal-hooks.js';
const myHandler: HookHandler = async (event) => {
const myHandler: InternalHookHandler = async (event) => {
// Only trigger on 'new' command
if (event.type !== 'command' || event.action !== 'new') {
return;
@@ -260,9 +260,9 @@ This hook does something useful when you issue `/new`.
### 4. Create handler.ts
```typescript
import type { HookHandler } from '../../src/hooks/hooks.js';
import type { InternalHookHandler } from '../../src/hooks/internal-hooks.js';
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
if (event.type !== 'command' || event.action !== 'new') {
return;
}
@@ -505,12 +505,12 @@ Hooks run during command processing. Keep them lightweight:
```typescript
// ✓ Good - async work, returns immediately
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
void processInBackground(event); // Fire and forget
};
// ✗ Bad - blocks command processing
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
await slowDatabaseQuery(event);
await evenSlowerAPICall(event);
};
@@ -521,7 +521,7 @@ const handler: HookHandler = async (event) => {
Always wrap risky operations:
```typescript
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
try {
await riskyOperation(event);
} catch (err) {
@@ -536,7 +536,7 @@ const handler: HookHandler = async (event) => {
Return early if the event isn't relevant:
```typescript
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
// Only handle 'new' commands
if (event.type !== 'command' || event.action !== 'new') {
return;
@@ -584,7 +584,7 @@ clawdbot hooks list --verbose
In your handler, log when it's called:
```typescript
const handler: HookHandler = async (event) => {
const handler: InternalHookHandler = async (event) => {
console.log('[my-handler] Triggered:', event.type, event.action);
// Your logic
};
@@ -620,11 +620,11 @@ Test your handlers in isolation:
```typescript
import { test } from 'vitest';
import { createHookEvent } from './src/hooks/hooks.js';
import { createInternalHookEvent } from './src/hooks/internal-hooks.js';
import myHandler from './hooks/my-hook/handler.js';
test('my handler works', async () => {
const event = createHookEvent('command', 'new', 'test-session', {
const event = createInternalHookEvent('command', 'new', 'test-session', {
foo: 'bar'
});
-9
View File
@@ -256,15 +256,6 @@ When `mode: "all"`, outputs are labeled `[Image 1/2]`, `[Audio 2/2]`, etc.
}
```
## Status output
When media understanding runs, `/status` includes a short summary line:
```
📎 Media: image ok (openai/gpt-5.2) · audio skipped (maxBytes)
```
This shows per‑capability outcomes and the chosen provider/model when applicable.
## Notes
- Understanding is **best‑effort**. Errors do not block replies.
- Attachments are still passed to models even when understanding is disabled.
+6 -8
View File
@@ -164,15 +164,13 @@ export const matrixPlugin: ChannelPlugin<ResolvedMatrixAccount> = {
},
messaging: {
normalizeTarget: normalizeMatrixMessagingTarget,
targetResolver: {
looksLikeId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(matrix:)?[!#@]/i.test(trimmed)) return true;
return trimmed.includes(":");
},
hint: "<room|alias|user>",
looksLikeTargetId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(matrix:)?[!#@]/i.test(trimmed)) return true;
return trimmed.includes(":");
},
targetHint: "<room|alias|user>",
},
directory: {
self: async () => null,
+6 -8
View File
@@ -133,15 +133,13 @@ export const msteamsPlugin: ChannelPlugin<ResolvedMSTeamsAccount> = {
},
messaging: {
normalizeTarget: normalizeMSTeamsMessagingTarget,
targetResolver: {
looksLikeId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(conversation:|user:)/i.test(trimmed)) return true;
return trimmed.includes("@thread");
},
hint: "<conversationId|user:ID|conversation:ID>",
looksLikeTargetId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(conversation:|user:)/i.test(trimmed)) return true;
return trimmed.includes("@thread");
},
targetHint: "<conversationId|user:ID|conversation:ID>",
},
directory: {
self: async () => null,
+5 -7
View File
@@ -150,14 +150,12 @@ export const zaloPlugin: ChannelPlugin<ResolvedZaloAccount> = {
actions: zaloMessageActions,
messaging: {
normalizeTarget: normalizeZaloMessagingTarget,
targetResolver: {
looksLikeId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
return /^\d{3,}$/.test(trimmed);
},
hint: "<chatId>",
looksLikeTargetId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
return /^\d{3,}$/.test(trimmed);
},
targetHint: "<chatId>",
},
directory: {
self: async () => null,
+5 -7
View File
@@ -218,14 +218,12 @@ export const zalouserPlugin: ChannelPlugin<ResolvedZalouserAccount> = {
if (!trimmed) return undefined;
return trimmed.replace(/^(zalouser|zlu):/i, "");
},
targetResolver: {
looksLikeId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
return /^\d{3,}$/.test(trimmed);
},
hint: "<threadId>",
looksLikeTargetId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
return /^\d{3,}$/.test(trimmed);
},
targetHint: "<threadId>",
},
directory: {
self: async ({ cfg, accountId, runtime }) => {
@@ -50,7 +50,7 @@ test("process submit sends CR for pty sessions", async () => {
const processTool = createProcessTool();
const result = await execTool.execute("toolcall", {
command:
"node -e \"process.stdin.on('data', d => { if (d.includes(10) || d.includes(13)) { process.stdout.write('submitted'); process.exit(0); } });\"",
"node -e \"process.stdin.on('data', d => { if (d.includes(13)) { process.stdout.write('submitted'); process.exit(0); } });\"",
pty: true,
background: true,
});
+1 -1
View File
@@ -1,4 +1,4 @@
const ESC = "\u001b";
const ESC = String.fromCharCode(0x1b);
const DSR_PATTERN = new RegExp(`${ESC}\\[\\??6n`, "g");
export function stripDsrRequests(input: string): { cleaned: string; requests: number } {
-31
View File
@@ -1,31 +0,0 @@
import { describe, expect, it, vi } from "vitest";
const watchMock = vi.fn(() => ({
on: vi.fn(),
close: vi.fn(async () => undefined),
}));
vi.mock("chokidar", () => {
return {
default: { watch: watchMock },
};
});
describe("ensureSkillsWatcher", () => {
it("ignores node_modules, dist, and .git by default", async () => {
const mod = await import("./refresh.js");
mod.ensureSkillsWatcher({ workspaceDir: "/tmp/workspace" });
expect(watchMock).toHaveBeenCalledTimes(1);
const opts = watchMock.mock.calls[0]?.[1] as { ignored?: unknown };
expect(Array.isArray(opts.ignored)).toBe(true);
const ignored = opts.ignored as RegExp[];
expect(ignored.some((re) => re.test("/tmp/workspace/skills/node_modules/pkg/index.js"))).toBe(
true,
);
expect(ignored.some((re) => re.test("/tmp/workspace/skills/dist/index.js"))).toBe(true);
expect(ignored.some((re) => re.test("/tmp/workspace/skills/.git/config"))).toBe(true);
expect(ignored.some((re) => re.test("/tmp/.hidden/skills/index.md"))).toBe(false);
});
});
-7
View File
@@ -125,13 +125,6 @@ export function ensureSkillsWatcher(params: { workspaceDir: string; config?: Cla
stabilityThreshold: debounceMs,
pollInterval: 100,
},
// Avoid FD exhaustion on macOS when a workspace contains huge trees.
// This watcher only needs to react to skill changes.
ignored: [
/(^|[\\/])\.git([\\/]|$)/,
/(^|[\\/])node_modules([\\/]|$)/,
/(^|[\\/])dist([\\/]|$)/,
],
});
const state: SkillsWatchState = { watcher, pathsKey, debounceMs };
-65
View File
@@ -41,69 +41,4 @@ describe("buildInboundMediaNote", () => {
});
expect(note).toBe("[media attached: /tmp/b.png | https://example.com/b.png]");
});
it("only suppresses attachments when media understanding succeeded", () => {
const note = buildInboundMediaNote({
MediaPaths: ["/tmp/a.png", "/tmp/b.png"],
MediaUrls: ["https://example.com/a.png", "https://example.com/b.png"],
MediaUnderstandingDecisions: [
{
capability: "image",
outcome: "skipped",
attachments: [
{
attachmentIndex: 0,
attempts: [
{
type: "provider",
outcome: "skipped",
reason: "maxBytes: too large",
},
],
},
],
},
],
});
expect(note).toBe(
[
"[media attached: 2 files]",
"[media attached 1/2: /tmp/a.png | https://example.com/a.png]",
"[media attached 2/2: /tmp/b.png | https://example.com/b.png]",
].join("\n"),
);
});
it("suppresses attachments when media understanding succeeds via decisions", () => {
const note = buildInboundMediaNote({
MediaPaths: ["/tmp/a.png", "/tmp/b.png"],
MediaUrls: ["https://example.com/a.png", "https://example.com/b.png"],
MediaUnderstandingDecisions: [
{
capability: "image",
outcome: "success",
attachments: [
{
attachmentIndex: 0,
attempts: [
{
type: "provider",
outcome: "success",
provider: "openai",
model: "gpt-5.2",
},
],
chosen: {
type: "provider",
outcome: "success",
provider: "openai",
model: "gpt-5.2",
},
},
],
},
],
});
expect(note).toBe("[media attached: /tmp/b.png | https://example.com/b.png]");
});
});
+5 -16
View File
@@ -19,22 +19,11 @@ function formatMediaAttachedLine(params: {
export function buildInboundMediaNote(ctx: MsgContext): string | undefined {
// Attachment indices follow MediaPaths/MediaUrls ordering as supplied by the channel.
const suppressed = new Set<number>();
if (Array.isArray(ctx.MediaUnderstanding)) {
for (const output of ctx.MediaUnderstanding) {
suppressed.add(output.attachmentIndex);
}
}
if (Array.isArray(ctx.MediaUnderstandingDecisions)) {
for (const decision of ctx.MediaUnderstandingDecisions) {
if (decision.outcome !== "success") continue;
for (const attachment of decision.attachments) {
if (attachment.chosen?.outcome === "success") {
suppressed.add(attachment.attachmentIndex);
}
}
}
}
const suppressed = new Set(
Array.isArray(ctx.MediaUnderstanding)
? ctx.MediaUnderstanding.map((output) => output.attachmentIndex)
: [],
);
const pathsFromArray = Array.isArray(ctx.MediaPaths) ? ctx.MediaPaths : undefined;
const paths =
pathsFromArray && pathsFromArray.length > 0
-1
View File
@@ -68,7 +68,6 @@ export const handleStatusCommand: CommandHandler = async (params, allowTextComma
resolveDefaultThinkingLevel: params.resolveDefaultThinkingLevel,
isGroup: params.isGroup,
defaultGroupActivation: params.defaultGroupActivation,
mediaDecisions: params.ctx.MediaUnderstandingDecisions,
});
return { shouldContinue: false, reply };
};
-3
View File
@@ -24,7 +24,6 @@ import type { ElevatedLevel, ReasoningLevel, ThinkLevel, VerboseLevel } from "..
import type { ReplyPayload } from "../types.js";
import type { CommandContext } from "./commands-types.js";
import { getFollowupQueueDepth, resolveQueueSettings } from "./queue.js";
import type { MediaUnderstandingDecision } from "../../media-understanding/types.js";
function formatApiKeySnippet(apiKey: string): string {
const compact = apiKey.replace(/\s+/g, "");
@@ -106,7 +105,6 @@ export async function buildStatusReply(params: {
resolveDefaultThinkingLevel: () => Promise<ThinkLevel | undefined>;
isGroup: boolean;
defaultGroupActivation: () => "always" | "mention";
mediaDecisions?: MediaUnderstandingDecision[];
}): Promise<ReplyPayload | undefined> {
const {
cfg,
@@ -202,7 +200,6 @@ export async function buildStatusReply(params: {
dropPolicy: queueSettings.dropPolicy,
showDetails: queueOverrides,
},
mediaDecisions: params.mediaDecisions,
includeTranscriptUsage: false,
});
+1 -1
View File
@@ -144,7 +144,7 @@ describe("handleCommands identity", () => {
});
});
describe("handleCommands hooks", () => {
describe("handleCommands internal hooks", () => {
it("triggers hooks for /new with arguments", async () => {
const cfg = {
commands: { text: true },
@@ -188,7 +188,6 @@ export async function applyInlineDirectiveOverrides(params: {
resolveDefaultThinkingLevel: async () => resolvedDefaultThinkLevel,
isGroup,
defaultGroupActivation: defaultActivation,
mediaDecisions: ctx.MediaUnderstandingDecisions,
});
}
typing.cleanup();
@@ -185,7 +185,6 @@ export async function handleInlineActions(params: {
resolveDefaultThinkingLevel,
isGroup,
defaultGroupActivation: defaultActivation,
mediaDecisions: ctx.MediaUnderstandingDecisions,
});
await sendInlineReply(inlineStatusReply);
directives = { ...directives, hasStatusDirective: false };
-53
View File
@@ -90,59 +90,6 @@ describe("buildStatusMessage", () => {
expect(text).toContain("elevated");
});
it("includes media understanding decisions when present", () => {
const text = buildStatusMessage({
agent: { model: "anthropic/claude-opus-4-5" },
sessionEntry: { sessionId: "media", updatedAt: 0 },
sessionKey: "agent:main:main",
queue: { mode: "none" },
mediaDecisions: [
{
capability: "image",
outcome: "success",
attachments: [
{
attachmentIndex: 0,
attempts: [
{
type: "provider",
outcome: "success",
provider: "openai",
model: "gpt-5.2",
},
],
chosen: {
type: "provider",
outcome: "success",
provider: "openai",
model: "gpt-5.2",
},
},
],
},
{
capability: "audio",
outcome: "skipped",
attachments: [
{
attachmentIndex: 1,
attempts: [
{
type: "provider",
outcome: "skipped",
reason: "maxBytes: too large",
},
],
},
],
},
],
});
const normalized = normalizeTestText(text);
expect(normalized).toContain("Media: image ok (openai/gpt-5.2) · audio skipped (maxBytes)");
});
it("does not show elevated label when session explicitly disables it", () => {
const text = buildStatusMessage({
agent: { model: "anthropic/claude-opus-4-5", elevatedDefault: "on" },
-40
View File
@@ -24,7 +24,6 @@ import { VERSION } from "../version.js";
import { listChatCommands, listChatCommandsForConfig } from "./commands-registry.js";
import type { SkillCommandSpec } from "../agents/skills.js";
import type { ElevatedLevel, ReasoningLevel, ThinkLevel, VerboseLevel } from "./thinking.js";
import type { MediaUnderstandingDecision } from "../media-understanding/types.js";
type AgentConfig = Partial<NonNullable<NonNullable<ClawdbotConfig["agents"]>["defaults"]>>;
@@ -53,7 +52,6 @@ type StatusArgs = {
modelAuth?: string;
usageLine?: string;
queue?: QueueStatus;
mediaDecisions?: MediaUnderstandingDecision[];
includeTranscriptUsage?: boolean;
now?: number;
};
@@ -169,42 +167,6 @@ const formatUsagePair = (input?: number | null, output?: number | null) => {
return `🧮 Tokens: ${inputLabel} in / ${outputLabel} out`;
};
const formatMediaUnderstandingLine = (decisions?: MediaUnderstandingDecision[]) => {
if (!decisions || decisions.length === 0) return null;
const parts = decisions
.map((decision) => {
const count = decision.attachments.length;
const countLabel = count > 1 ? ` x${count}` : "";
if (decision.outcome === "success") {
const chosen = decision.attachments.find((entry) => entry.chosen)?.chosen;
const provider = chosen?.provider?.trim();
const model = chosen?.model?.trim();
const modelLabel = provider ? (model ? `${provider}/${model}` : provider) : null;
return `${decision.capability}${countLabel} ok${modelLabel ? ` (${modelLabel})` : ""}`;
}
if (decision.outcome === "no-attachment") {
return `${decision.capability} none`;
}
if (decision.outcome === "disabled") {
return `${decision.capability} off`;
}
if (decision.outcome === "scope-deny") {
return `${decision.capability} denied`;
}
if (decision.outcome === "skipped") {
const reason = decision.attachments
.flatMap((entry) => entry.attempts.map((attempt) => attempt.reason).filter(Boolean))
.find(Boolean);
const shortReason = reason ? reason.split(":")[0]?.trim() : undefined;
return `${decision.capability} skipped${shortReason ? ` (${shortReason})` : ""}`;
}
return null;
})
.filter(Boolean);
if (parts.length === 0) return null;
return `📎 Media: ${parts.join(" · ")}`;
};
export function buildStatusMessage(args: StatusArgs): string {
const now = args.now ?? Date.now();
const entry = args.sessionEntry;
@@ -358,14 +320,12 @@ export function buildStatusMessage(args: StatusArgs): string {
const costLine = costLabel ? `💵 Cost: ${costLabel}` : null;
const usageCostLine =
usagePair && costLine ? `${usagePair} · ${costLine}` : (usagePair ?? costLine);
const mediaLine = formatMediaUnderstandingLine(args.mediaDecisions);
return [
versionLine,
modelLine,
usageCostLine,
`📚 ${contextLine}`,
mediaLine,
args.usageLine,
`🧵 ${sessionLine}`,
`⚙️ ${optionsLine}`,
+1 -1
View File
@@ -96,7 +96,7 @@ export type MsgContext = {
*/
OriginatingTo?: string;
/**
* Messages from hooks to be included in the response.
* Messages from internal hooks to be included in the response.
* Used for hook confirmation messages like "Session context saved to memory".
*/
HookMessages?: string[];
+24 -18
View File
@@ -1,17 +1,9 @@
import { describe, expect, it } from "vitest";
import {
listDiscordDirectoryGroupsFromConfig,
listDiscordDirectoryPeersFromConfig,
listSlackDirectoryGroupsFromConfig,
listSlackDirectoryPeersFromConfig,
listTelegramDirectoryGroupsFromConfig,
listTelegramDirectoryPeersFromConfig,
listWhatsAppDirectoryGroupsFromConfig,
listWhatsAppDirectoryPeersFromConfig,
} from "./directory-config.js";
describe("directory (config-backed)", () => {
it("lists Slack peers/groups from config", async () => {
const { slackPlugin } = await import("./slack.js");
const runtime = { log: () => {}, error: () => {}, exit: () => {} } as any;
const cfg = {
channels: {
slack: {
@@ -24,11 +16,12 @@ describe("directory (config-backed)", () => {
},
} as any;
const peers = await listSlackDirectoryPeersFromConfig({
const peers = await slackPlugin.directory?.listPeers?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(peers?.map((e) => e.id).sort()).toEqual([
"user:u123",
@@ -37,16 +30,19 @@ describe("directory (config-backed)", () => {
"user:u999",
]);
const groups = await listSlackDirectoryGroupsFromConfig({
const groups = await slackPlugin.directory?.listGroups?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(groups?.map((e) => e.id)).toEqual(["channel:c111"]);
});
it("lists Discord peers/groups from config (numeric ids only)", async () => {
const { discordPlugin } = await import("./discord.js");
const runtime = { log: () => {}, error: () => {}, exit: () => {} } as any;
const cfg = {
channels: {
discord: {
@@ -67,24 +63,28 @@ describe("directory (config-backed)", () => {
},
} as any;
const peers = await listDiscordDirectoryPeersFromConfig({
const peers = await discordPlugin.directory?.listPeers?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(peers?.map((e) => e.id).sort()).toEqual(["user:111", "user:12345", "user:222"]);
const groups = await listDiscordDirectoryGroupsFromConfig({
const groups = await discordPlugin.directory?.listGroups?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(groups?.map((e) => e.id).sort()).toEqual(["channel:555", "channel:666"]);
});
it("lists Telegram peers/groups from config", async () => {
const { telegramPlugin } = await import("./telegram.js");
const runtime = { log: () => {}, error: () => {}, exit: () => {} } as any;
const cfg = {
channels: {
telegram: {
@@ -96,24 +96,28 @@ describe("directory (config-backed)", () => {
},
} as any;
const peers = await listTelegramDirectoryPeersFromConfig({
const peers = await telegramPlugin.directory?.listPeers?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(peers?.map((e) => e.id).sort()).toEqual(["123", "456", "@alice", "@bob"]);
const groups = await listTelegramDirectoryGroupsFromConfig({
const groups = await telegramPlugin.directory?.listGroups?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(groups?.map((e) => e.id)).toEqual(["-1001"]);
});
it("lists WhatsApp peers/groups from config", async () => {
const { whatsappPlugin } = await import("./whatsapp.js");
const runtime = { log: () => {}, error: () => {}, exit: () => {} } as any;
const cfg = {
channels: {
whatsapp: {
@@ -123,19 +127,21 @@ describe("directory (config-backed)", () => {
},
} as any;
const peers = await listWhatsAppDirectoryPeersFromConfig({
const peers = await whatsappPlugin.directory?.listPeers?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(peers?.map((e) => e.id)).toEqual(["+15550000000"]);
const groups = await listWhatsAppDirectoryGroupsFromConfig({
const groups = await whatsappPlugin.directory?.listGroups?.({
cfg,
accountId: "default",
query: null,
limit: null,
runtime,
});
expect(groups?.map((e) => e.id)).toEqual(["999@g.us"]);
});
-214
View File
@@ -1,214 +0,0 @@
import type { ClawdbotConfig } from "../../config/types.js";
import type { ChannelDirectoryEntry } from "./types.js";
import { resolveSlackAccount } from "../../slack/accounts.js";
import { resolveDiscordAccount } from "../../discord/accounts.js";
import { resolveTelegramAccount } from "../../telegram/accounts.js";
import { resolveWhatsAppAccount } from "../../web/accounts.js";
import { normalizeSlackMessagingTarget } from "./normalize-target.js";
import { isWhatsAppGroupJid, normalizeWhatsAppTarget } from "../../whatsapp/normalize.js";
export type DirectoryConfigParams = {
cfg: ClawdbotConfig;
accountId?: string | null;
query?: string | null;
limit?: number | null;
};
export async function listSlackDirectoryPeersFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveSlackAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const entry of account.dm?.allowFrom ?? []) {
const raw = String(entry).trim();
if (!raw || raw === "*") continue;
ids.add(raw);
}
for (const id of Object.keys(account.config.dms ?? {})) {
const trimmed = id.trim();
if (trimmed) ids.add(trimmed);
}
for (const channel of Object.values(account.config.channels ?? {})) {
for (const user of channel.users ?? []) {
const raw = String(user).trim();
if (raw) ids.add(raw);
}
}
return Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<@([A-Z0-9]+)>$/i);
const normalizedUserId = (mention?.[1] ?? raw).replace(/^(slack|user):/i, "").trim();
if (!normalizedUserId) return null;
const target = `user:${normalizedUserId}`;
return normalizeSlackMessagingTarget(target) ?? target.toLowerCase();
})
.filter((id): id is string => Boolean(id))
.filter((id) => id.startsWith("user:"))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
}
export async function listSlackDirectoryGroupsFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveSlackAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
return Object.keys(account.config.channels ?? {})
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => normalizeSlackMessagingTarget(raw) ?? raw.toLowerCase())
.filter((id) => id.startsWith("channel:"))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
}
export async function listDiscordDirectoryPeersFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveDiscordAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const entry of account.config.dm?.allowFrom ?? []) {
const raw = String(entry).trim();
if (!raw || raw === "*") continue;
ids.add(raw);
}
for (const id of Object.keys(account.config.dms ?? {})) {
const trimmed = id.trim();
if (trimmed) ids.add(trimmed);
}
for (const guild of Object.values(account.config.guilds ?? {})) {
for (const entry of guild.users ?? []) {
const raw = String(entry).trim();
if (raw) ids.add(raw);
}
for (const channel of Object.values(guild.channels ?? {})) {
for (const user of channel.users ?? []) {
const raw = String(user).trim();
if (raw) ids.add(raw);
}
}
}
return Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<@!?(\d+)>$/);
const cleaned = (mention?.[1] ?? raw).replace(/^(discord|user):/i, "").trim();
if (!/^\d+$/.test(cleaned)) return null;
return `user:${cleaned}`;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
}
export async function listDiscordDirectoryGroupsFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveDiscordAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const guild of Object.values(account.config.guilds ?? {})) {
for (const channelId of Object.keys(guild.channels ?? {})) {
const trimmed = channelId.trim();
if (trimmed) ids.add(trimmed);
}
}
return Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<#(\d+)>$/);
const cleaned = (mention?.[1] ?? raw).replace(/^(discord|channel|group):/i, "").trim();
if (!/^\d+$/.test(cleaned)) return null;
return `channel:${cleaned}`;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
}
export async function listTelegramDirectoryPeersFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveTelegramAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
const raw = [
...(account.config.allowFrom ?? []).map((entry) => String(entry)),
...Object.keys(account.config.dms ?? {}),
];
return Array.from(
new Set(
raw
.map((entry) => entry.trim())
.filter(Boolean)
.map((entry) => entry.replace(/^(telegram|tg):/i, "")),
),
)
.map((entry) => {
const trimmed = entry.trim();
if (!trimmed) return null;
if (/^-?\d+$/.test(trimmed)) return trimmed;
const withAt = trimmed.startsWith("@") ? trimmed : `@${trimmed}`;
return withAt;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
}
export async function listTelegramDirectoryGroupsFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveTelegramAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
return Object.keys(account.config.groups ?? {})
.map((id) => id.trim())
.filter((id) => Boolean(id) && id !== "*")
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
}
export async function listWhatsAppDirectoryPeersFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveWhatsAppAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
return (account.allowFrom ?? [])
.map((entry) => String(entry).trim())
.filter((entry) => Boolean(entry) && entry !== "*")
.map((entry) => normalizeWhatsAppTarget(entry) ?? "")
.filter(Boolean)
.filter((id) => !isWhatsAppGroupJid(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
}
export async function listWhatsAppDirectoryGroupsFromConfig(
params: DirectoryConfigParams,
): Promise<ChannelDirectoryEntry[]> {
const account = resolveWhatsAppAccount({ cfg: params.cfg, accountId: params.accountId });
const q = params.query?.trim().toLowerCase() || "";
return Object.keys(account.groups ?? {})
.map((id) => id.trim())
.filter((id) => Boolean(id) && id !== "*")
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, params.limit && params.limit > 0 ? params.limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
}
+70 -10
View File
@@ -38,10 +38,6 @@ import {
} from "./setup-helpers.js";
import { collectDiscordStatusIssues } from "./status-issues/discord.js";
import type { ChannelPlugin } from "./types.js";
import {
listDiscordDirectoryGroupsFromConfig,
listDiscordDirectoryPeersFromConfig,
} from "./directory-config.js";
const meta = getChatChannelMeta("discord");
@@ -156,15 +152,79 @@ export const discordPlugin: ChannelPlugin<ResolvedDiscordAccount> = {
},
messaging: {
normalizeTarget: normalizeDiscordMessagingTarget,
targetResolver: {
looksLikeId: looksLikeDiscordTargetId,
hint: "<channelId|user:ID|channel:ID>",
},
looksLikeTargetId: looksLikeDiscordTargetId,
targetHint: "<channelId|user:ID|channel:ID>",
},
directory: {
self: async () => null,
listPeers: async (params) => listDiscordDirectoryPeersFromConfig(params),
listGroups: async (params) => listDiscordDirectoryGroupsFromConfig(params),
listPeers: async ({ cfg, accountId, query, limit }) => {
const account = resolveDiscordAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const entry of account.config.dm?.allowFrom ?? []) {
const raw = String(entry).trim();
if (!raw || raw === "*") continue;
ids.add(raw);
}
for (const id of Object.keys(account.config.dms ?? {})) {
const trimmed = id.trim();
if (trimmed) ids.add(trimmed);
}
for (const guild of Object.values(account.config.guilds ?? {})) {
for (const entry of guild.users ?? []) {
const raw = String(entry).trim();
if (raw) ids.add(raw);
}
for (const channel of Object.values(guild.channels ?? {})) {
for (const user of channel.users ?? []) {
const raw = String(user).trim();
if (raw) ids.add(raw);
}
}
}
const peers = Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<@!?(\d+)>$/);
const cleaned = (mention?.[1] ?? raw).replace(/^(discord|user):/i, "").trim();
if (!/^\d+$/.test(cleaned)) return null;
return `user:${cleaned}`;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
return peers;
},
listGroups: async ({ cfg, accountId, query, limit }) => {
const account = resolveDiscordAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const guild of Object.values(account.config.guilds ?? {})) {
for (const channelId of Object.keys(guild.channels ?? {})) {
const trimmed = channelId.trim();
if (trimmed) ids.add(trimmed);
}
}
const groups = Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<#(\d+)>$/);
const cleaned = (mention?.[1] ?? raw).replace(/^(discord|channel|group):/i, "").trim();
if (!/^\d+$/.test(cleaned)) return null;
return `channel:${cleaned}`;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
return groups;
},
listGroupsLive: async ({ cfg, accountId, query, limit }) => {
const account = resolveDiscordAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
+7 -9
View File
@@ -107,16 +107,14 @@ export const imessagePlugin: ChannelPlugin<ResolvedIMessageAccount> = {
resolveRequireMention: resolveIMessageGroupRequireMention,
},
messaging: {
targetResolver: {
looksLikeId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(imessage:|chat_id:)/i.test(trimmed)) return true;
if (trimmed.includes("@")) return true;
return /^\+?\d{3,}$/.test(trimmed);
},
hint: "<handle|chat_id:ID>",
looksLikeTargetId: (raw) => {
const trimmed = raw.trim();
if (!trimmed) return false;
if (/^(imessage:|chat_id:)/i.test(trimmed)) return true;
if (trimmed.includes("@")) return true;
return /^\+?\d{3,}$/.test(trimmed);
},
targetHint: "<handle|chat_id:ID>",
},
setup: {
resolveAccountId: ({ accountId }) => normalizeAccountId(accountId),
-10
View File
@@ -74,14 +74,4 @@ export function normalizeChannelId(raw?: string | null): ChannelId | null {
}
export { discordPlugin, imessagePlugin, signalPlugin, slackPlugin, telegramPlugin, whatsappPlugin };
export {
listDiscordDirectoryGroupsFromConfig,
listDiscordDirectoryPeersFromConfig,
listSlackDirectoryGroupsFromConfig,
listSlackDirectoryPeersFromConfig,
listTelegramDirectoryGroupsFromConfig,
listTelegramDirectoryPeersFromConfig,
listWhatsAppDirectoryGroupsFromConfig,
listWhatsAppDirectoryPeersFromConfig,
} from "./directory-config.js";
export type { ChannelId, ChannelPlugin } from "./types.js";
+2 -4
View File
@@ -118,10 +118,8 @@ export const signalPlugin: ChannelPlugin<ResolvedSignalAccount> = {
},
messaging: {
normalizeTarget: normalizeSignalMessagingTarget,
targetResolver: {
looksLikeId: looksLikeSignalTargetId,
hint: "<E.164|group:ID|signal:group:ID|signal:+E.164>",
},
looksLikeTargetId: looksLikeSignalTargetId,
targetHint: "<E.164|group:ID|signal:group:ID|signal:+E.164>",
},
setup: {
resolveAccountId: ({ accountId }) => normalizeAccountId(accountId),
+53 -10
View File
@@ -28,10 +28,6 @@ import {
migrateBaseNameToDefaultAccount,
} from "./setup-helpers.js";
import type { ChannelMessageActionName, ChannelPlugin } from "./types.js";
import {
listSlackDirectoryGroupsFromConfig,
listSlackDirectoryPeersFromConfig,
} from "./directory-config.js";
const meta = getChatChannelMeta("slack");
@@ -181,15 +177,62 @@ export const slackPlugin: ChannelPlugin<ResolvedSlackAccount> = {
},
messaging: {
normalizeTarget: normalizeSlackMessagingTarget,
targetResolver: {
looksLikeId: looksLikeSlackTargetId,
hint: "<channelId|user:ID|channel:ID>",
},
looksLikeTargetId: looksLikeSlackTargetId,
targetHint: "<channelId|user:ID|channel:ID>",
},
directory: {
self: async () => null,
listPeers: async (params) => listSlackDirectoryPeersFromConfig(params),
listGroups: async (params) => listSlackDirectoryGroupsFromConfig(params),
listPeers: async ({ cfg, accountId, query, limit }) => {
const account = resolveSlackAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const ids = new Set<string>();
for (const entry of account.dm?.allowFrom ?? []) {
const raw = String(entry).trim();
if (!raw || raw === "*") continue;
ids.add(raw);
}
for (const id of Object.keys(account.config.dms ?? {})) {
const trimmed = id.trim();
if (trimmed) ids.add(trimmed);
}
for (const channel of Object.values(account.config.channels ?? {})) {
for (const user of channel.users ?? []) {
const raw = String(user).trim();
if (raw) ids.add(raw);
}
}
const peers = Array.from(ids)
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => {
const mention = raw.match(/^<@([A-Z0-9]+)>$/i);
const normalizedUserId = (mention?.[1] ?? raw).replace(/^(slack|user):/i, "").trim();
if (!normalizedUserId) return null;
const target = `user:${normalizedUserId}`;
return normalizeSlackMessagingTarget(target) ?? target.toLowerCase();
})
.filter((id): id is string => Boolean(id))
.filter((id) => id.startsWith("user:"))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
return peers;
},
listGroups: async ({ cfg, accountId, query, limit }) => {
const account = resolveSlackAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const groups = Object.keys(account.config.channels ?? {})
.map((raw) => raw.trim())
.filter(Boolean)
.map((raw) => normalizeSlackMessagingTarget(raw) ?? raw.toLowerCase())
.filter((id) => id.startsWith("channel:"))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
return groups;
},
},
actions: {
listActions: ({ cfg }) => {
+41 -10
View File
@@ -38,10 +38,6 @@ import {
} from "./setup-helpers.js";
import { collectTelegramStatusIssues } from "./status-issues/telegram.js";
import type { ChannelPlugin } from "./types.js";
import {
listTelegramDirectoryGroupsFromConfig,
listTelegramDirectoryPeersFromConfig,
} from "./directory-config.js";
const meta = getChatChannelMeta("telegram");
@@ -164,15 +160,50 @@ export const telegramPlugin: ChannelPlugin<ResolvedTelegramAccount> = {
},
messaging: {
normalizeTarget: normalizeTelegramMessagingTarget,
targetResolver: {
looksLikeId: looksLikeTelegramTargetId,
hint: "<chatId>",
},
looksLikeTargetId: looksLikeTelegramTargetId,
targetHint: "<chatId>",
},
directory: {
self: async () => null,
listPeers: async (params) => listTelegramDirectoryPeersFromConfig(params),
listGroups: async (params) => listTelegramDirectoryGroupsFromConfig(params),
listPeers: async ({ cfg, accountId, query, limit }) => {
const account = resolveTelegramAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const raw = [
...(account.config.allowFrom ?? []).map((entry) => String(entry)),
...Object.keys(account.config.dms ?? {}),
];
const peers = Array.from(
new Set(
raw
.map((entry) => entry.trim())
.filter(Boolean)
.map((entry) => entry.replace(/^(telegram|tg):/i, "")),
),
)
.map((entry) => {
const trimmed = entry.trim();
if (!trimmed) return null;
if (/^-?\d+$/.test(trimmed)) return trimmed;
const withAt = trimmed.startsWith("@") ? trimmed : `@${trimmed}`;
return withAt;
})
.filter((id): id is string => Boolean(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
return peers;
},
listGroups: async ({ cfg, accountId, query, limit }) => {
const account = resolveTelegramAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const groups = Object.keys(account.config.groups ?? {})
.map((id) => id.trim())
.filter((id) => Boolean(id) && id !== "*")
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
return groups;
},
},
actions: telegramMessageActions,
setup: {
+2 -4
View File
@@ -216,15 +216,13 @@ export type ChannelThreadingToolContext = {
export type ChannelMessagingAdapter = {
normalizeTarget?: (raw: string) => string | undefined;
targetResolver?: {
looksLikeId?: (raw: string, normalized?: string) => boolean;
hint?: string;
};
looksLikeTargetId?: (raw: string, normalized?: string) => boolean;
formatTargetDisplay?: (params: {
target: string;
display?: string;
kind?: ChannelDirectoryEntryKind;
}) => string;
targetHint?: string;
};
export type ChannelDirectoryEntryKind = "user" | "group" | "channel";
+27 -10
View File
@@ -39,10 +39,6 @@ import { collectWhatsAppStatusIssues } from "./status-issues/whatsapp.js";
import type { ChannelMessageActionName, ChannelPlugin } from "./types.js";
import { resolveWhatsAppHeartbeatRecipients } from "./whatsapp-heartbeat.js";
import { missingTargetError } from "../../infra/outbound/target-errors.js";
import {
listWhatsAppDirectoryGroupsFromConfig,
listWhatsAppDirectoryPeersFromConfig,
} from "./directory-config.js";
const meta = getChatChannelMeta("whatsapp");
@@ -226,10 +222,8 @@ export const whatsappPlugin: ChannelPlugin<ResolvedWhatsAppAccount> = {
},
messaging: {
normalizeTarget: normalizeWhatsAppMessagingTarget,
targetResolver: {
looksLikeId: looksLikeWhatsAppTargetId,
hint: "<E.164|group JID>",
},
looksLikeTargetId: looksLikeWhatsAppTargetId,
targetHint: "<E.164|group JID>",
},
directory: {
self: async ({ cfg, accountId }) => {
@@ -244,8 +238,31 @@ export const whatsappPlugin: ChannelPlugin<ResolvedWhatsAppAccount> = {
raw: { e164, jid },
};
},
listPeers: async (params) => listWhatsAppDirectoryPeersFromConfig(params),
listGroups: async (params) => listWhatsAppDirectoryGroupsFromConfig(params),
listPeers: async ({ cfg, accountId, query, limit }) => {
const account = resolveWhatsAppAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const peers = (account.allowFrom ?? [])
.map((entry) => String(entry).trim())
.filter((entry) => Boolean(entry) && entry !== "*")
.map((entry) => normalizeWhatsAppTarget(entry) ?? "")
.filter(Boolean)
.filter((id) => !isWhatsAppGroupJid(id))
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "user", id }) as const);
return peers;
},
listGroups: async ({ cfg, accountId, query, limit }) => {
const account = resolveWhatsAppAccount({ cfg, accountId });
const q = query?.trim().toLowerCase() || "";
const groups = Object.keys(account.groups ?? {})
.map((id) => id.trim())
.filter((id) => Boolean(id) && id !== "*")
.filter((id) => (q ? id.toLowerCase().includes(q) : true))
.slice(0, limit && limit > 0 ? limit : undefined)
.map((id) => ({ kind: "group", id }) as const);
return groups;
},
},
actions: {
listActions: ({ cfg }) => {
-54
View File
@@ -1,54 +0,0 @@
import { describe, expect, it } from "vitest";
import type { HookStatusReport } from "../hooks/hooks-status.js";
import { formatHooksCheck, formatHooksList } from "./hooks-cli.js";
const report: HookStatusReport = {
workspaceDir: "/tmp/workspace",
managedHooksDir: "/tmp/hooks",
hooks: [
{
name: "session-memory",
description: "Save session context to memory",
source: "clawdbot-bundled",
filePath: "/tmp/hooks/session-memory/HOOK.md",
baseDir: "/tmp/hooks/session-memory",
handlerPath: "/tmp/hooks/session-memory/handler.js",
hookKey: "session-memory",
emoji: "💾",
homepage: "https://docs.clawd.bot/hooks#session-memory",
events: ["command:new"],
always: false,
disabled: false,
eligible: true,
requirements: {
bins: [],
anyBins: [],
env: [],
config: [],
os: [],
},
missing: {
bins: [],
anyBins: [],
env: [],
config: [],
os: [],
},
configChecks: [],
install: [],
},
],
};
describe("hooks cli formatting", () => {
it("labels hooks list output", () => {
const output = formatHooksList(report, {});
expect(output).toContain("Hooks");
expect(output).not.toContain("Internal Hooks");
});
it("labels hooks status output", () => {
const output = formatHooksCheck(report, {});
expect(output).toContain("Hooks Status");
});
});
+5 -3
View File
@@ -125,7 +125,9 @@ export function formatHooksList(report: HookStatusReport, opts: HooksListOptions
const notEligible = hooks.filter((h) => !h.eligible);
const lines: string[] = [];
lines.push(chalk.bold.cyan("Hooks") + chalk.gray(` (${eligible.length}/${hooks.length} ready)`));
lines.push(
chalk.bold.cyan("Internal Hooks") + chalk.gray(` (${eligible.length}/${hooks.length} ready)`),
);
lines.push("");
if (eligible.length > 0) {
@@ -271,7 +273,7 @@ export function formatHooksCheck(report: HookStatusReport, opts: HooksCheckOptio
const notEligible = report.hooks.filter((h) => !h.eligible);
const lines: string[] = [];
lines.push(chalk.bold.cyan("Hooks Status"));
lines.push(chalk.bold.cyan("Internal Hooks Status"));
lines.push("");
lines.push(`Total hooks: ${report.hooks.length}`);
lines.push(chalk.green(`Ready: ${eligible.length}`));
@@ -371,7 +373,7 @@ export function registerHooksCli(program: Command): void {
hooks
.command("list")
.description("List all hooks")
.description("List all internal hooks")
.option("--eligible", "Show only eligible hooks", false)
.option("--json", "Output as JSON", false)
.option("-v, --verbose", "Show more details including missing requirements", false)
+4 -4
View File
@@ -59,7 +59,7 @@ describe("onboard-hooks", () => {
});
describe("setupInternalHooks", () => {
it("should enable hooks when user selects them", async () => {
it("should enable internal hooks when user selects them", async () => {
const { buildWorkspaceHookStatus } = await import("../hooks/hooks-status.js");
vi.mocked(buildWorkspaceHookStatus).mockReturnValue(createMockHookReport());
@@ -75,7 +75,7 @@ describe("onboard-hooks", () => {
});
expect(prompter.note).toHaveBeenCalledTimes(2);
expect(prompter.multiselect).toHaveBeenCalledWith({
message: "Enable hooks?",
message: "Enable internal hooks?",
options: [
{ value: "__skip__", label: "Skip for now" },
{
@@ -173,8 +173,8 @@ describe("onboard-hooks", () => {
const noteCalls = (prompter.note as ReturnType<typeof vi.fn>).mock.calls;
expect(noteCalls).toHaveLength(2);
// First note should explain what hooks are
expect(noteCalls[0][0]).toContain("Hooks let you automate actions");
// First note should explain what internal hooks are
expect(noteCalls[0][0]).toContain("Internal hooks");
expect(noteCalls[0][0]).toContain("automate actions");
// Second note should confirm configuration
+4 -4
View File
@@ -11,12 +11,12 @@ export async function setupInternalHooks(
): Promise<ClawdbotConfig> {
await prompter.note(
[
"Hooks let you automate actions when agent commands are issued.",
"Internal hooks let you automate actions when agent commands are issued.",
"Example: Save session context to memory when you issue /new.",
"",
"Learn more: https://docs.clawd.bot/hooks",
"Learn more: https://docs.clawd.bot/internal-hooks",
].join("\n"),
"Hooks",
"Internal Hooks",
);
// Discover available hooks using the hook discovery system
@@ -35,7 +35,7 @@ export async function setupInternalHooks(
}
const toEnable = await prompter.multiselect({
message: "Enable hooks?",
message: "Enable internal hooks?",
options: [
{ value: "__skip__", label: "Skip for now" },
...recommendedHooks.map((hook) => ({
+23
View File
@@ -125,6 +125,29 @@ describe("resolveChannelCapabilities", () => {
}),
).toBeUndefined();
});
it("falls back to channel capabilities when account capabilities use object format", () => {
const cfg = {
channels: {
telegram: {
capabilities: ["inlineButtons"],
accounts: {
default: {
capabilities: { inlineButtons: "dm" },
},
},
},
},
} satisfies Partial<ClawdbotConfig>;
expect(
resolveChannelCapabilities({
cfg: cfg as ClawdbotConfig,
channel: "telegram",
accountId: "default",
}),
).toEqual(["inlineButtons"]);
});
});
const createRegistry = (channels: PluginRegistry["channels"]): PluginRegistry => ({
+1 -1
View File
@@ -90,7 +90,7 @@ export type HookInstallRecord = {
};
export type InternalHooksConfig = {
/** Enable hooks system */
/** Enable internal hooks system */
enabled?: boolean;
/** Legacy: List of internal hook handlers to register (still supported) */
handlers?: InternalHookHandlerConfig[];
+1 -1
View File
@@ -103,7 +103,7 @@ export async function startGatewaySidecars(params: {
);
}
} catch (err) {
params.logHooks.error(`failed to load hooks: ${String(err)}`);
params.logHooks.error(`failed to load internal hooks: ${String(err)}`);
}
// Launch configured channels so gateway replies via the surface the message came from.
+6 -6
View File
@@ -1,6 +1,6 @@
# Bundled Hooks
# Bundled Internal Hooks
This directory contains hooks that ship with Clawdbot. These hooks are automatically discovered and can be enabled/disabled via CLI or configuration.
This directory contains internal hooks that ship with Clawdbot. These hooks are automatically discovered and can be enabled/disabled via CLI or configuration.
## Available Hooks
@@ -53,7 +53,7 @@ session-memory/
---
name: my-hook
description: "Short description"
homepage: https://docs.clawd.bot/hooks#my-hook
homepage: https://docs.clawd.bot/hooks/my-hook
metadata:
{ "clawdbot": { "emoji": "🔗", "events": ["command:new"], "requires": { "bins": ["node"] } } }
---
@@ -161,9 +161,9 @@ interface InternalHookEvent {
Example handler:
```typescript
import type { HookHandler } from "../../src/hooks/hooks.js";
import type { InternalHookHandler } from "../../src/hooks/internal-hooks.js";
const myHandler: HookHandler = async (event) => {
const myHandler: InternalHookHandler = async (event) => {
if (event.type !== "command" || event.action !== "new") {
return;
}
@@ -190,4 +190,4 @@ Test your hooks by:
## Documentation
Full documentation: https://docs.clawd.bot/hooks
Full documentation: https://docs.clawd.bot/internal-hooks
+1 -1
View File
@@ -1,7 +1,7 @@
---
name: command-logger
description: "Log all command events to a centralized audit file"
homepage: https://docs.clawd.bot/hooks#command-logger
homepage: https://docs.clawd.bot/internal-hooks#command-logger
metadata:
{
"clawdbot":
+3 -3
View File
@@ -1,5 +1,5 @@
/**
* Example hook handler: Log all commands to a file
* Example internal hook handler: Log all commands to a file
*
* This handler demonstrates how to create a hook that logs all command events
* to a centralized log file for audit/debugging purposes.
@@ -26,12 +26,12 @@
import fs from "node:fs/promises";
import path from "node:path";
import os from "node:os";
import type { HookHandler } from "../../hooks.js";
import type { InternalHookHandler } from "../../internal-hooks.js";
/**
* Log all command events to a file
*/
const logCommand: HookHandler = async (event) => {
const logCommand: InternalHookHandler = async (event) => {
// Only trigger on command events
if (event.type !== "command") {
return;
+1 -1
View File
@@ -1,7 +1,7 @@
---
name: session-memory
description: "Save session context to memory when /new command is issued"
homepage: https://docs.clawd.bot/hooks#session-memory
homepage: https://docs.clawd.bot/internal-hooks#session-memory
metadata:
{
"clawdbot":
+2 -2
View File
@@ -11,7 +11,7 @@ import os from "node:os";
import type { ClawdbotConfig } from "../../../config/config.js";
import { resolveAgentWorkspaceDir } from "../../../agents/agent-scope.js";
import { resolveAgentIdFromSessionKey } from "../../../routing/session-key.js";
import type { HookHandler } from "../../hooks.js";
import type { InternalHookHandler } from "../../internal-hooks.js";
/**
* Read recent messages from session file for slug generation
@@ -57,7 +57,7 @@ async function getRecentSessionContent(sessionFilePath: string): Promise<string
/**
* Save session context to memory when /new command is triggered
*/
const saveSessionToMemory: HookHandler = async (event) => {
const saveSessionToMemory: InternalHookHandler = async (event) => {
// Only trigger on 'new' command
if (event.type !== "command" || event.action !== "new") {
return;
-116
View File
@@ -1,116 +0,0 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const tempDirs: string[] = [];
async function makeTempDir() {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-hooks-e2e-"));
tempDirs.push(dir);
return dir;
}
describe("hooks install (e2e)", () => {
let prevStateDir: string | undefined;
let prevBundledDir: string | undefined;
let workspaceDir: string;
beforeEach(async () => {
const baseDir = await makeTempDir();
workspaceDir = path.join(baseDir, "workspace");
await fs.mkdir(workspaceDir, { recursive: true });
prevStateDir = process.env.CLAWDBOT_STATE_DIR;
prevBundledDir = process.env.CLAWDBOT_BUNDLED_HOOKS_DIR;
process.env.CLAWDBOT_STATE_DIR = path.join(baseDir, "state");
process.env.CLAWDBOT_BUNDLED_HOOKS_DIR = path.join(baseDir, "bundled-none");
vi.resetModules();
});
afterEach(async () => {
if (prevStateDir === undefined) {
delete process.env.CLAWDBOT_STATE_DIR;
} else {
process.env.CLAWDBOT_STATE_DIR = prevStateDir;
}
if (prevBundledDir === undefined) {
delete process.env.CLAWDBOT_BUNDLED_HOOKS_DIR;
} else {
process.env.CLAWDBOT_BUNDLED_HOOKS_DIR = prevBundledDir;
}
vi.resetModules();
for (const dir of tempDirs.splice(0)) {
try {
await fs.rm(dir, { recursive: true, force: true });
} catch {
// ignore cleanup failures
}
}
});
it("installs a hook pack and triggers the handler", async () => {
const baseDir = await makeTempDir();
const packDir = path.join(baseDir, "hook-pack");
const hookDir = path.join(packDir, "hooks", "hello-hook");
await fs.mkdir(hookDir, { recursive: true });
await fs.writeFile(
path.join(packDir, "package.json"),
JSON.stringify(
{
name: "@acme/hello-hooks",
version: "0.0.0",
clawdbot: { hooks: ["./hooks/hello-hook"] },
},
null,
2,
),
"utf-8",
);
await fs.writeFile(
path.join(hookDir, "HOOK.md"),
[
"---",
'name: "hello-hook"',
'description: "Test hook"',
'metadata: {"clawdbot":{"events":["command:new"]}}',
"---",
"",
"# Hello Hook",
"",
].join("\n"),
"utf-8",
);
await fs.writeFile(
path.join(hookDir, "handler.js"),
"export default async function(event) { event.messages.push('hook-ok'); }\n",
"utf-8",
);
const { installHooksFromPath } = await import("./install.js");
const installResult = await installHooksFromPath({ path: packDir });
expect(installResult.ok).toBe(true);
if (!installResult.ok) return;
const { clearInternalHooks, createInternalHookEvent, triggerInternalHook } = await import(
"./internal-hooks.js"
);
const { loadInternalHooks } = await import("./loader.js");
clearInternalHooks();
const loaded = await loadInternalHooks(
{ hooks: { internal: { enabled: true } } },
workspaceDir,
);
expect(loaded).toBe(1);
const event = createInternalHookEvent("command", "new", "test-session");
await triggerInternalHook(event);
expect(event.messages).toContain("hook-ok");
});
});
-14
View File
@@ -1,14 +0,0 @@
export * from "./internal-hooks.js";
export type HookEventType = import("./internal-hooks.js").InternalHookEventType;
export type HookEvent = import("./internal-hooks.js").InternalHookEvent;
export type HookHandler = import("./internal-hooks.js").InternalHookHandler;
export {
registerInternalHook as registerHook,
unregisterInternalHook as unregisterHook,
clearInternalHooks as clearHooks,
getRegisteredEventKeys as getRegisteredHookEventKeys,
triggerInternalHook as triggerHook,
createInternalHookEvent as createHookEvent,
} from "./internal-hooks.js";
-49
View File
@@ -3,7 +3,6 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import JSZip from "jszip";
import * as tar from "tar";
import { afterEach, describe, expect, it, vi } from "vitest";
const tempDirs: string[] = [];
@@ -86,54 +85,6 @@ describe("installHooksFromArchive", () => {
fs.existsSync(path.join(result.targetDir, "hooks", "zip-hook", "HOOK.md")),
).toBe(true);
});
it("installs hook packs from tar archives", async () => {
const stateDir = makeTempDir();
const workDir = makeTempDir();
const archivePath = path.join(workDir, "hooks.tar");
const pkgDir = path.join(workDir, "package");
fs.mkdirSync(path.join(pkgDir, "hooks", "tar-hook"), { recursive: true });
fs.writeFileSync(
path.join(pkgDir, "package.json"),
JSON.stringify({
name: "@clawdbot/tar-hooks",
version: "0.0.1",
clawdbot: { hooks: ["./hooks/tar-hook"] },
}),
"utf-8",
);
fs.writeFileSync(
path.join(pkgDir, "hooks", "tar-hook", "HOOK.md"),
[
"---",
"name: tar-hook",
"description: Tar hook",
"metadata: {\"clawdbot\":{\"events\":[\"command:new\"]}}",
"---",
"",
"# Tar Hook",
].join("\n"),
"utf-8",
);
fs.writeFileSync(
path.join(pkgDir, "hooks", "tar-hook", "handler.ts"),
"export default async () => {};\n",
"utf-8",
);
await tar.c({ cwd: workDir, file: archivePath }, ["package"]);
const result = await withStateDir(stateDir, async () => {
const { installHooksFromArchive } = await import("./install.js");
return await installHooksFromArchive({ archivePath });
});
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.hookPackId).toBe("tar-hooks");
expect(result.hooks).toContain("tar-hook");
expect(result.targetDir).toBe(path.join(stateDir, "hooks", "tar-hooks"));
});
});
describe("installHooksFromPath", () => {
+2 -2
View File
@@ -9,7 +9,7 @@ import {
type InternalHookEvent,
} from "./internal-hooks.js";
describe("hooks", () => {
describe("internal-hooks", () => {
beforeEach(() => {
clearInternalHooks();
});
@@ -131,7 +131,7 @@ describe("hooks", () => {
expect(errorHandler).toHaveBeenCalled();
expect(successHandler).toHaveBeenCalled();
expect(consoleError).toHaveBeenCalledWith(
expect.stringContaining("Hook error"),
expect.stringContaining("Internal hook error"),
expect.stringContaining("Handler failed"),
);
+5 -5
View File
@@ -1,7 +1,7 @@
/**
* Hook system for clawdbot agent events
* Internal hook system for clawdbot agent events
*
* Provides an extensible event-driven hook system for agent events
* Provides an extensible event-driven hook system for internal agent events
* like command processing, session lifecycle, etc.
*/
@@ -91,7 +91,7 @@ export function getRegisteredEventKeys(): string[] {
}
/**
* Trigger a hook event
* Trigger an internal hook event
*
* Calls all handlers registered for:
* 1. The general event type (e.g., 'command')
@@ -117,7 +117,7 @@ export async function triggerInternalHook(event: InternalHookEvent): Promise<voi
await handler(event);
} catch (err) {
console.error(
`Hook error [${event.type}:${event.action}]:`,
`Internal hook error [${event.type}:${event.action}]:`,
err instanceof Error ? err.message : String(err),
);
}
@@ -125,7 +125,7 @@ export async function triggerInternalHook(event: InternalHookEvent): Promise<voi
}
/**
* Create a hook event with common fields filled in
* Create an internal hook event with common fields filled in
*
* @param type - The event type
* @param action - The action within that type
+2 -2
View File
@@ -43,7 +43,7 @@ describe("loader", () => {
});
describe("loadInternalHooks", () => {
it("should return 0 when hooks are not enabled", async () => {
it("should return 0 when internal hooks are not enabled", async () => {
const cfg: ClawdbotConfig = {
hooks: {
internal: {
@@ -170,7 +170,7 @@ describe("loader", () => {
const count = await loadInternalHooks(cfg, tmpDir);
expect(count).toBe(0);
expect(consoleError).toHaveBeenCalledWith(
expect.stringContaining("Failed to load hook handler"),
expect.stringContaining("Failed to load internal hook handler"),
expect.any(String),
);
+11 -11
View File
@@ -1,5 +1,5 @@
/**
* Dynamic loader for hook handlers
* Dynamic loader for internal hook handlers
*
* Loads hook handlers from external modules based on configuration
* and from directory-based discovery (bundled, managed, workspace)
@@ -15,7 +15,7 @@ import { resolveHookConfig } from "./config.js";
import { shouldIncludeHook } from "./config.js";
/**
* Load and register all hook handlers
* Load and register all internal hook handlers
*
* Loads hooks from both:
* 1. Directory-based discovery (bundled, managed, workspace)
@@ -30,14 +30,14 @@ import { shouldIncludeHook } from "./config.js";
* const config = await loadConfig();
* const workspaceDir = resolveAgentWorkspaceDir(config, agentId);
* const count = await loadInternalHooks(config, workspaceDir);
* console.log(`Loaded ${count} hook handlers`);
* console.log(`Loaded ${count} internal hook handlers`);
* ```
*/
export async function loadInternalHooks(
cfg: ClawdbotConfig,
workspaceDir: string,
): Promise<number> {
// Check if hooks are enabled
// Check if internal hooks are enabled
if (!cfg.hooks?.internal?.enabled) {
return 0;
}
@@ -71,7 +71,7 @@ export async function loadInternalHooks(
if (typeof handler !== "function") {
console.error(
`Hook error: Handler '${exportName}' from ${entry.hook.name} is not a function`,
`Internal hook error: Handler '${exportName}' from ${entry.hook.name} is not a function`,
);
continue;
}
@@ -80,7 +80,7 @@ export async function loadInternalHooks(
const events = entry.clawdbot?.events ?? [];
if (events.length === 0) {
console.warn(
`Hook warning: Hook '${entry.hook.name}' has no events defined in metadata`,
`Internal hook warning: Hook '${entry.hook.name}' has no events defined in metadata`,
);
continue;
}
@@ -90,12 +90,12 @@ export async function loadInternalHooks(
}
console.log(
`Registered hook: ${entry.hook.name} -> ${events.join(", ")}${exportName !== "default" ? ` (export: ${exportName})` : ""}`,
`Registered internal hook: ${entry.hook.name} -> ${events.join(", ")}${exportName !== "default" ? ` (export: ${exportName})` : ""}`,
);
loadedCount++;
} catch (err) {
console.error(
`Failed to load hook ${entry.hook.name}:`,
`Failed to load internal hook ${entry.hook.name}:`,
err instanceof Error ? err.message : String(err),
);
}
@@ -127,7 +127,7 @@ export async function loadInternalHooks(
if (typeof handler !== "function") {
console.error(
`Hook error: Handler '${exportName}' from ${modulePath} is not a function`,
`Internal hook error: Handler '${exportName}' from ${modulePath} is not a function`,
);
continue;
}
@@ -135,12 +135,12 @@ export async function loadInternalHooks(
// Register the handler
registerInternalHook(handlerConfig.event, handler as InternalHookHandler);
console.log(
`Registered hook (legacy): ${handlerConfig.event} -> ${modulePath}${exportName !== "default" ? `#${exportName}` : ""}`,
`Registered internal hook (legacy): ${handlerConfig.event} -> ${modulePath}${exportName !== "default" ? `#${exportName}` : ""}`,
);
loadedCount++;
} catch (err) {
console.error(
`Failed to load hook handler from ${handlerConfig.module}:`,
`Failed to load internal hook handler from ${handlerConfig.module}:`,
err instanceof Error ? err.message : String(err),
);
}
-68
View File
@@ -1,68 +0,0 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import JSZip from "jszip";
import * as tar from "tar";
import { afterEach, describe, expect, it } from "vitest";
import { extractArchive, resolveArchiveKind, resolvePackedRootDir } from "./archive.js";
const tempDirs: string[] = [];
async function makeTempDir() {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-archive-"));
tempDirs.push(dir);
return dir;
}
afterEach(async () => {
for (const dir of tempDirs.splice(0)) {
try {
await fs.rm(dir, { recursive: true, force: true });
} catch {
// ignore cleanup failures
}
}
});
describe("archive utils", () => {
it("detects archive kinds", () => {
expect(resolveArchiveKind("/tmp/file.zip")).toBe("zip");
expect(resolveArchiveKind("/tmp/file.tgz")).toBe("tar");
expect(resolveArchiveKind("/tmp/file.tar.gz")).toBe("tar");
expect(resolveArchiveKind("/tmp/file.tar")).toBe("tar");
expect(resolveArchiveKind("/tmp/file.txt")).toBeNull();
});
it("extracts zip archives", async () => {
const workDir = await makeTempDir();
const archivePath = path.join(workDir, "bundle.zip");
const extractDir = path.join(workDir, "extract");
const zip = new JSZip();
zip.file("package/hello.txt", "hi");
await fs.writeFile(archivePath, await zip.generateAsync({ type: "nodebuffer" }));
await fs.mkdir(extractDir, { recursive: true });
await extractArchive({ archivePath, destDir: extractDir, timeoutMs: 5_000 });
const rootDir = await resolvePackedRootDir(extractDir);
const content = await fs.readFile(path.join(rootDir, "hello.txt"), "utf-8");
expect(content).toBe("hi");
});
it("extracts tar archives", async () => {
const workDir = await makeTempDir();
const archivePath = path.join(workDir, "bundle.tar");
const extractDir = path.join(workDir, "extract");
const packageDir = path.join(workDir, "package");
await fs.mkdir(packageDir, { recursive: true });
await fs.writeFile(path.join(packageDir, "hello.txt"), "yo");
await tar.c({ cwd: workDir, file: archivePath }, ["package"]);
await fs.mkdir(extractDir, { recursive: true });
await extractArchive({ archivePath, destDir: extractDir, timeoutMs: 5_000 });
const rootDir = await resolvePackedRootDir(extractDir);
const content = await fs.readFile(path.join(rootDir, "hello.txt"), "utf-8");
expect(content).toBe("yo");
});
});
+4 -4
View File
@@ -1,5 +1,5 @@
export function missingTargetMessage(provider: string, hint?: string): string {
return `Delivering to ${provider} requires target${formatTargetHint(hint)}`;
return `Delivering to ${provider} requires target${formatHint(hint)}`;
}
export function missingTargetError(provider: string, hint?: string): Error {
@@ -7,7 +7,7 @@ export function missingTargetError(provider: string, hint?: string): Error {
}
export function ambiguousTargetMessage(provider: string, raw: string, hint?: string): string {
return `Ambiguous target "${raw}" for ${provider}. Provide a unique name or an explicit id.${formatTargetHint(hint, true)}`;
return `Ambiguous target "${raw}" for ${provider}. Provide a unique name or an explicit id.${formatHint(hint, true)}`;
}
export function ambiguousTargetError(provider: string, raw: string, hint?: string): Error {
@@ -15,14 +15,14 @@ export function ambiguousTargetError(provider: string, raw: string, hint?: strin
}
export function unknownTargetMessage(provider: string, raw: string, hint?: string): string {
return `Unknown target "${raw}" for ${provider}.${formatTargetHint(hint, true)}`;
return `Unknown target "${raw}" for ${provider}.${formatHint(hint, true)}`;
}
export function unknownTargetError(provider: string, raw: string, hint?: string): Error {
return new Error(unknownTargetMessage(provider, raw, hint));
}
function formatTargetHint(hint?: string, withLabel = false): string {
function formatHint(hint?: string, withLabel = false): string {
if (!hint) return "";
return withLabel ? ` Hint: ${hint}` : ` ${hint}`;
}
+2 -3
View File
@@ -16,9 +16,8 @@ export function normalizeTargetForProvider(provider: string, raw?: string): stri
export function buildTargetResolverSignature(channel: ChannelId): string {
const plugin = getChannelPlugin(channel);
const resolver = plugin?.messaging?.targetResolver;
const hint = resolver?.hint ?? "";
const looksLike = resolver?.looksLikeId;
const hint = plugin?.messaging?.targetHint ?? "";
const looksLike = plugin?.messaging?.looksLikeTargetId;
const source = looksLike ? looksLike.toString() : "";
return hashSignature(`${hint}|${source}`);
}
+20 -14
View File
@@ -164,6 +164,24 @@ function resolveMatch(params: {
return { kind: "ambiguous" as const, entries: matches };
}
function looksLikeTargetId(params: {
channel: ChannelId;
raw: string;
normalized: string;
}): boolean {
const raw = params.raw.trim();
if (!raw) return false;
const plugin = getChannelPlugin(params.channel);
const lookup = plugin?.messaging?.looksLikeTargetId;
if (lookup) return lookup(raw, params.normalized);
if (/^(channel|group|user):/i.test(raw)) return true;
if (/^[@#]/.test(raw)) return true;
if (/^\+?\d{6,}$/.test(raw)) return true;
if (raw.includes("@thread")) return true;
if (/^(conversation|user):/i.test(raw)) return true;
return false;
}
async function listDirectoryEntries(params: {
cfg: ClawdbotConfig;
channel: ChannelId;
@@ -267,22 +285,10 @@ export async function resolveMessagingTarget(params: {
}
const plugin = getChannelPlugin(params.channel);
const providerLabel = plugin?.meta?.label ?? params.channel;
const hint = plugin?.messaging?.targetResolver?.hint;
const hint = plugin?.messaging?.targetHint;
const kind = detectTargetKind(raw, params.preferredKind);
const normalized = normalizeTargetForProvider(params.channel, raw) ?? raw;
const looksLikeTargetId = (): boolean => {
const trimmed = raw.trim();
if (!trimmed) return false;
const lookup = plugin?.messaging?.targetResolver?.looksLikeId;
if (lookup) return lookup(trimmed, normalized);
if (/^(channel|group|user):/i.test(trimmed)) return true;
if (/^[@#]/.test(trimmed)) return true;
if (/^\+?\d{6,}$/.test(trimmed)) return true;
if (trimmed.includes("@thread")) return true;
if (/^(conversation|user):/i.test(trimmed)) return true;
return false;
};
if (looksLikeTargetId()) {
if (looksLikeTargetId({ channel: params.channel, raw, normalized })) {
const directTarget = preserveTargetCase(params.channel, raw, normalized);
return {
ok: true,
+1 -1
View File
@@ -146,7 +146,7 @@ export function resolveOutboundTarget(params: {
if (trimmed) {
return { ok: true, to: trimmed };
}
const hint = plugin.messaging?.targetResolver?.hint;
const hint = plugin.messaging?.targetHint;
return {
ok: false,
error: missingTargetError(plugin.meta.label ?? params.channel, hint),
-60
View File
@@ -424,64 +424,4 @@ describe("applyMediaUnderstanding", () => {
["[Audio 1/2]\nTranscript:\nnote-a.ogg", "[Audio 2/2]\nTranscript:\nnote-b.ogg"].join("\n\n"),
);
});
it("orders mixed media outputs as image, audio, video", async () => {
const { applyMediaUnderstanding } = await loadApply();
const dir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-media-"));
const imagePath = path.join(dir, "photo.jpg");
const audioPath = path.join(dir, "note.ogg");
const videoPath = path.join(dir, "clip.mp4");
await fs.writeFile(imagePath, "image-bytes");
await fs.writeFile(audioPath, "audio-bytes");
await fs.writeFile(videoPath, "video-bytes");
const ctx: MsgContext = {
Body: "<media:mixed>",
MediaPaths: [imagePath, audioPath, videoPath],
MediaTypes: ["image/jpeg", "audio/ogg", "video/mp4"],
};
const cfg: ClawdbotConfig = {
tools: {
media: {
image: { enabled: true, models: [{ provider: "openai", model: "gpt-5.2" }] },
audio: { enabled: true, models: [{ provider: "groq" }] },
video: { enabled: true, models: [{ provider: "google", model: "gemini-3" }] },
},
},
};
const result = await applyMediaUnderstanding({
ctx,
cfg,
agentDir: dir,
providers: {
openai: {
id: "openai",
describeImage: async () => ({ text: "image ok" }),
},
groq: {
id: "groq",
transcribeAudio: async () => ({ text: "audio ok" }),
},
google: {
id: "google",
describeVideo: async () => ({ text: "video ok" }),
},
},
});
expect(result.appliedImage).toBe(true);
expect(result.appliedAudio).toBe(true);
expect(result.appliedVideo).toBe(true);
expect(ctx.Body).toBe(
[
"[Image]\nDescription:\nimage ok",
"[Audio]\nTranscript:\naudio ok",
"[Video]\nDescription:\nvideo ok",
].join("\n\n"),
);
expect(ctx.Transcript).toBe("audio ok");
expect(ctx.CommandBody).toBe("audio ok");
expect(ctx.BodyForCommands).toBe("audio ok");
});
});
+517 -16
View File
@@ -1,26 +1,52 @@
import type { ClawdbotConfig } from "../config/config.js";
import type { MsgContext } from "../auto-reply/templating.js";
import { applyTemplate } from "../auto-reply/templating.js";
import { finalizeInboundContext } from "../auto-reply/reply/inbound-context.js";
import { resolveApiKeyForProvider } from "../agents/model-auth.js";
import { logVerbose, shouldLogVerbose } from "../globals.js";
import { runExec } from "../process/exec.js";
import type {
MediaUnderstandingConfig,
MediaUnderstandingModelConfig,
} from "../config/types.tools.js";
import { MediaAttachmentCache, normalizeAttachments, selectAttachments } from "./attachments.js";
import {
CLI_OUTPUT_MAX_BUFFER,
DEFAULT_AUDIO_MODELS,
DEFAULT_TIMEOUT_SECONDS,
} from "./defaults.js";
import { isMediaUnderstandingSkipError, MediaUnderstandingSkipError } from "./errors.js";
import {
extractMediaUserText,
formatAudioTranscripts,
formatMediaUnderstandingBody,
} from "./format.js";
import {
buildMediaUnderstandingRegistry,
getMediaUnderstandingProvider,
normalizeMediaProviderId,
} from "./providers/index.js";
import { describeImageWithModel } from "./providers/image.js";
import {
resolveCapabilityConfig,
inferProviderCapabilities,
resolveConcurrency,
resolveMaxBytes,
resolveMaxChars,
resolveModelEntries,
resolvePrompt,
resolveScopeDecision,
resolveTimeoutMs,
} from "./resolve.js";
import type {
MediaUnderstandingCapability,
MediaUnderstandingDecision,
MediaUnderstandingModelDecision,
MediaUnderstandingOutput,
MediaUnderstandingProvider,
} from "./types.js";
import { runWithConcurrency } from "./concurrency.js";
import { resolveConcurrency } from "./resolve.js";
import {
type ActiveMediaModel,
buildProviderRegistry,
createMediaAttachmentCache,
normalizeMediaAttachments,
runCapability,
} from "./runner.js";
import { estimateBase64Size, resolveVideoMaxBase64Bytes } from "./video.js";
export type ApplyMediaUnderstandingResult = {
outputs: MediaUnderstandingOutput[];
@@ -32,6 +58,476 @@ export type ApplyMediaUnderstandingResult = {
const CAPABILITY_ORDER: MediaUnderstandingCapability[] = ["image", "audio", "video"];
type ActiveMediaModel = {
provider: string;
model?: string;
};
function trimOutput(text: string, maxChars?: number): string {
const trimmed = text.trim();
if (!maxChars || trimmed.length <= maxChars) return trimmed;
return trimmed.slice(0, maxChars).trim();
}
function resolveEntriesWithActiveFallback(params: {
cfg: ClawdbotConfig;
capability: MediaUnderstandingCapability;
config?: MediaUnderstandingConfig;
activeModel?: ActiveMediaModel;
}): MediaUnderstandingModelConfig[] {
const entries = resolveModelEntries({
cfg: params.cfg,
capability: params.capability,
config: params.config,
});
if (entries.length > 0) return entries;
if (params.config?.enabled !== true) return entries;
const activeProvider = params.activeModel?.provider?.trim();
if (!activeProvider) return entries;
const capabilities = inferProviderCapabilities(activeProvider);
if (!capabilities || !capabilities.includes(params.capability)) return entries;
return [
{
type: "provider",
provider: activeProvider,
model: params.activeModel?.model,
},
];
}
function buildModelDecision(params: {
entry: MediaUnderstandingModelConfig;
entryType: "provider" | "cli";
outcome: MediaUnderstandingModelDecision["outcome"];
reason?: string;
}): MediaUnderstandingModelDecision {
if (params.entryType === "cli") {
const command = params.entry.command?.trim();
return {
type: "cli",
provider: command ?? "cli",
model: params.entry.model ?? command,
outcome: params.outcome,
reason: params.reason,
};
}
const providerIdRaw = params.entry.provider?.trim();
const providerId = providerIdRaw ? normalizeMediaProviderId(providerIdRaw) : undefined;
return {
type: "provider",
provider: providerId ?? providerIdRaw,
model: params.entry.model,
outcome: params.outcome,
reason: params.reason,
};
}
async function runProviderEntry(params: {
capability: MediaUnderstandingCapability;
entry: MediaUnderstandingModelConfig;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
cache: MediaAttachmentCache;
agentDir?: string;
providerRegistry: Map<string, MediaUnderstandingProvider>;
config?: MediaUnderstandingConfig;
}): Promise<MediaUnderstandingOutput | null> {
const { entry, capability, cfg } = params;
const providerIdRaw = entry.provider?.trim();
if (!providerIdRaw) {
throw new Error(`Provider entry missing provider for ${capability}`);
}
const providerId = normalizeMediaProviderId(providerIdRaw);
const maxBytes = resolveMaxBytes({ capability, entry, cfg, config: params.config });
const maxChars = resolveMaxChars({ capability, entry, cfg, config: params.config });
const timeoutMs = resolveTimeoutMs(
entry.timeoutSeconds ??
params.config?.timeoutSeconds ??
cfg.tools?.media?.[capability]?.timeoutSeconds,
DEFAULT_TIMEOUT_SECONDS[capability],
);
const prompt = resolvePrompt(
capability,
entry.prompt ?? params.config?.prompt ?? cfg.tools?.media?.[capability]?.prompt,
maxChars,
);
if (capability === "image") {
if (!params.agentDir) {
throw new Error("Image understanding requires agentDir");
}
const modelId = entry.model?.trim();
if (!modelId) {
throw new Error("Image understanding requires model id");
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const provider = getMediaUnderstandingProvider(providerId, params.providerRegistry);
const result = provider?.describeImage
? await provider.describeImage({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
model: modelId,
provider: providerId,
prompt,
timeoutMs,
profile: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
cfg: params.cfg,
})
: await describeImageWithModel({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
model: modelId,
provider: providerId,
prompt,
timeoutMs,
profile: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
cfg: params.cfg,
});
return {
kind: "image.description",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? modelId,
};
}
const provider = getMediaUnderstandingProvider(providerId, params.providerRegistry);
if (!provider) {
throw new Error(`Media provider not available: ${providerId}`);
}
if (capability === "audio") {
if (!provider.transcribeAudio) {
throw new Error(`Audio transcription provider "${providerId}" not available.`);
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const key = await resolveApiKeyForProvider({
provider: providerId,
cfg,
profileId: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
});
const providerConfig = cfg.models?.providers?.[providerId];
const model = entry.model?.trim() || DEFAULT_AUDIO_MODELS[providerId] || entry.model;
const result = await provider.transcribeAudio({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
apiKey: key.apiKey,
baseUrl: providerConfig?.baseUrl,
headers: providerConfig?.headers,
model,
language: entry.language ?? params.config?.language ?? cfg.tools?.media?.audio?.language,
prompt,
timeoutMs,
});
return {
kind: "audio.transcription",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? model,
};
}
if (!provider.describeVideo) {
throw new Error(`Video understanding provider "${providerId}" not available.`);
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const estimatedBase64Bytes = estimateBase64Size(media.size);
const maxBase64Bytes = resolveVideoMaxBase64Bytes(maxBytes);
if (estimatedBase64Bytes > maxBase64Bytes) {
throw new MediaUnderstandingSkipError(
"maxBytes",
`Video attachment ${params.attachmentIndex + 1} base64 payload ${estimatedBase64Bytes} exceeds ${maxBase64Bytes}`,
);
}
const key = await resolveApiKeyForProvider({
provider: providerId,
cfg,
profileId: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
});
const providerConfig = cfg.models?.providers?.[providerId];
const result = await provider.describeVideo({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
apiKey: key.apiKey,
baseUrl: providerConfig?.baseUrl,
headers: providerConfig?.headers,
model: entry.model,
prompt,
timeoutMs,
});
return {
kind: "video.description",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? entry.model,
};
}
async function runCliEntry(params: {
capability: MediaUnderstandingCapability;
entry: MediaUnderstandingModelConfig;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
cache: MediaAttachmentCache;
config?: MediaUnderstandingConfig;
}): Promise<MediaUnderstandingOutput | null> {
const { entry, capability, cfg, ctx } = params;
const command = entry.command?.trim();
const args = entry.args ?? [];
if (!command) {
throw new Error(`CLI entry missing command for ${capability}`);
}
const maxBytes = resolveMaxBytes({ capability, entry, cfg, config: params.config });
const maxChars = resolveMaxChars({ capability, entry, cfg, config: params.config });
const timeoutMs = resolveTimeoutMs(
entry.timeoutSeconds ??
params.config?.timeoutSeconds ??
cfg.tools?.media?.[capability]?.timeoutSeconds,
DEFAULT_TIMEOUT_SECONDS[capability],
);
const prompt = resolvePrompt(
capability,
entry.prompt ?? params.config?.prompt ?? cfg.tools?.media?.[capability]?.prompt,
maxChars,
);
const pathResult = await params.cache.getPath({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const templCtx: MsgContext = {
...ctx,
MediaPath: pathResult.path,
Prompt: prompt,
MaxChars: maxChars,
};
const argv = [command, ...args].map((part, index) =>
index === 0 ? part : applyTemplate(part, templCtx),
);
if (shouldLogVerbose()) {
logVerbose(`Media understanding via CLI: ${argv.join(" ")}`);
}
const { stdout } = await runExec(argv[0], argv.slice(1), {
timeoutMs,
maxBuffer: CLI_OUTPUT_MAX_BUFFER,
});
const text = trimOutput(stdout, maxChars);
if (!text) return null;
return {
kind: capability === "audio" ? "audio.transcription" : `${capability}.description`,
attachmentIndex: params.attachmentIndex,
text,
provider: "cli",
model: command,
};
}
async function runAttachmentEntries(params: {
capability: MediaUnderstandingCapability;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
agentDir?: string;
providerRegistry: Map<string, MediaUnderstandingProvider>;
cache: MediaAttachmentCache;
entries: MediaUnderstandingModelConfig[];
config?: MediaUnderstandingConfig;
}): Promise<{ output: MediaUnderstandingOutput | null; attempts: MediaUnderstandingModelDecision[] }> {
const { entries, capability } = params;
const attempts: MediaUnderstandingModelDecision[] = [];
for (const entry of entries) {
try {
const entryType = entry.type ?? (entry.command ? "cli" : "provider");
const result =
entryType === "cli"
? await runCliEntry({
capability,
entry,
cfg: params.cfg,
ctx: params.ctx,
attachmentIndex: params.attachmentIndex,
cache: params.cache,
config: params.config,
})
: await runProviderEntry({
capability,
entry,
cfg: params.cfg,
ctx: params.ctx,
attachmentIndex: params.attachmentIndex,
cache: params.cache,
agentDir: params.agentDir,
providerRegistry: params.providerRegistry,
config: params.config,
});
if (result) {
const decision = buildModelDecision({ entry, entryType, outcome: "success" });
if (result.provider) decision.provider = result.provider;
if (result.model) decision.model = result.model;
attempts.push(decision);
return { output: result, attempts };
}
attempts.push(
buildModelDecision({ entry, entryType, outcome: "skipped", reason: "empty output" }),
);
} catch (err) {
if (isMediaUnderstandingSkipError(err)) {
attempts.push(
buildModelDecision({
entry,
entryType: entry.type ?? (entry.command ? "cli" : "provider"),
outcome: "skipped",
reason: `${err.reason}: ${err.message}`,
}),
);
if (shouldLogVerbose()) {
logVerbose(`Skipping ${capability} model due to ${err.reason}: ${err.message}`);
}
continue;
}
attempts.push(
buildModelDecision({
entry,
entryType: entry.type ?? (entry.command ? "cli" : "provider"),
outcome: "failed",
reason: String(err),
}),
);
if (shouldLogVerbose()) {
logVerbose(`${capability} understanding failed: ${String(err)}`);
}
}
}
return { output: null, attempts };
}
async function runCapability(params: {
capability: MediaUnderstandingCapability;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachments: MediaAttachmentCache;
media: ReturnType<typeof normalizeAttachments>;
agentDir?: string;
providerRegistry: Map<string, MediaUnderstandingProvider>;
config?: MediaUnderstandingConfig;
activeModel?: ActiveMediaModel;
}): Promise<{ outputs: MediaUnderstandingOutput[]; decision: MediaUnderstandingDecision }> {
const { capability, cfg, ctx } = params;
const config = params.config ?? resolveCapabilityConfig(cfg, capability);
if (config?.enabled === false) {
return {
outputs: [],
decision: { capability, outcome: "disabled", attachments: [] },
};
}
const attachmentPolicy = config?.attachments;
const selected = selectAttachments({
capability,
attachments: params.media,
policy: attachmentPolicy,
});
if (selected.length === 0) {
return {
outputs: [],
decision: { capability, outcome: "no-attachment", attachments: [] },
};
}
const scopeDecision = resolveScopeDecision({ scope: config?.scope, ctx });
if (scopeDecision === "deny") {
if (shouldLogVerbose()) {
logVerbose(`${capability} understanding disabled by scope policy.`);
}
return {
outputs: [],
decision: {
capability,
outcome: "scope-deny",
attachments: selected.map((item) => ({ attachmentIndex: item.index, attempts: [] })),
},
};
}
const entries = resolveEntriesWithActiveFallback({
cfg,
capability,
config,
activeModel: params.activeModel,
});
if (entries.length === 0) {
return {
outputs: [],
decision: {
capability,
outcome: "skipped",
attachments: selected.map((item) => ({ attachmentIndex: item.index, attempts: [] })),
},
};
}
const outputs: MediaUnderstandingOutput[] = [];
const attachmentDecisions: MediaUnderstandingDecision["attachments"] = [];
for (const attachment of selected) {
const { output, attempts } = await runAttachmentEntries({
capability,
cfg,
ctx,
attachmentIndex: attachment.index,
agentDir: params.agentDir,
providerRegistry: params.providerRegistry,
cache: params.attachments,
entries,
config,
});
if (output) outputs.push(output);
attachmentDecisions.push({
attachmentIndex: attachment.index,
attempts,
chosen: attempts.find((attempt) => attempt.outcome === "success"),
});
}
return {
outputs,
decision: {
capability,
outcome: outputs.length > 0 ? "success" : "skipped",
attachments: attachmentDecisions,
},
};
}
export async function applyMediaUnderstanding(params: {
ctx: MsgContext;
cfg: ClawdbotConfig;
@@ -46,13 +542,13 @@ export async function applyMediaUnderstanding(params: {
.map((value) => extractMediaUserText(value))
.find((value) => value && value.trim()) ?? undefined;
const attachments = normalizeMediaAttachments(ctx);
const providerRegistry = buildProviderRegistry(params.providers);
const cache = createMediaAttachmentCache(attachments);
const attachments = normalizeAttachments(ctx);
const providerRegistry = buildMediaUnderstandingRegistry(params.providers);
const cache = new MediaAttachmentCache(attachments);
try {
const tasks = CAPABILITY_ORDER.map((capability) => async () => {
const config = cfg.tools?.media?.[capability];
const config = resolveCapabilityConfig(cfg, capability);
return await runCapability({
capability,
cfg,
@@ -69,12 +565,17 @@ export async function applyMediaUnderstanding(params: {
const results = await runWithConcurrency(tasks, resolveConcurrency(cfg));
const outputs: MediaUnderstandingOutput[] = [];
const decisions: MediaUnderstandingDecision[] = [];
for (const entry of results) {
for (const [index] of CAPABILITY_ORDER.entries()) {
const entry = results[index];
if (!entry) continue;
for (const output of entry.outputs) {
outputs.push(output);
if (Array.isArray(entry.outputs)) {
for (const output of entry.outputs) {
outputs.push(output);
}
}
if (entry.decision) {
decisions.push(entry.decision);
}
decisions.push(entry.decision);
}
if (decisions.length > 0) {
+11 -20
View File
@@ -7,7 +7,7 @@ import { fileURLToPath } from "node:url";
import type { MsgContext } from "../auto-reply/templating.js";
import type { MediaUnderstandingAttachmentsConfig } from "../config/types.tools.js";
import { fetchRemoteMedia, MediaFetchError } from "../media/fetch.js";
import { detectMime, getFileExtension, isAudioFileName, kindFromMime } from "../media/mime.js";
import { detectMime, getFileExtension, isAudioFileName } from "../media/mime.js";
import { logVerbose, shouldLogVerbose } from "../globals.js";
import { fetchWithTimeout } from "./providers/shared.js";
import type { MediaAttachment, MediaUnderstandingCapability } from "./types.js";
@@ -100,32 +100,23 @@ export function normalizeAttachments(ctx: MsgContext): MediaAttachment[] {
];
}
export function resolveAttachmentKind(
attachment: MediaAttachment,
): "image" | "audio" | "video" | "document" | "unknown" {
const kind = kindFromMime(attachment.mime);
if (kind !== "unknown") return kind;
const ext = getFileExtension(attachment.path ?? attachment.url);
if (!ext) return "unknown";
if ([".mp4", ".mov", ".mkv", ".webm", ".avi", ".m4v"].includes(ext)) return "video";
if (isAudioFileName(attachment.path ?? attachment.url)) return "audio";
if ([".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".tiff", ".tif"].includes(ext)) {
return "image";
}
return "unknown";
}
export function isVideoAttachment(attachment: MediaAttachment): boolean {
return resolveAttachmentKind(attachment) === "video";
if (attachment.mime?.startsWith("video/")) return true;
const ext = getFileExtension(attachment.path ?? attachment.url);
if (!ext) return false;
return [".mp4", ".mov", ".mkv", ".webm", ".avi", ".m4v"].includes(ext);
}
export function isAudioAttachment(attachment: MediaAttachment): boolean {
return resolveAttachmentKind(attachment) === "audio";
if (attachment.mime?.startsWith("audio/")) return true;
return isAudioFileName(attachment.path ?? attachment.url);
}
export function isImageAttachment(attachment: MediaAttachment): boolean {
return resolveAttachmentKind(attachment) === "image";
if (attachment.mime?.startsWith("image/")) return true;
const ext = getFileExtension(attachment.path ?? attachment.url);
if (!ext) return false;
return [".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ".tiff", ".tif"].includes(ext);
}
function isAbortError(err: unknown): boolean {
@@ -3,6 +3,5 @@ import { describeImageWithModel } from "../image.js";
export const anthropicProvider: MediaUnderstandingProvider = {
id: "anthropic",
capabilities: ["image"],
describeImage: describeImageWithModel,
};
@@ -4,7 +4,6 @@ import { describeGeminiVideo } from "./video.js";
export const googleProvider: MediaUnderstandingProvider = {
id: "google",
capabilities: ["image", "audio", "video"],
describeImage: describeImageWithModel,
describeVideo: describeGeminiVideo,
};
@@ -5,7 +5,6 @@ const DEFAULT_GROQ_AUDIO_BASE_URL = "https://api.groq.com/openai/v1";
export const groqProvider: MediaUnderstandingProvider = {
id: "groq",
capabilities: ["audio"],
transcribeAudio: (req) =>
transcribeOpenAiCompatibleAudio({
...req,
+1 -10
View File
@@ -29,16 +29,7 @@ export function buildMediaUnderstandingRegistry(
}
if (overrides) {
for (const [key, provider] of Object.entries(overrides)) {
const normalizedKey = normalizeMediaProviderId(key);
const existing = registry.get(normalizedKey);
const merged = existing
? {
...existing,
...provider,
capabilities: provider.capabilities ?? existing.capabilities,
}
: provider;
registry.set(normalizedKey, merged);
registry.set(normalizeMediaProviderId(key), provider);
}
}
return registry;
@@ -3,6 +3,5 @@ import { describeImageWithModel } from "../image.js";
export const minimaxProvider: MediaUnderstandingProvider = {
id: "minimax",
capabilities: ["image"],
describeImage: describeImageWithModel,
};
@@ -4,7 +4,6 @@ import { transcribeOpenAiCompatibleAudio } from "./audio.js";
export const openaiProvider: MediaUnderstandingProvider = {
id: "openai",
capabilities: ["image"],
describeImage: describeImageWithModel,
transcribeAudio: transcribeOpenAiCompatibleAudio,
};
-136
View File
@@ -1,136 +0,0 @@
import { describe, expect, it } from "vitest";
import type { ClawdbotConfig } from "../config/config.js";
import {
resolveEntriesWithActiveFallback,
resolveModelEntries,
} from "./resolve.js";
const providerRegistry = new Map([
["openai", { capabilities: ["image"] }],
["groq", { capabilities: ["audio"] }],
]);
describe("resolveModelEntries", () => {
it("uses provider capabilities for shared entries without explicit caps", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
models: [{ provider: "openai", model: "gpt-5.2" }],
},
},
};
const imageEntries = resolveModelEntries({
cfg,
capability: "image",
providerRegistry,
});
expect(imageEntries).toHaveLength(1);
const audioEntries = resolveModelEntries({
cfg,
capability: "audio",
providerRegistry,
});
expect(audioEntries).toHaveLength(0);
});
it("keeps per-capability entries even without explicit caps", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
image: {
models: [{ provider: "openai", model: "gpt-5.2" }],
},
},
},
};
const imageEntries = resolveModelEntries({
cfg,
capability: "image",
config: cfg.tools?.media?.image,
providerRegistry,
});
expect(imageEntries).toHaveLength(1);
});
it("skips shared CLI entries without capabilities", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
models: [{ type: "cli", command: "gemini", args: ["--file", "{{MediaPath}}"] }],
},
},
};
const entries = resolveModelEntries({
cfg,
capability: "image",
providerRegistry,
});
expect(entries).toHaveLength(0);
});
});
describe("resolveEntriesWithActiveFallback", () => {
it("uses active model when enabled and no models are configured", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
audio: { enabled: true },
},
},
};
const entries = resolveEntriesWithActiveFallback({
cfg,
capability: "audio",
config: cfg.tools?.media?.audio,
providerRegistry,
activeModel: { provider: "groq", model: "whisper-large-v3" },
});
expect(entries).toHaveLength(1);
expect(entries[0]?.provider).toBe("groq");
});
it("ignores active model when configured entries exist", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
audio: { enabled: true, models: [{ provider: "openai", model: "whisper-1" }] },
},
},
};
const entries = resolveEntriesWithActiveFallback({
cfg,
capability: "audio",
config: cfg.tools?.media?.audio,
providerRegistry,
activeModel: { provider: "groq", model: "whisper-large-v3" },
});
expect(entries).toHaveLength(1);
expect(entries[0]?.provider).toBe("openai");
});
it("skips active model when provider lacks capability", () => {
const cfg: ClawdbotConfig = {
tools: {
media: {
video: { enabled: true },
},
},
};
const entries = resolveEntriesWithActiveFallback({
cfg,
capability: "video",
config: cfg.tools?.media?.video,
providerRegistry,
activeModel: { provider: "groq", model: "whisper-large-v3" },
});
expect(entries).toHaveLength(0);
});
});
+32 -37
View File
@@ -77,22 +77,36 @@ export function resolveScopeDecision(params: {
});
}
function resolveEntryCapabilities(params: {
entry: MediaUnderstandingModelConfig;
providerRegistry: Map<string, { capabilities?: MediaUnderstandingCapability[] }>;
}): MediaUnderstandingCapability[] | undefined {
const entryType = params.entry.type ?? (params.entry.command ? "cli" : "provider");
if (entryType === "cli") return undefined;
const providerId = normalizeMediaProviderId(params.entry.provider ?? "");
if (!providerId) return undefined;
return params.providerRegistry.get(providerId)?.capabilities;
export function inferProviderCapabilities(
providerId?: string,
): MediaUnderstandingCapability[] | undefined {
const provider = normalizeMediaProviderId(providerId ?? "");
if (!provider) return undefined;
if (provider === "openai" || provider === "anthropic" || provider === "minimax") {
return ["image"];
}
if (provider === "google") {
return ["image", "audio", "video"];
}
if (provider === "groq") {
return ["audio"];
}
return undefined;
}
function inferCapabilities(
entry: MediaUnderstandingModelConfig,
): MediaUnderstandingCapability[] | undefined {
if ((entry.type ?? (entry.command ? "cli" : "provider")) === "cli") {
return undefined;
}
return inferProviderCapabilities(entry.provider);
}
export function resolveModelEntries(params: {
cfg: ClawdbotConfig;
capability: MediaUnderstandingCapability;
config?: MediaUnderstandingConfig;
providerRegistry: Map<string, { capabilities?: MediaUnderstandingCapability[] }>;
}): MediaUnderstandingModelConfig[] {
const { cfg, capability, config } = params;
const sharedModels = cfg.tools?.media?.models ?? [];
@@ -108,7 +122,7 @@ export function resolveModelEntries(params: {
entry.capabilities && entry.capabilities.length > 0
? entry.capabilities
: source === "shared"
? resolveEntryCapabilities({ entry, providerRegistry: params.providerRegistry })
? inferCapabilities(entry)
: undefined;
if (!caps || caps.length === 0) {
if (source === "shared") {
@@ -134,32 +148,13 @@ export function resolveConcurrency(cfg: ClawdbotConfig): number {
return DEFAULT_MEDIA_CONCURRENCY;
}
export function resolveEntriesWithActiveFallback(params: {
export function resolveCapabilityEnabled(params: {
cfg: ClawdbotConfig;
capability: MediaUnderstandingCapability;
config?: MediaUnderstandingConfig;
providerRegistry: Map<string, { capabilities?: MediaUnderstandingCapability[] }>;
activeModel?: { provider: string; model?: string };
}): MediaUnderstandingModelConfig[] {
const entries = resolveModelEntries({
cfg: params.cfg,
capability: params.capability,
config: params.config,
providerRegistry: params.providerRegistry,
});
if (entries.length > 0) return entries;
if (params.config?.enabled !== true) return entries;
const activeProviderRaw = params.activeModel?.provider?.trim();
if (!activeProviderRaw) return entries;
const activeProvider = normalizeMediaProviderId(activeProviderRaw);
if (!activeProvider) return entries;
const capabilities = params.providerRegistry.get(activeProvider)?.capabilities;
if (!capabilities || !capabilities.includes(params.capability)) return entries;
return [
{
type: "provider",
provider: activeProvider,
model: params.activeModel?.model,
},
];
}): boolean {
if (params.config?.enabled === false) return false;
const sharedModels = params.cfg.tools?.media?.models ?? [];
const hasModels = (params.config?.models?.length ?? 0) > 0 || sharedModels.length > 0;
if (!hasModels) return false;
return true;
}
-527
View File
@@ -1,527 +0,0 @@
import type { ClawdbotConfig } from "../config/config.js";
import type { MsgContext } from "../auto-reply/templating.js";
import { applyTemplate } from "../auto-reply/templating.js";
import { resolveApiKeyForProvider } from "../agents/model-auth.js";
import { logVerbose, shouldLogVerbose } from "../globals.js";
import { runExec } from "../process/exec.js";
import type {
MediaUnderstandingConfig,
MediaUnderstandingModelConfig,
} from "../config/types.tools.js";
import { MediaAttachmentCache, normalizeAttachments, selectAttachments } from "./attachments.js";
import {
CLI_OUTPUT_MAX_BUFFER,
DEFAULT_AUDIO_MODELS,
DEFAULT_TIMEOUT_SECONDS,
} from "./defaults.js";
import { isMediaUnderstandingSkipError, MediaUnderstandingSkipError } from "./errors.js";
import {
resolveEntriesWithActiveFallback,
resolveMaxBytes,
resolveMaxChars,
resolvePrompt,
resolveScopeDecision,
resolveTimeoutMs,
} from "./resolve.js";
import type {
MediaAttachment,
MediaUnderstandingCapability,
MediaUnderstandingDecision,
MediaUnderstandingModelDecision,
MediaUnderstandingOutput,
MediaUnderstandingProvider,
} from "./types.js";
import {
buildMediaUnderstandingRegistry,
getMediaUnderstandingProvider,
normalizeMediaProviderId,
} from "./providers/index.js";
import { describeImageWithModel } from "./providers/image.js";
import { estimateBase64Size, resolveVideoMaxBase64Bytes } from "./video.js";
export type ActiveMediaModel = {
provider: string;
model?: string;
};
type ProviderRegistry = Map<string, MediaUnderstandingProvider>;
export type RunCapabilityResult = {
outputs: MediaUnderstandingOutput[];
decision: MediaUnderstandingDecision;
};
export function buildProviderRegistry(
overrides?: Record<string, MediaUnderstandingProvider>,
): ProviderRegistry {
return buildMediaUnderstandingRegistry(overrides);
}
export function normalizeMediaAttachments(ctx: MsgContext): MediaAttachment[] {
return normalizeAttachments(ctx);
}
export function createMediaAttachmentCache(attachments: MediaAttachment[]): MediaAttachmentCache {
return new MediaAttachmentCache(attachments);
}
function trimOutput(text: string, maxChars?: number): string {
const trimmed = text.trim();
if (!maxChars || trimmed.length <= maxChars) return trimmed;
return trimmed.slice(0, maxChars).trim();
}
function buildModelDecision(params: {
entry: MediaUnderstandingModelConfig;
entryType: "provider" | "cli";
outcome: MediaUnderstandingModelDecision["outcome"];
reason?: string;
}): MediaUnderstandingModelDecision {
if (params.entryType === "cli") {
const command = params.entry.command?.trim();
return {
type: "cli",
provider: command ?? "cli",
model: params.entry.model ?? command,
outcome: params.outcome,
reason: params.reason,
};
}
const providerIdRaw = params.entry.provider?.trim();
const providerId = providerIdRaw ? normalizeMediaProviderId(providerIdRaw) : undefined;
return {
type: "provider",
provider: providerId ?? providerIdRaw,
model: params.entry.model,
outcome: params.outcome,
reason: params.reason,
};
}
function formatDecisionSummary(decision: MediaUnderstandingDecision): string {
const total = decision.attachments.length;
const success = decision.attachments.filter((entry) => entry.chosen?.outcome === "success").length;
const chosen = decision.attachments.find((entry) => entry.chosen)?.chosen;
const provider = chosen?.provider?.trim();
const model = chosen?.model?.trim();
const modelLabel = provider ? (model ? `${provider}/${model}` : provider) : undefined;
const reason = decision.attachments
.flatMap((entry) => entry.attempts.map((attempt) => attempt.reason).filter(Boolean))
.find(Boolean);
const shortReason = reason ? reason.split(":")[0]?.trim() : undefined;
const countLabel = total > 0 ? ` (${success}/${total})` : "";
const viaLabel = modelLabel ? ` via ${modelLabel}` : "";
const reasonLabel = shortReason ? ` reason=${shortReason}` : "";
return `${decision.capability}: ${decision.outcome}${countLabel}${viaLabel}${reasonLabel}`;
}
async function runProviderEntry(params: {
capability: MediaUnderstandingCapability;
entry: MediaUnderstandingModelConfig;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
cache: MediaAttachmentCache;
agentDir?: string;
providerRegistry: ProviderRegistry;
config?: MediaUnderstandingConfig;
}): Promise<MediaUnderstandingOutput | null> {
const { entry, capability, cfg } = params;
const providerIdRaw = entry.provider?.trim();
if (!providerIdRaw) {
throw new Error(`Provider entry missing provider for ${capability}`);
}
const providerId = normalizeMediaProviderId(providerIdRaw);
const maxBytes = resolveMaxBytes({ capability, entry, cfg, config: params.config });
const maxChars = resolveMaxChars({ capability, entry, cfg, config: params.config });
const timeoutMs = resolveTimeoutMs(
entry.timeoutSeconds ??
params.config?.timeoutSeconds ??
cfg.tools?.media?.[capability]?.timeoutSeconds,
DEFAULT_TIMEOUT_SECONDS[capability],
);
const prompt = resolvePrompt(
capability,
entry.prompt ?? params.config?.prompt ?? cfg.tools?.media?.[capability]?.prompt,
maxChars,
);
if (capability === "image") {
if (!params.agentDir) {
throw new Error("Image understanding requires agentDir");
}
const modelId = entry.model?.trim();
if (!modelId) {
throw new Error("Image understanding requires model id");
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const provider = getMediaUnderstandingProvider(providerId, params.providerRegistry);
const result = provider?.describeImage
? await provider.describeImage({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
model: modelId,
provider: providerId,
prompt,
timeoutMs,
profile: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
cfg: params.cfg,
})
: await describeImageWithModel({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
model: modelId,
provider: providerId,
prompt,
timeoutMs,
profile: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
cfg: params.cfg,
});
return {
kind: "image.description",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? modelId,
};
}
const provider = getMediaUnderstandingProvider(providerId, params.providerRegistry);
if (!provider) {
throw new Error(`Media provider not available: ${providerId}`);
}
if (capability === "audio") {
if (!provider.transcribeAudio) {
throw new Error(`Audio transcription provider "${providerId}" not available.`);
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const key = await resolveApiKeyForProvider({
provider: providerId,
cfg,
profileId: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
});
const providerConfig = cfg.models?.providers?.[providerId];
const model = entry.model?.trim() || DEFAULT_AUDIO_MODELS[providerId] || entry.model;
const result = await provider.transcribeAudio({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
apiKey: key.apiKey,
baseUrl: providerConfig?.baseUrl,
headers: providerConfig?.headers,
model,
language: entry.language ?? params.config?.language ?? cfg.tools?.media?.audio?.language,
prompt,
timeoutMs,
});
return {
kind: "audio.transcription",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? model,
};
}
if (!provider.describeVideo) {
throw new Error(`Video understanding provider "${providerId}" not available.`);
}
const media = await params.cache.getBuffer({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const estimatedBase64Bytes = estimateBase64Size(media.size);
const maxBase64Bytes = resolveVideoMaxBase64Bytes(maxBytes);
if (estimatedBase64Bytes > maxBase64Bytes) {
throw new MediaUnderstandingSkipError(
"maxBytes",
`Video attachment ${params.attachmentIndex + 1} base64 payload ${estimatedBase64Bytes} exceeds ${maxBase64Bytes}`,
);
}
const key = await resolveApiKeyForProvider({
provider: providerId,
cfg,
profileId: entry.profile,
preferredProfile: entry.preferredProfile,
agentDir: params.agentDir,
});
const providerConfig = cfg.models?.providers?.[providerId];
const result = await provider.describeVideo({
buffer: media.buffer,
fileName: media.fileName,
mime: media.mime,
apiKey: key.apiKey,
baseUrl: providerConfig?.baseUrl,
headers: providerConfig?.headers,
model: entry.model,
prompt,
timeoutMs,
});
return {
kind: "video.description",
attachmentIndex: params.attachmentIndex,
text: trimOutput(result.text, maxChars),
provider: providerId,
model: result.model ?? entry.model,
};
}
async function runCliEntry(params: {
capability: MediaUnderstandingCapability;
entry: MediaUnderstandingModelConfig;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
cache: MediaAttachmentCache;
config?: MediaUnderstandingConfig;
}): Promise<MediaUnderstandingOutput | null> {
const { entry, capability, cfg, ctx } = params;
const command = entry.command?.trim();
const args = entry.args ?? [];
if (!command) {
throw new Error(`CLI entry missing command for ${capability}`);
}
const maxBytes = resolveMaxBytes({ capability, entry, cfg, config: params.config });
const maxChars = resolveMaxChars({ capability, entry, cfg, config: params.config });
const timeoutMs = resolveTimeoutMs(
entry.timeoutSeconds ??
params.config?.timeoutSeconds ??
cfg.tools?.media?.[capability]?.timeoutSeconds,
DEFAULT_TIMEOUT_SECONDS[capability],
);
const prompt = resolvePrompt(
capability,
entry.prompt ?? params.config?.prompt ?? cfg.tools?.media?.[capability]?.prompt,
maxChars,
);
const pathResult = await params.cache.getPath({
attachmentIndex: params.attachmentIndex,
maxBytes,
timeoutMs,
});
const templCtx: MsgContext = {
...ctx,
MediaPath: pathResult.path,
Prompt: prompt,
MaxChars: maxChars,
};
const argv = [command, ...args].map((part, index) =>
index === 0 ? part : applyTemplate(part, templCtx),
);
if (shouldLogVerbose()) {
logVerbose(`Media understanding via CLI: ${argv.join(" ")}`);
}
const { stdout } = await runExec(argv[0], argv.slice(1), {
timeoutMs,
maxBuffer: CLI_OUTPUT_MAX_BUFFER,
});
const text = trimOutput(stdout, maxChars);
if (!text) return null;
return {
kind: capability === "audio" ? "audio.transcription" : `${capability}.description`,
attachmentIndex: params.attachmentIndex,
text,
provider: "cli",
model: command,
};
}
async function runAttachmentEntries(params: {
capability: MediaUnderstandingCapability;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachmentIndex: number;
agentDir?: string;
providerRegistry: ProviderRegistry;
cache: MediaAttachmentCache;
entries: MediaUnderstandingModelConfig[];
config?: MediaUnderstandingConfig;
}): Promise<{ output: MediaUnderstandingOutput | null; attempts: MediaUnderstandingModelDecision[] }> {
const { entries, capability } = params;
const attempts: MediaUnderstandingModelDecision[] = [];
for (const entry of entries) {
const entryType = entry.type ?? (entry.command ? "cli" : "provider");
try {
const result =
entryType === "cli"
? await runCliEntry({
capability,
entry,
cfg: params.cfg,
ctx: params.ctx,
attachmentIndex: params.attachmentIndex,
cache: params.cache,
config: params.config,
})
: await runProviderEntry({
capability,
entry,
cfg: params.cfg,
ctx: params.ctx,
attachmentIndex: params.attachmentIndex,
cache: params.cache,
agentDir: params.agentDir,
providerRegistry: params.providerRegistry,
config: params.config,
});
if (result) {
const decision = buildModelDecision({ entry, entryType, outcome: "success" });
if (result.provider) decision.provider = result.provider;
if (result.model) decision.model = result.model;
attempts.push(decision);
return { output: result, attempts };
}
attempts.push(
buildModelDecision({ entry, entryType, outcome: "skipped", reason: "empty output" }),
);
} catch (err) {
if (isMediaUnderstandingSkipError(err)) {
attempts.push(
buildModelDecision({
entry,
entryType,
outcome: "skipped",
reason: `${err.reason}: ${err.message}`,
}),
);
if (shouldLogVerbose()) {
logVerbose(`Skipping ${capability} model due to ${err.reason}: ${err.message}`);
}
continue;
}
attempts.push(
buildModelDecision({
entry,
entryType,
outcome: "failed",
reason: String(err),
}),
);
if (shouldLogVerbose()) {
logVerbose(`${capability} understanding failed: ${String(err)}`);
}
}
}
return { output: null, attempts };
}
export async function runCapability(params: {
capability: MediaUnderstandingCapability;
cfg: ClawdbotConfig;
ctx: MsgContext;
attachments: MediaAttachmentCache;
media: MediaAttachment[];
agentDir?: string;
providerRegistry: ProviderRegistry;
config?: MediaUnderstandingConfig;
activeModel?: ActiveMediaModel;
}): Promise<RunCapabilityResult> {
const { capability, cfg, ctx } = params;
const config = params.config ?? cfg.tools?.media?.[capability];
if (config?.enabled === false) {
return {
outputs: [],
decision: { capability, outcome: "disabled", attachments: [] },
};
}
const attachmentPolicy = config?.attachments;
const selected = selectAttachments({
capability,
attachments: params.media,
policy: attachmentPolicy,
});
if (selected.length === 0) {
return {
outputs: [],
decision: { capability, outcome: "no-attachment", attachments: [] },
};
}
const scopeDecision = resolveScopeDecision({ scope: config?.scope, ctx });
if (scopeDecision === "deny") {
if (shouldLogVerbose()) {
logVerbose(`${capability} understanding disabled by scope policy.`);
}
return {
outputs: [],
decision: {
capability,
outcome: "scope-deny",
attachments: selected.map((item) => ({ attachmentIndex: item.index, attempts: [] })),
},
};
}
const entries = resolveEntriesWithActiveFallback({
cfg,
capability,
config,
providerRegistry: params.providerRegistry,
activeModel: params.activeModel,
});
if (entries.length === 0) {
return {
outputs: [],
decision: {
capability,
outcome: "skipped",
attachments: selected.map((item) => ({ attachmentIndex: item.index, attempts: [] })),
},
};
}
const outputs: MediaUnderstandingOutput[] = [];
const attachmentDecisions: MediaUnderstandingDecision["attachments"] = [];
for (const attachment of selected) {
const { output, attempts } = await runAttachmentEntries({
capability,
cfg,
ctx,
attachmentIndex: attachment.index,
agentDir: params.agentDir,
providerRegistry: params.providerRegistry,
cache: params.attachments,
entries,
config,
});
if (output) outputs.push(output);
attachmentDecisions.push({
attachmentIndex: attachment.index,
attempts,
chosen: attempts.find((attempt) => attempt.outcome === "success"),
});
}
const decision: MediaUnderstandingDecision = {
capability,
outcome: outputs.length > 0 ? "success" : "skipped",
attachments: attachmentDecisions,
};
if (shouldLogVerbose()) {
logVerbose(`Media understanding ${formatDecisionSummary(decision)}`);
}
return {
outputs,
decision,
};
}
-1
View File
@@ -106,7 +106,6 @@ export type ImageDescriptionResult = {
export type MediaUnderstandingProvider = {
id: string;
capabilities?: MediaUnderstandingCapability[];
transcribeAudio?: (req: AudioTranscriptionRequest) => Promise<AudioTranscriptionResult>;
describeVideo?: (req: VideoDescriptionRequest) => Promise<VideoDescriptionResult>;
describeImage?: (req: ImageDescriptionRequest) => Promise<ImageDescriptionResult>;
+1 -1
View File
@@ -405,7 +405,7 @@ export async function runOnboardingWizard(
nextConfig = await setupSkills(nextConfig, workspaceDir, runtime, prompter);
}
// Setup hooks (session memory on /new)
// Setup internal hooks (session memory on /new)
nextConfig = await setupInternalHooks(nextConfig, runtime, prompter);
nextConfig = applyWizardMetadata(nextConfig, { command: "onboard", mode });