Compare commits

..
Author SHA1 Message Date
Peter Steinberger 63ac6ccd20 fix: resolve TwiML/status callbacks (#1180) (thanks @andrew-kurin) 2026-01-18 16:34:18 +00:00
Ghost 340a054f73 fix(voice-call): resolve StatusCallback with inline TwiML
- Switch from inline to URL-based TwiML for outbound calls
- Store TwiML content temporarily and serve on webhook request
- Add twimlStorage map and cleanup helper methods
- Fix TwiML serving to handle CallStatus='in-progress' on initial request

Closes #864
2026-01-18 16:11:19 +00:00
Peter Steinberger 810394f43b fix: improve remote bin probe logging 2026-01-18 16:09:48 +00:00
Peter Steinberger 835162fb62 fix: retry openai batch indexing 2026-01-18 16:08:22 +00:00
Peter Steinberger 82883095fe docs: explain Copilot provider options 2026-01-18 16:06:48 +00:00
Peter Steinberger 49d8ad3049 feat: surface node core/ui versions in macOS 2026-01-18 16:00:36 +00:00
Peter Steinberger 1721d04405 feat: add node core/ui versions in bridge 2026-01-18 15:59:54 +00:00
Peter Steinberger 633e0d9382 Merge pull request #1164 from ngutman/feat/boot-md
feat(hooks): run BOOT.md on gateway startup
2026-01-18 15:59:53 +00:00
Peter Steinberger f06ce98312 refactor: rename lancedb memory plugin 2026-01-18 15:48:05 +00:00
Nimrod Gutman 11b07f4a29 feat(hooks): run boot.md on gateway startup 2026-01-18 11:50:25 +02:00
32 changed files with 737 additions and 87 deletions

No files matched your search

+41 -2
View File
@@ -19,13 +19,52 @@ Docs: https://docs.clawd.bot
- Memory: add native Gemini embeddings provider for memory search. (#1151) - Memory: add native Gemini embeddings provider for memory search. (#1151)
- Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt. - Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt.
- Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne. - Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne.
- Nodes: report core/ui versions in node list + presence; surface both in CLI + macOS UI.
<<<<<<< HEAD
- Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt.
- Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne.
||||||| parent of 903e9be49 (feat: surface node core/ui versions in macOS)
### Fixes
- Auth profiles: keep auto-pinned preference while allowing rotation on failover; user pins stay locked. (#1138) — thanks @cheeeee.
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Voice call: separate TwiML fetches from status callbacks to avoid stuck calls. (#1180) — thanks @andrew-kurin.
## 2026.1.18-5
### Changes
- Dependencies: update core + plugin deps (grammy, vitest, openai, Microsoft agents hosting, etc.).
## 2026.1.18-3
### Changes
=======
- Nodes: report core/ui versions in node list + presence; surface both in CLI + macOS UI.
### Fixes
- Auth profiles: keep auto-pinned preference while allowing rotation on failover; user pins stay locked. (#1138) — thanks @cheeeee.
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Voice call: separate TwiML fetches from status callbacks to avoid stuck calls. (#1180) — thanks @andrew-kurin.
## 2026.1.18-5
### Changes
- Dependencies: update core + plugin deps (grammy, vitest, openai, Microsoft agents hosting, etc.).
## 2026.1.18-3
### Changes
>>>>>>> 903e9be49 (feat: surface node core/ui versions in macOS)
- Exec: add host/security/ask routing for gateway + node exec. - Exec: add host/security/ask routing for gateway + node exec.
- Exec: add `/exec` directive for per-session exec defaults (host/security/ask/node). - Exec: add `/exec` directive for per-session exec defaults (host/security/ask/node).
- macOS: migrate exec approvals to `~/.clawdbot/exec-approvals.json` with per-agent allowlists and skill auto-allow toggle. - macOS: migrate exec approvals to `~/.clawdbot/exec-approvals.json` with per-agent allowlists and skill auto-allow toggle.
- macOS: add approvals socket UI server + node exec lifecycle events. - macOS: add approvals socket UI server + node exec lifecycle events.
- Nodes: add headless node host (`clawdbot node start`) for `system.run`/`system.which`. - Nodes: add headless node host (`clawdbot node start`) for `system.run`/`system.which`.
- Nodes: add node daemon service install/status/start/stop/restart. - Nodes: add node daemon service install/status/start/stop/restart.
- Hooks: run `BOOT.md` on gateway startup with the boot-md hook. (#1164) — thanks @ngutman.
- Bridge: add `skills.bins` RPC to support node host auto-allow skill bins. - Bridge: add `skills.bins` RPC to support node host auto-allow skill bins.
- Slash commands: replace `/cost` with `/usage off|tokens|full` to control per-response usage footer; `/usage` no longer aliases `/status`. (Supersedes #1140) — thanks @Nachx639. - Slash commands: replace `/cost` with `/usage off|tokens|full` to control per-response usage footer; `/usage` no longer aliases `/status`. (Supersedes #1140) — thanks @Nachx639.
- Sessions: add daily reset policy with per-type overrides and idle windows (default 4am local), preserving legacy idle-only configs. (#1146) — thanks @austinm911. - Sessions: add daily reset policy with per-type overrides and idle windows (default 4am local), preserving legacy idle-only configs. (#1146) — thanks @austinm911.
@@ -47,8 +86,8 @@ Docs: https://docs.clawd.bot
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105) - macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151) - Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151) - Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Skills: improve remote bin probe logging with node labels + connectivity hints.
- Exec approvals: enforce allowlist when ask is off; prefer raw command for node approvals/events. - Exec approvals: enforce allowlist when ask is off; prefer raw command for node approvals/events.
- Exec approvals: parse command tokens correctly for PATH and relative resolution.
- Tools: return a companion-app-required message when node exec is requested with no paired node. - Tools: return a companion-app-required message when node exec is requested with no paired node.
- Streaming: emit assistant deltas for OpenAI-compatible SSE chunks. (#1147) — thanks @alauppe. - Streaming: emit assistant deltas for OpenAI-compatible SSE chunks. (#1147) — thanks @alauppe.
- Model fallback: treat timeout aborts as failover while preserving user aborts. (#1137) — thanks @cheeeee. - Model fallback: treat timeout aborts as failover while preserving user aborts. (#1137) — thanks @cheeeee.
@@ -8,6 +8,8 @@ struct BridgeNodeInfo: Sendable {
var displayName: String? var displayName: String?
var platform: String? var platform: String?
var version: String? var version: String?
var coreVersion: String?
var uiVersion: String?
var deviceFamily: String? var deviceFamily: String?
var modelIdentifier: String? var modelIdentifier: String?
var remoteAddress: String? var remoteAddress: String?
@@ -147,6 +149,8 @@ actor BridgeConnectionHandler {
displayName: hello.displayName, displayName: hello.displayName,
platform: hello.platform, platform: hello.platform,
version: hello.version, version: hello.version,
coreVersion: hello.coreVersion,
uiVersion: hello.uiVersion,
deviceFamily: hello.deviceFamily, deviceFamily: hello.deviceFamily,
modelIdentifier: hello.modelIdentifier, modelIdentifier: hello.modelIdentifier,
remoteAddress: self.remoteAddressString(), remoteAddress: self.remoteAddressString(),
@@ -171,6 +175,8 @@ actor BridgeConnectionHandler {
displayName: req.displayName, displayName: req.displayName,
platform: req.platform, platform: req.platform,
version: req.version, version: req.version,
coreVersion: req.coreVersion,
uiVersion: req.uiVersion,
deviceFamily: req.deviceFamily, deviceFamily: req.deviceFamily,
modelIdentifier: req.modelIdentifier, modelIdentifier: req.modelIdentifier,
caps: req.caps, caps: req.caps,
@@ -186,6 +192,8 @@ actor BridgeConnectionHandler {
displayName: enriched.displayName, displayName: enriched.displayName,
platform: enriched.platform, platform: enriched.platform,
version: enriched.version, version: enriched.version,
coreVersion: enriched.coreVersion,
uiVersion: enriched.uiVersion,
deviceFamily: enriched.deviceFamily, deviceFamily: enriched.deviceFamily,
modelIdentifier: enriched.modelIdentifier, modelIdentifier: enriched.modelIdentifier,
remoteAddress: enriched.remoteAddress, remoteAddress: enriched.remoteAddress,
@@ -249,6 +249,13 @@ actor GatewayConnection {
return trimmed.isEmpty ? nil : trimmed return trimmed.isEmpty ? nil : trimmed
} }
func cachedGatewayVersion() -> String? {
guard let snapshot = self.lastSnapshot else { return nil }
let raw = snapshot.server["version"]?.value as? String
let trimmed = raw?.trimmingCharacters(in: CharacterSet.whitespacesAndNewlines) ?? ""
return trimmed.isEmpty ? nil : trimmed
}
func snapshotPaths() -> (configPath: String?, stateDir: String?) { func snapshotPaths() -> (configPath: String?, stateDir: String?) {
guard let snapshot = self.lastSnapshot else { return (nil, nil) } guard let snapshot = self.lastSnapshot else { return (nil, nil) }
let configPath = snapshot.snapshot.configpath?.trimmingCharacters(in: .whitespacesAndNewlines) let configPath = snapshot.snapshot.configpath?.trimmingCharacters(in: .whitespacesAndNewlines)
@@ -747,8 +747,8 @@ extension MenuSessionsInjector {
menu.addItem(self.makeNodeCopyItem(label: "Platform", value: platform)) menu.addItem(self.makeNodeCopyItem(label: "Platform", value: platform))
} }
if let version = entry.version?.nonEmpty { if let version = NodeMenuEntryFormatter.detailRightVersion(entry)?.nonEmpty {
menu.addItem(self.makeNodeCopyItem(label: "Version", value: self.formatVersionLabel(version))) menu.addItem(self.makeNodeCopyItem(label: "Version", value: version))
} }
menu.addItem(self.makeNodeDetailItem(label: "Connected", value: entry.isConnected ? "Yes" : "No")) menu.addItem(self.makeNodeDetailItem(label: "Connected", value: entry.isConnected ? "Yes" : "No"))
@@ -95,6 +95,8 @@ actor MacNodeBridgePairingClient {
displayName: hello.displayName, displayName: hello.displayName,
platform: hello.platform, platform: hello.platform,
version: hello.version, version: hello.version,
coreVersion: hello.coreVersion,
uiVersion: hello.uiVersion,
deviceFamily: hello.deviceFamily, deviceFamily: hello.deviceFamily,
modelIdentifier: hello.modelIdentifier, modelIdentifier: hello.modelIdentifier,
caps: hello.caps, caps: hello.caps,
@@ -114,12 +114,19 @@ final class MacNodeModeCoordinator {
let caps = self.currentCaps() let caps = self.currentCaps()
let commands = self.currentCommands(caps: caps) let commands = self.currentCommands(caps: caps)
let permissions = await self.currentPermissions() let permissions = await self.currentPermissions()
let uiVersion = Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String
let liveGatewayVersion = await GatewayConnection.shared.cachedGatewayVersion()
let fallbackGatewayVersion = GatewayProcessManager.shared.environmentStatus.gatewayVersion
let coreVersion = (liveGatewayVersion ?? fallbackGatewayVersion)?
.trimmingCharacters(in: .whitespacesAndNewlines)
return BridgeHello( return BridgeHello(
nodeId: Self.nodeId(), nodeId: Self.nodeId(),
displayName: InstanceIdentity.displayName, displayName: InstanceIdentity.displayName,
token: token, token: token,
platform: "macos", platform: "macos",
version: Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String, version: uiVersion,
coreVersion: coreVersion?.isEmpty == false ? coreVersion : nil,
uiVersion: uiVersion,
deviceFamily: "Mac", deviceFamily: "Mac",
modelIdentifier: InstanceIdentity.modelIdentifier, modelIdentifier: InstanceIdentity.modelIdentifier,
caps: caps, caps: caps,
+39 -4
View File
@@ -35,8 +35,9 @@ struct NodeMenuEntryFormatter {
if let platform = self.platformText(entry) { if let platform = self.platformText(entry) {
parts.append("platform \(platform)") parts.append("platform \(platform)")
} }
if let version = entry.version?.nonEmpty { let versionLabels = self.versionLabels(entry)
parts.append("app \(self.compactVersion(version))") if !versionLabels.isEmpty {
parts.append(versionLabels.joined(separator: " · "))
} }
parts.append("status \(self.roleText(entry))") parts.append("status \(self.roleText(entry))")
return parts.joined(separator: " · ") return parts.joined(separator: " · ")
@@ -60,8 +61,9 @@ struct NodeMenuEntryFormatter {
} }
static func detailRightVersion(_ entry: NodeInfo) -> String? { static func detailRightVersion(_ entry: NodeInfo) -> String? {
guard let version = entry.version?.nonEmpty else { return nil } let labels = self.versionLabels(entry, compact: false)
return self.shortVersionLabel(version) if labels.isEmpty { return nil }
return labels.joined(separator: " · ")
} }
static func platformText(_ entry: NodeInfo) -> String? { static func platformText(_ entry: NodeInfo) -> String? {
@@ -127,6 +129,39 @@ struct NodeMenuEntryFormatter {
return compact return compact
} }
private static func versionLabels(_ entry: NodeInfo, compact: Bool = true) -> [String] {
let (core, ui) = self.resolveVersions(entry)
var labels: [String] = []
if let core {
let label = compact ? self.compactVersion(core) : self.shortVersionLabel(core)
labels.append("core \(label)")
}
if let ui {
let label = compact ? self.compactVersion(ui) : self.shortVersionLabel(ui)
labels.append("ui \(label)")
}
return labels
}
private static func resolveVersions(_ entry: NodeInfo) -> (core: String?, ui: String?) {
let core = entry.coreVersion?.nonEmpty
let ui = entry.uiVersion?.nonEmpty
if core != nil || ui != nil {
return (core, ui)
}
guard let legacy = entry.version?.nonEmpty else { return (nil, nil) }
if self.isHeadlessPlatform(entry) {
return (legacy, nil)
}
return (nil, legacy)
}
private static func isHeadlessPlatform(_ entry: NodeInfo) -> Bool {
let raw = entry.platform?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() ?? ""
if raw == "darwin" || raw == "linux" || raw == "win32" || raw == "windows" { return true }
return false
}
static func leadingSymbol(_ entry: NodeInfo) -> String { static func leadingSymbol(_ entry: NodeInfo) -> String {
if self.isGateway(entry) { if self.isGateway(entry) {
return self.safeSystemSymbol( return self.safeSystemSymbol(
@@ -7,6 +7,8 @@ struct NodeInfo: Identifiable, Codable {
let displayName: String? let displayName: String?
let platform: String? let platform: String?
let version: String? let version: String?
let coreVersion: String?
let uiVersion: String?
let deviceFamily: String? let deviceFamily: String?
let modelIdentifier: String? let modelIdentifier: String?
let remoteIp: String? let remoteIp: String?
@@ -530,6 +530,8 @@ public struct NodePairRequestParams: Codable, Sendable {
public let displayname: String? public let displayname: String?
public let platform: String? public let platform: String?
public let version: String? public let version: String?
public let coreversion: String?
public let uiversion: String?
public let devicefamily: String? public let devicefamily: String?
public let modelidentifier: String? public let modelidentifier: String?
public let caps: [String]? public let caps: [String]?
@@ -542,6 +544,8 @@ public struct NodePairRequestParams: Codable, Sendable {
displayname: String?, displayname: String?,
platform: String?, platform: String?,
version: String?, version: String?,
coreversion: String?,
uiversion: String?,
devicefamily: String?, devicefamily: String?,
modelidentifier: String?, modelidentifier: String?,
caps: [String]?, caps: [String]?,
@@ -553,6 +557,8 @@ public struct NodePairRequestParams: Codable, Sendable {
self.displayname = displayname self.displayname = displayname
self.platform = platform self.platform = platform
self.version = version self.version = version
self.coreversion = coreversion
self.uiversion = uiversion
self.devicefamily = devicefamily self.devicefamily = devicefamily
self.modelidentifier = modelidentifier self.modelidentifier = modelidentifier
self.caps = caps self.caps = caps
@@ -565,6 +571,8 @@ public struct NodePairRequestParams: Codable, Sendable {
case displayname = "displayName" case displayname = "displayName"
case platform case platform
case version case version
case coreversion = "coreVersion"
case uiversion = "uiVersion"
case devicefamily = "deviceFamily" case devicefamily = "deviceFamily"
case modelidentifier = "modelIdentifier" case modelidentifier = "modelIdentifier"
case caps case caps
@@ -63,6 +63,8 @@ public struct BridgeHello: Codable, Sendable {
public let token: String? public let token: String?
public let platform: String? public let platform: String?
public let version: String? public let version: String?
public let coreVersion: String?
public let uiVersion: String?
public let deviceFamily: String? public let deviceFamily: String?
public let modelIdentifier: String? public let modelIdentifier: String?
public let caps: [String]? public let caps: [String]?
@@ -76,6 +78,8 @@ public struct BridgeHello: Codable, Sendable {
token: String?, token: String?,
platform: String?, platform: String?,
version: String?, version: String?,
coreVersion: String? = nil,
uiVersion: String? = nil,
deviceFamily: String? = nil, deviceFamily: String? = nil,
modelIdentifier: String? = nil, modelIdentifier: String? = nil,
caps: [String]? = nil, caps: [String]? = nil,
@@ -88,6 +92,8 @@ public struct BridgeHello: Codable, Sendable {
self.token = token self.token = token
self.platform = platform self.platform = platform
self.version = version self.version = version
self.coreVersion = coreVersion
self.uiVersion = uiVersion
self.deviceFamily = deviceFamily self.deviceFamily = deviceFamily
self.modelIdentifier = modelIdentifier self.modelIdentifier = modelIdentifier
self.caps = caps self.caps = caps
@@ -121,6 +127,8 @@ public struct BridgePairRequest: Codable, Sendable {
public let displayName: String? public let displayName: String?
public let platform: String? public let platform: String?
public let version: String? public let version: String?
public let coreVersion: String?
public let uiVersion: String?
public let deviceFamily: String? public let deviceFamily: String?
public let modelIdentifier: String? public let modelIdentifier: String?
public let caps: [String]? public let caps: [String]?
@@ -135,6 +143,8 @@ public struct BridgePairRequest: Codable, Sendable {
displayName: String?, displayName: String?,
platform: String?, platform: String?,
version: String?, version: String?,
coreVersion: String? = nil,
uiVersion: String? = nil,
deviceFamily: String? = nil, deviceFamily: String? = nil,
modelIdentifier: String? = nil, modelIdentifier: String? = nil,
caps: [String]? = nil, caps: [String]? = nil,
@@ -148,6 +158,8 @@ public struct BridgePairRequest: Codable, Sendable {
self.displayName = displayName self.displayName = displayName
self.platform = platform self.platform = platform
self.version = version self.version = version
self.coreVersion = coreVersion
self.uiVersion = uiVersion
self.deviceFamily = deviceFamily self.deviceFamily = deviceFamily
self.modelIdentifier = modelIdentifier self.modelIdentifier = modelIdentifier
self.caps = caps self.caps = caps
+2 -1
View File
@@ -37,6 +37,7 @@ See [Voice Call](/plugins/voice-call) for a concrete example plugin.
- Microsoft Teams is plugin-only as of 2026.1.15; install `@clawdbot/msteams` if you use Teams. - Microsoft Teams is plugin-only as of 2026.1.15; install `@clawdbot/msteams` if you use Teams.
- Memory (Core) — bundled memory search plugin (enabled by default via `plugins.slots.memory`) - Memory (Core) — bundled memory search plugin (enabled by default via `plugins.slots.memory`)
- Memory (LanceDB) — bundled long-term memory plugin (auto-recall/capture; set `plugins.slots.memory = "memory-lancedb"`)
- [Voice Call](/plugins/voice-call) — `@clawdbot/voice-call` - [Voice Call](/plugins/voice-call) — `@clawdbot/voice-call`
- [Zalo Personal](/plugins/zalouser) — `@clawdbot/zalouser` - [Zalo Personal](/plugins/zalouser) — `@clawdbot/zalouser`
- [Matrix](/channels/matrix) — `@clawdbot/matrix` - [Matrix](/channels/matrix) — `@clawdbot/matrix`
@@ -45,7 +46,7 @@ See [Voice Call](/plugins/voice-call) for a concrete example plugin.
- Google Antigravity OAuth (provider auth) — bundled as `google-antigravity-auth` (disabled by default) - Google Antigravity OAuth (provider auth) — bundled as `google-antigravity-auth` (disabled by default)
- Gemini CLI OAuth (provider auth) — bundled as `google-gemini-cli-auth` (disabled by default) - Gemini CLI OAuth (provider auth) — bundled as `google-gemini-cli-auth` (disabled by default)
- Qwen OAuth (provider auth) — bundled as `qwen-portal-auth` (disabled by default) - Qwen OAuth (provider auth) — bundled as `qwen-portal-auth` (disabled by default)
- Copilot Proxy (provider auth) — bundled as `copilot-proxy` (disabled by default) - Copilot Proxy (provider auth) — local VS Code Copilot Proxy bridge; distinct from built-in `github-copilot` device login (bundled, disabled by default)
Clawdbot plugins are **TypeScript modules** loaded at runtime via jiti. They can Clawdbot plugins are **TypeScript modules** loaded at runtime via jiti. They can
register: register:
+21
View File
@@ -6,6 +6,27 @@ read_when:
--- ---
# Github Copilot # Github Copilot
## What is GitHub Copilot?
GitHub Copilot is GitHub's AI coding assistant. It provides access to Copilot
models for your GitHub account and plan. Clawdbot can use Copilot as a model
provider in two different ways.
## Two ways to use Copilot in Clawdbot
### 1) Built-in GitHub Copilot provider (`github-copilot`)
Use the native device-login flow to obtain a GitHub token, then exchange it for
Copilot API tokens when Clawdbot runs. This is the **default** and simplest path
because it does not require VS Code.
### 2) Copilot Proxy plugin (`copilot-proxy`)
Use the **Copilot Proxy** VS Code extension as a local bridge. Clawdbot talks to
the proxy’s `/v1` endpoint and uses the model list you configure there. Choose
this when you already run Copilot Proxy in VS Code or need to route through it.
You must enable the plugin and keep the VS Code extension running.
Use GitHub Copilot as a model provider (`github-copilot`). The login command runs Use GitHub Copilot as a model provider (`github-copilot`). The login command runs
the GitHub device flow, saves an auth profile, and updates your config to use that the GitHub device flow, saves an auth profile, and updates your config to use that
profile. profile.
+4 -4
View File
@@ -37,8 +37,8 @@ describeWithKey("memory plugin e2e", () => {
// Dynamic import to avoid loading LanceDB when not testing // Dynamic import to avoid loading LanceDB when not testing
const { default: memoryPlugin } = await import("./index.js"); const { default: memoryPlugin } = await import("./index.js");
expect(memoryPlugin.id).toBe("memory"); expect(memoryPlugin.id).toBe("memory-lancedb");
expect(memoryPlugin.name).toBe("Memory (Vector)"); expect(memoryPlugin.name).toBe("Memory (LanceDB)");
expect(memoryPlugin.kind).toBe("memory"); expect(memoryPlugin.kind).toBe("memory");
expect(memoryPlugin.configSchema).toBeDefined(); expect(memoryPlugin.configSchema).toBeDefined();
expect(memoryPlugin.register).toBeInstanceOf(Function); expect(memoryPlugin.register).toBeInstanceOf(Function);
@@ -185,8 +185,8 @@ describeWithKey("memory plugin live tests", () => {
const logs: string[] = []; const logs: string[] = [];
const mockApi = { const mockApi = {
id: "memory", id: "memory-lancedb",
name: "Memory (Vector)", name: "Memory (LanceDB)",
source: "test", source: "test",
config: {}, config: {},
pluginConfig: { pluginConfig: {
+15 -13
View File
@@ -1,5 +1,5 @@
/** /**
* Clawdbot Memory Plugin * Clawdbot Memory (LanceDB) Plugin
* *
* Long-term memory with vector search for AI conversations. * Long-term memory with vector search for AI conversations.
* Uses LanceDB for storage and OpenAI for embeddings. * Uses LanceDB for storage and OpenAI for embeddings.
@@ -214,9 +214,9 @@ function detectCategory(text: string): MemoryCategory {
// ============================================================================ // ============================================================================
const memoryPlugin = { const memoryPlugin = {
id: "memory", id: "memory-lancedb",
name: "Memory (Vector)", name: "Memory (LanceDB)",
description: "Long-term memory with vector search and seamless auto-recall/capture", description: "LanceDB-backed long-term memory with auto-recall/capture",
kind: "memory" as const, kind: "memory" as const,
configSchema: memoryConfigSchema, configSchema: memoryConfigSchema,
@@ -227,7 +227,9 @@ const memoryPlugin = {
const db = new MemoryDB(resolvedDbPath, vectorDim); const db = new MemoryDB(resolvedDbPath, vectorDim);
const embeddings = new Embeddings(cfg.embedding.apiKey, cfg.embedding.model!); const embeddings = new Embeddings(cfg.embedding.apiKey, cfg.embedding.model!);
api.logger.info(`memory: plugin registered (db: ${resolvedDbPath}, lazy init)`); api.logger.info(
`memory-lancedb: plugin registered (db: ${resolvedDbPath}, lazy init)`,
);
// ======================================================================== // ========================================================================
// Tools // Tools
@@ -417,7 +419,7 @@ const memoryPlugin = {
({ program }) => { ({ program }) => {
const memory = program const memory = program
.command("ltm") .command("ltm")
.description("Long-term memory plugin commands"); .description("LanceDB memory plugin commands");
memory memory
.command("list") .command("list")
@@ -477,14 +479,14 @@ const memoryPlugin = {
.join("\n"); .join("\n");
api.logger.info?.( api.logger.info?.(
`memory: injecting ${results.length} memories into context`, `memory-lancedb: injecting ${results.length} memories into context`,
); );
return { return {
prependContext: `<relevant-memories>\nThe following memories may be relevant to this conversation:\n${memoryContext}\n</relevant-memories>`, prependContext: `<relevant-memories>\nThe following memories may be relevant to this conversation:\n${memoryContext}\n</relevant-memories>`,
}; };
} catch (err) { } catch (err) {
api.logger.warn(`memory: recall failed: ${String(err)}`); api.logger.warn(`memory-lancedb: recall failed: ${String(err)}`);
} }
}); });
} }
@@ -559,10 +561,10 @@ const memoryPlugin = {
} }
if (stored > 0) { if (stored > 0) {
api.logger.info(`memory: auto-captured ${stored} memories`); api.logger.info(`memory-lancedb: auto-captured ${stored} memories`);
} }
} catch (err) { } catch (err) {
api.logger.warn(`memory: capture failed: ${String(err)}`); api.logger.warn(`memory-lancedb: capture failed: ${String(err)}`);
} }
}); });
} }
@@ -572,14 +574,14 @@ const memoryPlugin = {
// ======================================================================== // ========================================================================
api.registerService({ api.registerService({
id: "memory", id: "memory-lancedb",
start: () => { start: () => {
api.logger.info( api.logger.info(
`memory: initialized (db: ${resolvedDbPath}, model: ${cfg.embedding.model})`, `memory-lancedb: initialized (db: ${resolvedDbPath}, model: ${cfg.embedding.model})`,
); );
}, },
stop: () => { stop: () => {
api.logger.info("memory: stopped"); api.logger.info("memory-lancedb: stopped");
}, },
}); });
}, },
+2 -2
View File
@@ -1,8 +1,8 @@
{ {
"name": "@clawdbot/memory", "name": "@clawdbot/memory-lancedb",
"version": "0.0.1", "version": "0.0.1",
"type": "module", "type": "module",
"description": "Clawdbot long-term memory plugin with vector search and seamless auto-recall/capture", "description": "Clawdbot LanceDB-backed long-term memory plugin with auto-recall/capture",
"dependencies": { "dependencies": {
"@lancedb/lancedb": "^0.15.0", "@lancedb/lancedb": "^0.15.0",
"@sinclair/typebox": "0.34.47", "@sinclair/typebox": "0.34.47",
@@ -0,0 +1,92 @@
import { describe, expect, it } from "vitest";
import { TwilioProvider } from "./twilio.js";
const mockTwilioConfig = {
accountSid: "AC00000000000000000000000000000000",
authToken: "test-token",
};
const callId = "internal-call-id";
const twimlPayload =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><Response><Say>Hi</Say></Response>";
describe("TwilioProvider", () => {
it("serves stored TwiML on twiml requests without emitting events", async () => {
const provider = new TwilioProvider(mockTwilioConfig);
(provider as unknown as { apiRequest: (endpoint: string, params: Record<string, string>) => Promise<unknown> }).apiRequest =
async () => ({
sid: "CA00000000000000000000000000000000",
status: "queued",
direction: "outbound-api",
from: "+15550000000",
to: "+15550000001",
uri: "/Calls/CA00000000000000000000000000000000.json",
});
await provider.initiateCall({
callId,
to: "+15550000000",
from: "+15550000001",
webhookUrl: "https://example.com/voice/webhook?provider=twilio",
inlineTwiml: twimlPayload,
});
const result = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=twiml`,
method: "POST",
query: { provider: "twilio", callId, type: "twiml" },
});
expect(result.events).toHaveLength(0);
expect(result.providerResponseBody).toBe(twimlPayload);
});
it("does not consume stored TwiML on status callbacks", async () => {
const provider = new TwilioProvider(mockTwilioConfig);
(provider as unknown as { apiRequest: (endpoint: string, params: Record<string, string>) => Promise<unknown> }).apiRequest =
async () => ({
sid: "CA00000000000000000000000000000000",
status: "queued",
direction: "outbound-api",
from: "+15550000000",
to: "+15550000001",
uri: "/Calls/CA00000000000000000000000000000000.json",
});
await provider.initiateCall({
callId,
to: "+15550000000",
from: "+15550000001",
webhookUrl: "https://example.com/voice/webhook?provider=twilio",
inlineTwiml: twimlPayload,
});
const statusResult = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api&From=%2B15550000000&To=%2B15550000001",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=status`,
method: "POST",
query: { provider: "twilio", callId, type: "status" },
});
expect(statusResult.events).toHaveLength(1);
expect(statusResult.events[0]?.type).toBe("call.initiated");
expect(statusResult.providerResponseBody).not.toBe(twimlPayload);
const twimlResult = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=twiml`,
method: "POST",
query: { provider: "twilio", callId, type: "twiml" },
});
expect(twimlResult.providerResponseBody).toBe(twimlPayload);
});
});
+88 -16
View File
@@ -62,6 +62,37 @@ export class TwilioProvider implements VoiceCallProvider {
/** Map of call SID to stream SID for media streams */ /** Map of call SID to stream SID for media streams */
private callStreamMap = new Map<string, string>(); private callStreamMap = new Map<string, string>();
/** Storage for TwiML content (for notify mode with URL-based TwiML) */
private readonly twimlStorage = new Map<string, string>();
/**
* Delete stored TwiML for a given `callId`.
*
* We keep TwiML in-memory only long enough to satisfy the initial Twilio
* webhook request (notify mode). Subsequent webhooks should not reuse it.
*/
private deleteStoredTwiml(callId: string): void {
this.twimlStorage.delete(callId);
}
/**
* Delete stored TwiML for a call, addressed by Twilio's provider call SID.
*
* This is used when we only have `providerCallId` (e.g. hangup).
*/
private deleteStoredTwimlForProviderCall(providerCallId: string): void {
const webhookUrl = this.callWebhookUrls.get(providerCallId);
if (!webhookUrl) return;
try {
const callId = new URL(webhookUrl).searchParams.get("callId");
if (!callId) return;
this.deleteStoredTwiml(callId);
} catch {
// Ignore malformed URLs; best-effort cleanup only.
}
}
constructor(config: TwilioConfig, options: TwilioProviderOptions = {}) { constructor(config: TwilioConfig, options: TwilioProviderOptions = {}) {
if (!config.accountSid) { if (!config.accountSid) {
throw new Error("Twilio Account SID is required"); throw new Error("Twilio Account SID is required");
@@ -149,7 +180,14 @@ export class TwilioProvider implements VoiceCallProvider {
typeof ctx.query?.callId === "string" && ctx.query.callId.trim() typeof ctx.query?.callId === "string" && ctx.query.callId.trim()
? ctx.query.callId.trim() ? ctx.query.callId.trim()
: undefined; : undefined;
const event = this.normalizeEvent(params, callIdFromQuery); const requestType =
typeof ctx.query?.type === "string" && ctx.query.type.trim()
? ctx.query.type.trim()
: undefined;
const isTwimlRequest = requestType === "twiml";
const event = isTwimlRequest
? null
: this.normalizeEvent(params, callIdFromQuery);
// For Twilio, we must return TwiML. Most actions are driven by Calls API updates, // For Twilio, we must return TwiML. Most actions are driven by Calls API updates,
// so the webhook response is typically a pause to keep the call alive. // so the webhook response is typically a pause to keep the call alive.
@@ -228,8 +266,14 @@ export class TwilioProvider implements VoiceCallProvider {
case "busy": case "busy":
case "no-answer": case "no-answer":
case "failed": case "failed":
if (callIdOverride) {
this.deleteStoredTwiml(callIdOverride);
}
return { ...baseEvent, type: "call.ended", reason: callStatus }; return { ...baseEvent, type: "call.ended", reason: callStatus };
case "canceled": case "canceled":
if (callIdOverride) {
this.deleteStoredTwiml(callIdOverride);
}
return { ...baseEvent, type: "call.ended", reason: "hangup-bot" }; return { ...baseEvent, type: "call.ended", reason: "hangup-bot" };
default: default:
return null; return null;
@@ -254,11 +298,29 @@ export class TwilioProvider implements VoiceCallProvider {
const params = new URLSearchParams(ctx.rawBody); const params = new URLSearchParams(ctx.rawBody);
const callStatus = params.get("CallStatus"); const callStatus = params.get("CallStatus");
const direction = params.get("Direction"); const direction = params.get("Direction");
const callIdFromQuery =
typeof ctx.query?.callId === "string" && ctx.query.callId.trim()
? ctx.query.callId.trim()
: undefined;
const requestType =
typeof ctx.query?.type === "string" && ctx.query.type.trim()
? ctx.query.type.trim()
: undefined;
console.log( // Avoid logging webhook params/TwiML (may contain PII).
`[voice-call] generateTwimlResponse: status=${callStatus} direction=${direction}`,
);
// Handle initial TwiML request (when Twilio first initiates the call)
// Check if we have stored TwiML for this call (notify mode)
if (callIdFromQuery && requestType !== "status") {
const storedTwiml = this.twimlStorage.get(callIdFromQuery);
if (storedTwiml) {
// Clean up after serving (one-time use)
this.deleteStoredTwiml(callIdFromQuery);
return storedTwiml;
}
}
// Handle subsequent webhook requests (status callbacks, etc.)
// For inbound calls, answer immediately with stream // For inbound calls, answer immediately with stream
if (direction === "inbound") { if (direction === "inbound") {
const streamUrl = this.getStreamUrl(); const streamUrl = this.getStreamUrl();
@@ -325,31 +387,39 @@ export class TwilioProvider implements VoiceCallProvider {
* Otherwise, uses webhook URL for dynamic TwiML. * Otherwise, uses webhook URL for dynamic TwiML.
*/ */
async initiateCall(input: InitiateCallInput): Promise<InitiateCallResult> { async initiateCall(input: InitiateCallInput): Promise<InitiateCallResult> {
const url = new URL(input.webhookUrl); const webhookUrl = new URL(input.webhookUrl);
url.searchParams.set("callId", input.callId); webhookUrl.searchParams.set("callId", input.callId);
// Build request params const twimlUrl = new URL(webhookUrl);
twimlUrl.searchParams.set("type", "twiml");
// Create separate URL for status callbacks (required by Twilio)
const statusUrl = new URL(webhookUrl);
statusUrl.searchParams.set("type", "status"); // Differentiate from TwiML requests
// Store TwiML content if provided (for notify mode)
// We now serve it from the webhook endpoint instead of sending inline
if (input.inlineTwiml) {
this.twimlStorage.set(input.callId, input.inlineTwiml);
}
// Build request params - always use URL-based TwiML.
// Twilio silently ignores `StatusCallback` when using the inline `Twiml` parameter.
const params: Record<string, string> = { const params: Record<string, string> = {
To: input.to, To: input.to,
From: input.from, From: input.from,
StatusCallback: url.toString(), Url: twimlUrl.toString(), // TwiML serving endpoint
StatusCallback: statusUrl.toString(), // Separate status callback endpoint
StatusCallbackEvent: "initiated ringing answered completed", StatusCallbackEvent: "initiated ringing answered completed",
Timeout: "30", Timeout: "30",
}; };
// Use inline TwiML for notify mode (simpler, no webhook needed)
if (input.inlineTwiml) {
params.Twiml = input.inlineTwiml;
} else {
params.Url = url.toString();
}
const result = await this.apiRequest<TwilioCallResponse>( const result = await this.apiRequest<TwilioCallResponse>(
"/Calls.json", "/Calls.json",
params, params,
); );
this.callWebhookUrls.set(result.sid, url.toString()); this.callWebhookUrls.set(result.sid, webhookUrl.toString());
return { return {
providerCallId: result.sid, providerCallId: result.sid,
@@ -361,6 +431,8 @@ export class TwilioProvider implements VoiceCallProvider {
* Hang up a call via Twilio API. * Hang up a call via Twilio API.
*/ */
async hangupCall(input: HangupCallInput): Promise<void> { async hangupCall(input: HangupCallInput): Promise<void> {
this.deleteStoredTwimlForProviderCall(input.providerCallId);
this.callWebhookUrls.delete(input.providerCallId); this.callWebhookUrls.delete(input.providerCallId);
await this.apiRequest( await this.apiRequest(
+7
View File
@@ -4,6 +4,9 @@ export type NodeListNode = {
nodeId: string; nodeId: string;
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
@@ -20,6 +23,8 @@ type PendingRequest = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
isRepair?: boolean; isRepair?: boolean;
ts: number; ts: number;
@@ -31,6 +36,8 @@ type PairedNode = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
permissions?: Record<string, boolean>; permissions?: Record<string, boolean>;
createdAtMs?: number; createdAtMs?: number;
+47 -1
View File
@@ -4,6 +4,43 @@ import { formatAge, formatPermissions, parseNodeList, parsePairingList } from ".
import { callGatewayCli, nodesCallOpts, resolveNodeId } from "./rpc.js"; import { callGatewayCli, nodesCallOpts, resolveNodeId } from "./rpc.js";
import type { NodesRpcOpts } from "./types.js"; import type { NodesRpcOpts } from "./types.js";
function formatVersionLabel(raw: string) {
const trimmed = raw.trim();
if (!trimmed) return raw;
if (trimmed.toLowerCase().startsWith("v")) return trimmed;
return /^\d/.test(trimmed) ? `v${trimmed}` : trimmed;
}
function resolveNodeVersions(node: {
platform?: string;
version?: string;
coreVersion?: string;
uiVersion?: string;
}) {
const core = node.coreVersion?.trim() || undefined;
const ui = node.uiVersion?.trim() || undefined;
if (core || ui) return { core, ui };
const legacy = node.version?.trim();
if (!legacy) return { core: undefined, ui: undefined };
const platform = node.platform?.trim().toLowerCase() ?? "";
const headless =
platform === "darwin" || platform === "linux" || platform === "win32" || platform === "windows";
return headless ? { core: legacy, ui: undefined } : { core: undefined, ui: legacy };
}
function formatNodeVersions(node: {
platform?: string;
version?: string;
coreVersion?: string;
uiVersion?: string;
}) {
const { core, ui } = resolveNodeVersions(node);
const parts: string[] = [];
if (core) parts.push(`core ${formatVersionLabel(core)}`);
if (ui) parts.push(`ui ${formatVersionLabel(ui)}`);
return parts.length > 0 ? parts.join(" · ") : null;
}
export function registerNodesStatusCommands(nodes: Command) { export function registerNodesStatusCommands(nodes: Command) {
nodesCallOpts( nodesCallOpts(
nodes nodes
@@ -29,6 +66,8 @@ export function registerNodesStatusCommands(nodes: Command) {
const hw = n.modelIdentifier ? ` · hw: ${n.modelIdentifier}` : ""; const hw = n.modelIdentifier ? ` · hw: ${n.modelIdentifier}` : "";
const perms = formatPermissions(n.permissions); const perms = formatPermissions(n.permissions);
const permsText = perms ? ` · perms: ${perms}` : ""; const permsText = perms ? ` · perms: ${perms}` : "";
const versions = formatNodeVersions(n);
const versionText = versions ? ` · ${versions}` : "";
const caps = const caps =
Array.isArray(n.caps) && n.caps.length > 0 Array.isArray(n.caps) && n.caps.length > 0
? `[${n.caps.map(String).filter(Boolean).sort().join(",")}]` ? `[${n.caps.map(String).filter(Boolean).sort().join(",")}]`
@@ -37,7 +76,7 @@ export function registerNodesStatusCommands(nodes: Command) {
: "?"; : "?";
const pairing = n.paired ? "paired" : "unpaired"; const pairing = n.paired ? "paired" : "unpaired";
defaultRuntime.log( defaultRuntime.log(
`- ${name} · ${n.nodeId}${ip}${device}${hw}${permsText} · ${pairing} · ${n.connected ? "connected" : "disconnected"} · caps: ${caps}`, `- ${name} · ${n.nodeId}${ip}${device}${hw}${permsText}${versionText} · ${pairing} · ${n.connected ? "connected" : "disconnected"} · caps: ${caps}`,
); );
} }
} catch (err) { } catch (err) {
@@ -77,12 +116,19 @@ export function registerNodesStatusCommands(nodes: Command) {
const family = typeof obj.deviceFamily === "string" ? obj.deviceFamily : null; const family = typeof obj.deviceFamily === "string" ? obj.deviceFamily : null;
const model = typeof obj.modelIdentifier === "string" ? obj.modelIdentifier : null; const model = typeof obj.modelIdentifier === "string" ? obj.modelIdentifier : null;
const ip = typeof obj.remoteIp === "string" ? obj.remoteIp : null; const ip = typeof obj.remoteIp === "string" ? obj.remoteIp : null;
const versions = formatNodeVersions(obj as {
platform?: string;
version?: string;
coreVersion?: string;
uiVersion?: string;
});
const parts: string[] = ["Node:", displayName, nodeId]; const parts: string[] = ["Node:", displayName, nodeId];
if (ip) parts.push(ip); if (ip) parts.push(ip);
if (family) parts.push(`device: ${family}`); if (family) parts.push(`device: ${family}`);
if (model) parts.push(`hw: ${model}`); if (model) parts.push(`hw: ${model}`);
if (perms) parts.push(`perms: ${perms}`); if (perms) parts.push(`perms: ${perms}`);
if (versions) parts.push(versions);
parts.push(connected ? "connected" : "disconnected"); parts.push(connected ? "connected" : "disconnected");
parts.push(`caps: ${caps ? `[${caps.join(",")}]` : "?"}`); parts.push(`caps: ${caps ? `[${caps.join(",")}]` : "?"}`);
defaultRuntime.log(parts.join(" · ")); defaultRuntime.log(parts.join(" · "));
+6
View File
@@ -46,6 +46,8 @@ export type NodeListNode = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
@@ -62,6 +64,8 @@ export type PendingRequest = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
isRepair?: boolean; isRepair?: boolean;
ts: number; ts: number;
@@ -73,6 +77,8 @@ export type PairedNode = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
remoteIp?: string; remoteIp?: string;
permissions?: Record<string, boolean>; permissions?: Record<string, boolean>;
createdAtMs?: number; createdAtMs?: number;
+2
View File
@@ -8,6 +8,8 @@ export const NodePairRequestParamsSchema = Type.Object(
displayName: Type.Optional(NonEmptyString), displayName: Type.Optional(NonEmptyString),
platform: Type.Optional(NonEmptyString), platform: Type.Optional(NonEmptyString),
version: Type.Optional(NonEmptyString), version: Type.Optional(NonEmptyString),
coreVersion: Type.Optional(NonEmptyString),
uiVersion: Type.Optional(NonEmptyString),
deviceFamily: Type.Optional(NonEmptyString), deviceFamily: Type.Optional(NonEmptyString),
modelIdentifier: Type.Optional(NonEmptyString), modelIdentifier: Type.Optional(NonEmptyString),
caps: Type.Optional(Type.Array(NonEmptyString)), caps: Type.Optional(Type.Array(NonEmptyString)),
+8
View File
@@ -42,6 +42,8 @@ export const nodeHandlers: GatewayRequestHandlers = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -55,6 +57,8 @@ export const nodeHandlers: GatewayRequestHandlers = {
displayName: p.displayName, displayName: p.displayName,
platform: p.platform, platform: p.platform,
version: p.version, version: p.version,
coreVersion: p.coreVersion,
uiVersion: p.uiVersion,
deviceFamily: p.deviceFamily, deviceFamily: p.deviceFamily,
modelIdentifier: p.modelIdentifier, modelIdentifier: p.modelIdentifier,
caps: p.caps, caps: p.caps,
@@ -215,6 +219,8 @@ export const nodeHandlers: GatewayRequestHandlers = {
displayName: live?.displayName ?? paired?.displayName, displayName: live?.displayName ?? paired?.displayName,
platform: live?.platform ?? paired?.platform, platform: live?.platform ?? paired?.platform,
version: live?.version ?? paired?.version, version: live?.version ?? paired?.version,
coreVersion: live?.coreVersion ?? paired?.coreVersion,
uiVersion: live?.uiVersion ?? paired?.uiVersion,
deviceFamily: live?.deviceFamily ?? paired?.deviceFamily, deviceFamily: live?.deviceFamily ?? paired?.deviceFamily,
modelIdentifier: live?.modelIdentifier ?? paired?.modelIdentifier, modelIdentifier: live?.modelIdentifier ?? paired?.modelIdentifier,
remoteIp: live?.remoteIp ?? paired?.remoteIp, remoteIp: live?.remoteIp ?? paired?.remoteIp,
@@ -275,6 +281,8 @@ export const nodeHandlers: GatewayRequestHandlers = {
displayName: live?.displayName ?? paired?.displayName, displayName: live?.displayName ?? paired?.displayName,
platform: live?.platform ?? paired?.platform, platform: live?.platform ?? paired?.platform,
version: live?.version ?? paired?.version, version: live?.version ?? paired?.version,
coreVersion: live?.coreVersion ?? paired?.coreVersion,
uiVersion: live?.uiVersion ?? paired?.uiVersion,
deviceFamily: live?.deviceFamily ?? paired?.deviceFamily, deviceFamily: live?.deviceFamily ?? paired?.deviceFamily,
modelIdentifier: live?.modelIdentifier ?? paired?.modelIdentifier, modelIdentifier: live?.modelIdentifier ?? paired?.modelIdentifier,
remoteIp: live?.remoteIp ?? paired?.remoteIp, remoteIp: live?.remoteIp ?? paired?.remoteIp,
+23 -1
View File
@@ -43,6 +43,25 @@ export async function startGatewayNodeBridge(params: {
}): Promise<GatewayNodeBridgeRuntime> { }): Promise<GatewayNodeBridgeRuntime> {
const nodePresenceTimers = new Map<string, ReturnType<typeof setInterval>>(); const nodePresenceTimers = new Map<string, ReturnType<typeof setInterval>>();
const formatVersionLabel = (raw: string): string => {
const trimmed = raw.trim();
if (!trimmed) return raw;
if (trimmed.toLowerCase().startsWith("v")) return trimmed;
return /^\d/.test(trimmed) ? `v${trimmed}` : trimmed;
};
const resolveNodeVersionLabel = (node: {
coreVersion?: string;
uiVersion?: string;
}): string | null => {
const core = node.coreVersion?.trim();
const ui = node.uiVersion?.trim();
const parts: string[] = [];
if (core) parts.push(`core ${formatVersionLabel(core)}`);
if (ui) parts.push(`ui ${formatVersionLabel(ui)}`);
return parts.length > 0 ? parts.join(" · ") : null;
};
const stopNodePresenceTimer = (nodeId: string) => { const stopNodePresenceTimer = (nodeId: string) => {
const timer = nodePresenceTimers.get(nodeId); const timer = nodePresenceTimers.get(nodeId);
if (timer) { if (timer) {
@@ -57,6 +76,8 @@ export async function startGatewayNodeBridge(params: {
displayName?: string; displayName?: string;
remoteIp?: string; remoteIp?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
platform?: string; platform?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
@@ -66,7 +87,7 @@ export async function startGatewayNodeBridge(params: {
const host = node.displayName?.trim() || node.nodeId; const host = node.displayName?.trim() || node.nodeId;
const rawIp = node.remoteIp?.trim(); const rawIp = node.remoteIp?.trim();
const ip = rawIp && !isLoopbackAddress(rawIp) ? rawIp : undefined; const ip = rawIp && !isLoopbackAddress(rawIp) ? rawIp : undefined;
const version = node.version?.trim() || "unknown"; const version = resolveNodeVersionLabel(node) ?? node.version?.trim() ?? "unknown";
const platform = node.platform?.trim() || undefined; const platform = node.platform?.trim() || undefined;
const deviceFamily = node.deviceFamily?.trim() || undefined; const deviceFamily = node.deviceFamily?.trim() || undefined;
const modelIdentifier = node.modelIdentifier?.trim() || undefined; const modelIdentifier = node.modelIdentifier?.trim() || undefined;
@@ -127,6 +148,7 @@ export async function startGatewayNodeBridge(params: {
platform: node.platform, platform: node.platform,
deviceFamily: node.deviceFamily, deviceFamily: node.deviceFamily,
commands: node.commands, commands: node.commands,
remoteIp: node.remoteIp,
}); });
bumpSkillsSnapshotVersion({ reason: "remote-node" }); bumpSkillsSnapshotVersion({ reason: "remote-node" });
await refreshRemoteNodeBins({ await refreshRemoteNodeBins({
+8
View File
@@ -168,6 +168,8 @@ export function createNodeBridgeConnectionHandler(params: {
displayName: verified.node.displayName ?? hello.displayName, displayName: verified.node.displayName ?? hello.displayName,
platform: verified.node.platform ?? hello.platform, platform: verified.node.platform ?? hello.platform,
version: verified.node.version ?? hello.version, version: verified.node.version ?? hello.version,
coreVersion: verified.node.coreVersion ?? hello.coreVersion,
uiVersion: verified.node.uiVersion ?? hello.uiVersion,
deviceFamily: verified.node.deviceFamily ?? hello.deviceFamily, deviceFamily: verified.node.deviceFamily ?? hello.deviceFamily,
modelIdentifier: verified.node.modelIdentifier ?? hello.modelIdentifier, modelIdentifier: verified.node.modelIdentifier ?? hello.modelIdentifier,
caps, caps,
@@ -181,6 +183,8 @@ export function createNodeBridgeConnectionHandler(params: {
displayName: nodeInfo.displayName, displayName: nodeInfo.displayName,
platform: nodeInfo.platform, platform: nodeInfo.platform,
version: nodeInfo.version, version: nodeInfo.version,
coreVersion: nodeInfo.coreVersion,
uiVersion: nodeInfo.uiVersion,
deviceFamily: nodeInfo.deviceFamily, deviceFamily: nodeInfo.deviceFamily,
modelIdentifier: nodeInfo.modelIdentifier, modelIdentifier: nodeInfo.modelIdentifier,
remoteIp: nodeInfo.remoteIp, remoteIp: nodeInfo.remoteIp,
@@ -243,6 +247,8 @@ export function createNodeBridgeConnectionHandler(params: {
displayName: req.displayName, displayName: req.displayName,
platform: req.platform, platform: req.platform,
version: req.version, version: req.version,
coreVersion: req.coreVersion,
uiVersion: req.uiVersion,
deviceFamily: req.deviceFamily, deviceFamily: req.deviceFamily,
modelIdentifier: req.modelIdentifier, modelIdentifier: req.modelIdentifier,
caps: Array.isArray(req.caps) caps: Array.isArray(req.caps)
@@ -286,6 +292,8 @@ export function createNodeBridgeConnectionHandler(params: {
displayName: req.displayName, displayName: req.displayName,
platform: req.platform, platform: req.platform,
version: req.version, version: req.version,
coreVersion: req.coreVersion,
uiVersion: req.uiVersion,
deviceFamily: req.deviceFamily, deviceFamily: req.deviceFamily,
modelIdentifier: req.modelIdentifier, modelIdentifier: req.modelIdentifier,
caps: Array.isArray(req.caps) ? req.caps.map((c) => String(c)).filter(Boolean) : undefined, caps: Array.isArray(req.caps) ? req.caps.map((c) => String(c)).filter(Boolean) : undefined,
+6
View File
@@ -9,6 +9,8 @@ export type BridgeHelloFrame = {
token?: string; token?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -22,6 +24,8 @@ export type BridgePairRequestFrame = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -113,6 +117,8 @@ export type NodeBridgeClientInfo = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
remoteIp?: string; remoteIp?: string;
+1 -1
View File
@@ -242,7 +242,7 @@ function parseFirstToken(command: string): string | null {
if (end > 1) return trimmed.slice(1, end); if (end > 1) return trimmed.slice(1, end);
return trimmed.slice(1); return trimmed.slice(1);
} }
const match = /^[^\s]+/.exec(trimmed); const match = /^\S+/.exec(trimmed);
return match ? match[0] : null; return match ? match[0] : null;
} }
+10
View File
@@ -9,6 +9,8 @@ export type NodePairingPendingRequest = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -26,6 +28,8 @@ export type NodePairingPairedNode = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -186,6 +190,8 @@ export async function requestNodePairing(
displayName: req.displayName, displayName: req.displayName,
platform: req.platform, platform: req.platform,
version: req.version, version: req.version,
coreVersion: req.coreVersion,
uiVersion: req.uiVersion,
deviceFamily: req.deviceFamily, deviceFamily: req.deviceFamily,
modelIdentifier: req.modelIdentifier, modelIdentifier: req.modelIdentifier,
caps: req.caps, caps: req.caps,
@@ -219,6 +225,8 @@ export async function approveNodePairing(
displayName: pending.displayName, displayName: pending.displayName,
platform: pending.platform, platform: pending.platform,
version: pending.version, version: pending.version,
coreVersion: pending.coreVersion,
uiVersion: pending.uiVersion,
deviceFamily: pending.deviceFamily, deviceFamily: pending.deviceFamily,
modelIdentifier: pending.modelIdentifier, modelIdentifier: pending.modelIdentifier,
caps: pending.caps, caps: pending.caps,
@@ -278,6 +286,8 @@ export async function updatePairedNodeMetadata(
displayName: patch.displayName ?? existing.displayName, displayName: patch.displayName ?? existing.displayName,
platform: patch.platform ?? existing.platform, platform: patch.platform ?? existing.platform,
version: patch.version ?? existing.version, version: patch.version ?? existing.version,
coreVersion: patch.coreVersion ?? existing.coreVersion,
uiVersion: patch.uiVersion ?? existing.uiVersion,
deviceFamily: patch.deviceFamily ?? existing.deviceFamily, deviceFamily: patch.deviceFamily ?? existing.deviceFamily,
modelIdentifier: patch.modelIdentifier ?? existing.modelIdentifier, modelIdentifier: patch.modelIdentifier ?? existing.modelIdentifier,
remoteIp: patch.remoteIp ?? existing.remoteIp, remoteIp: patch.remoteIp ?? existing.remoteIp,
+51 -2
View File
@@ -14,12 +14,57 @@ type RemoteNodeRecord = {
deviceFamily?: string; deviceFamily?: string;
commands?: string[]; commands?: string[];
bins: Set<string>; bins: Set<string>;
remoteIp?: string;
}; };
const log = createSubsystemLogger("gateway/skills-remote"); const log = createSubsystemLogger("gateway/skills-remote");
const remoteNodes = new Map<string, RemoteNodeRecord>(); const remoteNodes = new Map<string, RemoteNodeRecord>();
let remoteBridge: NodeBridgeServer | null = null; let remoteBridge: NodeBridgeServer | null = null;
function describeNode(nodeId: string): string {
const record = remoteNodes.get(nodeId);
const name = record?.displayName?.trim();
const base = name && name !== nodeId ? `${name} (${nodeId})` : nodeId;
const ip = record?.remoteIp?.trim();
return ip ? `${base} @ ${ip}` : base;
}
function extractErrorMessage(err: unknown): string | undefined {
if (!err) return undefined;
if (typeof err === "string") return err;
if (err instanceof Error) return err.message;
if (typeof err === "object" && "message" in err && typeof err.message === "string") {
return err.message;
}
try {
return JSON.stringify(err);
} catch {
return undefined;
}
}
function logRemoteBinProbeFailure(nodeId: string, err: unknown) {
const message = extractErrorMessage(err);
const label = describeNode(nodeId);
if (message?.includes("UNAVAILABLE: node not connected")) {
log.info(
`remote bin probe skipped: node not connected (${label}); check nodes list/status for ${label}`,
);
return;
}
if (message?.includes("UNAVAILABLE: invoke timeout")) {
log.warn(`remote bin probe timed out (${label}); check node connectivity for ${label}`);
return;
}
if (message?.includes("bridge connection closed")) {
log.warn(
`remote bin probe aborted: bridge connection closed (${label}); check nodes list/status for ${label}`,
);
return;
}
log.warn(`remote bin probe error (${label}): ${message ?? "unknown"}`);
}
function isMacPlatform(platform?: string, deviceFamily?: string): boolean { function isMacPlatform(platform?: string, deviceFamily?: string): boolean {
const platformNorm = String(platform ?? "") const platformNorm = String(platform ?? "")
.trim() .trim()
@@ -47,6 +92,7 @@ function upsertNode(record: {
platform?: string; platform?: string;
deviceFamily?: string; deviceFamily?: string;
commands?: string[]; commands?: string[];
remoteIp?: string;
bins?: string[]; bins?: string[];
}) { }) {
const existing = remoteNodes.get(record.nodeId); const existing = remoteNodes.get(record.nodeId);
@@ -57,6 +103,7 @@ function upsertNode(record: {
platform: record.platform ?? existing?.platform, platform: record.platform ?? existing?.platform,
deviceFamily: record.deviceFamily ?? existing?.deviceFamily, deviceFamily: record.deviceFamily ?? existing?.deviceFamily,
commands: record.commands ?? existing?.commands, commands: record.commands ?? existing?.commands,
remoteIp: record.remoteIp ?? existing?.remoteIp,
bins, bins,
}); });
} }
@@ -76,6 +123,7 @@ export async function primeRemoteSkillsCache() {
platform: node.platform, platform: node.platform,
deviceFamily: node.deviceFamily, deviceFamily: node.deviceFamily,
commands: node.commands, commands: node.commands,
remoteIp: node.remoteIp,
bins: node.bins, bins: node.bins,
}); });
if (isMacPlatform(node.platform, node.deviceFamily) && supportsSystemRun(node.commands)) { if (isMacPlatform(node.platform, node.deviceFamily) && supportsSystemRun(node.commands)) {
@@ -96,6 +144,7 @@ export function recordRemoteNodeInfo(node: {
platform?: string; platform?: string;
deviceFamily?: string; deviceFamily?: string;
commands?: string[]; commands?: string[];
remoteIp?: string;
}) { }) {
upsertNode(node); upsertNode(node);
} }
@@ -203,7 +252,7 @@ export async function refreshRemoteNodeBins(params: {
}, },
); );
if (!res.ok) { if (!res.ok) {
log.warn(`remote bin probe failed (${params.nodeId}): ${res.error?.message ?? "unknown"}`); logRemoteBinProbeFailure(params.nodeId, res.error?.message ?? "unknown");
return; return;
} }
const bins = parseBinProbePayload(res.payloadJSON); const bins = parseBinProbePayload(res.payloadJSON);
@@ -211,7 +260,7 @@ export async function refreshRemoteNodeBins(params: {
await updatePairedNodeMetadata(params.nodeId, { bins }); await updatePairedNodeMetadata(params.nodeId, { bins });
bumpSkillsSnapshotVersion({ reason: "remote-node" }); bumpSkillsSnapshotVersion({ reason: "remote-node" });
} catch (err) { } catch (err) {
log.warn(`remote bin probe error (${params.nodeId}): ${String(err)}`); logRemoteBinProbeFailure(params.nodeId, err);
} }
} }
+98
View File
@@ -148,4 +148,102 @@ describe("memory indexing with OpenAI batches", () => {
expect(fetchMock).toHaveBeenCalled(); expect(fetchMock).toHaveBeenCalled();
expect(labels.some((label) => label.toLowerCase().includes("batch"))).toBe(true); expect(labels.some((label) => label.toLowerCase().includes("batch"))).toBe(true);
}); });
it("retries OpenAI batch create on transient failures", async () => {
const content = ["retry", "the", "batch"].join("\n\n");
await fs.writeFile(path.join(workspaceDir, "memory", "2026-01-08.md"), content);
let uploadedRequests: Array<{ custom_id?: string }> = [];
let batchCreates = 0;
const fetchMock = vi.fn(async (input: RequestInfo | URL, init?: RequestInit) => {
const url =
typeof input === "string" ? input : input instanceof URL ? input.toString() : input.url;
if (url.endsWith("/files")) {
const body = init?.body;
if (!(body instanceof FormData)) {
throw new Error("expected FormData upload");
}
for (const [key, value] of body.entries()) {
if (key !== "file") continue;
if (typeof value === "string") {
uploadedRequests = value
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line) as { custom_id?: string });
} else {
const text = await value.text();
uploadedRequests = text
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line) as { custom_id?: string });
}
}
return new Response(JSON.stringify({ id: "file_1" }), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
if (url.endsWith("/batches")) {
batchCreates += 1;
if (batchCreates === 1) {
return new Response("upstream connect error", { status: 503 });
}
return new Response(JSON.stringify({ id: "batch_1", status: "in_progress" }), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
if (url.endsWith("/batches/batch_1")) {
return new Response(
JSON.stringify({ id: "batch_1", status: "completed", output_file_id: "file_out" }),
{ status: 200, headers: { "Content-Type": "application/json" } },
);
}
if (url.endsWith("/files/file_out/content")) {
const lines = uploadedRequests.map((request, index) =>
JSON.stringify({
custom_id: request.custom_id,
response: {
status_code: 200,
body: { data: [{ embedding: [index + 1, 0, 0], index: 0 }] },
},
}),
);
return new Response(lines.join("\n"), {
status: 200,
headers: { "Content-Type": "application/jsonl" },
});
}
throw new Error(`unexpected fetch ${url}`);
});
vi.stubGlobal("fetch", fetchMock);
const cfg = {
agents: {
defaults: {
workspace: workspaceDir,
memorySearch: {
provider: "openai",
model: "text-embedding-3-small",
store: { path: indexPath },
sync: { watch: false, onSessionStart: false, onSearch: false },
query: { minScore: 0 },
remote: { batch: { enabled: true, wait: true } },
},
},
list: [{ id: "main", default: true }],
},
};
const result = await getMemorySearchManager({ cfg, agentId: "main" });
expect(result.manager).not.toBeNull();
if (!result.manager) throw new Error("manager missing");
manager = result.manager;
await manager.sync({ force: true });
const status = manager.status();
expect(status.chunks).toBeGreaterThan(0);
expect(batchCreates).toBe(2);
});
}); });
+110 -37
View File
@@ -1,3 +1,4 @@
import { retryAsync } from "../infra/retry.js";
import type { OpenAiEmbeddingClient } from "./embeddings.js"; import type { OpenAiEmbeddingClient } from "./embeddings.js";
import { hashText } from "./internal.js"; import { hashText } from "./internal.js";
@@ -33,6 +34,30 @@ export type OpenAiBatchOutputLine = {
export const OPENAI_BATCH_ENDPOINT = "/v1/embeddings"; export const OPENAI_BATCH_ENDPOINT = "/v1/embeddings";
const OPENAI_BATCH_COMPLETION_WINDOW = "24h"; const OPENAI_BATCH_COMPLETION_WINDOW = "24h";
const OPENAI_BATCH_MAX_REQUESTS = 50000; const OPENAI_BATCH_MAX_REQUESTS = 50000;
const OPENAI_BATCH_RETRY = {
attempts: 3,
minDelayMs: 500,
maxDelayMs: 5000,
jitter: 0.1,
};
type RetryableError = Error & { status?: number };
function isRetryableBatchError(err: unknown): boolean {
const status =
typeof (err as RetryableError)?.status === "number" ? (err as RetryableError).status : undefined;
if (typeof status === "number") {
return status === 429 || status >= 500;
}
const message = err instanceof Error ? err.message : String(err);
return /timeout|timed out|ECONNRESET|ECONNREFUSED|EHOSTUNREACH|ENOTFOUND|EAI_AGAIN|network|fetch failed|upstream connect/i.test(
message,
);
}
function formatRetryError(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
function getOpenAiBaseUrl(openAi: OpenAiEmbeddingClient): string { function getOpenAiBaseUrl(openAi: OpenAiEmbeddingClient): string {
return openAi.baseUrl?.replace(/\/$/, "") ?? ""; return openAi.baseUrl?.replace(/\/$/, "") ?? "";
@@ -63,10 +88,43 @@ function splitOpenAiBatchRequests(requests: OpenAiBatchRequest[]): OpenAiBatchRe
return groups; return groups;
} }
async function fetchOpenAiWithRetry(params: {
openAi: OpenAiEmbeddingClient;
url: string;
init?: RequestInit;
label: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<Response> {
return await retryAsync(
async () => {
const res = await fetch(params.url, params.init);
if (!res.ok) {
const text = await res.text();
const err = new Error(`openai batch ${params.label} failed: ${res.status} ${text}`);
(err as RetryableError).status = res.status;
throw err;
}
return res;
},
{
...OPENAI_BATCH_RETRY,
label: params.label,
shouldRetry: isRetryableBatchError,
onRetry: (info) => {
params.debug?.(
`openai batch ${params.label} retry ${info.attempt}/${info.maxAttempts} in ${info.delayMs}ms`,
{ error: formatRetryError(info.err) },
);
},
},
);
}
async function submitOpenAiBatch(params: { async function submitOpenAiBatch(params: {
openAi: OpenAiEmbeddingClient; openAi: OpenAiEmbeddingClient;
requests: OpenAiBatchRequest[]; requests: OpenAiBatchRequest[];
agentId: string; agentId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<OpenAiBatchStatus> { }): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi); const baseUrl = getOpenAiBaseUrl(params.openAi);
const jsonl = params.requests.map((request) => JSON.stringify(request)).join("\n"); const jsonl = params.requests.map((request) => JSON.stringify(request)).join("\n");
@@ -78,67 +136,73 @@ async function submitOpenAiBatch(params: {
`memory-embeddings.${hashText(String(Date.now()))}.jsonl`, `memory-embeddings.${hashText(String(Date.now()))}.jsonl`,
); );
const fileRes = await fetch(`${baseUrl}/files`, { const fileRes = await fetchOpenAiWithRetry({
method: "POST", openAi: params.openAi,
headers: getOpenAiHeaders(params.openAi, { json: false }), url: `${baseUrl}/files`,
body: form, init: {
method: "POST",
headers: getOpenAiHeaders(params.openAi, { json: false }),
body: form,
},
label: "file upload",
debug: params.debug,
}); });
if (!fileRes.ok) {
const text = await fileRes.text();
throw new Error(`openai batch file upload failed: ${fileRes.status} ${text}`);
}
const filePayload = (await fileRes.json()) as { id?: string }; const filePayload = (await fileRes.json()) as { id?: string };
if (!filePayload.id) { if (!filePayload.id) {
throw new Error("openai batch file upload failed: missing file id"); throw new Error("openai batch file upload failed: missing file id");
} }
const batchRes = await fetch(`${baseUrl}/batches`, { const batchRes = await fetchOpenAiWithRetry({
method: "POST", openAi: params.openAi,
headers: getOpenAiHeaders(params.openAi, { json: true }), url: `${baseUrl}/batches`,
body: JSON.stringify({ init: {
input_file_id: filePayload.id, method: "POST",
endpoint: OPENAI_BATCH_ENDPOINT, headers: getOpenAiHeaders(params.openAi, { json: true }),
completion_window: OPENAI_BATCH_COMPLETION_WINDOW, body: JSON.stringify({
metadata: { input_file_id: filePayload.id,
source: "clawdbot-memory", endpoint: OPENAI_BATCH_ENDPOINT,
agent: params.agentId, completion_window: OPENAI_BATCH_COMPLETION_WINDOW,
}, metadata: {
}), source: "clawdbot-memory",
agent: params.agentId,
},
}),
},
label: "create",
debug: params.debug,
}); });
if (!batchRes.ok) {
const text = await batchRes.text();
throw new Error(`openai batch create failed: ${batchRes.status} ${text}`);
}
return (await batchRes.json()) as OpenAiBatchStatus; return (await batchRes.json()) as OpenAiBatchStatus;
} }
async function fetchOpenAiBatchStatus(params: { async function fetchOpenAiBatchStatus(params: {
openAi: OpenAiEmbeddingClient; openAi: OpenAiEmbeddingClient;
batchId: string; batchId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<OpenAiBatchStatus> { }): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi); const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetch(`${baseUrl}/batches/${params.batchId}`, { const res = await fetchOpenAiWithRetry({
headers: getOpenAiHeaders(params.openAi, { json: true }), openAi: params.openAi,
url: `${baseUrl}/batches/${params.batchId}`,
init: { headers: getOpenAiHeaders(params.openAi, { json: true }) },
label: "status",
debug: params.debug,
}); });
if (!res.ok) {
const text = await res.text();
throw new Error(`openai batch status failed: ${res.status} ${text}`);
}
return (await res.json()) as OpenAiBatchStatus; return (await res.json()) as OpenAiBatchStatus;
} }
async function fetchOpenAiFileContent(params: { async function fetchOpenAiFileContent(params: {
openAi: OpenAiEmbeddingClient; openAi: OpenAiEmbeddingClient;
fileId: string; fileId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<string> { }): Promise<string> {
const baseUrl = getOpenAiBaseUrl(params.openAi); const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetch(`${baseUrl}/files/${params.fileId}/content`, { const res = await fetchOpenAiWithRetry({
headers: getOpenAiHeaders(params.openAi, { json: true }), openAi: params.openAi,
url: `${baseUrl}/files/${params.fileId}/content`,
init: { headers: getOpenAiHeaders(params.openAi, { json: true }) },
label: "file content",
debug: params.debug,
}); });
if (!res.ok) {
const text = await res.text();
throw new Error(`openai batch file content failed: ${res.status} ${text}`);
}
return await res.text(); return await res.text();
} }
@@ -154,11 +218,13 @@ function parseOpenAiBatchOutput(text: string): OpenAiBatchOutputLine[] {
async function readOpenAiBatchError(params: { async function readOpenAiBatchError(params: {
openAi: OpenAiEmbeddingClient; openAi: OpenAiEmbeddingClient;
errorFileId: string; errorFileId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<string | undefined> { }): Promise<string | undefined> {
try { try {
const content = await fetchOpenAiFileContent({ const content = await fetchOpenAiFileContent({
openAi: params.openAi, openAi: params.openAi,
fileId: params.errorFileId, fileId: params.errorFileId,
debug: params.debug,
}); });
const lines = parseOpenAiBatchOutput(content); const lines = parseOpenAiBatchOutput(content);
const first = lines.find((line) => line.error?.message || line.response?.body?.error); const first = lines.find((line) => line.error?.message || line.response?.body?.error);
@@ -191,6 +257,7 @@ async function waitForOpenAiBatch(params: {
(await fetchOpenAiBatchStatus({ (await fetchOpenAiBatchStatus({
openAi: params.openAi, openAi: params.openAi,
batchId: params.batchId, batchId: params.batchId,
debug: params.debug,
})); }));
const state = status.status ?? "unknown"; const state = status.status ?? "unknown";
if (state === "completed") { if (state === "completed") {
@@ -204,7 +271,11 @@ async function waitForOpenAiBatch(params: {
} }
if (["failed", "expired", "cancelled", "canceled"].includes(state)) { if (["failed", "expired", "cancelled", "canceled"].includes(state)) {
const detail = status.error_file_id const detail = status.error_file_id
? await readOpenAiBatchError({ openAi: params.openAi, errorFileId: status.error_file_id }) ? await readOpenAiBatchError({
openAi: params.openAi,
errorFileId: status.error_file_id,
debug: params.debug,
})
: undefined; : undefined;
const suffix = detail ? `: ${detail}` : ""; const suffix = detail ? `: ${detail}` : "";
throw new Error(`openai batch ${params.batchId} ${state}${suffix}`); throw new Error(`openai batch ${params.batchId} ${state}${suffix}`);
@@ -267,6 +338,7 @@ export async function runOpenAiEmbeddingBatches(params: {
openAi: params.openAi, openAi: params.openAi,
requests: group, requests: group,
agentId: params.agentId, agentId: params.agentId,
debug: params.debug,
}); });
if (!batchInfo.id) { if (!batchInfo.id) {
throw new Error("openai batch create failed: missing batch id"); throw new Error("openai batch create failed: missing batch id");
@@ -308,6 +380,7 @@ export async function runOpenAiEmbeddingBatches(params: {
const content = await fetchOpenAiFileContent({ const content = await fetchOpenAiFileContent({
openAi: params.openAi, openAi: params.openAi,
fileId: completed.outputFileId, fileId: completed.outputFileId,
debug: params.debug,
}); });
const outputLines = parseOpenAiBatchOutput(content); const outputLines = parseOpenAiBatchOutput(content);
const errors: string[] = []; const errors: string[] = [];
+6
View File
@@ -27,6 +27,8 @@ export type BridgeClientOptions = {
displayName?: string; displayName?: string;
platform?: string; platform?: string;
version?: string; version?: string;
coreVersion?: string;
uiVersion?: string;
deviceFamily?: string; deviceFamily?: string;
modelIdentifier?: string; modelIdentifier?: string;
caps?: string[]; caps?: string[];
@@ -172,6 +174,8 @@ export class BridgeClient {
displayName: this.opts.displayName, displayName: this.opts.displayName,
platform: this.opts.platform, platform: this.opts.platform,
version: this.opts.version, version: this.opts.version,
coreVersion: this.opts.coreVersion,
uiVersion: this.opts.uiVersion,
deviceFamily: this.opts.deviceFamily, deviceFamily: this.opts.deviceFamily,
modelIdentifier: this.opts.modelIdentifier, modelIdentifier: this.opts.modelIdentifier,
caps: this.opts.caps, caps: this.opts.caps,
@@ -188,6 +192,8 @@ export class BridgeClient {
displayName: this.opts.displayName, displayName: this.opts.displayName,
platform: this.opts.platform, platform: this.opts.platform,
version: this.opts.version, version: this.opts.version,
coreVersion: this.opts.coreVersion,
uiVersion: this.opts.uiVersion,
deviceFamily: this.opts.deviceFamily, deviceFamily: this.opts.deviceFamily,
modelIdentifier: this.opts.modelIdentifier, modelIdentifier: this.opts.modelIdentifier,
caps: this.opts.caps, caps: this.opts.caps,
+1
View File
@@ -346,6 +346,7 @@ export async function runNodeHost(opts: NodeHostRunOptions): Promise<void> {
displayName, displayName,
platform: process.platform, platform: process.platform,
version: VERSION, version: VERSION,
coreVersion: VERSION,
deviceFamily: os.platform(), deviceFamily: os.platform(),
modelIdentifier: os.hostname(), modelIdentifier: os.hostname(),
caps: ["system"], caps: ["system"],