Compare commits

..
Author SHA1 Message Date
Peter Steinberger 63ac6ccd20 fix: resolve TwiML/status callbacks (#1180) (thanks @andrew-kurin) 2026-01-18 16:34:18 +00:00
Ghost 340a054f73 fix(voice-call): resolve StatusCallback with inline TwiML
- Switch from inline to URL-based TwiML for outbound calls
- Store TwiML content temporarily and serve on webhook request
- Add twimlStorage map and cleanup helper methods
- Fix TwiML serving to handle CallStatus='in-progress' on initial request

Closes #864
2026-01-18 16:11:19 +00:00
73 changed files with 1942 additions and 4621 deletions

No files matched your search

+46 -52
View File
@@ -2,40 +2,63 @@
Docs: https://docs.clawd.bot
## 2026.1.17-7
### Changes
- Exec approvals: add `clawdbot approvals` CLI for viewing and updating gateway/node allowlists.
- CLI: add `clawdbot service` gateway/node management and a `clawdbot node status` alias.
- Status: show gateway + node service summaries in `clawdbot status` and `status --all`.
- Control UI: add gateway/node target selector for exec approvals.
- Docs: add approvals/service references and refresh node/control UI docs.
- Dependencies: update core + plugin deps (grammy, vitest, openai, Microsoft agents hosting, etc.).
- macOS: switch PeekabooBridge integration to the tagged Swift Package Manager release (no submodule).
- macOS: stop syncing Peekaboo as a git submodule in postinstall.
- Swabble: use the tagged Commander Swift package release.
- CLI: add `clawdbot acp client` interactive ACP harness for debugging.
- Plugins: route command detection/text chunking helpers through the plugin runtime and drop runtime exports from the SDK.
- Memory: add native Gemini embeddings provider for memory search. (#1151)
- Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt.
- Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne.
- Nodes: report core/ui versions in node list + presence; surface both in CLI + macOS UI.
<<<<<<< HEAD
- Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt.
- Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne.
||||||| parent of 903e9be49 (feat: surface node core/ui versions in macOS)
### Fixes
- Auth profiles: keep auto-pinned preference while allowing rotation on failover; user pins stay locked. (#1138) — thanks @cheeeee.
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Voice call: separate TwiML fetches from status callbacks to avoid stuck calls. (#1180) — thanks @andrew-kurin.
## 2026.1.18-5
### Changes
- Dependencies: update core + plugin deps (grammy, vitest, openai, Microsoft agents hosting, etc.).
### Fixes
- Plugins: auto-select exclusive slots when enabling/installing plugins. (#1181) — thanks @sebslight.
## 2026.1.18-4
## 2026.1.18-3
### Changes
- macOS: switch PeekabooBridge integration to the tagged Swift Package Manager release (no submodule).
- macOS: stop syncing Peekaboo as a git submodule in postinstall.
- Swabble: use the tagged Commander Swift package release.
- CLI: add `clawdbot acp client` interactive ACP harness for debugging.
- Plugins: route command detection/text chunking helpers through the plugin runtime and drop runtime exports from the SDK.
- Plugins: auto-enable bundled channel/provider plugins when configuration is present.
- Config: stamp last-touched metadata on write and warn if the config is newer than the running build.
- macOS: hide usage section when usage is unavailable instead of showing provider errors.
- Memory: add native Gemini embeddings provider for memory search. (#1151)
- Agents: add local docs path resolution and include docs/mirror/source/community pointers in the system prompt.
- Slack: add HTTP webhook mode via Bolt HTTP receiver for Events API deployments. (#1143) — thanks @jdrhyne.
=======
- Nodes: report core/ui versions in node list + presence; surface both in CLI + macOS UI.
### Fixes
- Auth profiles: keep auto-pinned preference while allowing rotation on failover; user pins stay locked. (#1138) — thanks @cheeeee.
- Agents: sanitize oversized image payloads before send and surface image-dimension errors.
- macOS: Doctor repairs LaunchAgent bootstrap issues for Gateway + Node when listed but not loaded. (#1166) — thanks @AlexMikhalev.
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Voice call: separate TwiML fetches from status callbacks to avoid stuck calls. (#1180) — thanks @andrew-kurin.
## 2026.1.18-5
### Changes
- Dependencies: update core + plugin deps (grammy, vitest, openai, Microsoft agents hosting, etc.).
## 2026.1.18-3
### Changes
>>>>>>> 903e9be49 (feat: surface node core/ui versions in macOS)
- Exec: add host/security/ask routing for gateway + node exec.
- Exec: add `/exec` directive for per-session exec defaults (host/security/ask/node).
- macOS: migrate exec approvals to `~/.clawdbot/exec-approvals.json` with per-agent allowlists and skill auto-allow toggle.
@@ -56,34 +79,21 @@ Docs: https://docs.clawd.bot
- Memory: allow parallel OpenAI batch indexing jobs (default concurrency: 2).
- macOS: add per-agent exec approvals with allowlists, skill CLI auto-allow, and settings UI.
- Docs: add exec approvals guide and link from tools index. https://docs.clawd.bot/tools/exec-approvals
- macOS: add exec-host IPC for node service `system.run` with HMAC + peer UID checks.
### Fixes
- Auth profiles: keep auto-pinned preference while allowing rotation on failover; user pins stay locked. (#1138) — thanks @cheeeee.
- Agents: sanitize oversized image payloads before send and surface image-dimension errors.
- macOS: avoid touching launchd in Remote over SSH so quitting the app no longer disables the remote gateway. (#1105)
- Memory: index atomically so failed reindex preserves the previous memory database. (#1151)
- Memory: avoid sqlite-vec unique constraint failures when reindexing duplicate chunk ids. (#1151)
- Skills: improve remote bin probe logging with node labels + connectivity hints.
- Exec approvals: enforce allowlist when ask is off; prefer raw command for node approvals/events.
- Tools: return a companion-app-required message when node exec is requested with no paired node.
- Streaming: emit assistant deltas for OpenAI-compatible SSE chunks. (#1147) — thanks @alauppe.
- Model fallback: treat timeout aborts as failover while preserving user aborts. (#1137) — thanks @cheeeee.
## 2026.1.18-2
### Fixes
- Tests: stabilize plugin SDK resolution and embedded agent timeouts.
## 2026.1.18-1
### Changes
- Tools: allow `sessions_spawn` to override thinking level for sub-agent runs.
- Channels: unify thread/topic allowlist matching + command/mention gating helpers across core providers.
- Models: add Qwen Portal OAuth provider support. (#1120) — thanks @mukhtharcm.
- Memory: add `--verbose` logging for memory status + batch indexing details.
- Memory: allow parallel OpenAI batch indexing jobs (default concurrency: 2).
- macOS: add per-agent exec approvals with allowlists, skill CLI auto-allow, and settings UI.
- Docs: add exec approvals guide and link from tools index. https://docs.clawd.bot/tools/exec-approvals
### Fixes
- Memory: apply OpenAI batch defaults even without explicit remote config.
- macOS: bundle Textual resources in packaged app builds to avoid code block crashes. (#1006)
- Tools: return a companion-app-required message when `system.run` is requested without a supporting node.
- Discord: only emit slow listener warnings after 30s.
## 2026.1.17-6
@@ -112,22 +122,6 @@ Docs: https://docs.clawd.bot
### Fixes
- Voice call: include request query in Twilio webhook verification when publicUrl is set. (#864)
## 2026.1.18-1
### Changes
- Tools: allow `sessions_spawn` to override thinking level for sub-agent runs.
- Channels: unify thread/topic allowlist matching + command/mention gating helpers across core providers.
- Models: add Qwen Portal OAuth provider support. (#1120) — thanks @mukhtharcm.
- Memory: add `--verbose` logging for memory status + batch indexing details.
- Memory: allow parallel OpenAI batch indexing jobs (default concurrency: 2).
- macOS: add per-agent exec approvals with allowlists, skill CLI auto-allow, and settings UI.
- Docs: add exec approvals guide and link from tools index. https://docs.clawd.bot/tools/exec-approvals
### Fixes
- Memory: apply OpenAI batch defaults even without explicit remote config.
- macOS: bundle Textual resources in packaged app builds to avoid code block crashes. (#1006)
- Tools: return a companion-app-required message when `system.run` is requested without a supporting node.
- Discord: only emit slow listener warnings after 30s.
## 2026.1.17-3
### Changes
-64
View File
@@ -1,64 +0,0 @@
# Clawdbot macOS app (dev + signing)
## Quick dev run
```bash
# from repo root
scripts/restart-mac.sh
```
Options:
```bash
scripts/restart-mac.sh --no-sign # fastest dev; ad-hoc signing (TCC permissions do not stick)
scripts/restart-mac.sh --sign # force code signing (requires cert)
```
## Packaging flow
```bash
scripts/package-mac-app.sh
```
Creates `dist/Clawdbot.app` and signs it via `scripts/codesign-mac-app.sh`.
## Signing behavior
Auto-selects identity (first match):
1) Developer ID Application
2) Apple Distribution
3) Apple Development
4) first available identity
If none found:
- errors by default
- set `ALLOW_ADHOC_SIGNING=1` or `SIGN_IDENTITY="-"` to ad-hoc sign
## Team ID audit (Sparkle mismatch guard)
After signing, we read the app bundle Team ID and compare every Mach-O inside the app.
If any embedded binary has a different Team ID, signing fails.
Skip the audit:
```bash
SKIP_TEAM_ID_CHECK=1 scripts/package-mac-app.sh
```
## Library validation workaround (dev only)
If Sparkle Team ID mismatch blocks loading (common with Apple Development certs), opt in:
```bash
DISABLE_LIBRARY_VALIDATION=1 scripts/package-mac-app.sh
```
This adds `com.apple.security.cs.disable-library-validation` to app entitlements.
Use for local dev only; keep off for release builds.
## Useful env flags
- `SIGN_IDENTITY="Apple Development: Your Name (TEAMID)"`
- `ALLOW_ADHOC_SIGNING=1` (ad-hoc, TCC permissions do not persist)
- `CODESIGN_TIMESTAMP=off` (offline debug)
- `DISABLE_LIBRARY_VALIDATION=1` (dev-only Sparkle workaround)
- `SKIP_TEAM_ID_CHECK=1` (bypass audit)
@@ -1,6 +1,5 @@
import AppKit
import ClawdbotKit
import CryptoKit
import Darwin
import Foundation
import OSLog
@@ -28,49 +27,6 @@ private struct ExecApprovalSocketDecision: Codable {
var decision: ExecApprovalDecision
}
fileprivate struct ExecHostSocketRequest: Codable {
var type: String
var id: String
var nonce: String
var ts: Int
var hmac: String
var requestJson: String
}
fileprivate struct ExecHostRequest: Codable {
var command: [String]
var rawCommand: String?
var cwd: String?
var env: [String: String]?
var timeoutMs: Int?
var needsScreenRecording: Bool?
var agentId: String?
var sessionKey: String?
}
fileprivate struct ExecHostRunResult: Codable {
var exitCode: Int?
var timedOut: Bool
var success: Bool
var stdout: String
var stderr: String
var error: String?
}
fileprivate struct ExecHostError: Codable {
var code: String
var message: String
var reason: String?
}
fileprivate struct ExecHostResponse: Codable {
var type: String
var id: String
var ok: Bool
var payload: ExecHostRunResult?
var error: ExecHostError?
}
enum ExecApprovalsSocketClient {
private struct TimeoutError: LocalizedError {
var message: String
@@ -190,9 +146,6 @@ final class ExecApprovalsPromptServer {
token: approvals.token,
onPrompt: { request in
await ExecApprovalsPromptPresenter.prompt(request)
},
onExec: { request in
await ExecHostExecutor.handle(request)
})
server.start()
self.server = server
@@ -253,182 +206,11 @@ enum ExecApprovalsPromptPresenter {
}
}
@MainActor
fileprivate enum ExecHostExecutor {
private static let blockedEnvKeys: Set<String> = [
"PATH",
"NODE_OPTIONS",
"PYTHONHOME",
"PYTHONPATH",
"PERL5LIB",
"PERL5OPT",
"RUBYOPT",
]
private static let blockedEnvPrefixes: [String] = [
"DYLD_",
"LD_",
]
static func handle(_ request: ExecHostRequest) async -> ExecHostResponse {
let command = request.command.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
guard !command.isEmpty else {
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: false,
payload: nil,
error: ExecHostError(code: "INVALID_REQUEST", message: "command required", reason: "invalid"))
}
let displayCommand = ExecCommandFormatter.displayString(
for: command,
rawCommand: request.rawCommand)
let agentId = request.agentId?.trimmingCharacters(in: .whitespacesAndNewlines)
let trimmedAgent = (agentId?.isEmpty == false) ? agentId : nil
let approvals = ExecApprovalsStore.resolve(agentId: trimmedAgent)
let security = approvals.agent.security
let ask = approvals.agent.ask
let autoAllowSkills = approvals.agent.autoAllowSkills
let env = self.sanitizedEnv(request.env)
let resolution = ExecCommandResolution.resolve(
command: command,
rawCommand: request.rawCommand,
cwd: request.cwd,
env: env)
let allowlistMatch = security == .allowlist
? ExecAllowlistMatcher.match(entries: approvals.allowlist, resolution: resolution)
: nil
let skillAllow: Bool
if autoAllowSkills, let name = resolution?.executableName {
let bins = await SkillBinsCache.shared.currentBins()
skillAllow = bins.contains(name)
} else {
skillAllow = false
}
if security == .deny {
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: false,
payload: nil,
error: ExecHostError(code: "UNAVAILABLE", message: "SYSTEM_RUN_DISABLED: security=deny", reason: "security=deny"))
}
let requiresAsk: Bool = {
if ask == .always { return true }
if ask == .onMiss && security == .allowlist && allowlistMatch == nil && !skillAllow { return true }
return false
}()
var approvedByAsk = false
if requiresAsk {
let decision = ExecApprovalsPromptPresenter.prompt(
ExecApprovalPromptRequest(
command: displayCommand,
cwd: request.cwd,
host: "node",
security: security.rawValue,
ask: ask.rawValue,
agentId: trimmedAgent,
resolvedPath: resolution?.resolvedPath))
switch decision {
case .deny:
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: false,
payload: nil,
error: ExecHostError(code: "UNAVAILABLE", message: "SYSTEM_RUN_DENIED: user denied", reason: "user-denied"))
case .allowAlways:
approvedByAsk = true
if security == .allowlist {
let pattern = resolution?.resolvedPath ?? resolution?.rawExecutable ?? command.first ?? ""
if !pattern.isEmpty {
ExecApprovalsStore.addAllowlistEntry(agentId: trimmedAgent, pattern: pattern)
}
}
case .allowOnce:
approvedByAsk = true
}
}
if security == .allowlist && allowlistMatch == nil && !skillAllow && !approvedByAsk {
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: false,
payload: nil,
error: ExecHostError(code: "UNAVAILABLE", message: "SYSTEM_RUN_DENIED: allowlist miss", reason: "allowlist-miss"))
}
if let match = allowlistMatch {
ExecApprovalsStore.recordAllowlistUse(
agentId: trimmedAgent,
pattern: match.pattern,
command: displayCommand,
resolvedPath: resolution?.resolvedPath)
}
if request.needsScreenRecording == true {
let authorized = await PermissionManager
.status([.screenRecording])[.screenRecording] ?? false
if !authorized {
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: false,
payload: nil,
error: ExecHostError(code: "UNAVAILABLE", message: "PERMISSION_MISSING: screenRecording", reason: "permission:screenRecording"))
}
}
let timeoutSec = request.timeoutMs.flatMap { Double($0) / 1000.0 }
let result = await Task.detached { () -> ShellExecutor.ShellResult in
await ShellExecutor.runDetailed(
command: command,
cwd: request.cwd,
env: env,
timeout: timeoutSec)
}.value
let payload = ExecHostRunResult(
exitCode: result.exitCode,
timedOut: result.timedOut,
success: result.success,
stdout: result.stdout,
stderr: result.stderr,
error: result.errorMessage)
return ExecHostResponse(
type: "exec-res",
id: UUID().uuidString,
ok: true,
payload: payload,
error: nil)
}
private static func sanitizedEnv(_ overrides: [String: String]?) -> [String: String]? {
guard let overrides else { return nil }
var merged = ProcessInfo.processInfo.environment
for (rawKey, value) in overrides {
let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines)
guard !key.isEmpty else { continue }
let upper = key.uppercased()
if self.blockedEnvKeys.contains(upper) { continue }
if self.blockedEnvPrefixes.contains(where: { upper.hasPrefix($0) }) { continue }
merged[key] = value
}
return merged
}
}
private final class ExecApprovalsSocketServer: @unchecked Sendable {
private let logger = Logger(subsystem: "com.clawdbot", category: "exec-approvals.socket")
private let socketPath: String
private let token: String
private let onPrompt: @Sendable (ExecApprovalPromptRequest) async -> ExecApprovalDecision
private let onExec: @Sendable (ExecHostRequest) async -> ExecHostResponse
private var socketFD: Int32 = -1
private var acceptTask: Task<Void, Never>?
private var isRunning = false
@@ -436,13 +218,11 @@ private final class ExecApprovalsSocketServer: @unchecked Sendable {
init(
socketPath: String,
token: String,
onPrompt: @escaping @Sendable (ExecApprovalPromptRequest) async -> ExecApprovalDecision,
onExec: @escaping @Sendable (ExecHostRequest) async -> ExecHostResponse)
onPrompt: @escaping @Sendable (ExecApprovalPromptRequest) async -> ExecApprovalDecision)
{
self.socketPath = socketPath
self.token = token
self.onPrompt = onPrompt
self.onExec = onExec
}
func start() {
@@ -537,39 +317,26 @@ private final class ExecApprovalsSocketServer: @unchecked Sendable {
private func handleClient(fd: Int32) async {
let handle = FileHandle(fileDescriptor: fd, closeOnDealloc: true)
do {
guard self.isAllowedPeer(fd: fd) else {
try self.sendApprovalResponse(handle: handle, id: UUID().uuidString, decision: .deny)
return
}
guard let line = try self.readLine(from: handle, maxBytes: 256_000),
let data = line.data(using: .utf8)
else {
return
}
guard
let envelope = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let type = envelope["type"] as? String
else {
return
}
if type == "request" {
let request = try JSONDecoder().decode(ExecApprovalSocketRequest.self, from: data)
guard request.token == self.token else {
try self.sendApprovalResponse(handle: handle, id: request.id, decision: .deny)
return
}
let decision = await self.onPrompt(request.request)
try self.sendApprovalResponse(handle: handle, id: request.id, decision: decision)
return
}
if type == "exec" {
let request = try JSONDecoder().decode(ExecHostSocketRequest.self, from: data)
let response = await self.handleExecRequest(request)
try self.sendExecResponse(handle: handle, response: response)
let request = try JSONDecoder().decode(ExecApprovalSocketRequest.self, from: data)
guard request.type == "request", request.token == self.token else {
let response = ExecApprovalSocketDecision(type: "decision", id: request.id, decision: .deny)
let data = try JSONEncoder().encode(response)
var payload = data
payload.append(0x0A)
try handle.write(contentsOf: payload)
return
}
let decision = await self.onPrompt(request.request)
let response = ExecApprovalSocketDecision(type: "decision", id: request.id, decision: decision)
let responseData = try JSONEncoder().encode(response)
var payload = responseData
payload.append(0x0A)
try handle.write(contentsOf: payload)
} catch {
self.logger.error("exec approvals socket handling failed: \(error.localizedDescription, privacy: .public)")
}
@@ -590,77 +357,4 @@ private final class ExecApprovalsSocketServer: @unchecked Sendable {
let lineData = buffer.subdata(in: 0..<newlineIndex)
return String(data: lineData, encoding: .utf8)
}
private func sendApprovalResponse(
handle: FileHandle,
id: String,
decision: ExecApprovalDecision) throws
{
let response = ExecApprovalSocketDecision(type: "decision", id: id, decision: decision)
let data = try JSONEncoder().encode(response)
var payload = data
payload.append(0x0A)
try handle.write(contentsOf: payload)
}
private func sendExecResponse(handle: FileHandle, response: ExecHostResponse) throws {
let data = try JSONEncoder().encode(response)
var payload = data
payload.append(0x0A)
try handle.write(contentsOf: payload)
}
private func isAllowedPeer(fd: Int32) -> Bool {
var uid = uid_t(0)
var gid = gid_t(0)
if getpeereid(fd, &uid, &gid) != 0 {
return false
}
return uid == geteuid()
}
private func handleExecRequest(_ request: ExecHostSocketRequest) async -> ExecHostResponse {
let nowMs = Int(Date().timeIntervalSince1970 * 1000)
if abs(nowMs - request.ts) > 10_000 {
return ExecHostResponse(
type: "exec-res",
id: request.id,
ok: false,
payload: nil,
error: ExecHostError(code: "INVALID_REQUEST", message: "expired request", reason: "ttl"))
}
let expected = self.hmacHex(nonce: request.nonce, ts: request.ts, requestJson: request.requestJson)
if expected != request.hmac {
return ExecHostResponse(
type: "exec-res",
id: request.id,
ok: false,
payload: nil,
error: ExecHostError(code: "INVALID_REQUEST", message: "invalid auth", reason: "hmac"))
}
guard let requestData = request.requestJson.data(using: .utf8),
let payload = try? JSONDecoder().decode(ExecHostRequest.self, from: requestData)
else {
return ExecHostResponse(
type: "exec-res",
id: request.id,
ok: false,
payload: nil,
error: ExecHostError(code: "INVALID_REQUEST", message: "invalid payload", reason: "json"))
}
let response = await self.onExec(payload)
return ExecHostResponse(
type: "exec-res",
id: request.id,
ok: response.ok,
payload: response.payload,
error: response.error)
}
private func hmacHex(nonce: String, ts: Int, requestJson: String) -> String {
let key = SymmetricKey(data: Data(self.token.utf8))
let message = "\(nonce):\(ts):\(requestJson)"
let mac = HMAC<SHA256>.authenticationCode(for: Data(message.utf8), using: key)
return mac.map { String(format: "%02x", $0) }.joined()
}
}
@@ -280,7 +280,9 @@ extension MenuSessionsInjector {
private func insertUsageSection(into menu: NSMenu, at cursor: Int, width: CGFloat) -> Int {
let rows = self.usageRows
if rows.isEmpty {
let errorText = self.cachedUsageErrorText
if rows.isEmpty, errorText == nil {
return cursor
}
@@ -304,6 +306,25 @@ extension MenuSessionsInjector {
menu.insertItem(headerItem, at: cursor)
cursor += 1
if let errorText = errorText?.nonEmpty, !rows.isEmpty {
menu.insertItem(
self.makeMessageItem(
text: errorText,
symbolName: "exclamationmark.triangle",
width: width,
maxLines: 2),
at: cursor)
cursor += 1
}
if rows.isEmpty {
menu.insertItem(
self.makeMessageItem(text: errorText ?? "No usage available", symbolName: "minus", width: width),
at: cursor)
cursor += 1
return cursor
}
if let selectedProvider = self.selectedUsageProviderId,
let primary = rows.first(where: { $0.providerId.lowercased() == selectedProvider }),
rows.count > 1
@@ -419,8 +440,6 @@ extension MenuSessionsInjector {
displayName: "Gateway",
platform: platform,
version: nil,
coreVersion: nil,
uiVersion: nil,
deviceFamily: nil,
modelIdentifier: nil,
remoteIp: host,
@@ -540,11 +559,14 @@ extension MenuSessionsInjector {
do {
self.cachedUsageSummary = try await UsageLoader.loadSummary()
} catch {
self.cachedUsageSummary = nil
self.cachedUsageErrorText = nil
self.usageCacheUpdatedAt = Date()
} catch {
if self.cachedUsageSummary == nil {
self.cachedUsageErrorText = self.compactUsageError(error)
}
self.usageCacheUpdatedAt = Date()
}
self.usageCacheUpdatedAt = Date()
}
private func compactUsageError(_ error: Error) -> String {
@@ -1660,40 +1660,6 @@ public struct ExecApprovalsSetParams: Codable, Sendable {
}
}
public struct ExecApprovalsNodeGetParams: Codable, Sendable {
public let nodeid: String
public init(
nodeid: String
) {
self.nodeid = nodeid
}
private enum CodingKeys: String, CodingKey {
case nodeid = "nodeId"
}
}
public struct ExecApprovalsNodeSetParams: Codable, Sendable {
public let nodeid: String
public let file: [String: AnyCodable]
public let basehash: String?
public init(
nodeid: String,
file: [String: AnyCodable],
basehash: String?
) {
self.nodeid = nodeid
self.file = file
self.basehash = basehash
}
private enum CodingKeys: String, CodingKey {
case nodeid = "nodeId"
case file
case basehash = "baseHash"
}
}
public struct ExecApprovalsSnapshot: Codable, Sendable {
public let path: String
public let exists: Bool
+5 -14
View File
@@ -7,8 +7,8 @@ read_when:
# `clawdbot memory`
Manage semantic memory indexing and search.
Provided by the active memory plugin (default: `memory-core`; set `plugins.slots.memory = "none"` to disable).
Memory search tools (semantic memory status/index/search).
Provided by the active memory plugin (default: `memory-core`; use `plugins.slots.memory = "none"` to disable).
Related:
- Memory concept: [Memory](/concepts/memory)
@@ -22,20 +22,11 @@ clawdbot memory status --deep
clawdbot memory status --deep --index
clawdbot memory status --deep --index --verbose
clawdbot memory index
clawdbot memory index --verbose
clawdbot memory search "release checklist"
clawdbot memory status --agent main
clawdbot memory index --agent main --verbose
```
## Options
Common:
- `--agent <id>`: scope to a single agent (default: all configured agents).
- `--verbose`: emit detailed logs during probes and indexing.
Notes:
- `memory status --deep` probes vector + embedding availability.
- `memory status --deep --index` runs a reindex if the store is dirty.
- `memory index --verbose` prints per-phase details (provider, model, sources, batch activity).
- `--verbose`: emit debug logs during memory probes and indexing.
- `--index-mode auto|batch|direct`: override batch usage when indexing (`direct` favors speed; `batch` favors OpenAI Batch pricing).
- `--progress auto|line|log|none`: progress output mode (`log` prints updates even without a TTY).
+18 -32
View File
@@ -79,46 +79,37 @@ semantic queries can find related notes even when wording differs.
Defaults:
- Enabled by default.
- Watches memory files for changes (debounced).
- Uses remote embeddings by default. If `memorySearch.provider` is not set, Clawdbot auto-selects:
1. `local` if a `memorySearch.local.modelPath` is configured and the file exists.
2. `openai` if an OpenAI key can be resolved.
3. `gemini` if a Gemini key can be resolved.
4. Otherwise memory search stays disabled until configured.
- Uses remote embeddings (OpenAI) unless configured for local.
- Local mode uses node-llama-cpp and may require `pnpm approve-builds`.
- Uses sqlite-vec (when available) to accelerate vector search inside SQLite.
Remote embeddings **require** an API key for the embedding provider. Clawdbot
resolves keys from auth profiles, `models.providers.*.apiKey`, or environment
variables. Codex OAuth only covers chat/completions and does **not** satisfy
embeddings for memory search. For Gemini, use `GEMINI_API_KEY` or
`models.providers.google.apiKey`. When using a custom OpenAI-compatible endpoint,
set `memorySearch.remote.apiKey` (and optional `memorySearch.remote.headers`).
Remote embeddings **require** an API key for the embedding provider. By default
this is OpenAI (`OPENAI_API_KEY` or `models.providers.openai.apiKey`). Codex
OAuth only covers chat/completions and does **not** satisfy embeddings for
memory search. When using a custom OpenAI-compatible endpoint, set
`memorySearch.remote.apiKey` (and optional `memorySearch.remote.headers`).
### Gemini embeddings (native)
Set the provider to `gemini` to use the Gemini embeddings API directly:
If you want to use **Gemini embeddings** directly, set the provider to `gemini`:
```json5
agents: {
defaults: {
memorySearch: {
provider: "gemini",
model: "gemini-embedding-001",
model: "gemini-embedding-001", // default
remote: {
apiKey: "YOUR_GEMINI_API_KEY"
apiKey: "${GEMINI_API_KEY}"
}
}
}
}
```
Notes:
- `remote.baseUrl` is optional (defaults to the Gemini API base URL).
- `remote.headers` lets you add extra headers if needed.
- Default model: `gemini-embedding-001`.
Gemini uses `GEMINI_API_KEY` (or `models.providers.google.apiKey`). Override
`memorySearch.remote.baseUrl` to point at a custom Gemini-compatible endpoint.
If you want to use a **custom OpenAI-compatible endpoint** (OpenRouter, vLLM, or a proxy),
you can use the `remote` configuration with the OpenAI provider:
If you want to use a **custom OpenAI-compatible endpoint** (like OpenRouter or a proxy),
you can use the `remote` configuration:
```json5
agents: {
@@ -127,8 +118,8 @@ agents: {
provider: "openai",
model: "text-embedding-3-small",
remote: {
baseUrl: "https://api.example.com/v1/",
apiKey: "YOUR_OPENAI_COMPAT_API_KEY",
baseUrl: "https://proxy.example/v1",
apiKey: "YOUR_PROXY_KEY",
headers: { "X-Custom-Header": "value" }
}
}
@@ -139,16 +130,11 @@ agents: {
If you don't want to set an API key, use `memorySearch.provider = "local"` or set
`memorySearch.fallback = "none"`.
Fallbacks:
- `memorySearch.fallback` can be `openai`, `gemini`, `local`, or `none`.
- The fallback provider is only used when the primary embedding provider fails.
Batch indexing (OpenAI + Gemini):
- Enabled by default for OpenAI and Gemini embeddings. Set `agents.defaults.memorySearch.remote.batch.enabled = false` to disable.
Batch indexing (OpenAI only):
- Enabled by default for OpenAI embeddings. Set `agents.defaults.memorySearch.remote.batch.enabled = false` to disable.
- Default behavior waits for batch completion; tune `remote.batch.wait`, `remote.batch.pollIntervalMs`, and `remote.batch.timeoutMinutes` if needed.
- Set `remote.batch.concurrency` to control how many batch jobs we submit in parallel (default: 2).
- Batch mode applies when `memorySearch.provider = "openai"` or `"gemini"` and uses the corresponding API key.
- Gemini batch jobs use the async embeddings batch endpoint and require Gemini Batch API availability.
- Batch mode currently applies only when `memorySearch.provider = "openai"` and uses your OpenAI API key.
Why OpenAI batch is fast + cheap:
- For large backfills, OpenAI is typically the fastest option we support because we can submit many embedding requests in a single batch job and let OpenAI process them asynchronously.
+2 -5
View File
@@ -9,9 +9,8 @@ read_when:
# Gateway on macOS (external launchd)
Clawdbot.app no longer bundles Node/Bun or the Gateway runtime. The macOS app
expects an **external** `clawdbot` CLI install, does not spawn the Gateway as a
child process, and manages a per‑user launchd service to keep the Gateway
running (or attaches to an existing local Gateway if one is already running).
expects an **external** `clawdbot` CLI install and manages a per‑user launchd
service to keep the Gateway running.
## Install the CLI (required for local mode)
@@ -39,8 +38,6 @@ Manager:
Behavior:
- “Clawdbot Active” enables/disables the LaunchAgent.
- App quit does **not** stop the gateway (launchd keeps it alive).
- If a Gateway is already running on the configured port, the app attaches to
it instead of starting a new one.
Logging:
- launchd stdout/err: `/tmp/clawdbot/clawdbot-gateway.log`
+3 -5
View File
@@ -5,11 +5,9 @@ read_when:
---
# Gateway lifecycle on macOS
The macOS app **manages the Gateway via launchd** by default and does not spawn
the Gateway as a child process. It first tries to attach to an already‑running
Gateway on the configured port; if none is reachable, it enables the launchd
service via the external `clawdbot` CLI (no embedded runtime). This gives you
reliable auto‑start at login and restart on crashes.
The macOS app **manages the Gateway via launchd** by default. The launchd job
uses the external `clawdbot` CLI (no embedded runtime). This gives you reliable
auto‑start at login and restart on crashes.
Child‑process mode (Gateway spawned directly by the app) is **not in use** today.
If you need tighter coupling to the UI, run the Gateway manually in a terminal.
-3
View File
@@ -32,9 +32,6 @@ To build the macOS app and package it into `dist/Clawdbot.app`, run:
If you don't have an Apple Developer ID certificate, the script will automatically use **ad-hoc signing** (`-`).
For dev run modes, signing flags, and Team ID troubleshooting, see the macOS app README:
https://github.com/clawdbot/clawdbot/blob/main/apps/macos/README.md
> **Note**: Ad-hoc signed apps may trigger security prompts. If the app crashes immediately with "Abort trap 6", see the [Troubleshooting](#troubleshooting) section.
## 3. Install the CLI
-2
View File
@@ -14,7 +14,6 @@ This app is usually built from [`scripts/package-mac-app.sh`](https://github.com
- inject build metadata into Info.plist: `ClawdbotBuildTimestamp` (UTC) and `ClawdbotGitCommit` (short hash) so the About pane can show build, git, and debug/release channel.
- **Packaging requires Node 22+**: the script runs TS builds and the Control UI build.
- reads `SIGN_IDENTITY` from the environment. Add `export SIGN_IDENTITY="Apple Development: Your Name (TEAMID)"` (or your Developer ID Application cert) to your shell rc to always sign with your cert. Ad-hoc signing requires explicit opt-in via `ALLOW_ADHOC_SIGNING=1` or `SIGN_IDENTITY="-"` (not recommended for permission testing).
- runs a Team ID audit after signing and fails if any Mach-O inside the app bundle is signed by a different Team ID. Set `SKIP_TEAM_ID_CHECK=1` to bypass.
## Usage
@@ -24,7 +23,6 @@ scripts/package-mac-app.sh # auto-selects identity; errors if none
SIGN_IDENTITY="Developer ID Application: Your Name" scripts/package-mac-app.sh # real cert
ALLOW_ADHOC_SIGNING=1 scripts/package-mac-app.sh # ad-hoc (permissions will not stick)
SIGN_IDENTITY="-" scripts/package-mac-app.sh # explicit ad-hoc (same caveat)
DISABLE_LIBRARY_VALIDATION=1 scripts/package-mac-app.sh # dev-only Sparkle Team ID mismatch workaround
```
### Ad-hoc Signing Note
-17
View File
@@ -7,8 +7,6 @@ read_when:
**Current model:** there is **no local control socket** and no `clawdbot-mac` CLI. All agent actions go through the Gateway WebSocket and `node.invoke`. UI automation still uses PeekabooBridge.
**Planned model:** add a local Unix socket between the **node service** and the **macOS app**. The app owns `system.run` (UI/TCC context); the node service forwards exec requests over IPC.
## Goals
- Single GUI app instance that owns all TCC-facing work (notifications, screen recording, mic, speech, AppleScript).
- A small surface for automation: Gateway + node commands, plus PeekabooBridge for UI automation.
@@ -19,19 +17,6 @@ read_when:
- The app runs the Gateway (local mode) and connects to it as a node.
- Agent actions are performed via `node.invoke` (e.g. `system.run`, `system.notify`, `canvas.*`).
### Node service + app IPC (planned)
- A headless node service connects to the Gateway bridge.
- `system.run` requests are forwarded to the macOS app over a local Unix socket.
- The app performs the exec in UI context, prompts if needed, and returns output.
Diagram (SCI):
```
Agent -> Gateway -> Bridge -> Node Service (TS)
| IPC (UDS + token + HMAC + TTL)
v
Mac App (UI + TCC + system.run)
```
### PeekabooBridge (UI automation)
- UI automation uses a separate UNIX socket named `bridge.sock` and the PeekabooBridge JSON protocol.
- Host preference order (client-side): Peekaboo.app → Claude.app → Clawdbot.app → local execution.
@@ -40,7 +25,6 @@ Agent -> Gateway -> Bridge -> Node Service (TS)
### Mach/XPC
- Not required for automation; `node.invoke` + PeekabooBridge cover current needs.
- Planned IPC keeps Unix sockets (no XPC helper).
## Operational flows
- Restart/rebuild: `SIGN_IDENTITY="Apple Development: <Developer Name> (<TEAMID>)" scripts/restart-mac.sh`
@@ -54,4 +38,3 @@ Agent -> Gateway -> Bridge -> Node Service (TS)
- PeekabooBridge: `PEEKABOO_ALLOW_UNSIGNED_SOCKET_CLIENTS=1` (DEBUG-only) may allow same-UID callers for local development.
- All communication remains local-only; no network sockets are exposed.
- TCC prompts originate only from the GUI app bundle; keep the signed bundle ID stable across rebuilds.
- Planned IPC hardening: socket mode `0600`, token, peer-UID checks, HMAC challenge/response, short TTL.
+5 -21
View File
@@ -7,8 +7,8 @@ read_when:
# Clawdbot macOS Companion (menu bar + gateway broker)
The macOS app is the **menu‑bar companion** for Clawdbot. It owns permissions,
manages/attaches to the Gateway locally (launchd or manual), and exposes macOS
capabilities to the agent as a node.
manages the Gateway locally, and exposes macOS capabilities to the agent as a
node.
## What it does
@@ -20,17 +20,13 @@ capabilities to the agent as a node.
- Optionally hosts **PeekabooBridge** for UI automation.
- Installs the global CLI (`clawdbot`) via npm/pnpm on request (bun not recommended for the Gateway runtime).
Planned:
- Run a headless **node service** locally (launchd).
- Keep `system.run` in the app (UI/TCC context), with the node service forwarding via IPC.
## Local vs remote mode
- **Local** (default): the app attaches to a running local Gateway if present;
otherwise it enables the launchd service via `clawdbot daemon`.
- **Local** (default): the app ensures a local Gateway is running via launchd.
- **Remote**: the app connects to a Gateway over SSH/Tailscale and never starts
a local process.
The app does not spawn the Gateway as a child process.
- **Attach‑only** (debug): the app connects to an already‑running local Gateway
and never spawns its own.
## Launchd control
@@ -58,18 +54,6 @@ The macOS app presents itself as a node. Common commands:
The node reports a `permissions` map so agents can decide what’s allowed.
Planned split:
- Node service advertises the node surface to the Gateway.
- macOS app performs `system.run` in UI context over IPC.
Diagram (SCI):
```
Gateway -> Bridge -> Node Service (TS)
| IPC (UDS + token + HMAC + TTL)
v
Mac App (UI + TCC + system.run)
```
## Exec approvals (system.run)
`system.run` is controlled by **Exec approvals** in the macOS app (Settings → Exec approvals).
+8 -21
View File
@@ -30,8 +30,6 @@ read_when:
- **Node identity:** use existing `nodeId`.
- **Socket auth:** Unix socket + token (cross-platform); split later if needed.
- **Node host state:** `~/.clawdbot/node.json` (node id + pairing token).
- **macOS exec host:** run `system.run` inside the macOS app; node service forwards requests over local IPC.
- **No XPC helper:** stick to Unix socket + token + peer checks.
## Key concepts
### Host
@@ -141,30 +139,19 @@ Notes:
## UI integration (macOS app)
### IPC
- Unix socket at `~/.clawdbot/exec-approvals.sock` (0600).
- Token stored in `exec-approvals.json` (0600).
- Peer checks: same-UID only.
- Challenge/response: nonce + HMAC(token, request-hash) to prevent replay.
- Short TTL (e.g., 10s) + max payload + rate limit.
- Unix socket at `~/.clawdbot/exec-approvals.sock`.
- Runner connects and sends an approval request; UI responds with a decision.
- Token stored in `exec-approvals.json`.
### Ask flow (macOS app exec host)
1) Node service receives `system.run` from gateway.
2) Node service connects to the local socket and sends the prompt/exec request.
3) App validates peer + token + HMAC + TTL, then shows dialog if needed.
4) App executes the command in UI context and returns output.
5) Node service returns output to gateway.
### Ask flow
1) Runner receives `system.run` from gateway.
2) If ask required, runner connects to the socket and sends a prompt request.
3) UI shows dialog; returns decision.
4) Runner enforces decision and proceeds.
If UI missing:
- Apply `askFallback` (`deny|allowlist|full`).
### Diagram (SCI)
```
Agent -> Gateway -> Bridge -> Node Service (TS)
| IPC (UDS + token + HMAC + TTL)
v
Mac App (UI + TCC + system.run)
```
## Node identity + binding
- Use existing `nodeId` from Bridge pairing.
- Binding model:
+36 -22
View File
@@ -60,6 +60,7 @@ Quick answers plus deeper troubleshooting for real-world setups (local dev, VPS,
- [Remote gateways + nodes](#remote-gateways-nodes)
- [How do commands propagate between Telegram, the gateway, and nodes?](#how-do-commands-propagate-between-telegram-the-gateway-and-nodes)
- [Do nodes run a gateway daemon?](#do-nodes-run-a-gateway-daemon)
- [Can I run a headless node host without the macOS app?](#can-i-run-a-headless-node-host-without-the-macos-app)
- [Is there an API / RPC way to apply config?](#is-there-an-api-rpc-way-to-apply-config)
- [What’s a minimal “sane” config for a first install?](#whats-a-minimal-sane-config-for-a-first-install)
- [How do I set up Tailscale on a VPS and connect from my Mac?](#how-do-i-set-up-tailscale-on-a-vps-and-connect-from-my-mac)
@@ -405,7 +406,7 @@ You have three supported patterns:
Run the Gateway where the macOS binaries exist, then connect from Linux in [remote mode](#how-do-i-run-clawdbot-in-remote-mode-client-connects-to-a-gateway-elsewhere) or over Tailscale. The skills load normally because the Gateway host is macOS.
**Option B - use a macOS node (no SSH).**
Run the Gateway on Linux, pair a macOS node (menubar app), and set **Node Run Commands** to "Always Ask" or "Always Allow" on the Mac. Clawdbot can treat macOS-only skills as eligible when the required binaries exist on the node. The agent runs those skills via the `nodes` tool. If you choose "Always Ask", approving "Always Allow" in the prompt adds that command to the allowlist.
Run the Gateway on Linux, pair a macOS node (menubar app), and configure **Exec approvals** (Settings → Exec approvals) to "Ask" or "Always Allow". Clawdbot can treat macOS-only skills as eligible when the required binaries exist on the node. The agent runs those skills via the `nodes` tool. If you choose "Ask", selecting "Always Allow" in the prompt adds that command to the allowlist.
**Option C - proxy macOS binaries over SSH (advanced).**
Keep the Gateway on Linux, but make the required CLI binaries resolve to SSH wrappers that run on a Mac. Then override the skill to allow Linux so it stays eligible.
@@ -501,23 +502,14 @@ is writable (read-only sandboxes skip it). See [Memory](/concepts/memory).
### Does semantic memory search require an OpenAI API key?
Only if you use **OpenAI embeddings**. Codex OAuth covers chat/completions and
does **not** grant embeddings access, so **signing in with Codex (OAuth or the
Codex CLI login)** does not help for semantic memory search. OpenAI embeddings
still need a real API key (`OPENAI_API_KEY` or `models.providers.openai.apiKey`).
If you don’t set a provider explicitly, Clawdbot auto-selects a provider when it
can resolve an API key (auth profiles, `models.providers.*.apiKey`, or env vars).
It prefers OpenAI if an OpenAI key resolves, otherwise Gemini if a Gemini key
resolves. If neither key is available, memory search stays disabled until you
configure it. If you have a local model path configured and present, Clawdbot
prefers `local`.
If you’d rather stay local, set `memorySearch.provider = "local"` (and optionally
`memorySearch.fallback = "none"`). If you want Gemini embeddings, set
`memorySearch.provider = "gemini"` and provide `GEMINI_API_KEY` (or
`memorySearch.remote.apiKey`). We support **OpenAI, Gemini, or local** embedding
models — see [Memory](/concepts/memory) for the setup details.
Only if you use **remote embeddings** (OpenAI). Codex OAuth covers
chat/completions and does **not** grant embeddings access, so **signing in with
Codex (OAuth or the Codex CLI login)** does not help for semantic memory search.
Remote memory search still needs a real OpenAI API key (`OPENAI_API_KEY` or
`models.providers.openai.apiKey`). If you’d rather stay local, set
`memorySearch.provider = "local"` (and optionally `memorySearch.fallback =
"none"`). We support **remote or local embedding models** — see [Memory](/concepts/memory)
for the setup details.
## Where things live on disk
@@ -751,6 +743,23 @@ to the gateway (iOS/Android nodes, or macOS “node mode” in the menubar app).
A full restart is required for `gateway`, `bridge`, `discovery`, and `canvasHost` changes.
### Can I run a headless node host without the macOS app?
Yes. The headless node host is a **command-only** node that exposes `system.run` / `system.which`
without any UI. It has no screen/camera/notify support (use the macOS app for those).
Start it:
```bash
clawdbot node start --host <gateway-host> --port 18790
```
Notes:
- Pairing is still required (`clawdbot nodes pending` → `clawdbot nodes approve <requestId>`).
- Exec approvals still apply via `~/.clawdbot/exec-approvals.json`.
- If prompts are enabled but no companion UI is reachable, `askFallback` decides (default: deny).
Docs: [Node CLI](/cli/node), [Nodes](/nodes), [Exec approvals](/tools/exec-approvals).
### Is there an API / RPC way to apply config?
Yes. `config.apply` validates + writes the full config and restarts the Gateway as part of the operation.
@@ -889,14 +898,19 @@ Send `/new` or `/reset` as a standalone message. See [Session management](/conce
### Do sessions reset automatically if I never send `/new`?
Yes. Sessions expire after `session.idleMinutes` (default **60**). The **next**
message starts a fresh session id for that chat key. This does not delete
transcripts — it just starts a new session.
Yes. By default sessions reset daily at **4:00 AM local time** on the gateway host.
You can also add an idle window; when both daily and idle resets are configured,
whichever expires first starts a new session id on the next message. This does
not delete transcripts — it just starts a new session.
```json5
{
session: {
idleMinutes: 240
reset: {
mode: "daily",
atHour: 4,
idleMinutes: 240
}
}
}
```
-17
View File
@@ -22,10 +22,6 @@ Exec approvals are enforced locally on the execution host:
- **gateway host** → `clawdbot` process on the gateway machine
- **node host** → node runner (macOS companion app or headless node host)
Planned macOS split:
- **node service** forwards `system.run` to the **macOS app** over local IPC.
- **macOS app** enforces approvals + executes the command in UI context.
## Settings and storage
Approvals live in a local JSON file on the execution host:
@@ -132,19 +128,6 @@ Actions:
- **Always allow** → add to allowlist + run
- **Deny** → block
### macOS IPC flow (planned)
```
Gateway -> Bridge -> Node Service (TS)
| IPC (UDS + token + HMAC + TTL)
v
Mac App (UI + approvals + system.run)
```
Security notes:
- Unix socket mode `0600`, token stored in `exec-approvals.json`.
- Same-UID peer check.
- Challenge/response (nonce + HMAC token + request hash) + short TTL.
## System events
Exec lifecycle is surfaced as system messages:
@@ -0,0 +1,92 @@
import { describe, expect, it } from "vitest";
import { TwilioProvider } from "./twilio.js";
const mockTwilioConfig = {
accountSid: "AC00000000000000000000000000000000",
authToken: "test-token",
};
const callId = "internal-call-id";
const twimlPayload =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><Response><Say>Hi</Say></Response>";
describe("TwilioProvider", () => {
it("serves stored TwiML on twiml requests without emitting events", async () => {
const provider = new TwilioProvider(mockTwilioConfig);
(provider as unknown as { apiRequest: (endpoint: string, params: Record<string, string>) => Promise<unknown> }).apiRequest =
async () => ({
sid: "CA00000000000000000000000000000000",
status: "queued",
direction: "outbound-api",
from: "+15550000000",
to: "+15550000001",
uri: "/Calls/CA00000000000000000000000000000000.json",
});
await provider.initiateCall({
callId,
to: "+15550000000",
from: "+15550000001",
webhookUrl: "https://example.com/voice/webhook?provider=twilio",
inlineTwiml: twimlPayload,
});
const result = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=twiml`,
method: "POST",
query: { provider: "twilio", callId, type: "twiml" },
});
expect(result.events).toHaveLength(0);
expect(result.providerResponseBody).toBe(twimlPayload);
});
it("does not consume stored TwiML on status callbacks", async () => {
const provider = new TwilioProvider(mockTwilioConfig);
(provider as unknown as { apiRequest: (endpoint: string, params: Record<string, string>) => Promise<unknown> }).apiRequest =
async () => ({
sid: "CA00000000000000000000000000000000",
status: "queued",
direction: "outbound-api",
from: "+15550000000",
to: "+15550000001",
uri: "/Calls/CA00000000000000000000000000000000.json",
});
await provider.initiateCall({
callId,
to: "+15550000000",
from: "+15550000001",
webhookUrl: "https://example.com/voice/webhook?provider=twilio",
inlineTwiml: twimlPayload,
});
const statusResult = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api&From=%2B15550000000&To=%2B15550000001",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=status`,
method: "POST",
query: { provider: "twilio", callId, type: "status" },
});
expect(statusResult.events).toHaveLength(1);
expect(statusResult.events[0]?.type).toBe("call.initiated");
expect(statusResult.providerResponseBody).not.toBe(twimlPayload);
const twimlResult = provider.parseWebhookEvent({
headers: { host: "example.com" },
rawBody:
"CallSid=CA00000000000000000000000000000000&CallStatus=initiated&Direction=outbound-api",
url: `https://example.com/voice/webhook?provider=twilio&callId=${callId}&type=twiml`,
method: "POST",
query: { provider: "twilio", callId, type: "twiml" },
});
expect(twimlResult.providerResponseBody).toBe(twimlPayload);
});
});
+28 -12
View File
@@ -84,10 +84,13 @@ export class TwilioProvider implements VoiceCallProvider {
const webhookUrl = this.callWebhookUrls.get(providerCallId);
if (!webhookUrl) return;
const callIdMatch = webhookUrl.match(/callId=([^&]+)/);
if (!callIdMatch) return;
this.deleteStoredTwiml(callIdMatch[1]);
try {
const callId = new URL(webhookUrl).searchParams.get("callId");
if (!callId) return;
this.deleteStoredTwiml(callId);
} catch {
// Ignore malformed URLs; best-effort cleanup only.
}
}
constructor(config: TwilioConfig, options: TwilioProviderOptions = {}) {
@@ -177,7 +180,14 @@ export class TwilioProvider implements VoiceCallProvider {
typeof ctx.query?.callId === "string" && ctx.query.callId.trim()
? ctx.query.callId.trim()
: undefined;
const event = this.normalizeEvent(params, callIdFromQuery);
const requestType =
typeof ctx.query?.type === "string" && ctx.query.type.trim()
? ctx.query.type.trim()
: undefined;
const isTwimlRequest = requestType === "twiml";
const event = isTwimlRequest
? null
: this.normalizeEvent(params, callIdFromQuery);
// For Twilio, we must return TwiML. Most actions are driven by Calls API updates,
// so the webhook response is typically a pause to keep the call alive.
@@ -292,12 +302,16 @@ export class TwilioProvider implements VoiceCallProvider {
typeof ctx.query?.callId === "string" && ctx.query.callId.trim()
? ctx.query.callId.trim()
: undefined;
const requestType =
typeof ctx.query?.type === "string" && ctx.query.type.trim()
? ctx.query.type.trim()
: undefined;
// Avoid logging webhook params/TwiML (may contain PII).
// Handle initial TwiML request (when Twilio first initiates the call)
// Check if we have stored TwiML for this call (notify mode)
if (callIdFromQuery) {
if (callIdFromQuery && requestType !== "status") {
const storedTwiml = this.twimlStorage.get(callIdFromQuery);
if (storedTwiml) {
// Clean up after serving (one-time use)
@@ -373,12 +387,14 @@ export class TwilioProvider implements VoiceCallProvider {
* Otherwise, uses webhook URL for dynamic TwiML.
*/
async initiateCall(input: InitiateCallInput): Promise<InitiateCallResult> {
const url = new URL(input.webhookUrl);
url.searchParams.set("callId", input.callId);
const webhookUrl = new URL(input.webhookUrl);
webhookUrl.searchParams.set("callId", input.callId);
const twimlUrl = new URL(webhookUrl);
twimlUrl.searchParams.set("type", "twiml");
// Create separate URL for status callbacks (required by Twilio)
const statusUrl = new URL(input.webhookUrl);
statusUrl.searchParams.set("callId", input.callId);
const statusUrl = new URL(webhookUrl);
statusUrl.searchParams.set("type", "status"); // Differentiate from TwiML requests
// Store TwiML content if provided (for notify mode)
@@ -392,7 +408,7 @@ export class TwilioProvider implements VoiceCallProvider {
const params: Record<string, string> = {
To: input.to,
From: input.from,
Url: url.toString(), // TwiML serving endpoint
Url: twimlUrl.toString(), // TwiML serving endpoint
StatusCallback: statusUrl.toString(), // Separate status callback endpoint
StatusCallbackEvent: "initiated ringing answered completed",
Timeout: "30",
@@ -403,7 +419,7 @@ export class TwilioProvider implements VoiceCallProvider {
params,
);
this.callWebhookUrls.set(result.sid, url.toString());
this.callWebhookUrls.set(result.sid, webhookUrl.toString());
return {
providerCallId: result.sid,
+16 -16
View File
@@ -65,27 +65,27 @@
"dist/whatsapp/**"
],
"scripts": {
"dev": "bun src/entry.ts",
"dev": "tsx src/entry.ts",
"postinstall": "node scripts/postinstall.js",
"prepack": "pnpm build",
"docs:list": "bun scripts/docs-list.ts",
"docs:list": "tsx scripts/docs-list.ts",
"docs:bin": "bun build scripts/docs-list.ts --compile --outfile bin/docs-list",
"docs:dev": "cd docs && mint dev",
"docs:build": "cd docs && pnpm dlx --reporter append-only mint broken-links",
"build": "tsc -p tsconfig.json && bun scripts/canvas-a2ui-copy.ts && bun scripts/copy-hook-metadata.ts && bun scripts/write-build-info.ts",
"plugins:sync": "bun scripts/sync-plugin-versions.ts",
"release:check": "bun scripts/release-check.ts",
"build": "tsc -p tsconfig.json && tsx scripts/canvas-a2ui-copy.ts && tsx scripts/copy-hook-metadata.ts && tsx scripts/write-build-info.ts",
"plugins:sync": "tsx scripts/sync-plugin-versions.ts",
"release:check": "tsx scripts/release-check.ts",
"ui:install": "node scripts/ui.js install",
"ui:dev": "node scripts/ui.js dev",
"ui:build": "node scripts/ui.js build",
"start": "bun src/entry.ts",
"clawdbot": "bun src/entry.ts",
"gateway:watch": "bun --watch src/entry.ts gateway --force",
"gateway:dev": "CLAWDBOT_SKIP_CHANNELS=1 bun src/entry.ts --dev gateway",
"gateway:dev:reset": "CLAWDBOT_SKIP_CHANNELS=1 bun src/entry.ts --dev gateway --reset",
"tui": "bun src/entry.ts tui",
"tui:dev": "CLAWDBOT_PROFILE=dev bun src/entry.ts tui",
"clawdbot:rpc": "bun src/entry.ts agent --mode rpc --json",
"start": "tsx src/entry.ts",
"clawdbot": "tsx src/entry.ts",
"gateway:watch": "tsx watch src/entry.ts gateway --force",
"gateway:dev": "CLAWDBOT_SKIP_CHANNELS=1 tsx src/entry.ts --dev gateway",
"gateway:dev:reset": "CLAWDBOT_SKIP_CHANNELS=1 tsx src/entry.ts --dev gateway --reset",
"tui": "tsx src/entry.ts tui",
"tui:dev": "CLAWDBOT_PROFILE=dev tsx src/entry.ts tui",
"clawdbot:rpc": "tsx src/entry.ts agent --mode rpc --json",
"ios:gen": "cd apps/ios && xcodegen generate",
"ios:open": "cd apps/ios && xcodegen generate && open Clawdbot.xcodeproj",
"ios:build": "bash -lc 'cd apps/ios && xcodegen generate && xcodebuild -project Clawdbot.xcodeproj -scheme Clawdbot -destination \"${IOS_DEST:-platform=iOS Simulator,name=iPhone 17}\" -configuration Debug build'",
@@ -108,7 +108,7 @@
"test": "vitest run",
"test:watch": "vitest",
"test:ui": "pnpm --dir ui test",
"test:force": "bun scripts/test-force.ts",
"test:force": "tsx scripts/test-force.ts",
"test:coverage": "vitest run --coverage",
"test:e2e": "vitest run --config vitest.e2e.config.ts",
"test:live": "CLAWDBOT_LIVE_TEST=1 vitest run --config vitest.live.config.ts",
@@ -126,8 +126,8 @@
"test:install:smoke": "bash scripts/test-install-sh-docker.sh",
"test:install:e2e:openai": "CLAWDBOT_E2E_MODELS=openai bash scripts/test-install-sh-e2e-docker.sh",
"test:install:e2e:anthropic": "CLAWDBOT_E2E_MODELS=anthropic bash scripts/test-install-sh-e2e-docker.sh",
"protocol:gen": "bun scripts/protocol-gen.ts",
"protocol:gen:swift": "bun scripts/protocol-gen-swift.ts",
"protocol:gen": "tsx scripts/protocol-gen.ts",
"protocol:gen:swift": "tsx scripts/protocol-gen-swift.ts",
"protocol:check": "pnpm protocol:gen && pnpm protocol:gen:swift && git diff --exit-code -- dist/protocol.schema.json apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift",
"canvas:a2ui:bundle": "bash scripts/bundle-a2ui.sh",
"check:loc": "tsx scripts/check-ts-max-loc.ts --max 500"
-77
View File
@@ -4,28 +4,11 @@ set -euo pipefail
APP_BUNDLE="${1:-dist/Clawdbot.app}"
IDENTITY="${SIGN_IDENTITY:-}"
TIMESTAMP_MODE="${CODESIGN_TIMESTAMP:-auto}"
DISABLE_LIBRARY_VALIDATION="${DISABLE_LIBRARY_VALIDATION:-0}"
SKIP_TEAM_ID_CHECK="${SKIP_TEAM_ID_CHECK:-0}"
ENT_TMP_BASE=$(mktemp -t clawdbot-entitlements-base.XXXXXX)
ENT_TMP_APP=$(mktemp -t clawdbot-entitlements-app.XXXXXX)
ENT_TMP_APP_BASE=$(mktemp -t clawdbot-entitlements-app-base.XXXXXX)
ENT_TMP_RUNTIME=$(mktemp -t clawdbot-entitlements-runtime.XXXXXX)
if [[ "${APP_BUNDLE}" == "--help" || "${APP_BUNDLE}" == "-h" ]]; then
cat <<'HELP'
Usage: scripts/codesign-mac-app.sh [app-bundle]
Env:
SIGN_IDENTITY="Apple Development: Your Name (TEAMID)"
ALLOW_ADHOC_SIGNING=1
CODESIGN_TIMESTAMP=auto|on|off
DISABLE_LIBRARY_VALIDATION=1 # dev-only Sparkle Team ID workaround
SKIP_TEAM_ID_CHECK=1 # bypass Team ID audit
ENABLE_TIME_SENSITIVE_NOTIFICATIONS=1
HELP
exit 0
fi
if [ ! -d "$APP_BUNDLE" ]; then
echo "App bundle not found: $APP_BUNDLE" >&2
exit 1
@@ -201,14 +184,6 @@ cat > "$ENT_TMP_APP" <<'PLIST'
</plist>
PLIST
if [[ "$DISABLE_LIBRARY_VALIDATION" == "1" ]]; then
/usr/libexec/PlistBuddy -c "Add :com.apple.security.cs.disable-library-validation bool true" "$ENT_TMP_APP_BASE" >/dev/null 2>&1 || \
/usr/libexec/PlistBuddy -c "Set :com.apple.security.cs.disable-library-validation true" "$ENT_TMP_APP_BASE"
/usr/libexec/PlistBuddy -c "Add :com.apple.security.cs.disable-library-validation bool true" "$ENT_TMP_APP" >/dev/null 2>&1 || \
/usr/libexec/PlistBuddy -c "Set :com.apple.security.cs.disable-library-validation true" "$ENT_TMP_APP"
echo "Note: disable-library-validation entitlement enabled (DISABLE_LIBRARY_VALIDATION=1)."
fi
# The time-sensitive entitlement is restricted and requires explicit enablement
# (and typically a matching provisioning profile). It is *not* safe to enable
# unconditionally for local debug packaging since AMFI will refuse to launch.
@@ -234,51 +209,6 @@ sign_plain_item() {
codesign --force ${options_args+"${options_args[@]}"} "${timestamp_args[@]}" --sign "$IDENTITY" "$target"
}
team_id_for() {
codesign -dv --verbose=4 "$1" 2>&1 | awk -F= '/^TeamIdentifier=/{print $2; exit}'
}
verify_team_ids() {
if [[ "$SKIP_TEAM_ID_CHECK" == "1" ]]; then
echo "Note: skipping Team ID audit (SKIP_TEAM_ID_CHECK=1)."
return 0
fi
local expected
expected="$(team_id_for "$APP_BUNDLE" || true)"
if [[ -z "$expected" ]]; then
echo "WARN: TeamIdentifier missing on app bundle; skipping Team ID audit."
return 0
fi
local mismatches=()
while IFS= read -r -d '' f; do
if /usr/bin/file "$f" | /usr/bin/grep -q "Mach-O"; then
local team
team="$(team_id_for "$f" || true)"
if [[ -z "$team" ]]; then
team="not set"
fi
if [[ "$expected" == "not set" ]]; then
if [[ "$team" != "not set" ]]; then
mismatches+=("$f (TeamIdentifier=$team)")
fi
elif [[ "$team" != "$expected" ]]; then
mismatches+=("$f (TeamIdentifier=$team)")
fi
fi
done < <(find "$APP_BUNDLE" -type f -print0)
if [[ "${#mismatches[@]}" -gt 0 ]]; then
echo "ERROR: Team ID mismatch detected (expected: $expected)"
for entry in "${mismatches[@]}"; do
echo " - $entry"
done
echo "Hint: re-sign embedded frameworks or set DISABLE_LIBRARY_VALIDATION=1 for dev builds."
exit 1
fi
}
# Sign main binary
if [ -f "$APP_BUNDLE/Contents/MacOS/Clawdbot" ]; then
echo "Signing main binary"; sign_item "$APP_BUNDLE/Contents/MacOS/Clawdbot" "$APP_ENTITLEMENTS"
@@ -288,11 +218,6 @@ fi
SPARKLE="$APP_BUNDLE/Contents/Frameworks/Sparkle.framework"
if [ -d "$SPARKLE" ]; then
echo "Signing Sparkle framework and helpers"
find "$SPARKLE" -type f -print0 | while IFS= read -r -d '' f; do
if /usr/bin/file "$f" | /usr/bin/grep -q "Mach-O"; then
sign_plain_item "$f"
fi
done
sign_plain_item "$SPARKLE/Versions/B/Sparkle"
sign_plain_item "$SPARKLE/Versions/B/Autoupdate"
sign_plain_item "$SPARKLE/Versions/B/Updater.app/Contents/MacOS/Updater"
@@ -315,7 +240,5 @@ fi
# Finally sign the bundle
sign_item "$APP_BUNDLE" "$APP_ENTITLEMENTS"
verify_team_ids
rm -f "$ENT_TMP_BASE" "$ENT_TMP_APP_BASE" "$ENT_TMP_APP" "$ENT_TMP_RUNTIME"
echo "Codesign complete for $APP_BUNDLE"
-18
View File
@@ -22,21 +22,6 @@ describe("memory search config", () => {
expect(resolved).toBeNull();
});
it("defaults provider to auto when unspecified", () => {
const cfg = {
agents: {
defaults: {
memorySearch: {
enabled: true,
},
},
},
};
const resolved = resolveMemorySearchConfig(cfg, "main");
expect(resolved?.provider).toBe("auto");
expect(resolved?.fallback).toBe("none");
});
it("merges defaults and overrides", () => {
const cfg = {
agents: {
@@ -141,7 +126,6 @@ describe("memory search config", () => {
agents: {
defaults: {
memorySearch: {
provider: "openai",
remote: {
baseUrl: "https://default.example/v1",
apiKey: "default-key",
@@ -182,7 +166,6 @@ describe("memory search config", () => {
agents: {
defaults: {
memorySearch: {
provider: "openai",
sources: ["memory", "sessions"],
},
},
@@ -206,7 +189,6 @@ describe("memory search config", () => {
agents: {
defaults: {
memorySearch: {
provider: "openai",
sources: ["memory", "sessions"],
experimental: { sessionMemory: true },
},
+24 -30
View File
@@ -3,13 +3,13 @@ import path from "node:path";
import type { ClawdbotConfig, MemorySearchConfig } from "../config/config.js";
import { resolveStateDir } from "../config/paths.js";
import { clampInt, clampNumber, resolveUserPath } from "../utils.js";
import { resolveUserPath } from "../utils.js";
import { resolveAgentConfig } from "./agent-scope.js";
export type ResolvedMemorySearchConfig = {
enabled: boolean;
sources: Array<"memory" | "sessions">;
provider: "openai" | "local" | "gemini" | "auto";
provider: "openai" | "gemini" | "local";
remote?: {
baseUrl?: string;
apiKey?: string;
@@ -25,7 +25,7 @@ export type ResolvedMemorySearchConfig = {
experimental: {
sessionMemory: boolean;
};
fallback: "openai" | "gemini" | "local" | "none";
fallback: "openai" | "none";
model: string;
local: {
modelPath?: string;
@@ -110,40 +110,34 @@ function mergeConfig(
const enabled = overrides?.enabled ?? defaults?.enabled ?? true;
const sessionMemory =
overrides?.experimental?.sessionMemory ?? defaults?.experimental?.sessionMemory ?? false;
const provider = overrides?.provider ?? defaults?.provider ?? "auto";
const defaultRemote = defaults?.remote;
const overrideRemote = overrides?.remote;
const hasRemote = Boolean(defaultRemote || overrideRemote);
const includeRemote =
hasRemote || provider === "openai" || provider === "gemini" || provider === "auto";
const provider = overrides?.provider ?? defaults?.provider ?? "openai";
const hasRemote = Boolean(defaults?.remote || overrides?.remote);
const includeRemote = hasRemote || provider === "openai" || provider === "gemini";
const batch = {
enabled: overrideRemote?.batch?.enabled ?? defaultRemote?.batch?.enabled ?? true,
wait: overrideRemote?.batch?.wait ?? defaultRemote?.batch?.wait ?? true,
enabled: overrides?.remote?.batch?.enabled ?? defaults?.remote?.batch?.enabled ?? true,
wait: overrides?.remote?.batch?.wait ?? defaults?.remote?.batch?.wait ?? true,
concurrency: Math.max(
1,
overrideRemote?.batch?.concurrency ?? defaultRemote?.batch?.concurrency ?? 2,
overrides?.remote?.batch?.concurrency ?? defaults?.remote?.batch?.concurrency ?? 2,
),
pollIntervalMs:
overrideRemote?.batch?.pollIntervalMs ?? defaultRemote?.batch?.pollIntervalMs ?? 2000,
overrides?.remote?.batch?.pollIntervalMs ?? defaults?.remote?.batch?.pollIntervalMs ?? 2000,
timeoutMinutes:
overrideRemote?.batch?.timeoutMinutes ?? defaultRemote?.batch?.timeoutMinutes ?? 60,
overrides?.remote?.batch?.timeoutMinutes ?? defaults?.remote?.batch?.timeoutMinutes ?? 60,
};
const remote = includeRemote
? {
baseUrl: overrideRemote?.baseUrl ?? defaultRemote?.baseUrl,
apiKey: overrideRemote?.apiKey ?? defaultRemote?.apiKey,
headers: overrideRemote?.headers ?? defaultRemote?.headers,
baseUrl: overrides?.remote?.baseUrl ?? defaults?.remote?.baseUrl,
apiKey: overrides?.remote?.apiKey ?? defaults?.remote?.apiKey,
headers: overrides?.remote?.headers ?? defaults?.remote?.headers,
batch,
}
: undefined;
const fallback = overrides?.fallback ?? defaults?.fallback ?? "none";
const modelDefault =
provider === "gemini"
? DEFAULT_GEMINI_MODEL
: provider === "openai"
? DEFAULT_OPENAI_MODEL
: undefined;
const model = overrides?.model ?? defaults?.model ?? modelDefault ?? "";
const fallback = overrides?.fallback ?? defaults?.fallback ?? "openai";
const model =
overrides?.model ??
defaults?.model ??
(provider === "gemini" ? DEFAULT_GEMINI_MODEL : DEFAULT_OPENAI_MODEL);
const local = {
modelPath: overrides?.local?.modelPath ?? defaults?.local?.modelPath,
modelCacheDir: overrides?.local?.modelCacheDir ?? defaults?.local?.modelCacheDir,
@@ -200,14 +194,14 @@ function mergeConfig(
maxEntries: overrides?.cache?.maxEntries ?? defaults?.cache?.maxEntries,
};
const overlap = clampNumber(chunking.overlap, 0, Math.max(0, chunking.tokens - 1));
const minScore = clampNumber(query.minScore, 0, 1);
const vectorWeight = clampNumber(hybrid.vectorWeight, 0, 1);
const textWeight = clampNumber(hybrid.textWeight, 0, 1);
const overlap = Math.max(0, Math.min(chunking.overlap, chunking.tokens - 1));
const minScore = Math.max(0, Math.min(1, query.minScore));
const vectorWeight = Math.max(0, Math.min(1, hybrid.vectorWeight));
const textWeight = Math.max(0, Math.min(1, hybrid.textWeight));
const sum = vectorWeight + textWeight;
const normalizedVectorWeight = sum > 0 ? vectorWeight / sum : DEFAULT_HYBRID_VECTOR_WEIGHT;
const normalizedTextWeight = sum > 0 ? textWeight / sum : DEFAULT_HYBRID_TEXT_WEIGHT;
const candidateMultiplier = clampInt(hybrid.candidateMultiplier, 1, 20);
const candidateMultiplier = Math.max(1, Math.min(20, Math.floor(hybrid.candidateMultiplier)));
return {
enabled,
sources,
-3
View File
@@ -97,9 +97,6 @@ describe("buildAgentSystemPrompt", () => {
"Use `Read` to load the SKILL.md at the location listed for that skill.",
);
expect(prompt).toContain("Clawdbot docs: /tmp/clawd/docs");
expect(prompt).toContain(
"For Clawdbot behavior, commands, config, or architecture: consult local docs first.",
);
});
it("includes docs guidance when docsPath is provided", () => {
+5 -2
View File
@@ -5,7 +5,6 @@ import { resolveSandboxRuntimeStatus } from "../../agents/sandbox.js";
import { killProcessTree } from "../../agents/shell-utils.js";
import type { ClawdbotConfig } from "../../config/config.js";
import { logVerbose } from "../../globals.js";
import { clampInt } from "../../utils.js";
import type { MsgContext } from "../templating.js";
import type { ReplyPayload } from "../types.js";
import { stripMentions, stripStructuralPrefixes } from "./mentions.js";
@@ -32,10 +31,14 @@ type ActiveBashJob =
let activeJob: ActiveBashJob | null = null;
function clampNumber(value: number, min: number, max: number) {
return Math.min(Math.max(value, min), max);
}
function resolveForegroundMs(cfg: ClawdbotConfig): number {
const raw = cfg.commands?.bashForegroundMs;
if (typeof raw !== "number" || Number.isNaN(raw)) return DEFAULT_FOREGROUND_MS;
return clampInt(raw, 0, MAX_FOREGROUND_MS);
return clampNumber(Math.floor(raw), 0, MAX_FOREGROUND_MS);
}
function formatSessionSnippet(sessionId: string) {
+89 -2
View File
@@ -248,6 +248,69 @@ describe("memory cli", () => {
expect(close).toHaveBeenCalled();
});
it("applies index mode overrides", async () => {
const { registerMemoryCli } = await import("./memory-cli.js");
const close = vi.fn(async () => {});
const sync = vi.fn(async () => {});
const probeEmbeddingAvailability = vi.fn(async () => ({ ok: true }));
loadConfig.mockReturnValueOnce({
agents: {
defaults: {
memorySearch: {
remote: {
batch: { enabled: true },
},
},
},
},
});
getMemorySearchManager.mockResolvedValueOnce({
manager: {
probeVectorAvailability: vi.fn(async () => true),
probeEmbeddingAvailability,
sync,
status: () => ({
files: 1,
chunks: 1,
dirty: false,
workspaceDir: "/tmp/clawd",
dbPath: "/tmp/memory.sqlite",
provider: "openai",
model: "text-embedding-3-small",
requestedProvider: "openai",
vector: { enabled: true, available: true },
}),
close,
},
});
const program = new Command();
program.name("test");
registerMemoryCli(program);
await program.parseAsync(["memory", "status", "--index", "--index-mode", "direct"], {
from: "user",
});
expect(getMemorySearchManager).toHaveBeenCalledWith(
expect.objectContaining({
cfg: expect.objectContaining({
agents: expect.objectContaining({
defaults: expect.objectContaining({
memorySearch: expect.objectContaining({
remote: expect.objectContaining({
batch: expect.objectContaining({ enabled: false }),
}),
}),
}),
}),
}),
}),
);
expect(sync).toHaveBeenCalled();
expect(probeEmbeddingAvailability).toHaveBeenCalled();
expect(close).toHaveBeenCalled();
});
it("closes manager after index", async () => {
const { registerMemoryCli } = await import("./memory-cli.js");
const { defaultRuntime } = await import("../runtime.js");
@@ -266,9 +329,31 @@ describe("memory cli", () => {
registerMemoryCli(program);
await program.parseAsync(["memory", "index"], { from: "user" });
expect(sync).toHaveBeenCalledWith(
expect.objectContaining({ reason: "cli", force: false, progress: expect.any(Function) }),
);
expect(close).toHaveBeenCalled();
expect(log).toHaveBeenCalledWith("Memory index updated.");
});
it("skips progress when --progress none", async () => {
const { registerMemoryCli } = await import("./memory-cli.js");
const close = vi.fn(async () => {});
const sync = vi.fn(async () => {});
getMemorySearchManager.mockResolvedValueOnce({
manager: {
sync,
close,
},
});
const program = new Command();
program.name("test");
registerMemoryCli(program);
await program.parseAsync(["memory", "index", "--progress", "none"], { from: "user" });
expect(sync).toHaveBeenCalledWith({ reason: "cli", force: false });
expect(close).toHaveBeenCalled();
expect(log).toHaveBeenCalledWith("Memory index updated (main).");
});
it("logs close failures without failing the command", async () => {
@@ -291,7 +376,9 @@ describe("memory cli", () => {
registerMemoryCli(program);
await program.parseAsync(["memory", "index"], { from: "user" });
expect(sync).toHaveBeenCalledWith({ reason: "cli", force: false });
expect(sync).toHaveBeenCalledWith(
expect.objectContaining({ reason: "cli", force: false, progress: expect.any(Function) }),
);
expect(close).toHaveBeenCalled();
expect(error).toHaveBeenCalledWith(
expect.stringContaining("Memory manager close failed: close boom"),
+277 -241
View File
@@ -1,39 +1,36 @@
import os from "node:os";
import path from "node:path";
import type { Command } from "commander";
import { resolveDefaultAgentId } from "../agents/agent-scope.js";
import type { ClawdbotConfig } from "../config/config.js";
import type { MemorySearchConfig } from "../config/types.tools.js";
import { loadConfig } from "../config/config.js";
import { setVerbose } from "../globals.js";
import { withProgress, withProgressTotals } from "./progress.js";
import { formatErrorMessage, withManager } from "./cli-utils.js";
import { getMemorySearchManager, type MemorySearchManagerResult } from "../memory/index.js";
import {
resolveMemoryCacheState,
resolveMemoryFtsState,
resolveMemoryVectorState,
type Tone,
} from "../memory/status-format.js";
import { defaultRuntime } from "../runtime.js";
import { formatDocsLink } from "../terminal/links.js";
import { colorize, isRich, theme } from "../terminal/theme.js";
import { resolveStateDir } from "../config/paths.js";
type MemoryCommandOptions = {
agent?: string;
json?: boolean;
deep?: boolean;
index?: boolean;
indexMode?: IndexMode;
progress?: ProgressMode;
verbose?: boolean;
};
type MemoryManager = NonNullable<MemorySearchManagerResult["manager"]>;
function formatSourceLabel(source: string, workspaceDir: string, agentId: string): string {
if (source === "memory") {
return `memory (MEMORY.md + ${path.join(workspaceDir, "memory")}${path.sep}*.md)`;
}
if (source === "sessions") {
const stateDir = resolveStateDir(process.env, os.homedir);
return `sessions (${path.join(stateDir, "agents", agentId, "sessions")}${path.sep}*.jsonl)`;
}
return source;
}
type IndexMode = "auto" | "batch" | "direct";
type ProgressMode = "auto" | "line" | "log" | "none";
function resolveAgent(cfg: ReturnType<typeof loadConfig>, agent?: string) {
const trimmed = agent?.trim();
@@ -41,14 +38,66 @@ function resolveAgent(cfg: ReturnType<typeof loadConfig>, agent?: string) {
return resolveDefaultAgentId(cfg);
}
function resolveAgentIds(cfg: ReturnType<typeof loadConfig>, agent?: string): string[] {
const trimmed = agent?.trim();
if (trimmed) return [trimmed];
const list = cfg.agents?.list ?? [];
if (list.length > 0) {
return list.map((entry) => entry.id).filter(Boolean);
function resolveIndexMode(raw?: string): IndexMode {
if (!raw) return "auto";
const trimmed = raw.trim().toLowerCase();
if (trimmed === "batch") return "batch";
if (trimmed === "direct") return "direct";
return "auto";
}
function resolveProgressMode(raw?: string): ProgressMode {
if (!raw) return "auto";
const trimmed = raw.trim().toLowerCase();
if (trimmed === "line") return "line";
if (trimmed === "log") return "log";
if (trimmed === "none") return "none";
return "auto";
}
function applyIndexMode(cfg: ClawdbotConfig, agentId: string, mode: IndexMode): ClawdbotConfig {
if (mode === "auto") return cfg;
const enabled = mode === "batch";
const patchMemorySearch = (memorySearch?: MemorySearchConfig) => {
const remote = memorySearch?.remote;
const batch = remote?.batch;
return {
...memorySearch,
remote: {
...remote,
batch: {
...batch,
enabled,
},
},
};
};
const nextAgents = { ...cfg.agents };
nextAgents.defaults = {
...cfg.agents?.defaults,
memorySearch: patchMemorySearch(cfg.agents?.defaults?.memorySearch),
};
if (cfg.agents?.list?.length) {
nextAgents.list = cfg.agents.list.map((agent) =>
agent.id === agentId
? {
...agent,
memorySearch: patchMemorySearch(agent.memorySearch),
}
: agent,
);
}
return [resolveDefaultAgentId(cfg)];
return { ...cfg, agents: nextAgents };
}
function resolveProgressOptions(
mode: ProgressMode,
verbose: boolean,
): { enabled?: boolean; fallback?: "spinner" | "line" | "log" | "none" } {
if (mode === "none") return { enabled: false, fallback: "none" };
if (mode === "line") return { fallback: "line" };
if (mode === "log") return { fallback: "log" };
return { fallback: verbose ? "line" : undefined };
}
export function registerMemoryCli(program: Command) {
@@ -68,194 +117,182 @@ export function registerMemoryCli(program: Command) {
.option("--json", "Print JSON")
.option("--deep", "Probe embedding provider availability")
.option("--index", "Reindex if dirty (implies --deep)")
.option("--index-mode <mode>", "Index mode (auto|batch|direct) when indexing", "auto")
.option("--progress <mode>", "Progress output (auto|line|log|none)", "auto")
.option("--verbose", "Verbose logging", false)
.action(async (opts: MemoryCommandOptions) => {
setVerbose(Boolean(opts.verbose));
const cfg = loadConfig();
const agentIds = resolveAgentIds(cfg, opts.agent);
const allResults: Array<{
agentId: string;
status: ReturnType<MemoryManager["status"]>;
embeddingProbe?: Awaited<ReturnType<MemoryManager["probeEmbeddingAvailability"]>>;
indexError?: string;
}> = [];
for (const agentId of agentIds) {
await withManager<MemoryManager>({
getManager: () => getMemorySearchManager({ cfg, agentId }),
onMissing: (error) => defaultRuntime.log(error ?? "Memory search disabled."),
onCloseError: (err) =>
defaultRuntime.error(`Memory manager close failed: ${formatErrorMessage(err)}`),
close: (manager) => manager.close(),
run: async (manager) => {
const deep = Boolean(opts.deep || opts.index);
let embeddingProbe:
| Awaited<ReturnType<typeof manager.probeEmbeddingAvailability>>
| undefined;
let indexError: string | undefined;
if (deep) {
await withProgress({ label: "Checking memory…", total: 2 }, async (progress) => {
const rawCfg = loadConfig();
const agentId = resolveAgent(rawCfg, opts.agent);
const indexMode = resolveIndexMode(opts.indexMode);
const progressMode = resolveProgressMode(opts.progress);
const progressOptions = resolveProgressOptions(progressMode, Boolean(opts.verbose));
const cfg = applyIndexMode(rawCfg, agentId, indexMode);
await withManager<MemoryManager>({
getManager: () => getMemorySearchManager({ cfg, agentId }),
onMissing: (error) => defaultRuntime.log(error ?? "Memory search disabled."),
onCloseError: (err) =>
defaultRuntime.error(`Memory manager close failed: ${formatErrorMessage(err)}`),
close: (manager) => manager.close(),
run: async (manager) => {
const deep = Boolean(opts.deep || opts.index);
let embeddingProbe:
| Awaited<ReturnType<typeof manager.probeEmbeddingAvailability>>
| undefined;
let indexError: string | undefined;
if (deep) {
await withProgress(
{ label: "Checking memory…", total: 2, ...progressOptions },
async (progress) => {
progress.setLabel("Probing vector…");
await manager.probeVectorAvailability();
progress.tick();
progress.setLabel("Probing embeddings…");
embeddingProbe = await manager.probeEmbeddingAvailability();
progress.tick();
});
if (opts.index) {
await withProgressTotals(
{
label: "Indexing memory…",
total: 0,
fallback: opts.verbose ? "line" : undefined,
},
async (update, progress) => {
try {
await manager.sync({
reason: "cli",
progress: (syncUpdate) => {
update({
completed: syncUpdate.completed,
total: syncUpdate.total,
label: syncUpdate.label,
});
if (syncUpdate.label) progress.setLabel(syncUpdate.label);
},
});
} catch (err) {
indexError = formatErrorMessage(err);
defaultRuntime.error(`Memory index failed: ${indexError}`);
process.exitCode = 1;
}
},
);
}
} else {
await manager.probeVectorAvailability();
},
);
if (opts.index) {
await withProgressTotals(
{
label: "Indexing memory…",
total: 0,
...progressOptions,
},
async (update, progress) => {
try {
await manager.sync({
reason: "cli",
progress: (syncUpdate) => {
update({
completed: syncUpdate.completed,
total: syncUpdate.total,
label: syncUpdate.label,
});
if (syncUpdate.label) progress.setLabel(syncUpdate.label);
},
});
} catch (err) {
indexError = formatErrorMessage(err);
defaultRuntime.error(`Memory index failed: ${indexError}`);
process.exitCode = 1;
}
},
);
}
const status = manager.status();
allResults.push({ agentId, status, embeddingProbe, indexError });
},
});
}
if (opts.json) {
defaultRuntime.log(JSON.stringify(allResults, null, 2));
return;
}
const rich = isRich();
const heading = (text: string) => colorize(rich, theme.heading, text);
const muted = (text: string) => colorize(rich, theme.muted, text);
const info = (text: string) => colorize(rich, theme.info, text);
const success = (text: string) => colorize(rich, theme.success, text);
const warn = (text: string) => colorize(rich, theme.warn, text);
const accent = (text: string) => colorize(rich, theme.accent, text);
const label = (text: string) => muted(`${text}:`);
for (const result of allResults) {
const { agentId, status, embeddingProbe, indexError } = result;
if (opts.index) {
const line = indexError
? `Memory index failed: ${indexError}`
: "Memory index complete.";
defaultRuntime.log(line);
}
const lines = [
`${heading("Memory Search")} ${muted(`(${agentId})`)}`,
`${label("Provider")} ${info(status.provider)} ${muted(
`(requested: ${status.requestedProvider})`,
)}`,
`${label("Model")} ${info(status.model)}`,
status.sources?.length
? `${label("Sources")} ${info(status.sources.join(", "))}`
: null,
`${label("Indexed")} ${success(`${status.files} files · ${status.chunks} chunks`)}`,
`${label("Dirty")} ${status.dirty ? warn("yes") : muted("no")}`,
`${label("Store")} ${info(status.dbPath)}`,
`${label("Workspace")} ${info(status.workspaceDir)}`,
].filter(Boolean) as string[];
if (embeddingProbe) {
const state = embeddingProbe.ok ? "ready" : "unavailable";
const stateColor = embeddingProbe.ok ? theme.success : theme.warn;
lines.push(`${label("Embeddings")} ${colorize(rich, stateColor, state)}`);
if (embeddingProbe.error) {
lines.push(`${label("Embeddings error")} ${warn(embeddingProbe.error)}`);
} else {
await manager.probeVectorAvailability();
}
}
if (status.sourceCounts?.length) {
lines.push(label("By source"));
for (const entry of status.sourceCounts) {
const counts = `${entry.files} files · ${entry.chunks} chunks`;
lines.push(` ${accent(entry.source)} ${muted("·")} ${muted(counts)}`);
const status = manager.status();
if (opts.json) {
defaultRuntime.log(
JSON.stringify(
{
...status,
embeddings: embeddingProbe
? { ok: embeddingProbe.ok, error: embeddingProbe.error }
: undefined,
indexError,
},
null,
2,
),
);
return;
}
}
if (status.fallback) {
lines.push(`${label("Fallback")} ${warn(status.fallback.from)}`);
}
if (status.vector) {
const vectorState = status.vector.enabled
? status.vector.available
? "ready"
: "unavailable"
: "disabled";
const vectorColor =
vectorState === "ready"
? theme.success
: vectorState === "unavailable"
? theme.warn
: theme.muted;
lines.push(`${label("Vector")} ${colorize(rich, vectorColor, vectorState)}`);
if (status.vector.dims) {
lines.push(`${label("Vector dims")} ${info(String(status.vector.dims))}`);
if (opts.index) {
const line = indexError
? `Memory index failed: ${indexError}`
: "Memory index complete.";
defaultRuntime.log(line);
}
if (status.vector.extensionPath) {
lines.push(`${label("Vector path")} ${info(status.vector.extensionPath)}`);
const rich = isRich();
const heading = (text: string) => colorize(rich, theme.heading, text);
const muted = (text: string) => colorize(rich, theme.muted, text);
const info = (text: string) => colorize(rich, theme.info, text);
const success = (text: string) => colorize(rich, theme.success, text);
const warn = (text: string) => colorize(rich, theme.warn, text);
const accent = (text: string) => colorize(rich, theme.accent, text);
const label = (text: string) => muted(`${text}:`);
const colorForTone = (tone: Tone) =>
tone === "ok" ? theme.success : tone === "warn" ? theme.warn : theme.muted;
const lines = [
`${heading("Memory Search")} ${muted(`(${agentId})`)}`,
`${label("Provider")} ${info(status.provider)} ${muted(
`(requested: ${status.requestedProvider})`,
)}`,
`${label("Model")} ${info(status.model)}`,
status.sources?.length
? `${label("Sources")} ${info(status.sources.join(", "))}`
: null,
`${label("Indexed")} ${success(`${status.files} files · ${status.chunks} chunks`)}`,
`${label("Dirty")} ${status.dirty ? warn("yes") : muted("no")}`,
`${label("Store")} ${info(status.dbPath)}`,
`${label("Workspace")} ${info(status.workspaceDir)}`,
].filter(Boolean) as string[];
if (embeddingProbe) {
const state = embeddingProbe.ok ? "ready" : "unavailable";
const stateColor = embeddingProbe.ok ? theme.success : theme.warn;
lines.push(`${label("Embeddings")} ${colorize(rich, stateColor, state)}`);
if (embeddingProbe.error) {
lines.push(`${label("Embeddings error")} ${warn(embeddingProbe.error)}`);
}
}
if (status.vector.loadError) {
lines.push(`${label("Vector error")} ${warn(status.vector.loadError)}`);
if (status.sourceCounts?.length) {
lines.push(label("By source"));
for (const entry of status.sourceCounts) {
const counts = `${entry.files} files · ${entry.chunks} chunks`;
lines.push(` ${accent(entry.source)} ${muted("·")} ${muted(counts)}`);
}
}
}
if (status.fts) {
const ftsState = status.fts.enabled
? status.fts.available
? "ready"
: "unavailable"
: "disabled";
const ftsColor =
ftsState === "ready"
? theme.success
: ftsState === "unavailable"
? theme.warn
: theme.muted;
lines.push(`${label("FTS")} ${colorize(rich, ftsColor, ftsState)}`);
if (status.fts.error) {
lines.push(`${label("FTS error")} ${warn(status.fts.error)}`);
if (status.fallback) {
lines.push(`${label("Fallback")} ${warn(status.fallback.from)}`);
}
}
if (status.cache) {
const cacheState = status.cache.enabled ? "enabled" : "disabled";
const cacheColor = status.cache.enabled ? theme.success : theme.muted;
const suffix =
status.cache.enabled && typeof status.cache.entries === "number"
? ` (${status.cache.entries} entries)`
: "";
lines.push(
`${label("Embedding cache")} ${colorize(rich, cacheColor, cacheState)}${suffix}`,
);
if (status.cache.enabled && typeof status.cache.maxEntries === "number") {
lines.push(`${label("Cache cap")} ${info(String(status.cache.maxEntries))}`);
if (status.vector) {
const vectorState = resolveMemoryVectorState(status.vector);
const vectorColor = colorForTone(vectorState.tone);
lines.push(`${label("Vector")} ${colorize(rich, vectorColor, vectorState.state)}`);
if (status.vector.dims) {
lines.push(`${label("Vector dims")} ${info(String(status.vector.dims))}`);
}
if (status.vector.extensionPath) {
lines.push(`${label("Vector path")} ${info(status.vector.extensionPath)}`);
}
if (status.vector.loadError) {
lines.push(`${label("Vector error")} ${warn(status.vector.loadError)}`);
}
}
}
if (status.fallback?.reason) {
lines.push(muted(status.fallback.reason));
}
if (indexError) {
lines.push(`${label("Index error")} ${warn(indexError)}`);
}
defaultRuntime.log(lines.join("\n"));
if (agentIds.length > 1) defaultRuntime.log("");
}
if (status.fts) {
const ftsState = resolveMemoryFtsState(status.fts);
const ftsColor = colorForTone(ftsState.tone);
lines.push(`${label("FTS")} ${colorize(rich, ftsColor, ftsState.state)}`);
if (status.fts.error) {
lines.push(`${label("FTS error")} ${warn(status.fts.error)}`);
}
}
if (status.cache) {
const cacheState = resolveMemoryCacheState(status.cache);
const cacheColor = colorForTone(cacheState.tone);
const suffix =
status.cache.enabled && typeof status.cache.entries === "number"
? ` (${status.cache.entries} entries)`
: "";
lines.push(
`${label("Embedding cache")} ${colorize(rich, cacheColor, cacheState.state)}${suffix}`,
);
if (status.cache.enabled && typeof status.cache.maxEntries === "number") {
lines.push(`${label("Cache cap")} ${info(String(status.cache.maxEntries))}`);
}
}
if (status.fallback?.reason) {
lines.push(muted(status.fallback.reason));
}
if (indexError) {
lines.push(`${label("Index error")} ${warn(indexError)}`);
}
defaultRuntime.log(lines.join("\n"));
},
});
});
memory
@@ -263,57 +300,56 @@ export function registerMemoryCli(program: Command) {
.description("Reindex memory files")
.option("--agent <id>", "Agent id (default: default agent)")
.option("--force", "Force full reindex", false)
.option("--verbose", "Verbose logging", false)
.option("--index-mode <mode>", "Index mode (auto|batch|direct) when indexing", "auto")
.option("--progress <mode>", "Progress output (auto|line|log|none)", "auto")
.action(async (opts: MemoryCommandOptions & { force?: boolean }) => {
setVerbose(Boolean(opts.verbose));
const cfg = loadConfig();
const agentIds = resolveAgentIds(cfg, opts.agent);
for (const agentId of agentIds) {
await withManager<MemoryManager>({
getManager: () => getMemorySearchManager({ cfg, agentId }),
onMissing: (error) => defaultRuntime.log(error ?? "Memory search disabled."),
onCloseError: (err) =>
defaultRuntime.error(`Memory manager close failed: ${formatErrorMessage(err)}`),
close: (manager) => manager.close(),
run: async (manager) => {
try {
if (opts.verbose) {
const status = manager.status();
const rich = isRich();
const heading = (text: string) => colorize(rich, theme.heading, text);
const muted = (text: string) => colorize(rich, theme.muted, text);
const info = (text: string) => colorize(rich, theme.info, text);
const warn = (text: string) => colorize(rich, theme.warn, text);
const label = (text: string) => muted(`${text}:`);
const sourceLabels = status.sources.map((source) =>
formatSourceLabel(source, status.workspaceDir, agentId),
);
const lines = [
`${heading("Memory Index")} ${muted(`(${agentId})`)}`,
`${label("Provider")} ${info(status.provider)} ${muted(
`(requested: ${status.requestedProvider})`,
)}`,
`${label("Model")} ${info(status.model)}`,
sourceLabels.length
? `${label("Sources")} ${info(sourceLabels.join(", "))}`
: null,
].filter(Boolean) as string[];
if (status.fallback) {
lines.push(`${label("Fallback")} ${warn(status.fallback.from)}`);
}
defaultRuntime.log(lines.join("\n"));
defaultRuntime.log("");
}
const rawCfg = loadConfig();
const agentId = resolveAgent(rawCfg, opts.agent);
const indexMode = resolveIndexMode(opts.indexMode);
const progressMode = resolveProgressMode(opts.progress);
const progressOptions = resolveProgressOptions(progressMode, Boolean(opts.verbose));
const cfg = applyIndexMode(rawCfg, agentId, indexMode);
await withManager<MemoryManager>({
getManager: () => getMemorySearchManager({ cfg, agentId }),
onMissing: (error) => defaultRuntime.log(error ?? "Memory search disabled."),
onCloseError: (err) =>
defaultRuntime.error(`Memory manager close failed: ${formatErrorMessage(err)}`),
close: (manager) => manager.close(),
run: async (manager) => {
try {
if (progressMode === "none") {
await manager.sync({ reason: "cli", force: opts.force });
defaultRuntime.log(`Memory index updated (${agentId}).`);
} catch (err) {
const message = formatErrorMessage(err);
defaultRuntime.error(`Memory index failed (${agentId}): ${message}`);
process.exitCode = 1;
} else {
await withProgressTotals(
{
label: "Indexing memory…",
total: 0,
...progressOptions,
},
async (update, progress) => {
await manager.sync({
reason: "cli",
force: opts.force,
progress: (syncUpdate) => {
update({
completed: syncUpdate.completed,
total: syncUpdate.total,
label: syncUpdate.label,
});
if (syncUpdate.label) progress.setLabel(syncUpdate.label);
},
});
},
);
}
},
});
}
defaultRuntime.log("Memory index updated.");
} catch (err) {
const message = formatErrorMessage(err);
defaultRuntime.error(`Memory index failed: ${message}`);
process.exitCode = 1;
}
},
});
});
memory
+1 -39
View File
@@ -13,7 +13,6 @@ import {
resolvePluginInstallDir,
} from "../plugins/install.js";
import { recordPluginInstall } from "../plugins/installs.js";
import { applyExclusiveSlotSelection } from "../plugins/slots.js";
import type { PluginRecord } from "../plugins/registry.js";
import { buildPluginStatusReport } from "../plugins/status.js";
import { defaultRuntime } from "../runtime.js";
@@ -80,31 +79,6 @@ async function readInstalledPackageVersion(dir: string): Promise<string | undefi
}
}
function applySlotSelectionForPlugin(
config: ClawdbotConfig,
pluginId: string,
): { config: ClawdbotConfig; warnings: string[] } {
const report = buildPluginStatusReport({ config });
const plugin = report.plugins.find((entry) => entry.id === pluginId);
if (!plugin) {
return { config, warnings: [] };
}
const result = applyExclusiveSlotSelection({
config,
selectedId: plugin.id,
selectedKind: plugin.kind,
registry: report,
});
return { config: result.config, warnings: result.warnings };
}
function logSlotWarnings(warnings: string[]) {
if (warnings.length === 0) return;
for (const warning of warnings) {
defaultRuntime.log(chalk.yellow(warning));
}
}
export function registerPluginsCli(program: Command) {
const plugins = program
.command("plugins")
@@ -223,7 +197,7 @@ export function registerPluginsCli(program: Command) {
.argument("<id>", "Plugin id")
.action(async (id: string) => {
const cfg = loadConfig();
let next: ClawdbotConfig = {
const next = {
...cfg,
plugins: {
...cfg.plugins,
@@ -236,10 +210,7 @@ export function registerPluginsCli(program: Command) {
},
},
};
const slotResult = applySlotSelectionForPlugin(next, id);
next = slotResult.config;
await writeConfigFile(next);
logSlotWarnings(slotResult.warnings);
defaultRuntime.log(`Enabled plugin "${id}". Restart the gateway to apply.`);
});
@@ -309,10 +280,7 @@ export function registerPluginsCli(program: Command) {
installPath: resolved,
version: probe.version,
});
const slotResult = applySlotSelectionForPlugin(next, probe.pluginId);
next = slotResult.config;
await writeConfigFile(next);
logSlotWarnings(slotResult.warnings);
defaultRuntime.log(`Linked plugin path: ${resolved}`);
defaultRuntime.log(`Restart the gateway to load plugins.`);
return;
@@ -351,10 +319,7 @@ export function registerPluginsCli(program: Command) {
installPath: result.targetDir,
version: result.version,
});
const slotResult = applySlotSelectionForPlugin(next, result.pluginId);
next = slotResult.config;
await writeConfigFile(next);
logSlotWarnings(slotResult.warnings);
defaultRuntime.log(`Installed plugin: ${result.pluginId}`);
defaultRuntime.log(`Restart the gateway to load plugins.`);
return;
@@ -414,10 +379,7 @@ export function registerPluginsCli(program: Command) {
installPath: result.targetDir,
version: result.version,
});
const slotResult = applySlotSelectionForPlugin(next, result.pluginId);
next = slotResult.config;
await writeConfigFile(next);
logSlotWarnings(slotResult.warnings);
defaultRuntime.log(`Installed plugin: ${result.pluginId}`);
defaultRuntime.log(`Restart the gateway to load plugins.`);
});
+2 -24
View File
@@ -13,7 +13,6 @@ export type AuthChoiceGroupId =
| "openai"
| "anthropic"
| "google"
| "copilot"
| "openrouter"
| "ai-gateway"
| "moonshot"
@@ -69,14 +68,8 @@ const AUTH_CHOICE_GROUP_DEFS: {
{
value: "google",
label: "Google",
hint: "Gemini API key + OAuth",
choices: ["gemini-api-key", "google-antigravity", "google-gemini-cli"],
},
{
value: "copilot",
label: "Copilot",
hint: "GitHub + local proxy",
choices: ["github-copilot", "copilot-proxy"],
hint: "Gemini API key",
choices: ["gemini-api-key"],
},
{
value: "openrouter",
@@ -202,23 +195,8 @@ export function buildAuthChoiceOptions(params: {
hint: "Uses GitHub device flow",
});
options.push({ value: "gemini-api-key", label: "Google Gemini API key" });
options.push({
value: "google-antigravity",
label: "Google Antigravity OAuth",
hint: "Uses the bundled Antigravity auth plugin",
});
options.push({
value: "google-gemini-cli",
label: "Google Gemini CLI OAuth",
hint: "Uses the bundled Gemini CLI auth plugin",
});
options.push({ value: "zai-api-key", label: "Z.AI (GLM 4.7) API key" });
options.push({ value: "qwen-portal", label: "Qwen OAuth" });
options.push({
value: "copilot-proxy",
label: "Copilot Proxy (local)",
hint: "Local proxy for VS Code Copilot models",
});
options.push({ value: "apiKey", label: "Anthropic API key" });
// Token flow is currently Anthropic-only; use CLI for advanced providers.
options.push({
@@ -1,14 +0,0 @@
import type { ApplyAuthChoiceParams, ApplyAuthChoiceResult } from "./auth-choice.apply.js";
import { applyAuthChoicePluginProvider } from "./auth-choice.apply.plugin-provider.js";
export async function applyAuthChoiceCopilotProxy(
params: ApplyAuthChoiceParams,
): Promise<ApplyAuthChoiceResult | null> {
return await applyAuthChoicePluginProvider(params, {
authChoice: "copilot-proxy",
pluginId: "copilot-proxy",
providerId: "copilot-proxy",
methodId: "local",
label: "Copilot Proxy",
});
}
@@ -1,14 +0,0 @@
import type { ApplyAuthChoiceParams, ApplyAuthChoiceResult } from "./auth-choice.apply.js";
import { applyAuthChoicePluginProvider } from "./auth-choice.apply.plugin-provider.js";
export async function applyAuthChoiceGoogleAntigravity(
params: ApplyAuthChoiceParams,
): Promise<ApplyAuthChoiceResult | null> {
return await applyAuthChoicePluginProvider(params, {
authChoice: "google-antigravity",
pluginId: "google-antigravity-auth",
providerId: "google-antigravity",
methodId: "oauth",
label: "Google Antigravity",
});
}
@@ -1,14 +0,0 @@
import type { ApplyAuthChoiceParams, ApplyAuthChoiceResult } from "./auth-choice.apply.js";
import { applyAuthChoicePluginProvider } from "./auth-choice.apply.plugin-provider.js";
export async function applyAuthChoiceGoogleGeminiCli(
params: ApplyAuthChoiceParams,
): Promise<ApplyAuthChoiceResult | null> {
return await applyAuthChoicePluginProvider(params, {
authChoice: "google-gemini-cli",
pluginId: "google-gemini-cli-auth",
providerId: "google-gemini-cli",
methodId: "oauth",
label: "Google Gemini CLI",
});
}
@@ -1,197 +0,0 @@
import { resolveClawdbotAgentDir } from "../agents/agent-paths.js";
import {
resolveDefaultAgentId,
resolveAgentDir,
resolveAgentWorkspaceDir,
} from "../agents/agent-scope.js";
import { upsertAuthProfile } from "../agents/auth-profiles.js";
import { normalizeProviderId } from "../agents/model-selection.js";
import { resolveDefaultAgentWorkspaceDir } from "../agents/workspace.js";
import type { ClawdbotConfig } from "../config/config.js";
import { enablePluginInConfig } from "../plugins/enable.js";
import { resolvePluginProviders } from "../plugins/providers.js";
import type { ProviderAuthMethod, ProviderPlugin } from "../plugins/types.js";
import type { ApplyAuthChoiceParams, ApplyAuthChoiceResult } from "./auth-choice.apply.js";
import { applyAuthProfileConfig } from "./onboard-auth.js";
import { openUrl } from "./onboard-helpers.js";
import { createVpsAwareOAuthHandlers } from "./oauth-flow.js";
import { isRemoteEnvironment } from "./oauth-env.js";
export type PluginProviderAuthChoiceOptions = {
authChoice: string;
pluginId: string;
providerId: string;
methodId?: string;
label: string;
};
function resolveProviderMatch(
providers: ProviderPlugin[],
rawProvider: string,
): ProviderPlugin | null {
const normalized = normalizeProviderId(rawProvider);
return (
providers.find((provider) => normalizeProviderId(provider.id) === normalized) ??
providers.find(
(provider) =>
provider.aliases?.some((alias) => normalizeProviderId(alias) === normalized) ?? false,
) ??
null
);
}
function pickAuthMethod(provider: ProviderPlugin, rawMethod?: string): ProviderAuthMethod | null {
const raw = rawMethod?.trim();
if (!raw) return null;
const normalized = raw.toLowerCase();
return (
provider.auth.find((method) => method.id.toLowerCase() === normalized) ??
provider.auth.find((method) => method.label.toLowerCase() === normalized) ??
null
);
}
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === "object" && !Array.isArray(value));
}
function mergeConfigPatch<T>(base: T, patch: unknown): T {
if (!isPlainRecord(base) || !isPlainRecord(patch)) {
return patch as T;
}
const next: Record<string, unknown> = { ...base };
for (const [key, value] of Object.entries(patch)) {
const existing = next[key];
if (isPlainRecord(existing) && isPlainRecord(value)) {
next[key] = mergeConfigPatch(existing, value);
} else {
next[key] = value;
}
}
return next as T;
}
function applyDefaultModel(cfg: ClawdbotConfig, model: string): ClawdbotConfig {
const models = { ...cfg.agents?.defaults?.models };
models[model] = models[model] ?? {};
const existingModel = cfg.agents?.defaults?.model;
return {
...cfg,
agents: {
...cfg.agents,
defaults: {
...cfg.agents?.defaults,
models,
model: {
...(existingModel && typeof existingModel === "object" && "fallbacks" in existingModel
? { fallbacks: (existingModel as { fallbacks?: string[] }).fallbacks }
: undefined),
primary: model,
},
},
},
};
}
export async function applyAuthChoicePluginProvider(
params: ApplyAuthChoiceParams,
options: PluginProviderAuthChoiceOptions,
): Promise<ApplyAuthChoiceResult | null> {
if (params.authChoice !== options.authChoice) return null;
const enableResult = enablePluginInConfig(params.config, options.pluginId);
let nextConfig = enableResult.config;
if (!enableResult.enabled) {
await params.prompter.note(
`${options.label} plugin is disabled (${enableResult.reason ?? "blocked"}).`,
options.label,
);
return { config: nextConfig };
}
const agentId = params.agentId ?? resolveDefaultAgentId(nextConfig);
const defaultAgentId = resolveDefaultAgentId(nextConfig);
const agentDir =
params.agentDir ??
(agentId === defaultAgentId ? resolveClawdbotAgentDir() : resolveAgentDir(nextConfig, agentId));
const workspaceDir =
resolveAgentWorkspaceDir(nextConfig, agentId) ?? resolveDefaultAgentWorkspaceDir();
const providers = resolvePluginProviders({ config: nextConfig, workspaceDir });
const provider = resolveProviderMatch(providers, options.providerId);
if (!provider) {
await params.prompter.note(
`${options.label} auth plugin is not available. Enable it and re-run the wizard.`,
options.label,
);
return { config: nextConfig };
}
const method = pickAuthMethod(provider, options.methodId) ?? provider.auth[0];
if (!method) {
await params.prompter.note(`${options.label} auth method missing.`, options.label);
return { config: nextConfig };
}
const isRemote = isRemoteEnvironment();
const result = await method.run({
config: nextConfig,
agentDir,
workspaceDir,
prompter: params.prompter,
runtime: params.runtime,
isRemote,
openUrl: async (url) => {
await openUrl(url);
},
oauth: {
createVpsAwareHandlers: (opts) => createVpsAwareOAuthHandlers(opts),
},
});
if (result.configPatch) {
nextConfig = mergeConfigPatch(nextConfig, result.configPatch);
}
for (const profile of result.profiles) {
upsertAuthProfile({
profileId: profile.profileId,
credential: profile.credential,
agentDir,
});
nextConfig = applyAuthProfileConfig(nextConfig, {
profileId: profile.profileId,
provider: profile.credential.provider,
mode: profile.credential.type === "token" ? "token" : profile.credential.type,
...("email" in profile.credential && profile.credential.email
? { email: profile.credential.email }
: {}),
});
}
let agentModelOverride: string | undefined;
if (result.defaultModel) {
if (params.setDefaultModel) {
nextConfig = applyDefaultModel(nextConfig, result.defaultModel);
await params.prompter.note(
`Default model set to ${result.defaultModel}`,
"Model configured",
);
} else if (params.agentId) {
agentModelOverride = result.defaultModel;
await params.prompter.note(
`Default model set to ${result.defaultModel} for agent "${params.agentId}".`,
"Model configured",
);
}
}
if (result.notes && result.notes.length > 0) {
await params.prompter.note(result.notes.join("\n"), "Provider notes");
}
return { config: nextConfig, agentModelOverride };
}
+188 -7
View File
@@ -1,14 +1,195 @@
import { resolveClawdbotAgentDir } from "../agents/agent-paths.js";
import {
resolveDefaultAgentId,
resolveAgentDir,
resolveAgentWorkspaceDir,
} from "../agents/agent-scope.js";
import { upsertAuthProfile } from "../agents/auth-profiles.js";
import { normalizeProviderId } from "../agents/model-selection.js";
import { resolveDefaultAgentWorkspaceDir } from "../agents/workspace.js";
import type { ClawdbotConfig } from "../config/config.js";
import { resolvePluginProviders } from "../plugins/providers.js";
import type { ProviderAuthMethod, ProviderPlugin } from "../plugins/types.js";
import type { ApplyAuthChoiceParams, ApplyAuthChoiceResult } from "./auth-choice.apply.js";
import { applyAuthChoicePluginProvider } from "./auth-choice.apply.plugin-provider.js";
import { applyAuthProfileConfig } from "./onboard-auth.js";
import { openUrl } from "./onboard-helpers.js";
import { createVpsAwareOAuthHandlers } from "./oauth-flow.js";
import { isRemoteEnvironment } from "./oauth-env.js";
const PLUGIN_ID = "qwen-portal-auth";
const PROVIDER_ID = "qwen-portal";
function enableBundledPlugin(cfg: ClawdbotConfig): ClawdbotConfig {
const existingEntry = cfg.plugins?.entries?.[PLUGIN_ID];
return {
...cfg,
plugins: {
...cfg.plugins,
entries: {
...cfg.plugins?.entries,
[PLUGIN_ID]: {
...existingEntry,
enabled: true,
},
},
},
};
}
function resolveProviderMatch(
providers: ProviderPlugin[],
rawProvider: string,
): ProviderPlugin | null {
const normalized = normalizeProviderId(rawProvider);
return (
providers.find((provider) => normalizeProviderId(provider.id) === normalized) ??
providers.find(
(provider) =>
provider.aliases?.some((alias) => normalizeProviderId(alias) === normalized) ?? false,
) ??
null
);
}
function pickAuthMethod(provider: ProviderPlugin, rawMethod?: string): ProviderAuthMethod | null {
const raw = rawMethod?.trim();
if (!raw) return null;
const normalized = raw.toLowerCase();
return (
provider.auth.find((method) => method.id.toLowerCase() === normalized) ??
provider.auth.find((method) => method.label.toLowerCase() === normalized) ??
null
);
}
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === "object" && !Array.isArray(value));
}
function mergeConfigPatch<T>(base: T, patch: unknown): T {
if (!isPlainRecord(base) || !isPlainRecord(patch)) {
return patch as T;
}
const next: Record<string, unknown> = { ...base };
for (const [key, value] of Object.entries(patch)) {
const existing = next[key];
if (isPlainRecord(existing) && isPlainRecord(value)) {
next[key] = mergeConfigPatch(existing, value);
} else {
next[key] = value;
}
}
return next as T;
}
function applyDefaultModel(cfg: ClawdbotConfig, model: string): ClawdbotConfig {
const models = { ...cfg.agents?.defaults?.models };
models[model] = models[model] ?? {};
const existingModel = cfg.agents?.defaults?.model;
return {
...cfg,
agents: {
...cfg.agents,
defaults: {
...cfg.agents?.defaults,
models,
model: {
...(existingModel && typeof existingModel === "object" && "fallbacks" in existingModel
? { fallbacks: (existingModel as { fallbacks?: string[] }).fallbacks }
: undefined),
primary: model,
},
},
},
};
}
export async function applyAuthChoiceQwenPortal(
params: ApplyAuthChoiceParams,
): Promise<ApplyAuthChoiceResult | null> {
return await applyAuthChoicePluginProvider(params, {
authChoice: "qwen-portal",
pluginId: "qwen-portal-auth",
providerId: "qwen-portal",
methodId: "device",
label: "Qwen",
if (params.authChoice !== "qwen-portal") return null;
let nextConfig = enableBundledPlugin(params.config);
const agentId = params.agentId ?? resolveDefaultAgentId(nextConfig);
const defaultAgentId = resolveDefaultAgentId(nextConfig);
const agentDir =
params.agentDir ??
(agentId === defaultAgentId ? resolveClawdbotAgentDir() : resolveAgentDir(nextConfig, agentId));
const workspaceDir =
resolveAgentWorkspaceDir(nextConfig, agentId) ?? resolveDefaultAgentWorkspaceDir();
const providers = resolvePluginProviders({ config: nextConfig, workspaceDir });
const provider = resolveProviderMatch(providers, PROVIDER_ID);
if (!provider) {
await params.prompter.note(
"Qwen auth plugin is not available. Run `clawdbot plugins enable qwen-portal-auth` and re-run the wizard.",
"Qwen",
);
return { config: nextConfig };
}
const method = pickAuthMethod(provider, "device") ?? provider.auth[0];
if (!method) {
await params.prompter.note("Qwen auth method missing.", "Qwen");
return { config: nextConfig };
}
const isRemote = isRemoteEnvironment();
const result = await method.run({
config: nextConfig,
agentDir,
workspaceDir,
prompter: params.prompter,
runtime: params.runtime,
isRemote,
openUrl: async (url) => {
await openUrl(url);
},
oauth: {
createVpsAwareHandlers: (opts) => createVpsAwareOAuthHandlers(opts),
},
});
if (result.configPatch) {
nextConfig = mergeConfigPatch(nextConfig, result.configPatch);
}
for (const profile of result.profiles) {
upsertAuthProfile({
profileId: profile.profileId,
credential: profile.credential,
agentDir,
});
nextConfig = applyAuthProfileConfig(nextConfig, {
profileId: profile.profileId,
provider: profile.credential.provider,
mode: profile.credential.type === "token" ? "token" : profile.credential.type,
...("email" in profile.credential && profile.credential.email
? { email: profile.credential.email }
: {}),
});
}
let agentModelOverride: string | undefined;
if (result.defaultModel) {
if (params.setDefaultModel) {
nextConfig = applyDefaultModel(nextConfig, result.defaultModel);
await params.prompter.note(`Default model set to ${result.defaultModel}`, "Model configured");
} else if (params.agentId) {
agentModelOverride = result.defaultModel;
await params.prompter.note(
`Default model set to ${result.defaultModel} for agent "${params.agentId}".`,
"Model configured",
);
}
}
if (result.notes && result.notes.length > 0) {
await params.prompter.note(result.notes.join("\n"), "Provider notes");
}
return { config: nextConfig, agentModelOverride };
}
-6
View File
@@ -3,10 +3,7 @@ import type { RuntimeEnv } from "../runtime.js";
import type { WizardPrompter } from "../wizard/prompts.js";
import { applyAuthChoiceAnthropic } from "./auth-choice.apply.anthropic.js";
import { applyAuthChoiceApiProviders } from "./auth-choice.apply.api-providers.js";
import { applyAuthChoiceCopilotProxy } from "./auth-choice.apply.copilot-proxy.js";
import { applyAuthChoiceGitHubCopilot } from "./auth-choice.apply.github-copilot.js";
import { applyAuthChoiceGoogleAntigravity } from "./auth-choice.apply.google-antigravity.js";
import { applyAuthChoiceGoogleGeminiCli } from "./auth-choice.apply.google-gemini-cli.js";
import { applyAuthChoiceMiniMax } from "./auth-choice.apply.minimax.js";
import { applyAuthChoiceOAuth } from "./auth-choice.apply.oauth.js";
import { applyAuthChoiceOpenAI } from "./auth-choice.apply.openai.js";
@@ -38,9 +35,6 @@ export async function applyAuthChoice(
applyAuthChoiceApiProviders,
applyAuthChoiceMiniMax,
applyAuthChoiceGitHubCopilot,
applyAuthChoiceGoogleAntigravity,
applyAuthChoiceGoogleGeminiCli,
applyAuthChoiceCopilotProxy,
applyAuthChoiceQwenPortal,
];
@@ -15,12 +15,9 @@ const PREFERRED_PROVIDER_BY_AUTH_CHOICE: Partial<Record<AuthChoice, string>> = {
"moonshot-api-key": "moonshot",
"kimi-code-api-key": "kimi-code",
"gemini-api-key": "google",
"google-antigravity": "google-antigravity",
"google-gemini-cli": "google-gemini-cli",
"zai-api-key": "zai",
"synthetic-api-key": "synthetic",
"github-copilot": "github-copilot",
"copilot-proxy": "copilot-proxy",
"minimax-cloud": "minimax",
"minimax-api": "minimax",
"minimax-api-lightning": "minimax",
+2 -76
View File
@@ -1,13 +1,7 @@
import type { ClawdbotConfig } from "../config/config.js";
import { resolveGatewayPort } from "../config/config.js";
import { resolveGatewayLaunchAgentLabel, resolveNodeLaunchAgentLabel } from "../daemon/constants.js";
import { resolveGatewayLaunchAgentLabel } from "../daemon/constants.js";
import { readLastGatewayErrorLine } from "../daemon/diagnostics.js";
import {
isLaunchAgentListed,
isLaunchAgentLoaded,
launchAgentPlistExists,
repairLaunchAgentBootstrap,
} from "../daemon/launchd.js";
import { resolveGatewayService } from "../daemon/service.js";
import { isSystemdUserServiceAvailable } from "../daemon/systemd.js";
import { renderSystemdUnavailableHints } from "../daemon/systemd-hints.js";
@@ -27,53 +21,6 @@ import type { DoctorOptions, DoctorPrompter } from "./doctor-prompter.js";
import { healthCommand } from "./health.js";
import { formatHealthCheckFailure } from "./health-format.js";
async function maybeRepairLaunchAgentBootstrap(params: {
env: Record<string, string | undefined>;
title: string;
runtime: RuntimeEnv;
prompter: DoctorPrompter;
}): Promise<boolean> {
if (process.platform !== "darwin") return false;
const listed = await isLaunchAgentListed({ env: params.env });
if (!listed) return false;
const loaded = await isLaunchAgentLoaded({ env: params.env });
if (loaded) return false;
const plistExists = await launchAgentPlistExists(params.env);
if (!plistExists) return false;
note(
"LaunchAgent is listed but not loaded in launchd.",
`${params.title} LaunchAgent`,
);
const shouldFix = await params.prompter.confirmSkipInNonInteractive({
message: `Repair ${params.title} LaunchAgent bootstrap now?`,
initialValue: true,
});
if (!shouldFix) return false;
params.runtime.log(`Bootstrapping ${params.title} LaunchAgent...`);
const repair = await repairLaunchAgentBootstrap({ env: params.env });
if (!repair.ok) {
params.runtime.error(
`${params.title} LaunchAgent bootstrap failed: ${repair.detail ?? "unknown error"}`,
);
return false;
}
const verified = await isLaunchAgentLoaded({ env: params.env });
if (!verified) {
params.runtime.error(`${params.title} LaunchAgent still not loaded after repair.`);
return false;
}
note(`${params.title} LaunchAgent repaired.`, `${params.title} LaunchAgent`);
return true;
}
export async function maybeRepairGatewayDaemon(params: {
cfg: ClawdbotConfig;
runtime: RuntimeEnv;
@@ -85,33 +32,12 @@ export async function maybeRepairGatewayDaemon(params: {
if (params.healthOk) return;
const service = resolveGatewayService();
let loaded = await service.isLoaded({ env: process.env });
const loaded = await service.isLoaded({ env: process.env });
let serviceRuntime: Awaited<ReturnType<typeof service.readRuntime>> | undefined;
if (loaded) {
serviceRuntime = await service.readRuntime(process.env).catch(() => undefined);
}
if (process.platform === "darwin" && params.cfg.gateway?.mode !== "remote") {
const gatewayRepaired = await maybeRepairLaunchAgentBootstrap({
env: process.env,
title: "Gateway",
runtime: params.runtime,
prompter: params.prompter,
});
await maybeRepairLaunchAgentBootstrap({
env: { ...process.env, CLAWDBOT_LAUNCHD_LABEL: resolveNodeLaunchAgentLabel() },
title: "Node",
runtime: params.runtime,
prompter: params.prompter,
});
if (gatewayRepaired) {
loaded = await service.isLoaded({ env: process.env });
if (loaded) {
serviceRuntime = await service.readRuntime(process.env).catch(() => undefined);
}
}
}
if (params.cfg.gateway?.mode !== "remote") {
const port = resolveGatewayPort(params.cfg, process.env);
const diagnostics = await inspectPortUsage(port);
+29 -98
View File
@@ -1,20 +1,13 @@
import { resolveAgentWorkspaceDir, resolveDefaultAgentId } from "../agents/agent-scope.js";
import { listChannelPluginCatalogEntries } from "../channels/plugins/catalog.js";
import { listChannelPlugins, getChannelPlugin } from "../channels/plugins/index.js";
import type { ChannelMeta } from "../channels/plugins/types.js";
import {
formatChannelPrimerLine,
formatChannelSelectionLine,
listChatChannels,
} from "../channels/registry.js";
import { formatChannelPrimerLine, formatChannelSelectionLine } from "../channels/registry.js";
import type { ClawdbotConfig } from "../config/config.js";
import { isChannelConfigured } from "../config/plugin-auto-enable.js";
import type { DmPolicy } from "../config/types.js";
import { resolveChannelDefaultAccountId } from "../channels/plugins/helpers.js";
import { DEFAULT_ACCOUNT_ID, normalizeAccountId } from "../routing/session-key.js";
import type { RuntimeEnv } from "../runtime.js";
import { formatDocsLink } from "../terminal/links.js";
import { enablePluginInConfig } from "../plugins/enable.js";
import type { WizardPrompter, WizardSelectOption } from "../wizard/prompts.js";
import type { ChannelChoice } from "./onboard-types.js";
import {
@@ -122,20 +115,6 @@ async function collectChannelStatus(params: {
}),
),
);
const statusByChannel = new Map(statusEntries.map((entry) => [entry.channel, entry]));
const fallbackStatuses = listChatChannels()
.filter((meta) => !statusByChannel.has(meta.id))
.map((meta) => {
const configured = isChannelConfigured(params.cfg, meta.id);
const statusLabel = configured ? "configured (plugin disabled)" : "not configured";
return {
channel: meta.id,
configured,
statusLines: [`${meta.label}: ${statusLabel}`],
selectionHint: configured ? "configured · plugin disabled" : "not configured",
quickstartScore: 0,
};
});
const catalogStatuses = catalogEntries.map((entry) => ({
channel: entry.id,
configured: false,
@@ -143,13 +122,13 @@ async function collectChannelStatus(params: {
selectionHint: "plugin · install",
quickstartScore: 0,
}));
const combinedStatuses = [...statusEntries, ...fallbackStatuses, ...catalogStatuses];
const mergedStatusByChannel = new Map(combinedStatuses.map((entry) => [entry.channel, entry]));
const combinedStatuses = [...statusEntries, ...catalogStatuses];
const statusByChannel = new Map(combinedStatuses.map((entry) => [entry.channel, entry]));
const statusLines = combinedStatuses.flatMap((entry) => entry.statusLines);
return {
installedPlugins,
catalogEntries,
statusByChannel: mergedStatusByChannel,
statusByChannel,
statusLines,
};
}
@@ -291,28 +270,17 @@ export async function setupChannels(
});
if (!shouldConfigure) return cfg;
const corePrimer = listChatChannels().map((meta) => ({
id: meta.id as ChannelChoice,
label: meta.label,
blurb: meta.blurb,
}));
const coreIds = new Set(corePrimer.map((entry) => entry.id));
const primerChannels = [
...corePrimer,
...installedPlugins
.filter((plugin) => !coreIds.has(plugin.id as ChannelChoice))
.map((plugin) => ({
id: plugin.id as ChannelChoice,
label: plugin.meta.label,
blurb: plugin.meta.blurb,
})),
...catalogEntries
.filter((entry) => !coreIds.has(entry.id as ChannelChoice))
.map((entry) => ({
id: entry.id as ChannelChoice,
label: entry.meta.label,
blurb: entry.meta.blurb,
})),
...installedPlugins.map((plugin) => ({
id: plugin.id as ChannelChoice,
label: plugin.meta.label,
blurb: plugin.meta.blurb,
})),
...catalogEntries.map((entry) => ({
id: entry.id as ChannelChoice,
label: entry.meta.label,
blurb: entry.meta.blurb,
})),
];
await noteChannelPrimer(prompter, primerChannels);
@@ -333,11 +301,7 @@ export async function setupChannels(
const resolveDisabledHint = (channel: ChannelChoice): string | undefined => {
const plugin = getChannelPlugin(channel);
if (!plugin) {
if (next.plugins?.entries?.[channel]?.enabled === false) return "plugin disabled";
if (next.plugins?.enabled === false) return "plugins disabled";
return undefined;
}
if (!plugin) return undefined;
const accountId = resolveChannelDefaultAccountId({ plugin, cfg: next });
const account = plugin.config.resolveAccount(next, accountId);
let enabled: boolean | undefined;
@@ -372,28 +336,21 @@ export async function setupChannels(
});
const getChannelEntries = () => {
const core = listChatChannels();
const installed = listChannelPlugins();
const installedIds = new Set(installed.map((plugin) => plugin.id));
const catalog = listChannelPluginCatalogEntries().filter(
(entry) => !installedIds.has(entry.id),
);
const metaById = new Map<string, ChannelMeta>();
for (const meta of core) {
metaById.set(meta.id, meta);
}
for (const plugin of installed) {
metaById.set(plugin.id, plugin.meta);
}
for (const entry of catalog) {
if (!metaById.has(entry.id)) {
metaById.set(entry.id, entry.meta);
}
}
const entries = Array.from(metaById, ([id, meta]) => ({
id: id as ChannelChoice,
meta,
}));
const entries = [
...installed.map((plugin) => ({
id: plugin.id as ChannelChoice,
meta: plugin.meta,
})),
...catalog.map((entry) => ({
id: entry.id as ChannelChoice,
meta: entry.meta,
})),
];
return {
entries,
catalog,
@@ -408,31 +365,6 @@ export async function setupChannels(
statusByChannel.set(channel, status);
};
const ensureBundledPluginEnabled = async (channel: ChannelChoice): Promise<boolean> => {
if (getChannelPlugin(channel)) return true;
const result = enablePluginInConfig(next, channel);
next = result.config;
if (!result.enabled) {
await prompter.note(
`Cannot enable ${channel}: ${result.reason ?? "plugin disabled"}.`,
"Channel setup",
);
return false;
}
const workspaceDir = resolveAgentWorkspaceDir(next, resolveDefaultAgentId(next));
reloadOnboardingPluginRegistry({
cfg: next,
runtime,
workspaceDir,
});
if (!getChannelPlugin(channel)) {
await prompter.note(`${channel} plugin not available.`, "Channel setup");
return false;
}
await refreshStatus(channel);
return true;
};
const configureChannel = async (channel: ChannelChoice) => {
const adapter = getChannelOnboardingAdapter(channel);
if (!adapter) {
@@ -544,9 +476,6 @@ export async function setupChannels(
workspaceDir,
});
await refreshStatus(channel);
} else {
const enabled = await ensureBundledPluginEnabled(channel);
if (!enabled) return;
}
const plugin = getChannelPlugin(channel);
@@ -602,8 +531,10 @@ export async function setupChannels(
options?.onSelection?.(selection);
const selectionNotes = new Map<string, string>();
const { entries: selectionEntries } = getChannelEntries();
for (const entry of selectionEntries) {
for (const plugin of listChannelPlugins()) {
selectionNotes.set(plugin.id, formatChannelSelectionLine(plugin.meta, formatDocsLink));
}
for (const entry of listChannelPluginCatalogEntries()) {
selectionNotes.set(entry.id, formatChannelSelectionLine(entry.meta, formatDocsLink));
}
const selectedLines = selection
-38
View File
@@ -77,39 +77,6 @@ describe("onboard-hooks", () => {
configChecks: [],
install: [],
},
{
name: "command-logger",
description: "Log all command events to a centralized audit file",
source: "clawdbot-bundled",
pluginId: undefined,
filePath: "/mock/workspace/hooks/command-logger/HOOK.md",
baseDir: "/mock/workspace/hooks/command-logger",
handlerPath: "/mock/workspace/hooks/command-logger/handler.js",
hookKey: "command-logger",
emoji: "📝",
events: ["command"],
homepage: undefined,
always: false,
disabled: false,
eligible,
managedByPlugin: false,
requirements: {
bins: [],
anyBins: [],
env: [],
config: ["workspace.dir"],
os: [],
},
missing: {
bins: [],
anyBins: [],
env: [],
config: eligible ? [] : ["workspace.dir"],
os: [],
},
configChecks: [],
install: [],
},
],
});
@@ -138,11 +105,6 @@ describe("onboard-hooks", () => {
label: "💾 session-memory",
hint: "Save session context to memory when /new command is issued",
},
{
value: "command-logger",
label: "📝 command-logger",
hint: "Log all command events to a centralized audit file",
},
],
});
});
+4 -4
View File
@@ -23,10 +23,10 @@ export async function setupInternalHooks(
const workspaceDir = resolveAgentWorkspaceDir(cfg, resolveDefaultAgentId(cfg));
const report = buildWorkspaceHookStatus(workspaceDir, { config: cfg });
// Show every eligible hook so users can opt in during onboarding.
const eligibleHooks = report.hooks.filter((h) => h.eligible);
// Filter for eligible and recommended hooks (session-memory is recommended)
const recommendedHooks = report.hooks.filter((h) => h.eligible && h.name === "session-memory");
if (eligibleHooks.length === 0) {
if (recommendedHooks.length === 0) {
await prompter.note(
"No eligible hooks found. You can configure hooks later in your config.",
"No Hooks Available",
@@ -38,7 +38,7 @@ export async function setupInternalHooks(
message: "Enable hooks?",
options: [
{ value: "__skip__", label: "Skip for now" },
...eligibleHooks.map((hook) => ({
...recommendedHooks.map((hook) => ({
value: hook.name,
label: `${hook.emoji ?? "🔗"} ${hook.name}`,
hint: hook.description,
-3
View File
@@ -19,8 +19,6 @@ export type AuthChoice =
| "codex-cli"
| "apiKey"
| "gemini-api-key"
| "google-antigravity"
| "google-gemini-cli"
| "zai-api-key"
| "minimax-cloud"
| "minimax"
@@ -28,7 +26,6 @@ export type AuthChoice =
| "minimax-api-lightning"
| "opencode-zen"
| "github-copilot"
| "copilot-proxy"
| "qwen-portal"
| "skip";
export type GatewayAuthChoice = "off" | "token" | "password";
+34 -4
View File
@@ -5,7 +5,6 @@ import type { ChannelPluginCatalogEntry } from "../../channels/plugins/catalog.j
import type { ClawdbotConfig } from "../../config/config.js";
import { createSubsystemLogger } from "../../logging.js";
import { recordPluginInstall } from "../../plugins/installs.js";
import { enablePluginInConfig } from "../../plugins/enable.js";
import { loadClawdbotPlugins } from "../../plugins/loader.js";
import { installPluginFromNpmSpec } from "../../plugins/install.js";
import type { RuntimeEnv } from "../../runtime.js";
@@ -49,6 +48,37 @@ function resolveLocalPath(
return null;
}
function ensurePluginEnabled(cfg: ClawdbotConfig, pluginId: string): ClawdbotConfig {
const entries = {
...cfg.plugins?.entries,
[pluginId]: {
...(cfg.plugins?.entries?.[pluginId] as Record<string, unknown> | undefined),
enabled: true,
},
};
const next: ClawdbotConfig = {
...cfg,
plugins: {
...cfg.plugins,
...(cfg.plugins?.enabled === false ? { enabled: true } : {}),
entries,
},
};
return ensurePluginAllowlist(next, pluginId);
}
function ensurePluginAllowlist(cfg: ClawdbotConfig, pluginId: string): ClawdbotConfig {
const allow = cfg.plugins?.allow;
if (!allow || allow.includes(pluginId)) return cfg;
return {
...cfg,
plugins: {
...cfg.plugins,
allow: [...allow, pluginId],
},
};
}
function addPluginLoadPath(cfg: ClawdbotConfig, pluginPath: string): ClawdbotConfig {
const existing = cfg.plugins?.load?.paths ?? [];
const merged = Array.from(new Set([...existing, pluginPath]));
@@ -115,7 +145,7 @@ export async function ensureOnboardingPluginInstalled(params: {
if (choice === "local" && localPath) {
next = addPluginLoadPath(next, localPath);
next = enablePluginInConfig(next, entry.id).config;
next = ensurePluginEnabled(next, entry.id);
return { cfg: next, installed: true };
}
@@ -128,7 +158,7 @@ export async function ensureOnboardingPluginInstalled(params: {
});
if (result.ok) {
next = enablePluginInConfig(next, result.pluginId).config;
next = ensurePluginEnabled(next, result.pluginId);
next = recordPluginInstall(next, {
pluginId: result.pluginId,
source: "npm",
@@ -151,7 +181,7 @@ export async function ensureOnboardingPluginInstalled(params: {
});
if (fallback) {
next = addPluginLoadPath(next, localPath);
next = enablePluginInConfig(next, entry.id).config;
next = ensurePluginEnabled(next, entry.id);
return { cfg: next, installed: true };
}
}
+8 -53
View File
@@ -19,18 +19,15 @@ import {
applySessionDefaults,
applyTalkApiKey,
} from "./defaults.js";
import { VERSION } from "../version.js";
import { MissingEnvVarError, resolveConfigEnvVars } from "./env-substitution.js";
import { ConfigIncludeError, resolveConfigIncludes } from "./includes.js";
import { applyLegacyMigrations, findLegacyConfigIssues } from "./legacy.js";
import { normalizeConfigPaths } from "./normalize-paths.js";
import { resolveConfigPath, resolveStateDir } from "./paths.js";
import { applyPluginAutoEnable } from "./plugin-auto-enable.js";
import { applyConfigOverrides } from "./runtime-overrides.js";
import type { ClawdbotConfig, ConfigFileSnapshot, LegacyConfigIssue } from "./types.js";
import { validateConfigObject } from "./validation.js";
import { ClawdbotSchema } from "./zod-schema.js";
import { compareClawdbotVersions } from "./version.js";
// Re-export for backwards compatibility
export { CircularIncludeError, ConfigIncludeError } from "./includes.js";
@@ -149,30 +146,6 @@ function formatLegacyMigrationLog(changes: string[]): string {
return `Auto-migrated config:\n${changes.map((entry) => `- ${entry}`).join("\n")}`;
}
function stampConfigVersion(cfg: ClawdbotConfig): ClawdbotConfig {
const now = new Date().toISOString();
return {
...cfg,
meta: {
...cfg.meta,
lastTouchedVersion: VERSION,
lastTouchedAt: now,
},
};
}
function warnIfConfigFromFuture(cfg: ClawdbotConfig, logger: Pick<typeof console, "warn">): void {
const touched = cfg.meta?.lastTouchedVersion;
if (!touched) return;
const cmp = compareClawdbotVersions(VERSION, touched);
if (cmp === null) return;
if (cmp < 0) {
logger.warn(
`Config was last written by a newer Clawdbot (${touched}); current version is ${VERSION}.`,
);
}
}
function applyConfigEnv(cfg: ClawdbotConfig, env: NodeJS.ProcessEnv): void {
const envConfig = cfg.env;
if (!envConfig) return;
@@ -232,9 +205,7 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
const writeConfigFileSync = (cfg: ClawdbotConfig) => {
const dir = path.dirname(configPath);
deps.fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
const json = JSON.stringify(applyModelDefaults(stampConfigVersion(cfg)), null, 2)
.trimEnd()
.concat("\n");
const json = JSON.stringify(applyModelDefaults(cfg), null, 2).trimEnd().concat("\n");
const tmp = path.join(
dir,
@@ -306,13 +277,7 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
const substituted = resolveConfigEnvVars(resolved, deps.env);
const migrated = applyLegacyMigrations(substituted);
let resolvedConfig = migrated.next ?? substituted;
const autoEnable = applyPluginAutoEnable({
config: coerceConfig(resolvedConfig),
env: deps.env,
});
resolvedConfig = autoEnable.config;
const migrationChanges = [...migrated.changes, ...autoEnable.changes];
const resolvedConfig = migrated.next ?? substituted;
warnOnConfigMiskeys(resolvedConfig, deps.logger);
if (typeof resolvedConfig !== "object" || resolvedConfig === null) return {};
const validated = ClawdbotSchema.safeParse(resolvedConfig);
@@ -323,9 +288,8 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
}
return {};
}
warnIfConfigFromFuture(validated.data as ClawdbotConfig, deps.logger);
if (migrationChanges.length > 0) {
deps.logger.warn(formatLegacyMigrationLog(migrationChanges));
if (migrated.next && migrated.changes.length > 0) {
deps.logger.warn(formatLegacyMigrationLog(migrated.changes));
try {
writeConfigFileSync(resolvedConfig as ClawdbotConfig);
} catch (err) {
@@ -462,13 +426,7 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
}
const migrated = applyLegacyMigrations(substituted);
let resolvedConfigRaw = migrated.next ?? substituted;
const autoEnable = applyPluginAutoEnable({
config: coerceConfig(resolvedConfigRaw),
env: deps.env,
});
resolvedConfigRaw = autoEnable.config;
const migrationChanges = [...migrated.changes, ...autoEnable.changes];
const resolvedConfigRaw = migrated.next ?? substituted;
const legacyIssues = findLegacyConfigIssues(resolvedConfigRaw);
const validated = validateConfigObject(resolvedConfigRaw);
@@ -486,9 +444,8 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
};
}
warnIfConfigFromFuture(validated.config, deps.logger);
if (migrationChanges.length > 0) {
deps.logger.warn(formatLegacyMigrationLog(migrationChanges));
if (migrated.next && migrated.changes.length > 0) {
deps.logger.warn(formatLegacyMigrationLog(migrated.changes));
await writeConfigFile(validated.config).catch((err) => {
deps.logger.warn(`Failed to write migrated config at ${configPath}: ${String(err)}`);
});
@@ -529,9 +486,7 @@ export function createConfigIO(overrides: ConfigIoDeps = {}) {
async function writeConfigFile(cfg: ClawdbotConfig) {
const dir = path.dirname(configPath);
await deps.fs.promises.mkdir(dir, { recursive: true, mode: 0o700 });
const json = JSON.stringify(applyModelDefaults(stampConfigVersion(cfg)), null, 2)
.trimEnd()
.concat("\n");
const json = JSON.stringify(applyModelDefaults(cfg), null, 2).trimEnd().concat("\n");
const tmp = path.join(
dir,
-58
View File
@@ -1,58 +0,0 @@
import { describe, expect, it } from "vitest";
import { applyPluginAutoEnable } from "./plugin-auto-enable.js";
describe("applyPluginAutoEnable", () => {
it("enables configured channel plugins and updates allowlist", () => {
const result = applyPluginAutoEnable({
config: {
channels: { slack: { botToken: "x" } },
plugins: { allow: ["telegram"] },
},
});
expect(result.config.plugins?.entries?.slack?.enabled).toBe(true);
expect(result.config.plugins?.allow).toEqual(["telegram", "slack"]);
expect(result.changes.join("\n")).toContain('Enabled plugin "slack"');
});
it("respects explicit disable", () => {
const result = applyPluginAutoEnable({
config: {
channels: { slack: { botToken: "x" } },
plugins: { entries: { slack: { enabled: false } } },
},
});
expect(result.config.plugins?.entries?.slack?.enabled).toBe(false);
expect(result.changes).toEqual([]);
});
it("enables provider auth plugins when profiles exist", () => {
const result = applyPluginAutoEnable({
config: {
auth: {
profiles: {
"google-antigravity:default": {
provider: "google-antigravity",
mode: "oauth",
},
},
},
},
});
expect(result.config.plugins?.entries?.["google-antigravity-auth"]?.enabled).toBe(true);
});
it("skips when plugins are globally disabled", () => {
const result = applyPluginAutoEnable({
config: {
channels: { slack: { botToken: "x" } },
plugins: { enabled: false },
},
});
expect(result.config.plugins?.entries?.slack?.enabled).toBeUndefined();
expect(result.changes).toEqual([]);
});
});
-328
View File
@@ -1,328 +0,0 @@
import type { ClawdbotConfig } from "./config.js";
import { hasAnyWhatsAppAuth } from "../web/accounts.js";
import { normalizeProviderId } from "../agents/model-selection.js";
type PluginEnableChange = {
pluginId: string;
reason: string;
};
export type PluginAutoEnableResult = {
config: ClawdbotConfig;
changes: string[];
};
const CHANNEL_PLUGIN_IDS = [
"whatsapp",
"telegram",
"discord",
"slack",
"signal",
"imessage",
"msteams",
"matrix",
"zalo",
"zalouser",
"bluebubbles",
] as const;
const PROVIDER_PLUGIN_IDS: Array<{ pluginId: string; providerId: string }> = [
{ pluginId: "google-antigravity-auth", providerId: "google-antigravity" },
{ pluginId: "google-gemini-cli-auth", providerId: "google-gemini-cli" },
{ pluginId: "qwen-portal-auth", providerId: "qwen-portal" },
{ pluginId: "copilot-proxy", providerId: "copilot-proxy" },
];
function isRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value && typeof value === "object" && !Array.isArray(value));
}
function hasNonEmptyString(value: unknown): boolean {
return typeof value === "string" && value.trim().length > 0;
}
function recordHasKeys(value: unknown): boolean {
return isRecord(value) && Object.keys(value).length > 0;
}
function accountsHaveKeys(
value: unknown,
keys: string[],
): boolean {
if (!isRecord(value)) return false;
for (const account of Object.values(value)) {
if (!isRecord(account)) continue;
for (const key of keys) {
if (hasNonEmptyString(account[key])) return true;
}
}
return false;
}
function resolveChannelConfig(cfg: ClawdbotConfig, channelId: string): Record<string, unknown> | null {
const channels = cfg.channels as Record<string, unknown> | undefined;
const entry = channels?.[channelId];
return isRecord(entry) ? entry : null;
}
function isTelegramConfigured(cfg: ClawdbotConfig, env: NodeJS.ProcessEnv): boolean {
if (hasNonEmptyString(env.TELEGRAM_BOT_TOKEN)) return true;
const entry = resolveChannelConfig(cfg, "telegram");
if (!entry) return false;
if (hasNonEmptyString(entry.botToken) || hasNonEmptyString(entry.tokenFile)) return true;
if (accountsHaveKeys(entry.accounts, ["botToken", "tokenFile"])) return true;
return recordHasKeys(entry);
}
function isDiscordConfigured(cfg: ClawdbotConfig, env: NodeJS.ProcessEnv): boolean {
if (hasNonEmptyString(env.DISCORD_BOT_TOKEN)) return true;
const entry = resolveChannelConfig(cfg, "discord");
if (!entry) return false;
if (hasNonEmptyString(entry.token)) return true;
if (accountsHaveKeys(entry.accounts, ["token"])) return true;
return recordHasKeys(entry);
}
function isSlackConfigured(cfg: ClawdbotConfig, env: NodeJS.ProcessEnv): boolean {
if (
hasNonEmptyString(env.SLACK_BOT_TOKEN) ||
hasNonEmptyString(env.SLACK_APP_TOKEN) ||
hasNonEmptyString(env.SLACK_USER_TOKEN)
) {
return true;
}
const entry = resolveChannelConfig(cfg, "slack");
if (!entry) return false;
if (
hasNonEmptyString(entry.botToken) ||
hasNonEmptyString(entry.appToken) ||
hasNonEmptyString(entry.userToken)
) {
return true;
}
if (accountsHaveKeys(entry.accounts, ["botToken", "appToken", "userToken"])) return true;
return recordHasKeys(entry);
}
function isSignalConfigured(cfg: ClawdbotConfig): boolean {
const entry = resolveChannelConfig(cfg, "signal");
if (!entry) return false;
if (
hasNonEmptyString(entry.account) ||
hasNonEmptyString(entry.httpUrl) ||
hasNonEmptyString(entry.httpHost) ||
typeof entry.httpPort === "number" ||
hasNonEmptyString(entry.cliPath)
) {
return true;
}
if (accountsHaveKeys(entry.accounts, ["account", "httpUrl", "httpHost", "cliPath"])) return true;
return recordHasKeys(entry);
}
function isIMessageConfigured(cfg: ClawdbotConfig): boolean {
const entry = resolveChannelConfig(cfg, "imessage");
if (!entry) return false;
if (hasNonEmptyString(entry.cliPath)) return true;
return recordHasKeys(entry);
}
function isWhatsAppConfigured(cfg: ClawdbotConfig): boolean {
if (hasAnyWhatsAppAuth(cfg)) return true;
const entry = resolveChannelConfig(cfg, "whatsapp");
if (!entry) return false;
return recordHasKeys(entry);
}
function isGenericChannelConfigured(cfg: ClawdbotConfig, channelId: string): boolean {
const entry = resolveChannelConfig(cfg, channelId);
return recordHasKeys(entry);
}
export function isChannelConfigured(
cfg: ClawdbotConfig,
channelId: string,
env: NodeJS.ProcessEnv = process.env,
): boolean {
switch (channelId) {
case "whatsapp":
return isWhatsAppConfigured(cfg);
case "telegram":
return isTelegramConfigured(cfg, env);
case "discord":
return isDiscordConfigured(cfg, env);
case "slack":
return isSlackConfigured(cfg, env);
case "signal":
return isSignalConfigured(cfg);
case "imessage":
return isIMessageConfigured(cfg);
default:
return isGenericChannelConfigured(cfg, channelId);
}
}
function collectModelRefs(cfg: ClawdbotConfig): string[] {
const refs: string[] = [];
const pushModelRef = (value: unknown) => {
if (typeof value === "string" && value.trim()) refs.push(value.trim());
};
const collectFromAgent = (agent: Record<string, unknown> | null | undefined) => {
if (!agent) return;
const model = agent.model;
if (typeof model === "string") {
pushModelRef(model);
} else if (isRecord(model)) {
pushModelRef(model.primary);
const fallbacks = model.fallbacks;
if (Array.isArray(fallbacks)) {
for (const entry of fallbacks) pushModelRef(entry);
}
}
const models = agent.models;
if (isRecord(models)) {
for (const key of Object.keys(models)) {
pushModelRef(key);
}
}
};
const defaults = cfg.agents?.defaults as Record<string, unknown> | undefined;
collectFromAgent(defaults);
const list = cfg.agents?.list;
if (Array.isArray(list)) {
for (const entry of list) {
if (isRecord(entry)) collectFromAgent(entry);
}
}
return refs;
}
function extractProviderFromModelRef(value: string): string | null {
const trimmed = value.trim();
const slash = trimmed.indexOf("/");
if (slash <= 0) return null;
return normalizeProviderId(trimmed.slice(0, slash));
}
function isProviderConfigured(cfg: ClawdbotConfig, providerId: string): boolean {
const normalized = normalizeProviderId(providerId);
const profiles = cfg.auth?.profiles;
if (profiles && typeof profiles === "object") {
for (const profile of Object.values(profiles)) {
if (!isRecord(profile)) continue;
const provider = normalizeProviderId(String(profile.provider ?? ""));
if (provider === normalized) return true;
}
}
const providerConfig = cfg.models?.providers;
if (providerConfig && typeof providerConfig === "object") {
for (const key of Object.keys(providerConfig)) {
if (normalizeProviderId(key) === normalized) return true;
}
}
const modelRefs = collectModelRefs(cfg);
for (const ref of modelRefs) {
const provider = extractProviderFromModelRef(ref);
if (provider && provider === normalized) return true;
}
return false;
}
function resolveConfiguredPlugins(cfg: ClawdbotConfig, env: NodeJS.ProcessEnv): PluginEnableChange[] {
const changes: PluginEnableChange[] = [];
for (const channelId of CHANNEL_PLUGIN_IDS) {
if (isChannelConfigured(cfg, channelId, env)) {
changes.push({
pluginId: channelId,
reason: `${channelId} configured`,
});
}
}
for (const mapping of PROVIDER_PLUGIN_IDS) {
if (isProviderConfigured(cfg, mapping.providerId)) {
changes.push({
pluginId: mapping.pluginId,
reason: `${mapping.providerId} auth configured`,
});
}
}
return changes;
}
function isPluginExplicitlyDisabled(cfg: ClawdbotConfig, pluginId: string): boolean {
const entry = cfg.plugins?.entries?.[pluginId];
return entry?.enabled === false;
}
function isPluginDenied(cfg: ClawdbotConfig, pluginId: string): boolean {
const deny = cfg.plugins?.deny;
return Array.isArray(deny) && deny.includes(pluginId);
}
function ensureAllowlisted(cfg: ClawdbotConfig, pluginId: string): ClawdbotConfig {
const allow = cfg.plugins?.allow;
if (!Array.isArray(allow) || allow.includes(pluginId)) return cfg;
return {
...cfg,
plugins: {
...cfg.plugins,
allow: [...allow, pluginId],
},
};
}
function enablePluginEntry(cfg: ClawdbotConfig, pluginId: string): ClawdbotConfig {
const entries = {
...cfg.plugins?.entries,
[pluginId]: {
...(cfg.plugins?.entries?.[pluginId] as Record<string, unknown> | undefined),
enabled: true,
},
};
return {
...cfg,
plugins: {
...cfg.plugins,
entries,
...(cfg.plugins?.enabled === false ? { enabled: true } : {}),
},
};
}
export function applyPluginAutoEnable(params: {
config: ClawdbotConfig;
env?: NodeJS.ProcessEnv;
}): PluginAutoEnableResult {
const env = params.env ?? process.env;
const configured = resolveConfiguredPlugins(params.config, env);
if (configured.length === 0) {
return { config: params.config, changes: [] };
}
let next = params.config;
const changes: string[] = [];
if (next.plugins?.enabled === false) {
return { config: next, changes };
}
for (const entry of configured) {
if (isPluginDenied(next, entry.pluginId)) continue;
if (isPluginExplicitlyDisabled(next, entry.pluginId)) continue;
const allow = next.plugins?.allow;
const allowMissing = Array.isArray(allow) && !allow.includes(entry.pluginId);
const alreadyEnabled = next.plugins?.entries?.[entry.pluginId]?.enabled === true;
if (alreadyEnabled && !allowMissing) continue;
next = enablePluginEntry(next, entry.pluginId);
next = ensureAllowlisted(next, entry.pluginId);
changes.push(`Enabled plugin "${entry.pluginId}" (${entry.reason}).`);
}
return { config: next, changes };
}
+6 -10
View File
@@ -97,8 +97,6 @@ const GROUP_ORDER: Record<string, number> = {
};
const FIELD_LABELS: Record<string, string> = {
"meta.lastTouchedVersion": "Config Last Touched Version",
"meta.lastTouchedAt": "Config Last Touched At",
"update.channel": "Update Channel",
"update.checkOnStart": "Update Check on Start",
"gateway.remote.url": "Remote Gateway URL",
@@ -298,8 +296,6 @@ const FIELD_LABELS: Record<string, string> = {
};
const FIELD_HELP: Record<string, string> = {
"meta.lastTouchedVersion": "Auto-set when Clawdbot writes the config.",
"meta.lastTouchedAt": "ISO timestamp of the last config write (auto-set).",
"update.channel": 'Update channel for npm installs ("stable" or "beta").',
"update.checkOnStart": "Check for npm updates when the gateway starts (default: true).",
"gateway.remote.url": "Remote Gateway WebSocket URL (ws:// or wss://).",
@@ -385,19 +381,19 @@ const FIELD_HELP: Record<string, string> = {
"agents.defaults.memorySearch.remote.headers":
"Extra headers for remote embeddings (merged; remote overrides OpenAI headers).",
"agents.defaults.memorySearch.remote.batch.enabled":
"Enable batch API for memory embeddings (OpenAI/Gemini; default: true).",
"Enable OpenAI Batch API for memory embeddings (default: true).",
"agents.defaults.memorySearch.remote.batch.wait":
"Wait for batch completion when indexing (default: true).",
"Wait for OpenAI batch completion when indexing (default: true).",
"agents.defaults.memorySearch.remote.batch.concurrency":
"Max concurrent embedding batch jobs for memory indexing (default: 2).",
"Max concurrent OpenAI batch jobs for memory indexing (default: 2).",
"agents.defaults.memorySearch.remote.batch.pollIntervalMs":
"Polling interval in ms for batch status (default: 2000).",
"Polling interval in ms for OpenAI batch status (default: 2000).",
"agents.defaults.memorySearch.remote.batch.timeoutMinutes":
"Timeout in minutes for batch indexing (default: 60).",
"Timeout in minutes for OpenAI batch indexing (default: 60).",
"agents.defaults.memorySearch.local.modelPath":
"Local GGUF model path or hf: URI (node-llama-cpp).",
"agents.defaults.memorySearch.fallback":
'Fallback provider when embeddings fail ("openai", "gemini", "local", or "none").',
'Fallback to OpenAI when local embeddings fail ("openai" or "none").',
"agents.defaults.memorySearch.store.path":
"SQLite index path (default: ~/.clawdbot/memory/{agentId}.sqlite).",
"agents.defaults.memorySearch.store.vector.enabled":
-6
View File
@@ -24,12 +24,6 @@ import type { SkillsConfig } from "./types.skills.js";
import type { ToolsConfig } from "./types.tools.js";
export type ClawdbotConfig = {
meta?: {
/** Last clawdbot version that wrote this config. */
lastTouchedVersion?: string;
/** ISO timestamp when this config was last written. */
lastTouchedAt?: string;
};
auth?: AuthConfig;
env?: {
/** Opt-in: import missing secrets from a login shell environment (exec `$SHELL -l -c 'env -0'`). */
+3 -17
View File
@@ -121,30 +121,16 @@ export type ToolPolicyConfig = {
};
export type ExecToolConfig = {
/** Exec host routing (default: sandbox). */
host?: "sandbox" | "gateway" | "node";
/** Exec security mode (default: deny). */
security?: "deny" | "allowlist" | "full";
/** Exec ask mode (default: on-miss). */
ask?: "off" | "on-miss" | "always";
/** Default node binding for exec.host=node (node id/name). */
node?: string;
/** Default time (ms) before an exec command auto-backgrounds. */
backgroundMs?: number;
/** Default timeout (seconds) before auto-killing exec commands. */
timeoutSec?: number;
/** How long to keep finished sessions in memory (ms). */
cleanupMs?: number;
/** Emit a system event and heartbeat when a backgrounded exec exits. */
notifyOnExit?: boolean;
/** apply_patch subtool configuration (experimental). */
applyPatch?: {
/** Enable apply_patch for OpenAI models (default: false). */
enabled?: boolean;
/**
* Optional allowlist of model ids that can use apply_patch.
* Accepts either raw ids (e.g. "gpt-5.2") or full ids (e.g. "openai/gpt-5.2").
*/
allowModels?: string[];
};
};
@@ -190,7 +176,7 @@ export type MemorySearchConfig = {
apiKey?: string;
headers?: Record<string, string>;
batch?: {
/** Enable batch API for embedding indexing (OpenAI/Gemini; default: true). */
/** Enable OpenAI Batch API for embedding indexing (default: true). */
enabled?: boolean;
/** Wait for batch completion (default: true). */
wait?: boolean;
@@ -202,8 +188,8 @@ export type MemorySearchConfig = {
timeoutMinutes?: number;
};
};
/** Fallback behavior when embeddings fail. */
fallback?: "openai" | "gemini" | "local" | "none";
/** Fallback behavior when local embeddings fail. */
fallback?: "openai" | "none";
/** Embedding model id (remote) or alias (local). */
model?: string;
/** Local embedding settings (node-llama-cpp). */
-32
View File
@@ -1,32 +0,0 @@
export type ClawdbotVersion = {
major: number;
minor: number;
patch: number;
revision: number;
};
const VERSION_RE = /^v?(\d+)\.(\d+)\.(\d+)(?:-(\d+))?/;
export function parseClawdbotVersion(raw: string | null | undefined): ClawdbotVersion | null {
if (!raw) return null;
const match = raw.trim().match(VERSION_RE);
if (!match) return null;
const [, major, minor, patch, revision] = match;
return {
major: Number.parseInt(major, 10),
minor: Number.parseInt(minor, 10),
patch: Number.parseInt(patch, 10),
revision: revision ? Number.parseInt(revision, 10) : 0,
};
}
export function compareClawdbotVersions(a: string | null | undefined, b: string | null | undefined): number | null {
const parsedA = parseClawdbotVersion(a);
const parsedB = parseClawdbotVersion(b);
if (!parsedA || !parsedB) return null;
if (parsedA.major !== parsedB.major) return parsedA.major < parsedB.major ? -1 : 1;
if (parsedA.minor !== parsedB.minor) return parsedA.minor < parsedB.minor ? -1 : 1;
if (parsedA.patch !== parsedB.patch) return parsedA.patch < parsedB.patch ? -1 : 1;
if (parsedA.revision !== parsedB.revision) return parsedA.revision < parsedB.revision ? -1 : 1;
return 0;
}
+20 -4
View File
@@ -183,6 +183,24 @@ export const AgentToolsSchema = z
allowFrom: ElevatedAllowFromSchema,
})
.optional(),
exec: z
.object({
host: z.enum(["sandbox", "gateway", "node"]).optional(),
security: z.enum(["deny", "allowlist", "full"]).optional(),
ask: z.enum(["off", "on-miss", "always"]).optional(),
node: z.string().optional(),
backgroundMs: z.number().int().positive().optional(),
timeoutSec: z.number().int().positive().optional(),
cleanupMs: z.number().int().positive().optional(),
notifyOnExit: z.boolean().optional(),
applyPatch: z
.object({
enabled: z.boolean().optional(),
allowModels: z.array(z.string()).optional(),
})
.optional(),
})
.optional(),
sandbox: z
.object({
tools: ToolPolicySchema,
@@ -200,7 +218,7 @@ export const MemorySearchSchema = z
sessionMemory: z.boolean().optional(),
})
.optional(),
provider: z.union([z.literal("openai"), z.literal("local"), z.literal("gemini")]).optional(),
provider: z.union([z.literal("openai"), z.literal("gemini"), z.literal("local")]).optional(),
remote: z
.object({
baseUrl: z.string().optional(),
@@ -217,9 +235,7 @@ export const MemorySearchSchema = z
.optional(),
})
.optional(),
fallback: z
.union([z.literal("openai"), z.literal("gemini"), z.literal("local"), z.literal("none")])
.optional(),
fallback: z.union([z.literal("openai"), z.literal("none")]).optional(),
model: z.string().optional(),
local: z
.object({
-6
View File
@@ -8,12 +8,6 @@ import { CommandsSchema, MessagesSchema, SessionSchema } from "./zod-schema.sess
export const ClawdbotSchema = z
.object({
meta: z
.object({
lastTouchedVersion: z.string().optional(),
lastTouchedAt: z.string().optional(),
})
.optional(),
env: z
.object({
shellEnv: z
+1 -122
View File
@@ -5,88 +5,7 @@ import { PassThrough } from "node:stream";
import { describe, expect, it } from "vitest";
import {
installLaunchAgent,
isLaunchAgentListed,
parseLaunchctlPrint,
repairLaunchAgentBootstrap,
resolveLaunchAgentPlistPath,
} from "./launchd.js";
async function withLaunchctlStub(
options: { listOutput?: string },
run: (context: { env: Record<string, string | undefined>; logPath: string }) => Promise<void>,
) {
const originalPath = process.env.PATH;
const originalLogPath = process.env.CLAWDBOT_TEST_LAUNCHCTL_LOG;
const originalListOutput = process.env.CLAWDBOT_TEST_LAUNCHCTL_LIST_OUTPUT;
const tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-launchctl-test-"));
try {
const binDir = path.join(tmpDir, "bin");
const homeDir = path.join(tmpDir, "home");
const logPath = path.join(tmpDir, "launchctl.log");
await fs.mkdir(binDir, { recursive: true });
await fs.mkdir(homeDir, { recursive: true });
const stubJsPath = path.join(binDir, "launchctl.js");
await fs.writeFile(
stubJsPath,
[
'import fs from "node:fs";',
"const args = process.argv.slice(2);",
"const logPath = process.env.CLAWDBOT_TEST_LAUNCHCTL_LOG;",
"if (logPath) {",
' fs.appendFileSync(logPath, JSON.stringify(args) + "\\n", "utf8");',
"}",
'if (args[0] === "list") {',
" const output = process.env.CLAWDBOT_TEST_LAUNCHCTL_LIST_OUTPUT || \"\";",
" process.stdout.write(output);",
"}",
"process.exit(0);",
"",
].join("\n"),
"utf8",
);
if (process.platform === "win32") {
await fs.writeFile(
path.join(binDir, "launchctl.cmd"),
`@echo off\r\nnode "%~dp0\\launchctl.js" %*\r\n`,
"utf8",
);
} else {
const shPath = path.join(binDir, "launchctl");
await fs.writeFile(shPath, `#!/bin/sh\nnode "$(dirname "$0")/launchctl.js" "$@"\n`, "utf8");
await fs.chmod(shPath, 0o755);
}
process.env.CLAWDBOT_TEST_LAUNCHCTL_LOG = logPath;
process.env.CLAWDBOT_TEST_LAUNCHCTL_LIST_OUTPUT = options.listOutput ?? "";
process.env.PATH = `${binDir}${path.delimiter}${originalPath ?? ""}`;
await run({
env: {
HOME: homeDir,
CLAWDBOT_PROFILE: "default",
},
logPath,
});
} finally {
process.env.PATH = originalPath;
if (originalLogPath === undefined) {
delete process.env.CLAWDBOT_TEST_LAUNCHCTL_LOG;
} else {
process.env.CLAWDBOT_TEST_LAUNCHCTL_LOG = originalLogPath;
}
if (originalListOutput === undefined) {
delete process.env.CLAWDBOT_TEST_LAUNCHCTL_LIST_OUTPUT;
} else {
process.env.CLAWDBOT_TEST_LAUNCHCTL_LIST_OUTPUT = originalListOutput;
}
await fs.rm(tmpDir, { recursive: true, force: true });
}
}
import { installLaunchAgent, parseLaunchctlPrint, resolveLaunchAgentPlistPath } from "./launchd.js";
describe("launchd runtime parsing", () => {
it("parses state, pid, and exit status", () => {
@@ -105,46 +24,6 @@ describe("launchd runtime parsing", () => {
});
});
describe("launchctl list detection", () => {
it("detects the resolved label in launchctl list", async () => {
await withLaunchctlStub(
{ listOutput: "123 0 com.clawdbot.gateway\n" },
async ({ env }) => {
const listed = await isLaunchAgentListed({ env });
expect(listed).toBe(true);
},
);
});
it("returns false when the label is missing", async () => {
await withLaunchctlStub({ listOutput: "123 0 com.other.service\n" }, async ({ env }) => {
const listed = await isLaunchAgentListed({ env });
expect(listed).toBe(false);
});
});
});
describe("launchd bootstrap repair", () => {
it("bootstraps and kickstarts the resolved label", async () => {
await withLaunchctlStub({}, async ({ env, logPath }) => {
const repair = await repairLaunchAgentBootstrap({ env });
expect(repair.ok).toBe(true);
const calls = (await fs.readFile(logPath, "utf8"))
.split("\n")
.filter(Boolean)
.map((line) => JSON.parse(line) as string[]);
const domain = typeof process.getuid === "function" ? `gui/${process.getuid()}` : "gui/501";
const label = "com.clawdbot.gateway";
const plistPath = resolveLaunchAgentPlistPath(env);
expect(calls).toContainEqual(["bootstrap", domain, plistPath]);
expect(calls).toContainEqual(["kickstart", "-k", `${domain}/${label}`]);
});
});
});
describe("launchd install", () => {
it("enables service before bootstrap (clears persisted disabled state)", async () => {
const originalPath = process.env.PATH;
+3 -34
View File
@@ -170,22 +170,9 @@ export async function isLaunchAgentLoaded(args: {
return res.code === 0;
}
export async function isLaunchAgentListed(args: {
env?: Record<string, string | undefined>;
}): Promise<boolean> {
const label = resolveLaunchAgentLabel({ env: args.env });
const res = await execLaunchctl(["list"]);
if (res.code !== 0) return false;
return res.stdout
.split(/\r?\n/)
.some((line) => line.trim().split(/\s+/).at(-1) === label);
}
export async function launchAgentPlistExists(
env: Record<string, string | undefined>,
): Promise<boolean> {
async function hasLaunchAgentPlist(env: Record<string, string | undefined>): Promise<boolean> {
const plistPath = resolveLaunchAgentPlistPath(env);
try {
const plistPath = resolveLaunchAgentPlistPath(env);
await fs.access(plistPath);
return true;
} catch {
@@ -207,7 +194,7 @@ export async function readLaunchAgentRuntime(
};
}
const parsed = parseLaunchctlPrint(res.stdout || res.stderr || "");
const plistExists = await launchAgentPlistExists(env);
const plistExists = await hasLaunchAgentPlist(env);
const state = parsed.state?.toLowerCase();
const status = state === "running" || parsed.pid ? "running" : state ? "stopped" : "unknown";
return {
@@ -220,24 +207,6 @@ export async function readLaunchAgentRuntime(
};
}
export async function repairLaunchAgentBootstrap(args: {
env?: Record<string, string | undefined>;
}): Promise<{ ok: boolean; detail?: string }> {
const env = args.env ?? (process.env as Record<string, string | undefined>);
const domain = resolveGuiDomain();
const label = resolveLaunchAgentLabel({ env });
const plistPath = resolveLaunchAgentPlistPath(env);
const boot = await execLaunchctl(["bootstrap", domain, plistPath]);
if (boot.code !== 0) {
return { ok: false, detail: (boot.stderr || boot.stdout).trim() || undefined };
}
const kick = await execLaunchctl(["kickstart", "-k", `${domain}/${label}`]);
if (kick.code !== 0) {
return { ok: false, detail: (kick.stderr || kick.stdout).trim() || undefined };
}
return { ok: true };
}
export type LegacyLaunchAgent = {
label: string;
plistPath: string;
+1 -3
View File
@@ -23,10 +23,8 @@ vi.mock("../logger.js", () => {
};
});
vi.mock("../logging.js", async () => {
const actual = await vi.importActual<typeof import("../logging.js")>("../logging.js");
vi.mock("../logging.js", () => {
return {
...actual,
getLogger: () => ({ info: (...args: unknown[]) => getLoggerInfo(...args) }),
};
});
+1 -1
View File
@@ -242,7 +242,7 @@ function parseFirstToken(command: string): string | null {
if (end > 1) return trimmed.slice(1, end);
return trimmed.slice(1);
}
const match = /^[^\s]+/.exec(trimmed);
const match = /^\S+/.exec(trimmed);
return match ? match[0] : null;
}
-109
View File
@@ -1,109 +0,0 @@
import crypto from "node:crypto";
import net from "node:net";
export type ExecHostRequest = {
command: string[];
rawCommand?: string | null;
cwd?: string | null;
env?: Record<string, string> | null;
timeoutMs?: number | null;
needsScreenRecording?: boolean | null;
agentId?: string | null;
sessionKey?: string | null;
};
export type ExecHostRunResult = {
exitCode?: number;
timedOut: boolean;
success: boolean;
stdout: string;
stderr: string;
error?: string | null;
};
export type ExecHostError = {
code: string;
message: string;
reason?: string;
};
export type ExecHostResponse =
| { ok: true; payload: ExecHostRunResult }
| { ok: false; error: ExecHostError };
export async function requestExecHostViaSocket(params: {
socketPath: string;
token: string;
request: ExecHostRequest;
timeoutMs?: number;
}): Promise<ExecHostResponse | null> {
const { socketPath, token, request } = params;
if (!socketPath || !token) return null;
const timeoutMs = params.timeoutMs ?? 20_000;
return await new Promise((resolve) => {
const client = new net.Socket();
let settled = false;
let buffer = "";
const finish = (value: ExecHostResponse | null) => {
if (settled) return;
settled = true;
try {
client.destroy();
} catch {
// ignore
}
resolve(value);
};
const requestJson = JSON.stringify(request);
const nonce = crypto.randomBytes(16).toString("hex");
const ts = Date.now();
const hmac = crypto
.createHmac("sha256", token)
.update(`${nonce}:${ts}:${requestJson}`)
.digest("hex");
const payload = JSON.stringify({
type: "exec",
id: crypto.randomUUID(),
nonce,
ts,
hmac,
requestJson,
});
const timer = setTimeout(() => finish(null), timeoutMs);
client.on("error", () => finish(null));
client.connect(socketPath, () => {
client.write(`${payload}\n`);
});
client.on("data", (data) => {
buffer += data.toString("utf8");
let idx = buffer.indexOf("\n");
while (idx !== -1) {
const line = buffer.slice(0, idx).trim();
buffer = buffer.slice(idx + 1);
idx = buffer.indexOf("\n");
if (!line) continue;
try {
const msg = JSON.parse(line) as { type?: string; ok?: boolean; payload?: unknown; error?: unknown };
if (msg?.type === "exec-res") {
clearTimeout(timer);
if (msg.ok === true && msg.payload) {
finish({ ok: true, payload: msg.payload as ExecHostRunResult });
return;
}
if (msg.ok === false && msg.error) {
finish({ ok: false, error: msg.error as ExecHostError });
return;
}
finish(null);
return;
}
} catch {
// ignore
}
}
});
});
}
+4 -13
View File
@@ -36,20 +36,11 @@ function extractErrorMessage(err: unknown): string | undefined {
if (typeof err === "object" && "message" in err && typeof err.message === "string") {
return err.message;
}
if (typeof err === "number" || typeof err === "boolean" || typeof err === "bigint") {
return String(err);
try {
return JSON.stringify(err);
} catch {
return undefined;
}
if (typeof err === "symbol") {
return err.toString();
}
if (typeof err === "object") {
try {
return JSON.stringify(err);
} catch {
return undefined;
}
}
return undefined;
}
function logRemoteBinProbeFailure(nodeId: string, err: unknown) {
-407
View File
@@ -1,407 +0,0 @@
import { createSubsystemLogger } from "../logging.js";
import type { GeminiEmbeddingClient } from "./embeddings-gemini.js";
import { hashText } from "./internal.js";
export type GeminiBatchRequest = {
custom_id: string;
content: { parts: Array<{ text: string }> };
taskType: "RETRIEVAL_DOCUMENT" | "RETRIEVAL_QUERY";
};
export type GeminiBatchStatus = {
name?: string;
state?: string;
outputConfig?: { file?: string; fileId?: string };
metadata?: {
output?: {
responsesFile?: string;
};
};
error?: { message?: string };
};
export type GeminiBatchOutputLine = {
key?: string;
custom_id?: string;
request_id?: string;
embedding?: { values?: number[] };
response?: {
embedding?: { values?: number[] };
error?: { message?: string };
};
error?: { message?: string };
};
const GEMINI_BATCH_MAX_REQUESTS = 50000;
const debugEmbeddings = process.env.CLAWDBOT_DEBUG_MEMORY_EMBEDDINGS === "1";
const log = createSubsystemLogger("memory/embeddings");
const debugLog = (message: string, meta?: Record<string, unknown>) => {
if (!debugEmbeddings) return;
const suffix = meta ? ` ${JSON.stringify(meta)}` : "";
log.raw(`${message}${suffix}`);
};
function getGeminiBaseUrl(gemini: GeminiEmbeddingClient): string {
return gemini.baseUrl?.replace(/\/$/, "") ?? "";
}
function getGeminiHeaders(
gemini: GeminiEmbeddingClient,
params: { json: boolean },
): Record<string, string> {
const headers = gemini.headers ? { ...gemini.headers } : {};
if (params.json) {
if (!headers["Content-Type"] && !headers["content-type"]) {
headers["Content-Type"] = "application/json";
}
} else {
delete headers["Content-Type"];
delete headers["content-type"];
}
return headers;
}
function getGeminiUploadUrl(baseUrl: string): string {
if (baseUrl.includes("/v1beta")) {
return baseUrl.replace(/\/v1beta\/?$/, "/upload/v1beta");
}
return `${baseUrl.replace(/\/$/, "")}/upload`;
}
function splitGeminiBatchRequests(requests: GeminiBatchRequest[]): GeminiBatchRequest[][] {
if (requests.length <= GEMINI_BATCH_MAX_REQUESTS) return [requests];
const groups: GeminiBatchRequest[][] = [];
for (let i = 0; i < requests.length; i += GEMINI_BATCH_MAX_REQUESTS) {
groups.push(requests.slice(i, i + GEMINI_BATCH_MAX_REQUESTS));
}
return groups;
}
function buildGeminiUploadBody(params: { jsonl: string; displayName: string }): {
body: Blob;
contentType: string;
} {
const boundary = `clawdbot-${hashText(params.displayName)}`;
const jsonPart = JSON.stringify({
file: {
displayName: params.displayName,
mimeType: "application/jsonl",
},
});
const delimiter = `--${boundary}\r\n`;
const closeDelimiter = `--${boundary}--\r\n`;
const parts = [
`${delimiter}Content-Type: application/json; charset=UTF-8\r\n\r\n${jsonPart}\r\n`,
`${delimiter}Content-Type: application/jsonl; charset=UTF-8\r\n\r\n${params.jsonl}\r\n`,
closeDelimiter,
];
const body = new Blob([parts.join("")], { type: "multipart/related" });
return {
body,
contentType: `multipart/related; boundary=${boundary}`,
};
}
async function submitGeminiBatch(params: {
gemini: GeminiEmbeddingClient;
requests: GeminiBatchRequest[];
agentId: string;
}): Promise<GeminiBatchStatus> {
const baseUrl = getGeminiBaseUrl(params.gemini);
const jsonl = params.requests
.map((request) =>
JSON.stringify({
key: request.custom_id,
request: {
content: request.content,
task_type: request.taskType,
},
}),
)
.join("\n");
const displayName = `memory-embeddings-${hashText(String(Date.now()))}`;
const uploadPayload = buildGeminiUploadBody({ jsonl, displayName });
const uploadUrl = `${getGeminiUploadUrl(baseUrl)}/files?uploadType=multipart`;
debugLog("memory embeddings: gemini batch upload", {
uploadUrl,
baseUrl,
requests: params.requests.length,
});
const fileRes = await fetch(uploadUrl, {
method: "POST",
headers: {
...getGeminiHeaders(params.gemini, { json: false }),
"Content-Type": uploadPayload.contentType,
},
body: uploadPayload.body,
});
if (!fileRes.ok) {
const text = await fileRes.text();
throw new Error(`gemini batch file upload failed: ${fileRes.status} ${text}`);
}
const filePayload = (await fileRes.json()) as { name?: string; file?: { name?: string } };
const fileId = filePayload.name ?? filePayload.file?.name;
if (!fileId) {
throw new Error("gemini batch file upload failed: missing file id");
}
const batchBody = {
batch: {
displayName: `memory-embeddings-${params.agentId}`,
inputConfig: {
file_name: fileId,
},
},
};
const batchEndpoint = `${baseUrl}/${params.gemini.modelPath}:asyncBatchEmbedContent`;
debugLog("memory embeddings: gemini batch create", {
batchEndpoint,
fileId,
});
const batchRes = await fetch(batchEndpoint, {
method: "POST",
headers: getGeminiHeaders(params.gemini, { json: true }),
body: JSON.stringify(batchBody),
});
if (batchRes.ok) {
return (await batchRes.json()) as GeminiBatchStatus;
}
const text = await batchRes.text();
if (batchRes.status === 404) {
throw new Error(
"gemini batch create failed: 404 (asyncBatchEmbedContent not available for this model/baseUrl). Disable remote.batch.enabled or switch providers.",
);
}
throw new Error(`gemini batch create failed: ${batchRes.status} ${text}`);
}
async function fetchGeminiBatchStatus(params: {
gemini: GeminiEmbeddingClient;
batchName: string;
}): Promise<GeminiBatchStatus> {
const baseUrl = getGeminiBaseUrl(params.gemini);
const name = params.batchName.startsWith("batches/") ? params.batchName : `batches/${params.batchName}`;
const statusUrl = `${baseUrl}/${name}`;
debugLog("memory embeddings: gemini batch status", { statusUrl });
const res = await fetch(statusUrl, {
headers: getGeminiHeaders(params.gemini, { json: true }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`gemini batch status failed: ${res.status} ${text}`);
}
return (await res.json()) as GeminiBatchStatus;
}
async function fetchGeminiFileContent(params: {
gemini: GeminiEmbeddingClient;
fileId: string;
}): Promise<string> {
const baseUrl = getGeminiBaseUrl(params.gemini);
const file = params.fileId.startsWith("files/") ? params.fileId : `files/${params.fileId}`;
const downloadUrl = `${baseUrl}/${file}:download`;
debugLog("memory embeddings: gemini batch download", { downloadUrl });
const res = await fetch(downloadUrl, {
headers: getGeminiHeaders(params.gemini, { json: true }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`gemini batch file content failed: ${res.status} ${text}`);
}
return await res.text();
}
function parseGeminiBatchOutput(text: string): GeminiBatchOutputLine[] {
if (!text.trim()) return [];
return text
.split("\n")
.map((line) => line.trim())
.filter(Boolean)
.map((line) => JSON.parse(line) as GeminiBatchOutputLine);
}
async function waitForGeminiBatch(params: {
gemini: GeminiEmbeddingClient;
batchName: string;
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
initial?: GeminiBatchStatus;
}): Promise<{ outputFileId: string }> {
const start = Date.now();
let current: GeminiBatchStatus | undefined = params.initial;
while (true) {
const status =
current ??
(await fetchGeminiBatchStatus({
gemini: params.gemini,
batchName: params.batchName,
}));
const state = status.state ?? "UNKNOWN";
if (["SUCCEEDED", "COMPLETED", "DONE"].includes(state)) {
const outputFileId =
status.outputConfig?.file ??
status.outputConfig?.fileId ??
status.metadata?.output?.responsesFile;
if (!outputFileId) {
throw new Error(`gemini batch ${params.batchName} completed without output file`);
}
return { outputFileId };
}
if (["FAILED", "CANCELLED", "CANCELED", "EXPIRED"].includes(state)) {
const message = status.error?.message ?? "unknown error";
throw new Error(`gemini batch ${params.batchName} ${state}: ${message}`);
}
if (!params.wait) {
throw new Error(`gemini batch ${params.batchName} still ${state}; wait disabled`);
}
if (Date.now() - start > params.timeoutMs) {
throw new Error(`gemini batch ${params.batchName} timed out after ${params.timeoutMs}ms`);
}
params.debug?.(`gemini batch ${params.batchName} ${state}; waiting ${params.pollIntervalMs}ms`);
await new Promise((resolve) => setTimeout(resolve, params.pollIntervalMs));
current = undefined;
}
}
async function runWithConcurrency<T>(tasks: Array<() => Promise<T>>, limit: number): Promise<T[]> {
if (tasks.length === 0) return [];
const resolvedLimit = Math.max(1, Math.min(limit, tasks.length));
const results: T[] = Array.from({ length: tasks.length });
let next = 0;
let firstError: unknown = null;
const workers = Array.from({ length: resolvedLimit }, async () => {
while (true) {
if (firstError) return;
const index = next;
next += 1;
if (index >= tasks.length) return;
try {
results[index] = await tasks[index]();
} catch (err) {
firstError = err;
return;
}
}
});
await Promise.allSettled(workers);
if (firstError) throw firstError;
return results;
}
export async function runGeminiEmbeddingBatches(params: {
gemini: GeminiEmbeddingClient;
agentId: string;
requests: GeminiBatchRequest[];
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
concurrency: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<Map<string, number[]>> {
if (params.requests.length === 0) return new Map();
const groups = splitGeminiBatchRequests(params.requests);
const byCustomId = new Map<string, number[]>();
const tasks = groups.map((group, groupIndex) => async () => {
const batchInfo = await submitGeminiBatch({
gemini: params.gemini,
requests: group,
agentId: params.agentId,
});
const batchName = batchInfo.name ?? "";
if (!batchName) {
throw new Error("gemini batch create failed: missing batch name");
}
params.debug?.("memory embeddings: gemini batch created", {
batchName,
state: batchInfo.state,
group: groupIndex + 1,
groups: groups.length,
requests: group.length,
});
if (!params.wait && batchInfo.state && !["SUCCEEDED", "COMPLETED", "DONE"].includes(batchInfo.state)) {
throw new Error(
`gemini batch ${batchName} submitted; enable remote.batch.wait to await completion`,
);
}
const completed =
batchInfo.state && ["SUCCEEDED", "COMPLETED", "DONE"].includes(batchInfo.state)
? {
outputFileId:
batchInfo.outputConfig?.file ??
batchInfo.outputConfig?.fileId ??
batchInfo.metadata?.output?.responsesFile ??
"",
}
: await waitForGeminiBatch({
gemini: params.gemini,
batchName,
wait: params.wait,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
debug: params.debug,
initial: batchInfo,
});
if (!completed.outputFileId) {
throw new Error(`gemini batch ${batchName} completed without output file`);
}
const content = await fetchGeminiFileContent({
gemini: params.gemini,
fileId: completed.outputFileId,
});
const outputLines = parseGeminiBatchOutput(content);
const errors: string[] = [];
const remaining = new Set(group.map((request) => request.custom_id));
for (const line of outputLines) {
const customId = line.key ?? line.custom_id ?? line.request_id;
if (!customId) continue;
remaining.delete(customId);
if (line.error?.message) {
errors.push(`${customId}: ${line.error.message}`);
continue;
}
if (line.response?.error?.message) {
errors.push(`${customId}: ${line.response.error.message}`);
continue;
}
const embedding =
line.embedding?.values ?? line.response?.embedding?.values ?? [];
if (embedding.length === 0) {
errors.push(`${customId}: empty embedding`);
continue;
}
byCustomId.set(customId, embedding);
}
if (errors.length > 0) {
throw new Error(`gemini batch ${batchName} failed: ${errors.join("; ")}`);
}
if (remaining.size > 0) {
throw new Error(`gemini batch ${batchName} missing ${remaining.size} embedding responses`);
}
});
params.debug?.("memory embeddings: gemini batch submit", {
requests: params.requests.length,
groups: groups.length,
wait: params.wait,
concurrency: params.concurrency,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
});
await runWithConcurrency(tasks, params.concurrency);
return byCustomId;
}
-382
View File
@@ -1,382 +0,0 @@
import { retryAsync } from "../infra/retry.js";
import type { OpenAiEmbeddingClient } from "./embeddings-openai.js";
import { hashText } from "./internal.js";
export type OpenAiBatchRequest = {
custom_id: string;
method: "POST";
url: "/v1/embeddings";
body: {
model: string;
input: string;
};
};
export type OpenAiBatchStatus = {
id?: string;
status?: string;
output_file_id?: string | null;
error_file_id?: string | null;
};
export type OpenAiBatchOutputLine = {
custom_id?: string;
response?: {
status_code?: number;
body?: {
data?: Array<{ embedding?: number[]; index?: number }>;
error?: { message?: string };
};
};
error?: { message?: string };
};
export const OPENAI_BATCH_ENDPOINT = "/v1/embeddings";
const OPENAI_BATCH_COMPLETION_WINDOW = "24h";
const OPENAI_BATCH_MAX_REQUESTS = 50000;
function getOpenAiBaseUrl(openAi: OpenAiEmbeddingClient): string {
return openAi.baseUrl?.replace(/\/$/, "") ?? "";
}
function getOpenAiHeaders(
openAi: OpenAiEmbeddingClient,
params: { json: boolean },
): Record<string, string> {
const headers = openAi.headers ? { ...openAi.headers } : {};
if (params.json) {
if (!headers["Content-Type"] && !headers["content-type"]) {
headers["Content-Type"] = "application/json";
}
} else {
delete headers["Content-Type"];
delete headers["content-type"];
}
return headers;
}
function splitOpenAiBatchRequests(requests: OpenAiBatchRequest[]): OpenAiBatchRequest[][] {
if (requests.length <= OPENAI_BATCH_MAX_REQUESTS) return [requests];
const groups: OpenAiBatchRequest[][] = [];
for (let i = 0; i < requests.length; i += OPENAI_BATCH_MAX_REQUESTS) {
groups.push(requests.slice(i, i + OPENAI_BATCH_MAX_REQUESTS));
}
return groups;
}
async function submitOpenAiBatch(params: {
openAi: OpenAiEmbeddingClient;
requests: OpenAiBatchRequest[];
agentId: string;
}): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const jsonl = params.requests.map((request) => JSON.stringify(request)).join("\n");
const form = new FormData();
form.append("purpose", "batch");
form.append(
"file",
new Blob([jsonl], { type: "application/jsonl" }),
`memory-embeddings.${hashText(String(Date.now()))}.jsonl`,
);
const fileRes = await fetch(`${baseUrl}/files`, {
method: "POST",
headers: getOpenAiHeaders(params.openAi, { json: false }),
body: form,
});
if (!fileRes.ok) {
const text = await fileRes.text();
throw new Error(`openai batch file upload failed: ${fileRes.status} ${text}`);
}
const filePayload = (await fileRes.json()) as { id?: string };
if (!filePayload.id) {
throw new Error("openai batch file upload failed: missing file id");
}
const batchRes = await retryAsync(
async () => {
const res = await fetch(`${baseUrl}/batches`, {
method: "POST",
headers: getOpenAiHeaders(params.openAi, { json: true }),
body: JSON.stringify({
input_file_id: filePayload.id,
endpoint: OPENAI_BATCH_ENDPOINT,
completion_window: OPENAI_BATCH_COMPLETION_WINDOW,
metadata: {
source: "clawdbot-memory",
agent: params.agentId,
},
}),
});
if (!res.ok) {
const text = await res.text();
const err = new Error(`openai batch create failed: ${res.status} ${text}`) as Error & {
status?: number;
};
err.status = res.status;
throw err;
}
return res;
},
{
attempts: 3,
minDelayMs: 300,
maxDelayMs: 2000,
jitter: 0.2,
shouldRetry: (err) => {
const status = (err as { status?: number }).status;
return status === 429 || (typeof status === "number" && status >= 500);
},
},
);
return (await batchRes.json()) as OpenAiBatchStatus;
}
async function fetchOpenAiBatchStatus(params: {
openAi: OpenAiEmbeddingClient;
batchId: string;
}): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetch(`${baseUrl}/batches/${params.batchId}`, {
headers: getOpenAiHeaders(params.openAi, { json: true }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`openai batch status failed: ${res.status} ${text}`);
}
return (await res.json()) as OpenAiBatchStatus;
}
async function fetchOpenAiFileContent(params: {
openAi: OpenAiEmbeddingClient;
fileId: string;
}): Promise<string> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetch(`${baseUrl}/files/${params.fileId}/content`, {
headers: getOpenAiHeaders(params.openAi, { json: true }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`openai batch file content failed: ${res.status} ${text}`);
}
return await res.text();
}
function parseOpenAiBatchOutput(text: string): OpenAiBatchOutputLine[] {
if (!text.trim()) return [];
return text
.split("\n")
.map((line) => line.trim())
.filter(Boolean)
.map((line) => JSON.parse(line) as OpenAiBatchOutputLine);
}
async function readOpenAiBatchError(params: {
openAi: OpenAiEmbeddingClient;
errorFileId: string;
}): Promise<string | undefined> {
try {
const content = await fetchOpenAiFileContent({
openAi: params.openAi,
fileId: params.errorFileId,
});
const lines = parseOpenAiBatchOutput(content);
const first = lines.find((line) => line.error?.message || line.response?.body?.error);
const message =
first?.error?.message ??
(typeof first?.response?.body?.error?.message === "string"
? first?.response?.body?.error?.message
: undefined);
return message;
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
return message ? `error file unavailable: ${message}` : undefined;
}
}
async function waitForOpenAiBatch(params: {
openAi: OpenAiEmbeddingClient;
batchId: string;
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
initial?: OpenAiBatchStatus;
}): Promise<{ outputFileId: string; errorFileId?: string }> {
const start = Date.now();
let current: OpenAiBatchStatus | undefined = params.initial;
while (true) {
const status =
current ??
(await fetchOpenAiBatchStatus({
openAi: params.openAi,
batchId: params.batchId,
}));
const state = status.status ?? "unknown";
if (state === "completed") {
if (!status.output_file_id) {
throw new Error(`openai batch ${params.batchId} completed without output file`);
}
return {
outputFileId: status.output_file_id,
errorFileId: status.error_file_id ?? undefined,
};
}
if (["failed", "expired", "cancelled", "canceled"].includes(state)) {
const detail = status.error_file_id
? await readOpenAiBatchError({ openAi: params.openAi, errorFileId: status.error_file_id })
: undefined;
const suffix = detail ? `: ${detail}` : "";
throw new Error(`openai batch ${params.batchId} ${state}${suffix}`);
}
if (!params.wait) {
throw new Error(`openai batch ${params.batchId} still ${state}; wait disabled`);
}
if (Date.now() - start > params.timeoutMs) {
throw new Error(`openai batch ${params.batchId} timed out after ${params.timeoutMs}ms`);
}
params.debug?.(`openai batch ${params.batchId} ${state}; waiting ${params.pollIntervalMs}ms`);
await new Promise((resolve) => setTimeout(resolve, params.pollIntervalMs));
current = undefined;
}
}
async function runWithConcurrency<T>(tasks: Array<() => Promise<T>>, limit: number): Promise<T[]> {
if (tasks.length === 0) return [];
const resolvedLimit = Math.max(1, Math.min(limit, tasks.length));
const results: T[] = Array.from({ length: tasks.length });
let next = 0;
let firstError: unknown = null;
const workers = Array.from({ length: resolvedLimit }, async () => {
while (true) {
if (firstError) return;
const index = next;
next += 1;
if (index >= tasks.length) return;
try {
results[index] = await tasks[index]();
} catch (err) {
firstError = err;
return;
}
}
});
await Promise.allSettled(workers);
if (firstError) throw firstError;
return results;
}
export async function runOpenAiEmbeddingBatches(params: {
openAi: OpenAiEmbeddingClient;
agentId: string;
requests: OpenAiBatchRequest[];
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
concurrency: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<Map<string, number[]>> {
if (params.requests.length === 0) return new Map();
const groups = splitOpenAiBatchRequests(params.requests);
const byCustomId = new Map<string, number[]>();
const tasks = groups.map((group, groupIndex) => async () => {
const batchInfo = await submitOpenAiBatch({
openAi: params.openAi,
requests: group,
agentId: params.agentId,
});
if (!batchInfo.id) {
throw new Error("openai batch create failed: missing batch id");
}
params.debug?.("memory embeddings: openai batch created", {
batchId: batchInfo.id,
status: batchInfo.status,
group: groupIndex + 1,
groups: groups.length,
requests: group.length,
});
if (!params.wait && batchInfo.status !== "completed") {
throw new Error(
`openai batch ${batchInfo.id} submitted; enable remote.batch.wait to await completion`,
);
}
const completed =
batchInfo.status === "completed"
? {
outputFileId: batchInfo.output_file_id ?? "",
errorFileId: batchInfo.error_file_id ?? undefined,
}
: await waitForOpenAiBatch({
openAi: params.openAi,
batchId: batchInfo.id,
wait: params.wait,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
debug: params.debug,
initial: batchInfo,
});
if (!completed.outputFileId) {
throw new Error(`openai batch ${batchInfo.id} completed without output file`);
}
const content = await fetchOpenAiFileContent({
openAi: params.openAi,
fileId: completed.outputFileId,
});
const outputLines = parseOpenAiBatchOutput(content);
const errors: string[] = [];
const remaining = new Set(group.map((request) => request.custom_id));
for (const line of outputLines) {
const customId = line.custom_id;
if (!customId) continue;
remaining.delete(customId);
if (line.error?.message) {
errors.push(`${customId}: ${line.error.message}`);
continue;
}
const response = line.response;
const statusCode = response?.status_code ?? 0;
if (statusCode >= 400) {
const message =
response?.body?.error?.message ??
(typeof response?.body === "string" ? response.body : undefined) ??
"unknown error";
errors.push(`${customId}: ${message}`);
continue;
}
const data = response?.body?.data ?? [];
const embedding = data[0]?.embedding ?? [];
if (embedding.length === 0) {
errors.push(`${customId}: empty embedding`);
continue;
}
byCustomId.set(customId, embedding);
}
if (errors.length > 0) {
throw new Error(`openai batch ${batchInfo.id} failed: ${errors.join("; ")}`);
}
if (remaining.size > 0) {
throw new Error(`openai batch ${batchInfo.id} missing ${remaining.size} embedding responses`);
}
});
params.debug?.("memory embeddings: openai batch submit", {
requests: params.requests.length,
groups: groups.length,
wait: params.wait,
concurrency: params.concurrency,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
});
await runWithConcurrency(tasks, params.concurrency);
return byCustomId;
}
-145
View File
@@ -1,145 +0,0 @@
import { resolveApiKeyForProvider } from "../agents/model-auth.js";
import { createSubsystemLogger } from "../logging.js";
import type { EmbeddingProvider, EmbeddingProviderOptions } from "./embeddings.js";
export type GeminiEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
model: string;
modelPath: string;
};
const DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
export const DEFAULT_GEMINI_EMBEDDING_MODEL = "gemini-embedding-001";
const debugEmbeddings = process.env.CLAWDBOT_DEBUG_MEMORY_EMBEDDINGS === "1";
const log = createSubsystemLogger("memory/embeddings");
const debugLog = (message: string, meta?: Record<string, unknown>) => {
if (!debugEmbeddings) return;
const suffix = meta ? ` ${JSON.stringify(meta)}` : "";
log.raw(`${message}${suffix}`);
};
function resolveRemoteApiKey(remoteApiKey?: string): string | undefined {
const trimmed = remoteApiKey?.trim();
if (!trimmed) return undefined;
if (trimmed === "GOOGLE_API_KEY" || trimmed === "GEMINI_API_KEY") {
return process.env[trimmed]?.trim();
}
return trimmed;
}
function normalizeGeminiModel(model: string): string {
const trimmed = model.trim();
if (!trimmed) return DEFAULT_GEMINI_EMBEDDING_MODEL;
const withoutPrefix = trimmed.replace(/^models\//, "");
if (withoutPrefix.startsWith("gemini/")) return withoutPrefix.slice("gemini/".length);
if (withoutPrefix.startsWith("google/")) return withoutPrefix.slice("google/".length);
return withoutPrefix;
}
function normalizeGeminiBaseUrl(raw: string): string {
const trimmed = raw.replace(/\/+$/, "");
const openAiIndex = trimmed.indexOf("/openai");
if (openAiIndex > -1) return trimmed.slice(0, openAiIndex);
return trimmed;
}
function buildGeminiModelPath(model: string): string {
return model.startsWith("models/") ? model : `models/${model}`;
}
export async function createGeminiEmbeddingProvider(
options: EmbeddingProviderOptions,
): Promise<{ provider: EmbeddingProvider; client: GeminiEmbeddingClient }> {
const client = await resolveGeminiEmbeddingClient(options);
const baseUrl = client.baseUrl.replace(/\/$/, "");
const embedUrl = `${baseUrl}/${client.modelPath}:embedContent`;
const batchUrl = `${baseUrl}/${client.modelPath}:batchEmbedContents`;
const embedQuery = async (text: string): Promise<number[]> => {
if (!text.trim()) return [];
const res = await fetch(embedUrl, {
method: "POST",
headers: client.headers,
body: JSON.stringify({
content: { parts: [{ text }] },
taskType: "RETRIEVAL_QUERY",
}),
});
if (!res.ok) {
const payload = await res.text();
throw new Error(`gemini embeddings failed: ${res.status} ${payload}`);
}
const payload = (await res.json()) as { embedding?: { values?: number[] } };
return payload.embedding?.values ?? [];
};
const embedBatch = async (texts: string[]): Promise<number[][]> => {
if (texts.length === 0) return [];
const requests = texts.map((text) => ({
model: client.modelPath,
content: { parts: [{ text }] },
taskType: "RETRIEVAL_DOCUMENT",
}));
const res = await fetch(batchUrl, {
method: "POST",
headers: client.headers,
body: JSON.stringify({ requests }),
});
if (!res.ok) {
const payload = await res.text();
throw new Error(`gemini embeddings failed: ${res.status} ${payload}`);
}
const payload = (await res.json()) as { embeddings?: Array<{ values?: number[] }> };
const embeddings = Array.isArray(payload.embeddings) ? payload.embeddings : [];
return texts.map((_, index) => embeddings[index]?.values ?? []);
};
return {
provider: {
id: "gemini",
model: client.model,
embedQuery,
embedBatch,
},
client,
};
}
export async function resolveGeminiEmbeddingClient(
options: EmbeddingProviderOptions,
): Promise<GeminiEmbeddingClient> {
const remote = options.remote;
const remoteApiKey = resolveRemoteApiKey(remote?.apiKey);
const remoteBaseUrl = remote?.baseUrl?.trim();
const { apiKey } = remoteApiKey
? { apiKey: remoteApiKey }
: await resolveApiKeyForProvider({
provider: "google",
cfg: options.config,
agentDir: options.agentDir,
});
const providerConfig = options.config.models?.providers?.google;
const rawBaseUrl = remoteBaseUrl || providerConfig?.baseUrl?.trim() || DEFAULT_GEMINI_BASE_URL;
const baseUrl = normalizeGeminiBaseUrl(rawBaseUrl);
const headerOverrides = Object.assign({}, providerConfig?.headers, remote?.headers);
const headers: Record<string, string> = {
"Content-Type": "application/json",
"x-goog-api-key": apiKey,
...headerOverrides,
};
const model = normalizeGeminiModel(options.model);
const modelPath = buildGeminiModelPath(model);
debugLog("memory embeddings: gemini client", {
rawBaseUrl,
baseUrl,
model,
modelPath,
embedEndpoint: `${baseUrl}/${modelPath}:embedContent`,
batchEndpoint: `${baseUrl}/${modelPath}:batchEmbedContents`,
});
return { baseUrl, headers, model, modelPath };
}
-83
View File
@@ -1,83 +0,0 @@
import { resolveApiKeyForProvider } from "../agents/model-auth.js";
import type { EmbeddingProvider, EmbeddingProviderOptions } from "./embeddings.js";
export type OpenAiEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
model: string;
};
export const DEFAULT_OPENAI_EMBEDDING_MODEL = "text-embedding-3-small";
const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1";
export function normalizeOpenAiModel(model: string): string {
const trimmed = model.trim();
if (!trimmed) return DEFAULT_OPENAI_EMBEDDING_MODEL;
if (trimmed.startsWith("openai/")) return trimmed.slice("openai/".length);
return trimmed;
}
export async function createOpenAiEmbeddingProvider(
options: EmbeddingProviderOptions,
): Promise<{ provider: EmbeddingProvider; client: OpenAiEmbeddingClient }> {
const client = await resolveOpenAiEmbeddingClient(options);
const url = `${client.baseUrl.replace(/\/$/, "")}/embeddings`;
const embed = async (input: string[]): Promise<number[][]> => {
if (input.length === 0) return [];
const res = await fetch(url, {
method: "POST",
headers: client.headers,
body: JSON.stringify({ model: client.model, input }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`openai embeddings failed: ${res.status} ${text}`);
}
const payload = (await res.json()) as {
data?: Array<{ embedding?: number[] }>;
};
const data = payload.data ?? [];
return data.map((entry) => entry.embedding ?? []);
};
return {
provider: {
id: "openai",
model: client.model,
embedQuery: async (text) => {
const [vec] = await embed([text]);
return vec ?? [];
},
embedBatch: embed,
},
client,
};
}
export async function resolveOpenAiEmbeddingClient(
options: EmbeddingProviderOptions,
): Promise<OpenAiEmbeddingClient> {
const remote = options.remote;
const remoteApiKey = remote?.apiKey?.trim();
const remoteBaseUrl = remote?.baseUrl?.trim();
const { apiKey } = remoteApiKey
? { apiKey: remoteApiKey }
: await resolveApiKeyForProvider({
provider: "openai",
cfg: options.config,
agentDir: options.agentDir,
});
const providerConfig = options.config.models?.providers?.openai;
const baseUrl = remoteBaseUrl || providerConfig?.baseUrl?.trim() || DEFAULT_OPENAI_BASE_URL;
const headerOverrides = Object.assign({}, providerConfig?.headers, remote?.headers);
const headers: Record<string, string> = {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
...headerOverrides,
};
const model = normalizeOpenAiModel(options.model);
return { baseUrl, headers, model };
}
+6 -71
View File
@@ -108,7 +108,7 @@ describe("embedding provider remote overrides", () => {
expect(headers.Authorization).toBe("Bearer provider-key");
});
it("builds Gemini embeddings requests with api key header", async () => {
it("uses gemini embedContent endpoint with x-goog-api-key", async () => {
const fetchMock = vi.fn(async () => ({
ok: true,
status: 200,
@@ -119,94 +119,29 @@ describe("embedding provider remote overrides", () => {
const { createEmbeddingProvider } = await import("./embeddings.js");
const authModule = await import("../agents/model-auth.js");
vi.mocked(authModule.resolveApiKeyForProvider).mockResolvedValue({
apiKey: "provider-key",
apiKey: "gemini-key",
});
const cfg = {
models: {
providers: {
google: {
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
},
},
},
};
const result = await createEmbeddingProvider({
config: cfg as never,
config: {} as never,
provider: "gemini",
remote: {
apiKey: "gemini-key",
baseUrl: "https://gemini.example/v1beta",
},
model: "text-embedding-004",
model: "gemini-embedding-001",
fallback: "openai",
});
await result.provider.embedQuery("hello");
const [url, init] = fetchMock.mock.calls[0] ?? [];
expect(url).toBe(
"https://generativelanguage.googleapis.com/v1beta/models/text-embedding-004:embedContent",
);
expect(url).toBe("https://gemini.example/v1beta/models/gemini-embedding-001:embedContent");
const headers = (init?.headers ?? {}) as Record<string, string>;
expect(headers["x-goog-api-key"]).toBe("gemini-key");
expect(headers["Content-Type"]).toBe("application/json");
});
});
describe("embedding provider auto selection", () => {
afterEach(() => {
vi.resetAllMocks();
vi.resetModules();
vi.unstubAllGlobals();
});
it("prefers openai when a key resolves", async () => {
const { createEmbeddingProvider } = await import("./embeddings.js");
const authModule = await import("../agents/model-auth.js");
vi.mocked(authModule.resolveApiKeyForProvider).mockImplementation(async ({ provider }) => {
if (provider === "openai") {
return { apiKey: "openai-key", source: "env: OPENAI_API_KEY" };
}
throw new Error(`No API key found for provider "${provider}".`);
});
const result = await createEmbeddingProvider({
config: {} as never,
provider: "auto",
model: "",
fallback: "none",
});
expect(result.requestedProvider).toBe("auto");
expect(result.provider.id).toBe("openai");
});
it("uses gemini when openai is missing", async () => {
const { createEmbeddingProvider } = await import("./embeddings.js");
const authModule = await import("../agents/model-auth.js");
vi.mocked(authModule.resolveApiKeyForProvider).mockImplementation(async ({ provider }) => {
if (provider === "openai") {
throw new Error('No API key found for provider "openai".');
}
if (provider === "google") {
return { apiKey: "gemini-key", source: "env: GEMINI_API_KEY" };
}
throw new Error(`Unexpected provider ${provider}`);
});
const result = await createEmbeddingProvider({
config: {} as never,
provider: "auto",
model: "",
fallback: "none",
});
expect(result.requestedProvider).toBe("auto");
expect(result.provider.id).toBe("gemini");
});
});
describe("embedding provider local fallback", () => {
afterEach(() => {
vi.resetAllMocks();
+222 -99
View File
@@ -1,21 +1,8 @@
import fsSync from "node:fs";
import type { Llama, LlamaEmbeddingContext, LlamaModel } from "node-llama-cpp";
import { resolveApiKeyForProvider } from "../agents/model-auth.js";
import type { ClawdbotConfig } from "../config/config.js";
import { resolveUserPath } from "../utils.js";
import {
createGeminiEmbeddingProvider,
type GeminiEmbeddingClient,
} from "./embeddings-gemini.js";
import {
createOpenAiEmbeddingProvider,
type OpenAiEmbeddingClient,
} from "./embeddings-openai.js";
import { importNodeLlamaCpp } from "./node-llama.js";
export type { GeminiEmbeddingClient } from "./embeddings-gemini.js";
export type { OpenAiEmbeddingClient } from "./embeddings-openai.js";
export type EmbeddingProvider = {
id: string;
model: string;
@@ -25,49 +12,230 @@ export type EmbeddingProvider = {
export type EmbeddingProviderResult = {
provider: EmbeddingProvider;
requestedProvider: "openai" | "local" | "gemini" | "auto";
fallbackFrom?: "openai" | "local" | "gemini";
requestedProvider: "openai" | "gemini" | "local";
fallbackFrom?: "local";
fallbackReason?: string;
openAi?: OpenAiEmbeddingClient;
gemini?: GeminiEmbeddingClient;
};
export type OpenAiEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
model: string;
};
export type GeminiEmbeddingClient = {
baseUrl: string;
headers: Record<string, string>;
model: string;
};
export type EmbeddingProviderOptions = {
config: ClawdbotConfig;
agentDir?: string;
provider: "openai" | "local" | "gemini" | "auto";
provider: "openai" | "gemini" | "local";
remote?: {
baseUrl?: string;
apiKey?: string;
headers?: Record<string, string>;
};
model: string;
fallback: "openai" | "gemini" | "local" | "none";
fallback: "openai" | "none";
local?: {
modelPath?: string;
modelCacheDir?: string;
};
};
const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1";
const DEFAULT_LOCAL_MODEL = "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf";
const DEFAULT_GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta";
const DEFAULT_GEMINI_MODEL = "gemini-embedding-001";
function canAutoSelectLocal(options: EmbeddingProviderOptions): boolean {
const modelPath = options.local?.modelPath?.trim();
if (!modelPath) return false;
if (/^(hf:|https?:)/i.test(modelPath)) return false;
const resolved = resolveUserPath(modelPath);
try {
return fsSync.statSync(resolved).isFile();
} catch {
return false;
function normalizeOpenAiModel(model: string): string {
const trimmed = model.trim();
if (!trimmed) return "text-embedding-3-small";
if (trimmed.startsWith("openai/")) return trimmed.slice("openai/".length);
return trimmed;
}
function normalizeGeminiModel(model: string): string {
const trimmed = model.trim();
if (!trimmed) return DEFAULT_GEMINI_MODEL;
if (trimmed.startsWith("models/")) return trimmed.slice("models/".length);
if (trimmed.startsWith("google/")) return trimmed.slice("google/".length);
return trimmed;
}
async function createOpenAiEmbeddingProvider(
options: EmbeddingProviderOptions,
): Promise<{ provider: EmbeddingProvider; client: OpenAiEmbeddingClient }> {
const client = await resolveOpenAiEmbeddingClient(options);
const url = `${client.baseUrl.replace(/\/$/, "")}/embeddings`;
const embed = async (input: string[]): Promise<number[][]> => {
if (input.length === 0) return [];
const res = await fetch(url, {
method: "POST",
headers: client.headers,
body: JSON.stringify({ model: client.model, input }),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`openai embeddings failed: ${res.status} ${text}`);
}
const payload = (await res.json()) as {
data?: Array<{ embedding?: number[] }>;
};
const data = payload.data ?? [];
return data.map((entry) => entry.embedding ?? []);
};
return {
provider: {
id: "openai",
model: client.model,
embedQuery: async (text) => {
const [vec] = await embed([text]);
return vec ?? [];
},
embedBatch: embed,
},
client,
};
}
function extractGeminiEmbeddingValues(entry: unknown): number[] {
if (!entry || typeof entry !== "object") return [];
const record = entry as { values?: unknown; embedding?: { values?: unknown } };
const values = record.values ?? record.embedding?.values;
if (!Array.isArray(values)) return [];
return values.filter((value): value is number => typeof value === "number");
}
function parseGeminiEmbeddings(payload: unknown): number[][] {
if (!payload || typeof payload !== "object") return [];
const data = payload as { embedding?: unknown; embeddings?: unknown[] };
if (Array.isArray(data.embeddings)) {
return data.embeddings.map((entry) => extractGeminiEmbeddingValues(entry));
}
if (data.embedding) {
return [extractGeminiEmbeddingValues(data.embedding)];
}
return [];
}
function isMissingApiKeyError(err: unknown): boolean {
const message = formatError(err);
return message.includes("No API key found for provider");
async function createGeminiEmbeddingProvider(
options: EmbeddingProviderOptions,
): Promise<{ provider: EmbeddingProvider; client: GeminiEmbeddingClient }> {
const client = await resolveGeminiEmbeddingClient(options);
const baseUrl = client.baseUrl.replace(/\/$/, "");
const model = `models/${client.model}`;
const embedContent = async (input: string): Promise<number[]> => {
const res = await fetch(`${baseUrl}/${model}:embedContent`, {
method: "POST",
headers: client.headers,
body: JSON.stringify({
model,
content: { parts: [{ text: input }] },
}),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`gemini embeddings failed: ${res.status} ${text}`);
}
const payload = await res.json();
const embeddings = parseGeminiEmbeddings(payload);
return embeddings[0] ?? [];
};
const embedBatch = async (input: string[]): Promise<number[][]> => {
if (input.length === 0) return [];
const res = await fetch(`${baseUrl}/${model}:batchEmbedContents`, {
method: "POST",
headers: client.headers,
body: JSON.stringify({
requests: input.map((text) => ({
model,
content: { parts: [{ text }] },
})),
}),
});
if (!res.ok) {
const text = await res.text();
throw new Error(`gemini embeddings failed: ${res.status} ${text}`);
}
const payload = await res.json();
const embeddings = parseGeminiEmbeddings(payload);
return embeddings;
};
return {
provider: {
id: "gemini",
model: client.model,
embedQuery: embedContent,
embedBatch,
},
client,
};
}
async function resolveOpenAiEmbeddingClient(
options: EmbeddingProviderOptions,
): Promise<OpenAiEmbeddingClient> {
const remote = options.remote;
const remoteApiKey = remote?.apiKey?.trim();
const remoteBaseUrl = remote?.baseUrl?.trim();
const { apiKey } = remoteApiKey
? { apiKey: remoteApiKey }
: await resolveApiKeyForProvider({
provider: "openai",
cfg: options.config,
agentDir: options.agentDir,
});
const providerConfig = options.config.models?.providers?.openai;
const baseUrl = remoteBaseUrl || providerConfig?.baseUrl?.trim() || DEFAULT_OPENAI_BASE_URL;
const headerOverrides = Object.assign({}, providerConfig?.headers, remote?.headers);
const headers: Record<string, string> = {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
...headerOverrides,
};
const model = normalizeOpenAiModel(options.model);
return { baseUrl, headers, model };
}
async function resolveGeminiEmbeddingClient(
options: EmbeddingProviderOptions,
): Promise<GeminiEmbeddingClient> {
const remote = options.remote;
const remoteApiKey = remote?.apiKey?.trim();
const remoteBaseUrl = remote?.baseUrl?.trim();
const { apiKey } = remoteApiKey
? { apiKey: remoteApiKey }
: await resolveApiKeyForProvider({
provider: "google",
cfg: options.config,
agentDir: options.agentDir,
});
const providerConfig = options.config.models?.providers?.google;
const baseUrl = remoteBaseUrl || providerConfig?.baseUrl?.trim() || DEFAULT_GEMINI_BASE_URL;
const headerOverrides = Object.assign({}, providerConfig?.headers, remote?.headers);
const headers: Record<string, string> = {
"Content-Type": "application/json",
"x-goog-api-key": apiKey,
...headerOverrides,
};
const model = normalizeGeminiModel(options.model);
return { baseUrl, headers, model };
}
async function createLocalEmbeddingProvider(
options: EmbeddingProviderOptions,
@@ -121,80 +289,35 @@ export async function createEmbeddingProvider(
options: EmbeddingProviderOptions,
): Promise<EmbeddingProviderResult> {
const requestedProvider = options.provider;
const fallback = options.fallback;
const createProvider = async (id: "openai" | "local" | "gemini") => {
if (id === "local") {
if (options.provider === "gemini") {
const { provider, client } = await createGeminiEmbeddingProvider(options);
return { provider, requestedProvider, gemini: client };
}
if (options.provider === "local") {
try {
const provider = await createLocalEmbeddingProvider(options);
return { provider };
}
if (id === "gemini") {
const { provider, client } = await createGeminiEmbeddingProvider(options);
return { provider, gemini: client };
}
const { provider, client } = await createOpenAiEmbeddingProvider(options);
return { provider, openAi: client };
};
const formatPrimaryError = (err: unknown, provider: "openai" | "local" | "gemini") =>
provider === "local" ? formatLocalSetupError(err) : formatError(err);
if (requestedProvider === "auto") {
const missingKeyErrors: string[] = [];
let localError: string | null = null;
if (canAutoSelectLocal(options)) {
try {
const local = await createProvider("local");
return { ...local, requestedProvider };
} catch (err) {
localError = formatLocalSetupError(err);
}
}
for (const provider of ["openai", "gemini"] as const) {
try {
const result = await createProvider(provider);
return { ...result, requestedProvider };
} catch (err) {
const message = formatPrimaryError(err, provider);
if (isMissingApiKeyError(err)) {
missingKeyErrors.push(message);
continue;
return { provider, requestedProvider };
} catch (err) {
const reason = formatLocalSetupError(err);
if (options.fallback === "openai") {
try {
const { provider, client } = await createOpenAiEmbeddingProvider(options);
return {
provider,
requestedProvider,
fallbackFrom: "local",
fallbackReason: reason,
openAi: client,
};
} catch (fallbackErr) {
throw new Error(`${reason}\n\nFallback to OpenAI failed: ${formatError(fallbackErr)}`);
}
throw new Error(message);
}
throw new Error(reason);
}
const details = [...missingKeyErrors, localError].filter(Boolean) as string[];
if (details.length > 0) {
throw new Error(details.join("\n\n"));
}
throw new Error("No embeddings provider available.");
}
try {
const primary = await createProvider(requestedProvider);
return { ...primary, requestedProvider };
} catch (primaryErr) {
const reason = formatPrimaryError(primaryErr, requestedProvider);
if (fallback && fallback !== "none" && fallback !== requestedProvider) {
try {
const fallbackResult = await createProvider(fallback);
return {
...fallbackResult,
requestedProvider,
fallbackFrom: requestedProvider,
fallbackReason: reason,
};
} catch (fallbackErr) {
throw new Error(
`${reason}\n\nFallback to ${fallback} failed: ${formatError(fallbackErr)}`,
);
}
}
throw new Error(reason);
}
const { provider, client } = await createOpenAiEmbeddingProvider(options);
return { provider, requestedProvider, openAi: client };
}
function formatError(err: unknown): string {
-42
View File
@@ -7,7 +7,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { getMemorySearchManager, type MemoryIndexManager } from "./index.js";
let embedBatchCalls = 0;
let failEmbeddings = false;
vi.mock("./embeddings.js", () => {
const embedText = (text: string) => {
@@ -25,9 +24,6 @@ vi.mock("./embeddings.js", () => {
embedQuery: async (text: string) => embedText(text),
embedBatch: async (texts: string[]) => {
embedBatchCalls += 1;
if (failEmbeddings) {
throw new Error("mock embeddings failed");
}
return texts.map(embedText);
},
},
@@ -42,7 +38,6 @@ describe("memory index", () => {
beforeEach(async () => {
embedBatchCalls = 0;
failEmbeddings = false;
workspaceDir = await fs.mkdtemp(path.join(os.tmpdir(), "clawdbot-mem-"));
indexPath = path.join(workspaceDir, "index.sqlite");
await fs.mkdir(path.join(workspaceDir, "memory"));
@@ -186,43 +181,6 @@ describe("memory index", () => {
expect(embedBatchCalls).toBe(afterFirst);
});
it("preserves existing index when forced reindex fails", async () => {
const cfg = {
agents: {
defaults: {
workspace: workspaceDir,
memorySearch: {
provider: "openai",
model: "mock-embed",
store: { path: indexPath, vector: { enabled: false } },
sync: { watch: false, onSessionStart: false, onSearch: false },
query: { minScore: 0 },
cache: { enabled: false },
},
},
list: [{ id: "main", default: true }],
},
};
const result = await getMemorySearchManager({ cfg, agentId: "main" });
expect(result.manager).not.toBeNull();
if (!result.manager) throw new Error("manager missing");
manager = result.manager;
await manager.sync({ force: true });
const before = manager.status();
expect(before.files).toBeGreaterThan(0);
failEmbeddings = true;
await expect(manager.sync({ force: true })).rejects.toThrow(/mock embeddings failed/i);
const after = manager.status();
expect(after.files).toBe(before.files);
expect(after.chunks).toBe(before.chunks);
const files = await fs.readdir(workspaceDir);
expect(files.some((name) => name.includes(".tmp-"))).toBe(false);
});
it("finds keyword matches via hybrid search when query embedding is zero", async () => {
const cfg = {
agents: {
+271 -651
View File
File diff suppressed because it is too large. Load diff
+435 -2
View File
@@ -1,2 +1,435 @@
// Deprecated: use ./batch-openai.js
export * from "./batch-openai.js";
import { retryAsync } from "../infra/retry.js";
import type { OpenAiEmbeddingClient } from "./embeddings.js";
import { hashText } from "./internal.js";
export type OpenAiBatchRequest = {
custom_id: string;
method: "POST";
url: "/v1/embeddings";
body: {
model: string;
input: string;
};
};
export type OpenAiBatchStatus = {
id?: string;
status?: string;
output_file_id?: string | null;
error_file_id?: string | null;
};
export type OpenAiBatchOutputLine = {
custom_id?: string;
response?: {
status_code?: number;
body?: {
data?: Array<{ embedding?: number[]; index?: number }>;
error?: { message?: string };
};
};
error?: { message?: string };
};
export const OPENAI_BATCH_ENDPOINT = "/v1/embeddings";
const OPENAI_BATCH_COMPLETION_WINDOW = "24h";
const OPENAI_BATCH_MAX_REQUESTS = 50000;
const OPENAI_BATCH_RETRY = {
attempts: 3,
minDelayMs: 500,
maxDelayMs: 5000,
jitter: 0.1,
};
type RetryableError = Error & { status?: number };
function isRetryableBatchError(err: unknown): boolean {
const status =
typeof (err as RetryableError)?.status === "number" ? (err as RetryableError).status : undefined;
if (typeof status === "number") {
return status === 429 || status >= 500;
}
const message = err instanceof Error ? err.message : String(err);
return /timeout|timed out|ECONNRESET|ECONNREFUSED|EHOSTUNREACH|ENOTFOUND|EAI_AGAIN|network|fetch failed|upstream connect/i.test(
message,
);
}
function formatRetryError(err: unknown): string {
return err instanceof Error ? err.message : String(err);
}
function getOpenAiBaseUrl(openAi: OpenAiEmbeddingClient): string {
return openAi.baseUrl?.replace(/\/$/, "") ?? "";
}
function getOpenAiHeaders(
openAi: OpenAiEmbeddingClient,
params: { json: boolean },
): Record<string, string> {
const headers = openAi.headers ? { ...openAi.headers } : {};
if (params.json) {
if (!headers["Content-Type"] && !headers["content-type"]) {
headers["Content-Type"] = "application/json";
}
} else {
delete headers["Content-Type"];
delete headers["content-type"];
}
return headers;
}
function splitOpenAiBatchRequests(requests: OpenAiBatchRequest[]): OpenAiBatchRequest[][] {
if (requests.length <= OPENAI_BATCH_MAX_REQUESTS) return [requests];
const groups: OpenAiBatchRequest[][] = [];
for (let i = 0; i < requests.length; i += OPENAI_BATCH_MAX_REQUESTS) {
groups.push(requests.slice(i, i + OPENAI_BATCH_MAX_REQUESTS));
}
return groups;
}
async function fetchOpenAiWithRetry(params: {
openAi: OpenAiEmbeddingClient;
url: string;
init?: RequestInit;
label: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<Response> {
return await retryAsync(
async () => {
const res = await fetch(params.url, params.init);
if (!res.ok) {
const text = await res.text();
const err = new Error(`openai batch ${params.label} failed: ${res.status} ${text}`);
(err as RetryableError).status = res.status;
throw err;
}
return res;
},
{
...OPENAI_BATCH_RETRY,
label: params.label,
shouldRetry: isRetryableBatchError,
onRetry: (info) => {
params.debug?.(
`openai batch ${params.label} retry ${info.attempt}/${info.maxAttempts} in ${info.delayMs}ms`,
{ error: formatRetryError(info.err) },
);
},
},
);
}
async function submitOpenAiBatch(params: {
openAi: OpenAiEmbeddingClient;
requests: OpenAiBatchRequest[];
agentId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const jsonl = params.requests.map((request) => JSON.stringify(request)).join("\n");
const form = new FormData();
form.append("purpose", "batch");
form.append(
"file",
new Blob([jsonl], { type: "application/jsonl" }),
`memory-embeddings.${hashText(String(Date.now()))}.jsonl`,
);
const fileRes = await fetchOpenAiWithRetry({
openAi: params.openAi,
url: `${baseUrl}/files`,
init: {
method: "POST",
headers: getOpenAiHeaders(params.openAi, { json: false }),
body: form,
},
label: "file upload",
debug: params.debug,
});
const filePayload = (await fileRes.json()) as { id?: string };
if (!filePayload.id) {
throw new Error("openai batch file upload failed: missing file id");
}
const batchRes = await fetchOpenAiWithRetry({
openAi: params.openAi,
url: `${baseUrl}/batches`,
init: {
method: "POST",
headers: getOpenAiHeaders(params.openAi, { json: true }),
body: JSON.stringify({
input_file_id: filePayload.id,
endpoint: OPENAI_BATCH_ENDPOINT,
completion_window: OPENAI_BATCH_COMPLETION_WINDOW,
metadata: {
source: "clawdbot-memory",
agent: params.agentId,
},
}),
},
label: "create",
debug: params.debug,
});
return (await batchRes.json()) as OpenAiBatchStatus;
}
async function fetchOpenAiBatchStatus(params: {
openAi: OpenAiEmbeddingClient;
batchId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<OpenAiBatchStatus> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetchOpenAiWithRetry({
openAi: params.openAi,
url: `${baseUrl}/batches/${params.batchId}`,
init: { headers: getOpenAiHeaders(params.openAi, { json: true }) },
label: "status",
debug: params.debug,
});
return (await res.json()) as OpenAiBatchStatus;
}
async function fetchOpenAiFileContent(params: {
openAi: OpenAiEmbeddingClient;
fileId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<string> {
const baseUrl = getOpenAiBaseUrl(params.openAi);
const res = await fetchOpenAiWithRetry({
openAi: params.openAi,
url: `${baseUrl}/files/${params.fileId}/content`,
init: { headers: getOpenAiHeaders(params.openAi, { json: true }) },
label: "file content",
debug: params.debug,
});
return await res.text();
}
function parseOpenAiBatchOutput(text: string): OpenAiBatchOutputLine[] {
if (!text.trim()) return [];
return text
.split("\n")
.map((line) => line.trim())
.filter(Boolean)
.map((line) => JSON.parse(line) as OpenAiBatchOutputLine);
}
async function readOpenAiBatchError(params: {
openAi: OpenAiEmbeddingClient;
errorFileId: string;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<string | undefined> {
try {
const content = await fetchOpenAiFileContent({
openAi: params.openAi,
fileId: params.errorFileId,
debug: params.debug,
});
const lines = parseOpenAiBatchOutput(content);
const first = lines.find((line) => line.error?.message || line.response?.body?.error);
const message =
first?.error?.message ??
(typeof first?.response?.body?.error?.message === "string"
? first?.response?.body?.error?.message
: undefined);
return message;
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
return message ? `error file unavailable: ${message}` : undefined;
}
}
async function waitForOpenAiBatch(params: {
openAi: OpenAiEmbeddingClient;
batchId: string;
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
initial?: OpenAiBatchStatus;
}): Promise<{ outputFileId: string; errorFileId?: string }> {
const start = Date.now();
let current: OpenAiBatchStatus | undefined = params.initial;
while (true) {
const status =
current ??
(await fetchOpenAiBatchStatus({
openAi: params.openAi,
batchId: params.batchId,
debug: params.debug,
}));
const state = status.status ?? "unknown";
if (state === "completed") {
if (!status.output_file_id) {
throw new Error(`openai batch ${params.batchId} completed without output file`);
}
return {
outputFileId: status.output_file_id,
errorFileId: status.error_file_id ?? undefined,
};
}
if (["failed", "expired", "cancelled", "canceled"].includes(state)) {
const detail = status.error_file_id
? await readOpenAiBatchError({
openAi: params.openAi,
errorFileId: status.error_file_id,
debug: params.debug,
})
: undefined;
const suffix = detail ? `: ${detail}` : "";
throw new Error(`openai batch ${params.batchId} ${state}${suffix}`);
}
if (!params.wait) {
throw new Error(`openai batch ${params.batchId} still ${state}; wait disabled`);
}
if (Date.now() - start > params.timeoutMs) {
throw new Error(`openai batch ${params.batchId} timed out after ${params.timeoutMs}ms`);
}
params.debug?.(`openai batch ${params.batchId} ${state}; waiting ${params.pollIntervalMs}ms`);
await new Promise((resolve) => setTimeout(resolve, params.pollIntervalMs));
current = undefined;
}
}
async function runWithConcurrency<T>(tasks: Array<() => Promise<T>>, limit: number): Promise<T[]> {
if (tasks.length === 0) return [];
const resolvedLimit = Math.max(1, Math.min(limit, tasks.length));
const results: T[] = Array.from({ length: tasks.length });
let next = 0;
let firstError: unknown = null;
const workers = Array.from({ length: resolvedLimit }, async () => {
while (true) {
if (firstError) return;
const index = next;
next += 1;
if (index >= tasks.length) return;
try {
results[index] = await tasks[index]();
} catch (err) {
firstError = err;
return;
}
}
});
await Promise.allSettled(workers);
if (firstError) throw firstError;
return results;
}
export async function runOpenAiEmbeddingBatches(params: {
openAi: OpenAiEmbeddingClient;
agentId: string;
requests: OpenAiBatchRequest[];
wait: boolean;
pollIntervalMs: number;
timeoutMs: number;
concurrency: number;
debug?: (message: string, data?: Record<string, unknown>) => void;
}): Promise<Map<string, number[]>> {
if (params.requests.length === 0) return new Map();
const groups = splitOpenAiBatchRequests(params.requests);
const byCustomId = new Map<string, number[]>();
const tasks = groups.map((group, groupIndex) => async () => {
const batchInfo = await submitOpenAiBatch({
openAi: params.openAi,
requests: group,
agentId: params.agentId,
debug: params.debug,
});
if (!batchInfo.id) {
throw new Error("openai batch create failed: missing batch id");
}
params.debug?.("memory embeddings: openai batch created", {
batchId: batchInfo.id,
status: batchInfo.status,
group: groupIndex + 1,
groups: groups.length,
requests: group.length,
});
if (!params.wait && batchInfo.status !== "completed") {
throw new Error(
`openai batch ${batchInfo.id} submitted; enable remote.batch.wait to await completion`,
);
}
const completed =
batchInfo.status === "completed"
? {
outputFileId: batchInfo.output_file_id ?? "",
errorFileId: batchInfo.error_file_id ?? undefined,
}
: await waitForOpenAiBatch({
openAi: params.openAi,
batchId: batchInfo.id,
wait: params.wait,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
debug: params.debug,
initial: batchInfo,
});
if (!completed.outputFileId) {
throw new Error(`openai batch ${batchInfo.id} completed without output file`);
}
const content = await fetchOpenAiFileContent({
openAi: params.openAi,
fileId: completed.outputFileId,
debug: params.debug,
});
const outputLines = parseOpenAiBatchOutput(content);
const errors: string[] = [];
const remaining = new Set(group.map((request) => request.custom_id));
for (const line of outputLines) {
const customId = line.custom_id;
if (!customId) continue;
remaining.delete(customId);
if (line.error?.message) {
errors.push(`${customId}: ${line.error.message}`);
continue;
}
const response = line.response;
const statusCode = response?.status_code ?? 0;
if (statusCode >= 400) {
const message =
response?.body?.error?.message ??
(typeof response?.body === "string" ? response.body : undefined) ??
"unknown error";
errors.push(`${customId}: ${message}`);
continue;
}
const data = response?.body?.data ?? [];
const embedding = data[0]?.embedding ?? [];
if (embedding.length === 0) {
errors.push(`${customId}: empty embedding`);
continue;
}
byCustomId.set(customId, embedding);
}
if (errors.length > 0) {
throw new Error(`openai batch ${batchInfo.id} failed: ${errors.join("; ")}`);
}
if (remaining.size > 0) {
throw new Error(`openai batch ${batchInfo.id} missing ${remaining.size} embedding responses`);
}
});
params.debug?.("memory embeddings: openai batch submit", {
requests: params.requests.length,
groups: groups.length,
wait: params.wait,
concurrency: params.concurrency,
pollIntervalMs: params.pollIntervalMs,
timeoutMs: params.timeoutMs,
});
await runWithConcurrency(tasks, params.concurrency);
return byCustomId;
}
-102
View File
@@ -19,12 +19,6 @@ import {
saveExecApprovals,
type ExecApprovalsFile,
} from "../infra/exec-approvals.js";
import {
requestExecHostViaSocket,
type ExecHostRequest,
type ExecHostResponse,
type ExecHostRunResult,
} from "../infra/exec-host.js";
import { getMachineDisplayName } from "../infra/machine-name.js";
import { VERSION } from "../version.js";
@@ -92,9 +86,6 @@ type ExecEventPayload = {
const OUTPUT_CAP = 200_000;
const OUTPUT_EVENT_TAIL = 20_000;
const execHostEnforced = process.env.CLAWDBOT_NODE_EXEC_HOST?.trim().toLowerCase() === "app";
const execHostFallbackAllowed = process.env.CLAWDBOT_NODE_EXEC_FALLBACK?.trim() === "1";
const blockedEnvKeys = new Set([
"PATH",
"NODE_OPTIONS",
@@ -314,18 +305,6 @@ function buildExecEventPayload(payload: ExecEventPayload): ExecEventPayload {
return { ...payload, output: text };
}
async function runViaMacAppExecHost(params: {
approvals: ReturnType<typeof resolveExecApprovals>;
request: ExecHostRequest;
}): Promise<ExecHostResponse | null> {
const { approvals, request } = params;
return await requestExecHostViaSocket({
socketPath: approvals.socketPath,
token: approvals.token,
request,
});
}
export async function runNodeHost(opts: NodeHostRunOptions): Promise<void> {
const config = await ensureNodeHostConfig();
const nodeId = opts.nodeId?.trim() || config.nodeId;
@@ -576,87 +555,6 @@ async function handleInvoke(
const skillAllow =
autoAllowSkills && resolution?.executableName ? bins.has(resolution.executableName) : false;
const useMacAppExec = process.platform === "darwin" && (execHostEnforced || !execHostFallbackAllowed);
if (useMacAppExec) {
const execRequest: ExecHostRequest = {
command: argv,
rawCommand: rawCommand || null,
cwd: params.cwd ?? null,
env: params.env ?? null,
timeoutMs: params.timeoutMs ?? null,
needsScreenRecording: params.needsScreenRecording ?? null,
agentId: agentId ?? null,
sessionKey: sessionKey ?? null,
};
const response = await runViaMacAppExecHost({ approvals, request: execRequest });
if (!response) {
client.sendEvent(
"exec.denied",
buildExecEventPayload({
sessionKey,
runId,
host: "node",
command: cmdText,
reason: "companion-unavailable",
}),
);
client.sendInvokeResponse({
type: "invoke-res",
id: frame.id,
ok: false,
error: {
code: "UNAVAILABLE",
message: "COMPANION_APP_UNAVAILABLE: macOS app exec host unreachable",
},
});
return;
}
if (!response.ok) {
const reason = response.error.reason ?? "approval-required";
client.sendEvent(
"exec.denied",
buildExecEventPayload({
sessionKey,
runId,
host: "node",
command: cmdText,
reason,
}),
);
client.sendInvokeResponse({
type: "invoke-res",
id: frame.id,
ok: false,
error: { code: "UNAVAILABLE", message: response.error.message },
});
return;
}
const result: ExecHostRunResult = response.payload;
const combined = [result.stdout, result.stderr, result.error].filter(Boolean).join("\n");
client.sendEvent(
"exec.finished",
buildExecEventPayload({
sessionKey,
runId,
host: "node",
command: cmdText,
exitCode: result.exitCode,
timedOut: result.timedOut,
success: result.success,
output: combined,
}),
);
client.sendInvokeResponse({
type: "invoke-res",
id: frame.id,
ok: true,
payloadJSON: JSON.stringify(result),
});
return;
}
if (security === "deny") {
client.sendEvent(
"exec.denied",
-45
View File
@@ -1,45 +0,0 @@
import type { ClawdbotConfig } from "../config/config.js";
export type PluginEnableResult = {
config: ClawdbotConfig;
enabled: boolean;
reason?: string;
};
function ensureAllowlisted(cfg: ClawdbotConfig, pluginId: string): ClawdbotConfig {
const allow = cfg.plugins?.allow;
if (!Array.isArray(allow) || allow.includes(pluginId)) return cfg;
return {
...cfg,
plugins: {
...cfg.plugins,
allow: [...allow, pluginId],
},
};
}
export function enablePluginInConfig(cfg: ClawdbotConfig, pluginId: string): PluginEnableResult {
if (cfg.plugins?.enabled === false) {
return { config: cfg, enabled: false, reason: "plugins disabled" };
}
if (cfg.plugins?.deny?.includes(pluginId)) {
return { config: cfg, enabled: false, reason: "blocked by denylist" };
}
const entries = {
...cfg.plugins?.entries,
[pluginId]: {
...(cfg.plugins?.entries?.[pluginId] as Record<string, unknown> | undefined),
enabled: true,
},
};
let next: ClawdbotConfig = {
...cfg,
plugins: {
...cfg.plugins,
entries,
},
};
next = ensureAllowlisted(next, pluginId);
return { config: next, enabled: true };
}
+2 -3
View File
@@ -12,7 +12,6 @@ import { initializeGlobalHookRunner } from "./hook-runner-global.js";
import { createPluginRegistry, type PluginRecord, type PluginRegistry } from "./registry.js";
import { createPluginRuntime } from "./runtime/index.js";
import { setActivePluginRegistry } from "./runtime.js";
import { defaultSlotIdForKey } from "./slots.js";
import type {
ClawdbotPluginConfigSchema,
ClawdbotPluginDefinition,
@@ -47,7 +46,7 @@ const registryCache = new Map<string, PluginRegistry>();
const defaultLogger = () => createSubsystemLogger("plugins");
const BUNDLED_ENABLED_BY_DEFAULT = new Set<string>();
const BUNDLED_ENABLED_BY_DEFAULT = new Set(["telegram", "whatsapp", "discord", "slack", "signal"]);
const normalizeList = (value: unknown): string[] => {
if (!Array.isArray(value)) return [];
@@ -93,7 +92,7 @@ const normalizePluginsConfig = (config?: ClawdbotConfig["plugins"]): NormalizedP
deny: normalizeList(config?.deny),
loadPaths: normalizeList(config?.load?.paths),
slots: {
memory: memorySlot ?? defaultSlotIdForKey("memory"),
memory: memorySlot ?? "memory-core",
},
entries: normalizePluginEntries(config?.entries),
};
-96
View File
@@ -1,96 +0,0 @@
import { describe, expect, it } from "vitest";
import type { ClawdbotConfig } from "../config/config.js";
import { applyExclusiveSlotSelection } from "./slots.js";
describe("applyExclusiveSlotSelection", () => {
it("selects the slot and disables other entries for the same kind", () => {
const config: ClawdbotConfig = {
plugins: {
slots: { memory: "memory-core" },
entries: {
"memory-core": { enabled: true },
memory: { enabled: true },
},
},
};
const result = applyExclusiveSlotSelection({
config,
selectedId: "memory",
selectedKind: "memory",
registry: {
plugins: [
{ id: "memory-core", kind: "memory" },
{ id: "memory", kind: "memory" },
],
},
});
expect(result.changed).toBe(true);
expect(result.config.plugins?.slots?.memory).toBe("memory");
expect(result.config.plugins?.entries?.["memory-core"]?.enabled).toBe(false);
expect(result.warnings).toContain(
'Exclusive slot "memory" switched from "memory-core" to "memory".',
);
expect(result.warnings).toContain(
'Disabled other "memory" slot plugins: memory-core.',
);
});
it("does nothing when the slot already matches", () => {
const config: ClawdbotConfig = {
plugins: {
slots: { memory: "memory" },
entries: {
memory: { enabled: true },
},
},
};
const result = applyExclusiveSlotSelection({
config,
selectedId: "memory",
selectedKind: "memory",
registry: { plugins: [{ id: "memory", kind: "memory" }] },
});
expect(result.changed).toBe(false);
expect(result.warnings).toHaveLength(0);
expect(result.config).toBe(config);
});
it("warns when the slot falls back to a default", () => {
const config: ClawdbotConfig = {
plugins: {
entries: {
memory: { enabled: true },
},
},
};
const result = applyExclusiveSlotSelection({
config,
selectedId: "memory",
selectedKind: "memory",
registry: { plugins: [{ id: "memory", kind: "memory" }] },
});
expect(result.changed).toBe(true);
expect(result.warnings).toContain(
'Exclusive slot "memory" switched from "memory-core" to "memory".',
);
});
it("skips changes when no exclusive slot applies", () => {
const config: ClawdbotConfig = {};
const result = applyExclusiveSlotSelection({
config,
selectedId: "custom",
});
expect(result.changed).toBe(false);
expect(result.warnings).toHaveLength(0);
expect(result.config).toBe(config);
});
});
-102
View File
@@ -1,102 +0,0 @@
import type { ClawdbotConfig } from "../config/config.js";
import type { PluginSlotsConfig } from "../config/types.plugins.js";
import type { PluginKind } from "./types.js";
export type PluginSlotKey = keyof PluginSlotsConfig;
type SlotPluginRecord = {
id: string;
kind?: PluginKind;
};
const SLOT_BY_KIND: Record<PluginKind, PluginSlotKey> = {
memory: "memory",
};
const DEFAULT_SLOT_BY_KEY: Record<PluginSlotKey, string> = {
memory: "memory-core",
};
export function slotKeyForPluginKind(kind?: PluginKind): PluginSlotKey | null {
if (!kind) return null;
return SLOT_BY_KIND[kind] ?? null;
}
export function defaultSlotIdForKey(slotKey: PluginSlotKey): string {
return DEFAULT_SLOT_BY_KEY[slotKey];
}
export type SlotSelectionResult = {
config: ClawdbotConfig;
warnings: string[];
changed: boolean;
};
export function applyExclusiveSlotSelection(params: {
config: ClawdbotConfig;
selectedId: string;
selectedKind?: PluginKind;
registry?: { plugins: SlotPluginRecord[] };
}): SlotSelectionResult {
const slotKey = slotKeyForPluginKind(params.selectedKind);
if (!slotKey) {
return { config: params.config, warnings: [], changed: false };
}
const warnings: string[] = [];
const pluginsConfig = params.config.plugins ?? {};
const prevSlot = pluginsConfig.slots?.[slotKey];
const slots = {
...pluginsConfig.slots,
[slotKey]: params.selectedId,
};
const inferredPrevSlot = prevSlot ?? defaultSlotIdForKey(slotKey);
if (inferredPrevSlot && inferredPrevSlot !== params.selectedId) {
warnings.push(
`Exclusive slot "${slotKey}" switched from "${inferredPrevSlot}" to "${params.selectedId}".`,
);
}
const entries = { ...pluginsConfig.entries };
const disabledIds: string[] = [];
if (params.registry) {
for (const plugin of params.registry.plugins) {
if (plugin.id === params.selectedId) continue;
if (plugin.kind !== params.selectedKind) continue;
const entry = entries[plugin.id];
if (!entry || entry.enabled !== false) {
entries[plugin.id] = {
...entry,
enabled: false,
};
disabledIds.push(plugin.id);
}
}
}
if (disabledIds.length > 0) {
warnings.push(
`Disabled other "${slotKey}" slot plugins: ${disabledIds.sort().join(", ")}.`,
);
}
const changed = prevSlot !== params.selectedId || disabledIds.length > 0;
if (!changed) {
return { config: params.config, warnings: [], changed: false };
}
return {
config: {
...params.config,
plugins: {
...pluginsConfig,
slots,
entries,
},
},
warnings,
changed: true,
};
}
+1 -9
View File
@@ -1,4 +1,4 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { describe, expect, it, vi } from "vitest";
const loadJsonFile = vi.fn();
const saveJsonFile = vi.fn();
@@ -14,14 +14,6 @@ vi.mock("../config/paths.js", () => ({
}));
describe("github-copilot token", () => {
beforeEach(() => {
vi.resetModules();
loadJsonFile.mockReset();
saveJsonFile.mockReset();
resolveStateDir.mockReset();
resolveStateDir.mockReturnValue("/tmp/clawdbot-state");
});
it("derives baseUrl from token", async () => {
const { deriveCopilotApiBaseUrlFromToken } = await import("./github-copilot-token.js");
+2 -1
View File
@@ -26,7 +26,8 @@ import { normalizeAllowList } from "./allow-list.js";
import type { MonitorSlackOpts } from "./types.js";
const { App, HTTPReceiver } = SlackBolt as typeof import("@slack/bolt");
const { App, HTTPReceiver } = SlackBolt;
function parseApiAppIdFromAppToken(raw?: string) {
const token = raw?.trim();
if (!token) return undefined;
-8
View File
@@ -8,14 +8,6 @@ export async function ensureDir(dir: string) {
await fs.promises.mkdir(dir, { recursive: true });
}
export function clampNumber(value: number, min: number, max: number): number {
return Math.max(min, Math.min(max, value));
}
export function clampInt(value: number, min: number, max: number): number {
return clampNumber(Math.floor(value), min, max);
}
export type WebChannel = "web";
export function assertWebChannel(input: string): asserts input is WebChannel {