Compare commits

..
Author SHA1 Message Date
Peter Steinberger 0cbc051019 fix: prevent voice-call TTS overlap (#1713) (thanks @dguido) 2026-01-25 11:55:37 +00:00
Dan Guido 1005934964 fix(voice-call): use iterative queue processing to prevent heap exhaustion
The recursive processQueue() pattern accumulated stack frames, causing
JavaScript heap out of memory errors on macOS CI. Convert to while loop
for constant stack usage regardless of queue depth.
2026-01-25 11:55:37 +00:00
Dan GuidoandClaude Opus 4.5 76014685eb fix(voice-call): prevent audio overlap with TTS queue
Add a TTS queue to serialize audio playback and prevent overlapping
speech during voice calls. Previously, concurrent speak() calls could
send audio chunks simultaneously, causing garbled/choppy output.

Changes:
- Add queueTts() to MediaStreamHandler for sequential TTS playback
- Wrap playTtsViaStream() audio sending in the queue
- Clear queue on barge-in (when user starts speaking)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 11:55:37 +00:00
Peter Steinberger 875b018ea1 fix: stop sending tool summaries to channels 2026-01-25 11:54:29 +00:00
Nimrod Gutman b6581e77f6 refactor(gateway): share request encoding 2026-01-25 11:48:22 +00:00
Nimrod Gutman 81e915110e fix(node): avoid invoke result deadlock 2026-01-25 11:48:22 +00:00
Peter SteinbergerandRadek Paclt 7e9aa3c275 fix(telegram): honor outbound proxy config (#1774, thanks @radek-paclt)
Co-authored-by: Radek Paclt <developer@muj-partak.cz>
2026-01-25 11:41:54 +00:00
DeveloperandClaude Opus 4.5 65e2d939e1 fix(telegram): use configured proxy for outbound API calls
The proxy configuration (`channels.telegram.proxy`) was only used for
the gateway monitor (polling), but not for outbound sends (sendMessage,
reactMessage, deleteMessage). This caused outbound messages to bypass
the configured proxy, which is problematic for users behind corporate
proxies or those who want to route all traffic through a specific proxy.

This change ensures that all three outbound functions use the same
proxy configuration as the monitor:
- sendMessageTelegram
- reactMessageTelegram
- deleteMessageTelegram

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 11:41:54 +00:00
24 changed files with 533 additions and 350 deletions

No files matched your search

+2 -1
View File
@@ -26,7 +26,6 @@ Docs: https://docs.clawd.bot
- Diagnostics: add diagnostic flags for targeted debug logs (config + env override). https://docs.clawd.bot/diagnostics/flags - Diagnostics: add diagnostic flags for targeted debug logs (config + env override). https://docs.clawd.bot/diagnostics/flags
### Fixes ### Fixes
- macOS: rearm gateway receive loop before push handling to avoid node invoke stalls. (#1752) Thanks @ngutman.
- Gateway: include inline config env vars in service install environments. (#1735) Thanks @Seredeep. - Gateway: include inline config env vars in service install environments. (#1735) Thanks @Seredeep.
- BlueBubbles: route phone-number targets to DMs, avoid leaking routing IDs, and auto-create missing DMs (Private API required). (#1751) Thanks @tyler6204. https://docs.clawd.bot/channels/bluebubbles - BlueBubbles: route phone-number targets to DMs, avoid leaking routing IDs, and auto-create missing DMs (Private API required). (#1751) Thanks @tyler6204. https://docs.clawd.bot/channels/bluebubbles
- BlueBubbles: keep part-index GUIDs in reply tags when short IDs are missing. - BlueBubbles: keep part-index GUIDs in reply tags when short IDs are missing.
@@ -37,6 +36,7 @@ Docs: https://docs.clawd.bot
- TUI: reload history after gateway reconnect to restore session state. (#1663) - TUI: reload history after gateway reconnect to restore session state. (#1663)
- Telegram: use wrapped fetch for long-polling on Node to normalize AbortSignal handling. (#1639) - Telegram: use wrapped fetch for long-polling on Node to normalize AbortSignal handling. (#1639)
- Telegram: set fetch duplex="half" for uploads on Node 22 to avoid sendPhoto failures. (#1684) Thanks @commdata2338. - Telegram: set fetch duplex="half" for uploads on Node 22 to avoid sendPhoto failures. (#1684) Thanks @commdata2338.
- Telegram: honor per-account proxy for outbound API calls. (#1774) Thanks @radek-paclt.
- Signal: repair reaction sends (group/UUID targets + CLI author flags). (#1651) Thanks @vilkasdev. - Signal: repair reaction sends (group/UUID targets + CLI author flags). (#1651) Thanks @vilkasdev.
- Signal: add configurable signal-cli startup timeout + external daemon mode docs. (#1677) https://docs.clawd.bot/channels/signal - Signal: add configurable signal-cli startup timeout + external daemon mode docs. (#1677) https://docs.clawd.bot/channels/signal
- Exec: keep approvals for elevated ask unless full mode. (#1616) Thanks @ivancasco. - Exec: keep approvals for elevated ask unless full mode. (#1616) Thanks @ivancasco.
@@ -51,6 +51,7 @@ Docs: https://docs.clawd.bot
- Gateway: store lock files in the temp directory to avoid stale locks on persistent volumes. (#1676) - Gateway: store lock files in the temp directory to avoid stale locks on persistent volumes. (#1676)
- macOS: default direct-transport `ws://` URLs to port 18789; document `gateway.remote.transport`. (#1603) Thanks @ngutman. - macOS: default direct-transport `ws://` URLs to port 18789; document `gateway.remote.transport`. (#1603) Thanks @ngutman.
- Voice Call: return stream TwiML for outbound conversation calls on initial Twilio webhook. (#1634) - Voice Call: return stream TwiML for outbound conversation calls on initial Twilio webhook. (#1634)
- Voice Call: serialize Twilio TTS playback and cancel on barge-in to prevent overlap. (#1713) Thanks @dguido.
- Google Chat: tighten email allowlist matching, typing cleanup, media caps, and onboarding/docs/tests. (#1635) Thanks @iHildy. - Google Chat: tighten email allowlist matching, typing cleanup, media caps, and onboarding/docs/tests. (#1635) Thanks @iHildy.
- Google Chat: normalize space targets without double `spaces/` prefix. - Google Chat: normalize space targets without double `spaces/` prefix.
- Messaging: keep newline chunking safe for fenced markdown blocks across channels. - Messaging: keep newline chunking safe for fenced markdown blocks across channels.
@@ -427,8 +427,8 @@ public actor GatewayChannelActor {
Task { await self.handleReceiveFailure(err) } Task { await self.handleReceiveFailure(err) }
case let .success(msg): case let .success(msg):
Task { Task {
await self.listen()
await self.handle(msg) await self.handle(msg)
await self.listen()
} }
} }
} }
@@ -619,6 +619,29 @@ public actor GatewayChannelActor {
return Data() // Should not happen, but tolerate empty payloads. return Data() // Should not happen, but tolerate empty payloads.
} }
public func send(method: String, params: [String: AnyCodable]?) async throws {
try await self.connectOrThrow(context: "gateway connect")
let payload = try self.encodeRequest(method: method, params: params, kind: "send")
guard let task = self.task else {
throw NSError(
domain: "Gateway",
code: 5,
userInfo: [NSLocalizedDescriptionKey: "gateway socket unavailable"])
}
do {
try await task.send(.data(payload.data))
} catch {
let wrapped = self.wrap(error, context: "gateway send \(method)")
self.connected = false
self.task?.cancel(with: .goingAway, reason: nil)
Task { [weak self] in
guard let self else { return }
await self.scheduleReconnect()
}
throw wrapped
}
}
// Wrap low-level URLSession/WebSocket errors with context so UI can surface them. // Wrap low-level URLSession/WebSocket errors with context so UI can surface them.
private func wrap(_ error: Error, context: String) -> Error { private func wrap(_ error: Error, context: String) -> Error {
if let urlError = error as? URLError { if let urlError = error as? URLError {
@@ -143,7 +143,7 @@ public actor GatewayNodeSession {
"payloadJSON": AnyCodable(payloadJSON ?? NSNull()), "payloadJSON": AnyCodable(payloadJSON ?? NSNull()),
] ]
do { do {
_ = try await channel.request(method: "node.event", params: params, timeoutMs: 8000) try await channel.send(method: "node.event", params: params)
} catch { } catch {
self.logger.error("node event failed: \(error.localizedDescription, privacy: .public)") self.logger.error("node event failed: \(error.localizedDescription, privacy: .public)")
} }
@@ -224,7 +224,7 @@ public actor GatewayNodeSession {
]) ])
} }
do { do {
_ = try await channel.request(method: "node.invoke.result", params: params, timeoutMs: 15000) try await channel.send(method: "node.invoke.result", params: params)
} catch { } catch {
self.logger.error("node invoke result failed: \(error.localizedDescription, privacy: .public)") self.logger.error("node invoke result failed: \(error.localizedDescription, privacy: .public)")
} }
@@ -1,212 +0,0 @@
import Foundation
import Testing
@testable import ClawdbotKit
import ClawdbotProtocol
private final class FakeWebSocketTask: WebSocketTasking, @unchecked Sendable {
private let lock = NSLock()
private var queue: [URLSessionWebSocketTask.Message] = []
private var pendingHandler: (@Sendable (Result<URLSessionWebSocketTask.Message, Error>) -> Void)?
private var pendingContinuation: CheckedContinuation<URLSessionWebSocketTask.Message, Error>?
private let encoder = JSONEncoder()
private let decoder = JSONDecoder()
var state: URLSessionTask.State = .running
func resume() {}
func cancel(with closeCode: URLSessionWebSocketTask.CloseCode, reason: Data?) {
state = .canceling
}
func send(_ message: URLSessionWebSocketTask.Message) async throws {
guard case let .data(data) = message else { return }
guard let frame = try? decoder.decode(RequestFrame.self, from: data) else { return }
switch frame.method {
case "connect":
enqueueResponse(id: frame.id, payload: helloOkPayload())
default:
enqueueResponse(id: frame.id, payload: ["ok": true])
}
}
func receive() async throws -> URLSessionWebSocketTask.Message {
try await withCheckedThrowingContinuation { cont in
lock.lock()
if !queue.isEmpty {
let msg = queue.removeFirst()
lock.unlock()
cont.resume(returning: msg)
return
}
pendingContinuation = cont
lock.unlock()
}
}
func receive(
completionHandler: @escaping @Sendable (Result<URLSessionWebSocketTask.Message, Error>) -> Void)
{
lock.lock()
if !queue.isEmpty {
let msg = queue.removeFirst()
lock.unlock()
completionHandler(.success(msg))
return
}
pendingHandler = completionHandler
lock.unlock()
}
func enqueue(_ message: URLSessionWebSocketTask.Message) {
lock.lock()
if let handler = pendingHandler {
pendingHandler = nil
lock.unlock()
handler(.success(message))
return
}
if let continuation = pendingContinuation {
pendingContinuation = nil
lock.unlock()
continuation.resume(returning: message)
return
}
queue.append(message)
lock.unlock()
}
private func enqueueResponse(id: String, payload: [String: Any]) {
let response = ResponseFrame(
type: "res",
id: id,
ok: true,
payload: ClawdbotProtocol.AnyCodable(payload),
error: nil)
guard let data = try? encoder.encode(response) else { return }
enqueue(.data(data))
}
private func helloOkPayload() -> [String: Any] {
[
"type": "hello.ok",
"protocol": 1,
"server": [:],
"features": [:],
"snapshot": [
"presence": [],
"health": [:],
"stateVersion": [
"presence": 0,
"health": 0,
],
"uptimeMs": 0,
],
"policy": [
"tickIntervalMs": 1000,
],
]
}
}
private final class FakeWebSocketSession: WebSocketSessioning {
let task: FakeWebSocketTask
init(task: FakeWebSocketTask) {
self.task = task
}
func makeWebSocketTask(url: URL) -> WebSocketTaskBox {
WebSocketTaskBox(task: task)
}
}
private actor AsyncSignal {
private var continuation: CheckedContinuation<Result<Void, Error>, Never>?
private var stored: Result<Void, Error>?
func finish(_ result: Result<Void, Error>) {
if let continuation {
self.continuation = nil
continuation.resume(returning: result)
return
}
stored = result
}
func wait() async throws {
let result = await withCheckedContinuation { cont in
if let stored {
self.stored = nil
cont.resume(returning: stored)
return
}
continuation = cont
}
switch result {
case .success:
return
case let .failure(error):
throw error
}
}
}
private enum TestError: Error {
case timeout
}
struct GatewayChannelTests {
@Test
func listenRearmsBeforePushHandler() async throws {
let task = FakeWebSocketTask()
let session = FakeWebSocketSession(task: task)
let signal = AsyncSignal()
let url = URL(string: "ws://example.invalid")!
final class ChannelBox { var channel: GatewayChannelActor? }
let box = ChannelBox()
let channel = GatewayChannelActor(
url: url,
token: nil,
session: WebSocketSessionBox(session: session),
pushHandler: { push in
guard case let .event(evt) = push, evt.event == "test.event" else { return }
guard let channel = box.channel else { return }
let params: [String: ClawdbotKit.AnyCodable] = [
"event": ClawdbotKit.AnyCodable("test"),
"payloadJSON": ClawdbotKit.AnyCodable(NSNull()),
]
do {
_ = try await channel.request(method: "node.event", params: params, timeoutMs: 50)
await signal.finish(.success(()))
} catch {
await signal.finish(.failure(error))
}
})
box.channel = channel
let challenge = EventFrame(
type: "event",
event: "connect.challenge",
payload: ClawdbotProtocol.AnyCodable(["nonce": "test-nonce"]),
seq: nil,
stateversion: nil)
let encoder = JSONEncoder()
task.enqueue(.data(try encoder.encode(challenge)))
try await channel.connect()
let event = EventFrame(
type: "event",
event: "test.event",
payload: ClawdbotProtocol.AnyCodable([:]),
seq: nil,
stateversion: nil)
task.enqueue(.data(try encoder.encode(event)))
try await AsyncTimeout.withTimeout(seconds: 1, onTimeout: { TestError.timeout }) {
try await signal.wait()
}
}
}
@@ -0,0 +1,97 @@
import { describe, expect, it } from "vitest";
import type {
OpenAIRealtimeSTTProvider,
RealtimeSTTSession,
} from "./providers/stt-openai-realtime.js";
import { MediaStreamHandler } from "./media-stream.js";
const createStubSession = (): RealtimeSTTSession => ({
connect: async () => {},
sendAudio: () => {},
waitForTranscript: async () => "",
onPartial: () => {},
onTranscript: () => {},
onSpeechStart: () => {},
close: () => {},
isConnected: () => true,
});
const createStubSttProvider = (): OpenAIRealtimeSTTProvider =>
({
createSession: () => createStubSession(),
}) as unknown as OpenAIRealtimeSTTProvider;
const flush = async (): Promise<void> => {
await new Promise((resolve) => setTimeout(resolve, 0));
};
const waitForAbort = (signal: AbortSignal): Promise<void> =>
new Promise((resolve) => {
if (signal.aborted) {
resolve();
return;
}
signal.addEventListener("abort", () => resolve(), { once: true });
});
describe("MediaStreamHandler TTS queue", () => {
it("serializes TTS playback and resolves in order", async () => {
const handler = new MediaStreamHandler({
sttProvider: createStubSttProvider(),
});
const started: number[] = [];
const finished: number[] = [];
let resolveFirst!: () => void;
const firstGate = new Promise<void>((resolve) => {
resolveFirst = resolve;
});
const first = handler.queueTts("stream-1", async () => {
started.push(1);
await firstGate;
finished.push(1);
});
const second = handler.queueTts("stream-1", async () => {
started.push(2);
finished.push(2);
});
await flush();
expect(started).toEqual([1]);
resolveFirst();
await first;
await second;
expect(started).toEqual([1, 2]);
expect(finished).toEqual([1, 2]);
});
it("cancels active playback and clears queued items", async () => {
const handler = new MediaStreamHandler({
sttProvider: createStubSttProvider(),
});
let queuedRan = false;
const started: string[] = [];
const active = handler.queueTts("stream-1", async (signal) => {
started.push("active");
await waitForAbort(signal);
});
void handler.queueTts("stream-1", async () => {
queuedRan = true;
});
await flush();
expect(started).toEqual(["active"]);
handler.clearTtsQueue("stream-1");
await active;
await flush();
expect(queuedRan).toBe(false);
});
});
+114
View File
@@ -29,6 +29,8 @@ export interface MediaStreamConfig {
onPartialTranscript?: (callId: string, partial: string) => void; onPartialTranscript?: (callId: string, partial: string) => void;
/** Callback when stream connects */ /** Callback when stream connects */
onConnect?: (callId: string, streamSid: string) => void; onConnect?: (callId: string, streamSid: string) => void;
/** Callback when speech starts (barge-in) */
onSpeechStart?: (callId: string) => void;
/** Callback when stream disconnects */ /** Callback when stream disconnects */
onDisconnect?: (callId: string) => void; onDisconnect?: (callId: string) => void;
} }
@@ -43,6 +45,13 @@ interface StreamSession {
sttSession: RealtimeSTTSession; sttSession: RealtimeSTTSession;
} }
type TtsQueueEntry = {
playFn: (signal: AbortSignal) => Promise<void>;
controller: AbortController;
resolve: () => void;
reject: (error: unknown) => void;
};
/** /**
* Manages WebSocket connections for Twilio media streams. * Manages WebSocket connections for Twilio media streams.
*/ */
@@ -50,6 +59,12 @@ export class MediaStreamHandler {
private wss: WebSocketServer | null = null; private wss: WebSocketServer | null = null;
private sessions = new Map<string, StreamSession>(); private sessions = new Map<string, StreamSession>();
private config: MediaStreamConfig; private config: MediaStreamConfig;
/** TTS playback queues per stream (serialize audio to prevent overlap) */
private ttsQueues = new Map<string, TtsQueueEntry[]>();
/** Whether TTS is currently playing per stream */
private ttsPlaying = new Map<string, boolean>();
/** Active TTS playback controllers per stream */
private ttsActiveControllers = new Map<string, AbortController>();
constructor(config: MediaStreamConfig) { constructor(config: MediaStreamConfig) {
this.config = config; this.config = config;
@@ -148,6 +163,10 @@ export class MediaStreamHandler {
this.config.onTranscript?.(callSid, transcript); this.config.onTranscript?.(callSid, transcript);
}); });
sttSession.onSpeechStart(() => {
this.config.onSpeechStart?.(callSid);
});
const session: StreamSession = { const session: StreamSession = {
callId: callSid, callId: callSid,
streamSid, streamSid,
@@ -177,6 +196,7 @@ export class MediaStreamHandler {
private handleStop(session: StreamSession): void { private handleStop(session: StreamSession): void {
console.log(`[MediaStream] Stream stopped: ${session.streamSid}`); console.log(`[MediaStream] Stream stopped: ${session.streamSid}`);
this.clearTtsState(session.streamSid);
session.sttSession.close(); session.sttSession.close();
this.sessions.delete(session.streamSid); this.sessions.delete(session.streamSid);
this.config.onDisconnect?.(session.callId); this.config.onDisconnect?.(session.callId);
@@ -228,6 +248,46 @@ export class MediaStreamHandler {
this.sendToStream(streamSid, { event: "clear", streamSid }); this.sendToStream(streamSid, { event: "clear", streamSid });
} }
/**
* Queue a TTS operation for sequential playback.
* Only one TTS operation plays at a time per stream to prevent overlap.
*/
async queueTts(
streamSid: string,
playFn: (signal: AbortSignal) => Promise<void>,
): Promise<void> {
const queue = this.getTtsQueue(streamSid);
let resolveEntry: () => void;
let rejectEntry: (error: unknown) => void;
const promise = new Promise<void>((resolve, reject) => {
resolveEntry = resolve;
rejectEntry = reject;
});
queue.push({
playFn,
controller: new AbortController(),
resolve: resolveEntry!,
reject: rejectEntry!,
});
if (!this.ttsPlaying.get(streamSid)) {
void this.processQueue(streamSid);
}
return promise;
}
/**
* Clear TTS queue and interrupt current playback (barge-in).
*/
clearTtsQueue(streamSid: string): void {
const queue = this.getTtsQueue(streamSid);
queue.length = 0;
this.ttsActiveControllers.get(streamSid)?.abort();
this.clearAudio(streamSid);
}
/** /**
* Get active session by call ID. * Get active session by call ID.
*/ */
@@ -242,11 +302,65 @@ export class MediaStreamHandler {
*/ */
closeAll(): void { closeAll(): void {
for (const session of this.sessions.values()) { for (const session of this.sessions.values()) {
this.clearTtsState(session.streamSid);
session.sttSession.close(); session.sttSession.close();
session.ws.close(); session.ws.close();
} }
this.sessions.clear(); this.sessions.clear();
} }
private getTtsQueue(streamSid: string): TtsQueueEntry[] {
const existing = this.ttsQueues.get(streamSid);
if (existing) return existing;
const queue: TtsQueueEntry[] = [];
this.ttsQueues.set(streamSid, queue);
return queue;
}
/**
* Process the TTS queue for a stream.
* Uses iterative approach to avoid stack accumulation from recursion.
*/
private async processQueue(streamSid: string): Promise<void> {
this.ttsPlaying.set(streamSid, true);
while (true) {
const queue = this.ttsQueues.get(streamSid);
if (!queue || queue.length === 0) {
this.ttsPlaying.set(streamSid, false);
this.ttsActiveControllers.delete(streamSid);
return;
}
const entry = queue.shift()!;
this.ttsActiveControllers.set(streamSid, entry.controller);
try {
await entry.playFn(entry.controller.signal);
entry.resolve();
} catch (error) {
if (entry.controller.signal.aborted) {
entry.resolve();
} else {
console.error("[MediaStream] TTS playback error:", error);
entry.reject(error);
}
} finally {
if (this.ttsActiveControllers.get(streamSid) === entry.controller) {
this.ttsActiveControllers.delete(streamSid);
}
}
}
}
private clearTtsState(streamSid: string): void {
const queue = this.ttsQueues.get(streamSid);
if (queue) queue.length = 0;
this.ttsActiveControllers.get(streamSid)?.abort();
this.ttsActiveControllers.delete(streamSid);
this.ttsPlaying.delete(streamSid);
this.ttsQueues.delete(streamSid);
}
} }
/** /**
@@ -38,6 +38,8 @@ export interface RealtimeSTTSession {
onPartial(callback: (partial: string) => void): void; onPartial(callback: (partial: string) => void): void;
/** Set callback for final transcripts */ /** Set callback for final transcripts */
onTranscript(callback: (transcript: string) => void): void; onTranscript(callback: (transcript: string) => void): void;
/** Set callback when speech starts (VAD) */
onSpeechStart(callback: () => void): void;
/** Close the session */ /** Close the session */
close(): void; close(): void;
/** Check if session is connected */ /** Check if session is connected */
@@ -91,6 +93,7 @@ class OpenAIRealtimeSTTSession implements RealtimeSTTSession {
private pendingTranscript = ""; private pendingTranscript = "";
private onTranscriptCallback: ((transcript: string) => void) | null = null; private onTranscriptCallback: ((transcript: string) => void) | null = null;
private onPartialCallback: ((partial: string) => void) | null = null; private onPartialCallback: ((partial: string) => void) | null = null;
private onSpeechStartCallback: (() => void) | null = null;
constructor( constructor(
private readonly apiKey: string, private readonly apiKey: string,
@@ -243,6 +246,7 @@ class OpenAIRealtimeSTTSession implements RealtimeSTTSession {
case "input_audio_buffer.speech_started": case "input_audio_buffer.speech_started":
console.log("[RealtimeSTT] Speech started"); console.log("[RealtimeSTT] Speech started");
this.pendingTranscript = ""; this.pendingTranscript = "";
this.onSpeechStartCallback?.();
break; break;
case "error": case "error":
@@ -273,6 +277,10 @@ class OpenAIRealtimeSTTSession implements RealtimeSTTSession {
this.onTranscriptCallback = callback; this.onTranscriptCallback = callback;
} }
onSpeechStart(callback: () => void): void {
this.onSpeechStartCallback = callback;
}
async waitForTranscript(timeoutMs = 30000): Promise<string> { async waitForTranscript(timeoutMs = 30000): Promise<string> {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const timeout = setTimeout(() => { const timeout = setTimeout(() => {
+29 -11
View File
@@ -135,6 +135,17 @@ export class TwilioProvider implements VoiceCallProvider {
this.callStreamMap.delete(callSid); this.callStreamMap.delete(callSid);
} }
/**
* Clear TTS queue for a call (barge-in).
* Used when user starts speaking to interrupt current TTS playback.
*/
clearTtsQueue(callSid: string): void {
const streamSid = this.callStreamMap.get(callSid);
if (streamSid && this.mediaStreamHandler) {
this.mediaStreamHandler.clearTtsQueue(streamSid);
}
}
/** /**
* Make an authenticated request to the Twilio API. * Make an authenticated request to the Twilio API.
*/ */
@@ -504,7 +515,7 @@ export class TwilioProvider implements VoiceCallProvider {
/** /**
* Play TTS via core TTS and Twilio Media Streams. * Play TTS via core TTS and Twilio Media Streams.
* Generates audio with core TTS, converts to mu-law, and streams via WebSocket. * Generates audio with core TTS, converts to mu-law, and streams via WebSocket.
* Uses a jitter buffer to smooth out timing variations. * Uses a queue to serialize playback and prevent overlapping audio.
*/ */
private async playTtsViaStream( private async playTtsViaStream(
text: string, text: string,
@@ -514,22 +525,29 @@ export class TwilioProvider implements VoiceCallProvider {
throw new Error("TTS provider and media stream handler required"); throw new Error("TTS provider and media stream handler required");
} }
// Generate audio with core TTS (returns mu-law at 8kHz)
const muLawAudio = await this.ttsProvider.synthesizeForTelephony(text);
// Stream audio in 20ms chunks (160 bytes at 8kHz mu-law) // Stream audio in 20ms chunks (160 bytes at 8kHz mu-law)
const CHUNK_SIZE = 160; const CHUNK_SIZE = 160;
const CHUNK_DELAY_MS = 20; const CHUNK_DELAY_MS = 20;
for (const chunk of chunkAudio(muLawAudio, CHUNK_SIZE)) { const handler = this.mediaStreamHandler;
this.mediaStreamHandler.sendAudio(streamSid, chunk); const ttsProvider = this.ttsProvider;
await handler.queueTts(streamSid, async (signal) => {
// Generate audio with core TTS (returns mu-law at 8kHz)
const muLawAudio = await ttsProvider.synthesizeForTelephony(text);
for (const chunk of chunkAudio(muLawAudio, CHUNK_SIZE)) {
if (signal.aborted) break;
handler.sendAudio(streamSid, chunk);
// Pace the audio to match real-time playback // Pace the audio to match real-time playback
await new Promise((resolve) => setTimeout(resolve, CHUNK_DELAY_MS)); await new Promise((resolve) => setTimeout(resolve, CHUNK_DELAY_MS));
} if (signal.aborted) break;
}
// Send a mark to track when audio finishes if (!signal.aborted) {
this.mediaStreamHandler.sendMark(streamSid, `tts-${Date.now()}`); // Send a mark to track when audio finishes
handler.sendMark(streamSid, `tts-${Date.now()}`);
}
});
} }
/** /**
+10
View File
@@ -78,6 +78,11 @@ export class VoiceCallWebhookServer {
`[voice-call] Transcript for ${providerCallId}: ${transcript}`, `[voice-call] Transcript for ${providerCallId}: ${transcript}`,
); );
// Clear TTS queue on barge-in (user started speaking, interrupt current playback)
if (this.provider.name === "twilio") {
(this.provider as TwilioProvider).clearTtsQueue(providerCallId);
}
// Look up our internal call ID from the provider call ID // Look up our internal call ID from the provider call ID
const call = this.manager.getCallByProviderCallId(providerCallId); const call = this.manager.getCallByProviderCallId(providerCallId);
if (!call) { if (!call) {
@@ -109,6 +114,11 @@ export class VoiceCallWebhookServer {
}); });
} }
}, },
onSpeechStart: (providerCallId) => {
if (this.provider.name === "twilio") {
(this.provider as TwilioProvider).clearTtsQueue(providerCallId);
}
},
onPartialTranscript: (callId, partial) => { onPartialTranscript: (callId, partial) => {
console.log(`[voice-call] Partial for ${callId}: ${partial}`); console.log(`[voice-call] Partial for ${callId}: ${partial}`);
}, },
+1 -1
View File
@@ -599,7 +599,7 @@ export async function runEmbeddedPiAgent(
verboseLevel: params.verboseLevel, verboseLevel: params.verboseLevel,
reasoningLevel: params.reasoningLevel, reasoningLevel: params.reasoningLevel,
toolResultFormat: resolvedToolResultFormat, toolResultFormat: resolvedToolResultFormat,
inlineToolResultsAllowed: !params.onPartialReply && !params.onToolResult, inlineToolResultsAllowed: false,
}); });
log.debug( log.debug(
@@ -138,6 +138,38 @@ describe("dispatchReplyFromConfig", () => {
); );
}); });
it("does not provide onToolResult when routing cross-provider", async () => {
mocks.tryFastAbortFromMessage.mockResolvedValue({
handled: false,
aborted: false,
});
mocks.routeReply.mockClear();
const cfg = {} as ClawdbotConfig;
const dispatcher = createDispatcher();
const ctx = buildTestCtx({
Provider: "slack",
OriginatingChannel: "telegram",
OriginatingTo: "telegram:999",
});
const replyResolver = async (
_ctx: MsgContext,
opts: GetReplyOptions | undefined,
_cfg: ClawdbotConfig,
) => {
expect(opts?.onToolResult).toBeUndefined();
return { text: "hi" } satisfies ReplyPayload;
};
await dispatchReplyFromConfig({ ctx, cfg, dispatcher, replyResolver });
expect(mocks.routeReply).toHaveBeenCalledWith(
expect.objectContaining({
payload: expect.objectContaining({ text: "hi" }),
}),
);
});
it("fast-aborts without calling the reply resolver", async () => { it("fast-aborts without calling the reply resolver", async () => {
mocks.tryFastAbortFromMessage.mockResolvedValue({ mocks.tryFastAbortFromMessage.mockResolvedValue({
handled: true, handled: true,
+3 -19
View File
@@ -206,6 +206,7 @@ export async function dispatchReplyFromConfig(params: {
const sendPayloadAsync = async ( const sendPayloadAsync = async (
payload: ReplyPayload, payload: ReplyPayload,
abortSignal?: AbortSignal, abortSignal?: AbortSignal,
mirror?: boolean,
): Promise<void> => { ): Promise<void> => {
// TypeScript doesn't narrow these from the shouldRouteToOriginating check, // TypeScript doesn't narrow these from the shouldRouteToOriginating check,
// but they're guaranteed non-null when this function is called. // but they're guaranteed non-null when this function is called.
@@ -220,6 +221,7 @@ export async function dispatchReplyFromConfig(params: {
threadId: ctx.MessageThreadId, threadId: ctx.MessageThreadId,
cfg, cfg,
abortSignal, abortSignal,
mirror,
}); });
if (!result.ok) { if (!result.ok) {
logVerbose(`dispatch-from-config: route-reply failed: ${result.error ?? "unknown error"}`); logVerbose(`dispatch-from-config: route-reply failed: ${result.error ?? "unknown error"}`);
@@ -268,24 +270,6 @@ export async function dispatchReplyFromConfig(params: {
ctx, ctx,
{ {
...params.replyOptions, ...params.replyOptions,
onToolResult: (payload: ReplyPayload) => {
const run = async () => {
const ttsPayload = await maybeApplyTtsToPayload({
payload,
cfg,
channel: ttsChannel,
kind: "tool",
inboundAudio,
ttsAuto: sessionTtsAuto,
});
if (shouldRouteToOriginating) {
await sendPayloadAsync(ttsPayload);
} else {
dispatcher.sendToolResult(ttsPayload);
}
};
return run();
},
onBlockReply: (payload: ReplyPayload, context) => { onBlockReply: (payload: ReplyPayload, context) => {
const run = async () => { const run = async () => {
const ttsPayload = await maybeApplyTtsToPayload({ const ttsPayload = await maybeApplyTtsToPayload({
@@ -297,7 +281,7 @@ export async function dispatchReplyFromConfig(params: {
ttsAuto: sessionTtsAuto, ttsAuto: sessionTtsAuto,
}); });
if (shouldRouteToOriginating) { if (shouldRouteToOriginating) {
await sendPayloadAsync(ttsPayload, context?.abortSignal); await sendPayloadAsync(ttsPayload, context?.abortSignal, false);
} else { } else {
dispatcher.sendBlockReply(ttsPayload); dispatcher.sendBlockReply(ttsPayload);
} }
+17
View File
@@ -379,6 +379,23 @@ describe("routeReply", () => {
}), }),
); );
}); });
it("skips mirror data when mirror is false", async () => {
mocks.deliverOutboundPayloads.mockResolvedValue([]);
await routeReply({
payload: { text: "hi" },
channel: "slack",
to: "channel:C123",
sessionKey: "agent:main:main",
mirror: false,
cfg: {} as never,
});
expect(mocks.deliverOutboundPayloads).toHaveBeenCalledWith(
expect.objectContaining({
mirror: undefined,
}),
);
});
}); });
const emptyRegistry = createRegistry([]); const emptyRegistry = createRegistry([]);
+11 -8
View File
@@ -33,6 +33,8 @@ export type RouteReplyParams = {
cfg: ClawdbotConfig; cfg: ClawdbotConfig;
/** Optional abort signal for cooperative cancellation. */ /** Optional abort signal for cooperative cancellation. */
abortSignal?: AbortSignal; abortSignal?: AbortSignal;
/** Mirror reply into session transcript (default: true when sessionKey is set). */
mirror?: boolean;
}; };
export type RouteReplyResult = { export type RouteReplyResult = {
@@ -118,14 +120,15 @@ export async function routeReply(params: RouteReplyParams): Promise<RouteReplyRe
replyToId: resolvedReplyToId ?? null, replyToId: resolvedReplyToId ?? null,
threadId: resolvedThreadId, threadId: resolvedThreadId,
abortSignal, abortSignal,
mirror: params.sessionKey mirror:
? { params.mirror !== false && params.sessionKey
sessionKey: params.sessionKey, ? {
agentId: resolveSessionAgentId({ sessionKey: params.sessionKey, config: cfg }), sessionKey: params.sessionKey,
text, agentId: resolveSessionAgentId({ sessionKey: params.sessionKey, config: cfg }),
mediaUrls, text,
} mediaUrls,
: undefined, }
: undefined,
}); });
const last = results.at(-1); const last = results.at(-1);
+4 -7
View File
@@ -34,15 +34,13 @@ vi.mock("../auto-reply/dispatch.js", async (importOriginal) => {
beforeEach(() => { beforeEach(() => {
dispatchMock.mockReset().mockImplementation(async (params) => { dispatchMock.mockReset().mockImplementation(async (params) => {
if ("dispatcher" in params && params.dispatcher) { if ("dispatcher" in params && params.dispatcher) {
params.dispatcher.sendToolResult({ text: "tool update" });
params.dispatcher.sendFinalReply({ text: "final reply" }); params.dispatcher.sendFinalReply({ text: "final reply" });
return { queuedFinal: true, counts: { tool: 1, block: 0, final: 1 } }; return { queuedFinal: true, counts: { tool: 0, block: 0, final: 1 } };
} }
if ("dispatcherOptions" in params && params.dispatcherOptions) { if ("dispatcherOptions" in params && params.dispatcherOptions) {
const { dispatcher, markDispatchIdle } = createReplyDispatcherWithTyping( const { dispatcher, markDispatchIdle } = createReplyDispatcherWithTyping(
params.dispatcherOptions, params.dispatcherOptions,
); );
dispatcher.sendToolResult({ text: "tool update" });
dispatcher.sendFinalReply({ text: "final reply" }); dispatcher.sendFinalReply({ text: "final reply" });
await dispatcher.waitForIdle(); await dispatcher.waitForIdle();
markDispatchIdle(); markDispatchIdle();
@@ -53,7 +51,7 @@ beforeEach(() => {
}); });
describe("discord native commands", () => { describe("discord native commands", () => {
it("streams tool results for native slash commands", { timeout: 60_000 }, async () => { it("skips tool results for native slash commands", { timeout: 60_000 }, async () => {
const { ChannelType } = await import("@buape/carbon"); const { ChannelType } = await import("@buape/carbon");
const { createDiscordNativeCommand } = await import("./monitor.js"); const { createDiscordNativeCommand } = await import("./monitor.js");
@@ -97,8 +95,7 @@ describe("discord native commands", () => {
expect(dispatchMock).toHaveBeenCalledTimes(1); expect(dispatchMock).toHaveBeenCalledTimes(1);
expect(reply).toHaveBeenCalledTimes(1); expect(reply).toHaveBeenCalledTimes(1);
expect(followUp).toHaveBeenCalledTimes(1); expect(followUp).toHaveBeenCalledTimes(0);
expect(reply.mock.calls[0]?.[0]?.content).toContain("tool"); expect(reply.mock.calls[0]?.[0]?.content).toContain("final");
expect(followUp.mock.calls[0]?.[0]?.content).toContain("final");
}); });
}); });
@@ -277,15 +277,12 @@ describe("monitorIMessageProvider", () => {
expect(ctx.SessionKey).toBe("agent:main:imessage:group:2"); expect(ctx.SessionKey).toBe("agent:main:imessage:group:2");
}); });
it("prefixes tool and final replies with responsePrefix", async () => { it("prefixes final replies with responsePrefix", async () => {
config = { config = {
...config, ...config,
messages: { responsePrefix: "PFX" }, messages: { responsePrefix: "PFX" },
}; };
replyMock.mockImplementation(async (_ctx, opts) => { replyMock.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool update" });
return { text: "final reply" };
});
const run = monitorIMessageProvider(); const run = monitorIMessageProvider();
await waitForSubscribe(); await waitForSubscribe();
@@ -307,9 +304,8 @@ describe("monitorIMessageProvider", () => {
closeResolve?.(); closeResolve?.();
await run; await run;
expect(sendMock).toHaveBeenCalledTimes(2); expect(sendMock).toHaveBeenCalledTimes(1);
expect(sendMock.mock.calls[0][1]).toBe("PFX tool update"); expect(sendMock.mock.calls[0][1]).toBe("PFX final reply");
expect(sendMock.mock.calls[1][1]).toBe("PFX final reply");
}); });
it("defaults to dmPolicy=pairing behavior when allowFrom is empty", async () => { it("defaults to dmPolicy=pairing behavior when allowFrom is empty", async () => {
@@ -210,12 +210,9 @@ describe("monitorSignalProvider tool results", () => {
); );
}); });
it("sends tool summaries with responsePrefix", async () => { it("skips tool summaries with responsePrefix", async () => {
const abortController = new AbortController(); const abortController = new AbortController();
replyMock.mockImplementation(async (_ctx, opts) => { replyMock.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool update" });
return { text: "final reply" };
});
streamMock.mockImplementation(async ({ onEvent }) => { streamMock.mockImplementation(async ({ onEvent }) => {
const payload = { const payload = {
@@ -243,9 +240,8 @@ describe("monitorSignalProvider tool results", () => {
await flush(); await flush();
expect(sendMock).toHaveBeenCalledTimes(2); expect(sendMock).toHaveBeenCalledTimes(1);
expect(sendMock.mock.calls[0][1]).toBe("PFX tool update"); expect(sendMock.mock.calls[0][1]).toBe("PFX final reply");
expect(sendMock.mock.calls[1][1]).toBe("PFX final reply");
}); });
it("replies with pairing code when dmPolicy is pairing and no allowFrom is set", async () => { it("replies with pairing code when dmPolicy is pairing and no allowFrom is set", async () => {
@@ -23,11 +23,8 @@ beforeEach(() => {
}); });
describe("monitorSlackProvider tool results", () => { describe("monitorSlackProvider tool results", () => {
it("sends tool summaries with responsePrefix", async () => { it("skips tool summaries with responsePrefix", async () => {
replyMock.mockImplementation(async (_ctx, opts) => { replyMock.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool update" });
return { text: "final reply" };
});
const controller = new AbortController(); const controller = new AbortController();
const run = monitorSlackProvider({ const run = monitorSlackProvider({
@@ -55,9 +52,8 @@ describe("monitorSlackProvider tool results", () => {
controller.abort(); controller.abort();
await run; await run;
expect(sendMock).toHaveBeenCalledTimes(2); expect(sendMock).toHaveBeenCalledTimes(1);
expect(sendMock.mock.calls[0][1]).toBe("PFX tool update"); expect(sendMock.mock.calls[0][1]).toBe("PFX final reply");
expect(sendMock.mock.calls[1][1]).toBe("PFX final reply");
}); });
it("drops events with mismatched api_app_id", async () => { it("drops events with mismatched api_app_id", async () => {
@@ -130,10 +126,7 @@ describe("monitorSlackProvider tool results", () => {
}, },
}; };
replyMock.mockImplementation(async (_ctx, opts) => { replyMock.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool update" });
return { text: "final reply" };
});
const controller = new AbortController(); const controller = new AbortController();
const run = monitorSlackProvider({ const run = monitorSlackProvider({
@@ -161,9 +154,8 @@ describe("monitorSlackProvider tool results", () => {
controller.abort(); controller.abort();
await run; await run;
expect(sendMock).toHaveBeenCalledTimes(2); expect(sendMock).toHaveBeenCalledTimes(1);
expect(sendMock.mock.calls[0][1]).toBe("tool update"); expect(sendMock.mock.calls[0][1]).toBe("final reply");
expect(sendMock.mock.calls[1][1]).toBe("final reply");
}); });
it("preserves RawBody without injecting processed room history", async () => { it("preserves RawBody without injecting processed room history", async () => {
@@ -300,7 +300,7 @@ describe("createTelegramBot", () => {
expect.objectContaining({ message_thread_id: 99 }), expect.objectContaining({ message_thread_id: 99 }),
); );
}); });
it("streams tool summaries for native slash commands", async () => { it("skips tool summaries for native slash commands", async () => {
onSpy.mockReset(); onSpy.mockReset();
sendMessageSpy.mockReset(); sendMessageSpy.mockReset();
commandSpy.mockReset(); commandSpy.mockReset();
@@ -338,9 +338,8 @@ describe("createTelegramBot", () => {
match: "on", match: "on",
}); });
expect(sendMessageSpy).toHaveBeenCalledTimes(2); expect(sendMessageSpy).toHaveBeenCalledTimes(1);
expect(sendMessageSpy.mock.calls[0]?.[1]).toContain("tool update"); expect(sendMessageSpy.mock.calls[0]?.[1]).toContain("final reply");
expect(sendMessageSpy.mock.calls[1]?.[1]).toContain("final reply");
}); });
it("dedupes duplicate message updates by update_id", async () => { it("dedupes duplicate message updates by update_id", async () => {
onSpy.mockReset(); onSpy.mockReset();
@@ -217,15 +217,12 @@ describe("createTelegramBot", () => {
expect(call[2]?.reply_to_message_id).toBeUndefined(); expect(call[2]?.reply_to_message_id).toBeUndefined();
} }
}); });
it("prefixes tool and final replies with responsePrefix", async () => { it("prefixes final replies with responsePrefix", async () => {
onSpy.mockReset(); onSpy.mockReset();
sendMessageSpy.mockReset(); sendMessageSpy.mockReset();
const replySpy = replyModule.__replySpy as unknown as ReturnType<typeof vi.fn>; const replySpy = replyModule.__replySpy as unknown as ReturnType<typeof vi.fn>;
replySpy.mockReset(); replySpy.mockReset();
replySpy.mockImplementation(async (_ctx, opts) => { replySpy.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool result" });
return { text: "final reply" };
});
loadConfig.mockReturnValue({ loadConfig.mockReturnValue({
channels: { channels: {
telegram: { dmPolicy: "open", allowFrom: ["*"] }, telegram: { dmPolicy: "open", allowFrom: ["*"] },
@@ -245,9 +242,8 @@ describe("createTelegramBot", () => {
getFile: async () => ({ download: async () => new Uint8Array() }), getFile: async () => ({ download: async () => new Uint8Array() }),
}); });
expect(sendMessageSpy).toHaveBeenCalledTimes(2); expect(sendMessageSpy).toHaveBeenCalledTimes(1);
expect(sendMessageSpy.mock.calls[0][1]).toBe("PFX tool result"); expect(sendMessageSpy.mock.calls[0][1]).toBe("PFX final reply");
expect(sendMessageSpy.mock.calls[1][1]).toBe("PFX final reply");
}); });
it("honors replyToMode=all for threaded replies", async () => { it("honors replyToMode=all for threaded replies", async () => {
onSpy.mockReset(); onSpy.mockReset();
+7 -12
View File
@@ -865,15 +865,12 @@ describe("createTelegramBot", () => {
} }
}); });
it("prefixes tool and final replies with responsePrefix", async () => { it("prefixes final replies with responsePrefix", async () => {
onSpy.mockReset(); onSpy.mockReset();
sendMessageSpy.mockReset(); sendMessageSpy.mockReset();
const replySpy = replyModule.__replySpy as unknown as ReturnType<typeof vi.fn>; const replySpy = replyModule.__replySpy as unknown as ReturnType<typeof vi.fn>;
replySpy.mockReset(); replySpy.mockReset();
replySpy.mockImplementation(async (_ctx, opts) => { replySpy.mockResolvedValue({ text: "final reply" });
await opts?.onToolResult?.({ text: "tool result" });
return { text: "final reply" };
});
loadConfig.mockReturnValue({ loadConfig.mockReturnValue({
channels: { channels: {
telegram: { dmPolicy: "open", allowFrom: ["*"] }, telegram: { dmPolicy: "open", allowFrom: ["*"] },
@@ -893,9 +890,8 @@ describe("createTelegramBot", () => {
getFile: async () => ({ download: async () => new Uint8Array() }), getFile: async () => ({ download: async () => new Uint8Array() }),
}); });
expect(sendMessageSpy).toHaveBeenCalledTimes(2); expect(sendMessageSpy).toHaveBeenCalledTimes(1);
expect(sendMessageSpy.mock.calls[0][1]).toBe("PFX tool result"); expect(sendMessageSpy.mock.calls[0][1]).toBe("PFX final reply");
expect(sendMessageSpy.mock.calls[1][1]).toBe("PFX final reply");
}); });
it("honors replyToMode=all for threaded replies", async () => { it("honors replyToMode=all for threaded replies", async () => {
@@ -2288,7 +2284,7 @@ describe("createTelegramBot", () => {
); );
}); });
it("streams tool summaries for native slash commands", async () => { it("skips tool summaries for native slash commands", async () => {
onSpy.mockReset(); onSpy.mockReset();
sendMessageSpy.mockReset(); sendMessageSpy.mockReset();
commandSpy.mockReset(); commandSpy.mockReset();
@@ -2326,9 +2322,8 @@ describe("createTelegramBot", () => {
match: "on", match: "on",
}); });
expect(sendMessageSpy).toHaveBeenCalledTimes(2); expect(sendMessageSpy).toHaveBeenCalledTimes(1);
expect(sendMessageSpy.mock.calls[0]?.[1]).toContain("tool update"); expect(sendMessageSpy.mock.calls[0]?.[1]).toContain("final reply");
expect(sendMessageSpy.mock.calls[1]?.[1]).toContain("final reply");
}); });
it("dedupes duplicate message updates by update_id", async () => { it("dedupes duplicate message updates by update_id", async () => {
+123
View File
@@ -0,0 +1,123 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
const { botApi, botCtorSpy } = vi.hoisted(() => ({
botApi: {
sendMessage: vi.fn(),
setMessageReaction: vi.fn(),
deleteMessage: vi.fn(),
},
botCtorSpy: vi.fn(),
}));
const { loadConfig } = vi.hoisted(() => ({
loadConfig: vi.fn(() => ({})),
}));
const { makeProxyFetch } = vi.hoisted(() => ({
makeProxyFetch: vi.fn(),
}));
const { resolveTelegramFetch } = vi.hoisted(() => ({
resolveTelegramFetch: vi.fn(),
}));
vi.mock("../config/config.js", async (importOriginal) => {
const actual = await importOriginal<typeof import("../config/config.js")>();
return {
...actual,
loadConfig,
};
});
vi.mock("./proxy.js", () => ({
makeProxyFetch,
}));
vi.mock("./fetch.js", () => ({
resolveTelegramFetch,
}));
vi.mock("grammy", () => ({
Bot: class {
api = botApi;
constructor(
public token: string,
public options?: { client?: { fetch?: typeof fetch; timeoutSeconds?: number } },
) {
botCtorSpy(token, options);
}
},
InputFile: class {},
}));
import { deleteMessageTelegram, reactMessageTelegram, sendMessageTelegram } from "./send.js";
describe("telegram proxy client", () => {
const proxyUrl = "http://proxy.test:8080";
beforeEach(() => {
botApi.sendMessage.mockResolvedValue({ message_id: 1, chat: { id: "123" } });
botApi.setMessageReaction.mockResolvedValue(undefined);
botApi.deleteMessage.mockResolvedValue(true);
botCtorSpy.mockReset();
loadConfig.mockReturnValue({
channels: { telegram: { accounts: { foo: { proxy: proxyUrl } } } },
});
makeProxyFetch.mockReset();
resolveTelegramFetch.mockReset();
});
it("uses proxy fetch for sendMessage", async () => {
const proxyFetch = vi.fn();
const fetchImpl = vi.fn();
makeProxyFetch.mockReturnValue(proxyFetch as unknown as typeof fetch);
resolveTelegramFetch.mockReturnValue(fetchImpl as unknown as typeof fetch);
await sendMessageTelegram("123", "hi", { token: "tok", accountId: "foo" });
expect(makeProxyFetch).toHaveBeenCalledWith(proxyUrl);
expect(resolveTelegramFetch).toHaveBeenCalledWith(proxyFetch);
expect(botCtorSpy).toHaveBeenCalledWith(
"tok",
expect.objectContaining({
client: expect.objectContaining({ fetch: fetchImpl }),
}),
);
});
it("uses proxy fetch for reactions", async () => {
const proxyFetch = vi.fn();
const fetchImpl = vi.fn();
makeProxyFetch.mockReturnValue(proxyFetch as unknown as typeof fetch);
resolveTelegramFetch.mockReturnValue(fetchImpl as unknown as typeof fetch);
await reactMessageTelegram("123", "456", "✅", { token: "tok", accountId: "foo" });
expect(makeProxyFetch).toHaveBeenCalledWith(proxyUrl);
expect(resolveTelegramFetch).toHaveBeenCalledWith(proxyFetch);
expect(botCtorSpy).toHaveBeenCalledWith(
"tok",
expect.objectContaining({
client: expect.objectContaining({ fetch: fetchImpl }),
}),
);
});
it("uses proxy fetch for deleteMessage", async () => {
const proxyFetch = vi.fn();
const fetchImpl = vi.fn();
makeProxyFetch.mockReturnValue(proxyFetch as unknown as typeof fetch);
resolveTelegramFetch.mockReturnValue(fetchImpl as unknown as typeof fetch);
await deleteMessageTelegram("123", "456", { token: "tok", accountId: "foo" });
expect(makeProxyFetch).toHaveBeenCalledWith(proxyUrl);
expect(resolveTelegramFetch).toHaveBeenCalledWith(proxyFetch);
expect(botCtorSpy).toHaveBeenCalledWith(
"tok",
expect.objectContaining({
client: expect.objectContaining({ fetch: fetchImpl }),
}),
);
});
});
+24 -22
View File
@@ -17,8 +17,9 @@ import { createSubsystemLogger } from "../logging/subsystem.js";
import { mediaKindFromMime } from "../media/constants.js"; import { mediaKindFromMime } from "../media/constants.js";
import { isGifMedia } from "../media/mime.js"; import { isGifMedia } from "../media/mime.js";
import { loadWebMedia } from "../web/media.js"; import { loadWebMedia } from "../web/media.js";
import { resolveTelegramAccount } from "./accounts.js"; import { type ResolvedTelegramAccount, resolveTelegramAccount } from "./accounts.js";
import { resolveTelegramFetch } from "./fetch.js"; import { resolveTelegramFetch } from "./fetch.js";
import { makeProxyFetch } from "./proxy.js";
import { renderTelegramHtmlText } from "./format.js"; import { renderTelegramHtmlText } from "./format.js";
import { resolveMarkdownTableMode } from "../config/markdown-tables.js"; import { resolveMarkdownTableMode } from "../config/markdown-tables.js";
import { splitTelegramCaption } from "./caption.js"; import { splitTelegramCaption } from "./caption.js";
@@ -76,6 +77,25 @@ function createTelegramHttpLogger(cfg: ReturnType<typeof loadConfig>) {
}; };
} }
function resolveTelegramClientOptions(
account: ResolvedTelegramAccount,
): ApiClientOptions | undefined {
const proxyUrl = account.config.proxy?.trim();
const proxyFetch = proxyUrl ? makeProxyFetch(proxyUrl) : undefined;
const fetchImpl = resolveTelegramFetch(proxyFetch);
const timeoutSeconds =
typeof account.config.timeoutSeconds === "number" &&
Number.isFinite(account.config.timeoutSeconds)
? Math.max(1, Math.floor(account.config.timeoutSeconds))
: undefined;
return fetchImpl || timeoutSeconds
? {
...(fetchImpl ? { fetch: fetchImpl as unknown as ApiClientOptions["fetch"] } : {}),
...(timeoutSeconds ? { timeoutSeconds } : {}),
}
: undefined;
}
function resolveToken(explicit: string | undefined, params: { accountId: string; token: string }) { function resolveToken(explicit: string | undefined, params: { accountId: string; token: string }) {
if (explicit?.trim()) return explicit.trim(); if (explicit?.trim()) return explicit.trim();
if (!params.token) { if (!params.token) {
@@ -162,19 +182,7 @@ export async function sendMessageTelegram(
const chatId = normalizeChatId(target.chatId); const chatId = normalizeChatId(target.chatId);
// Use provided api or create a new Bot instance. The nullish coalescing // Use provided api or create a new Bot instance. The nullish coalescing
// operator ensures api is always defined (Bot.api is always non-null). // operator ensures api is always defined (Bot.api is always non-null).
const fetchImpl = resolveTelegramFetch(); const client = resolveTelegramClientOptions(account);
const timeoutSeconds =
typeof account.config.timeoutSeconds === "number" &&
Number.isFinite(account.config.timeoutSeconds)
? Math.max(1, Math.floor(account.config.timeoutSeconds))
: undefined;
const client: ApiClientOptions | undefined =
fetchImpl || timeoutSeconds
? {
...(fetchImpl ? { fetch: fetchImpl as unknown as ApiClientOptions["fetch"] } : {}),
...(timeoutSeconds ? { timeoutSeconds } : {}),
}
: undefined;
const api = opts.api ?? new Bot(token, client ? { client } : undefined).api; const api = opts.api ?? new Bot(token, client ? { client } : undefined).api;
const mediaUrl = opts.mediaUrl?.trim(); const mediaUrl = opts.mediaUrl?.trim();
const replyMarkup = buildInlineKeyboard(opts.buttons); const replyMarkup = buildInlineKeyboard(opts.buttons);
@@ -416,10 +424,7 @@ export async function reactMessageTelegram(
const token = resolveToken(opts.token, account); const token = resolveToken(opts.token, account);
const chatId = normalizeChatId(String(chatIdInput)); const chatId = normalizeChatId(String(chatIdInput));
const messageId = normalizeMessageId(messageIdInput); const messageId = normalizeMessageId(messageIdInput);
const fetchImpl = resolveTelegramFetch(); const client = resolveTelegramClientOptions(account);
const client: ApiClientOptions | undefined = fetchImpl
? { fetch: fetchImpl as unknown as ApiClientOptions["fetch"] }
: undefined;
const api = opts.api ?? new Bot(token, client ? { client } : undefined).api; const api = opts.api ?? new Bot(token, client ? { client } : undefined).api;
const request = createTelegramRetryRunner({ const request = createTelegramRetryRunner({
retry: opts.retry, retry: opts.retry,
@@ -468,10 +473,7 @@ export async function deleteMessageTelegram(
const token = resolveToken(opts.token, account); const token = resolveToken(opts.token, account);
const chatId = normalizeChatId(String(chatIdInput)); const chatId = normalizeChatId(String(chatIdInput));
const messageId = normalizeMessageId(messageIdInput); const messageId = normalizeMessageId(messageIdInput);
const fetchImpl = resolveTelegramFetch(); const client = resolveTelegramClientOptions(account);
const client: ApiClientOptions | undefined = fetchImpl
? { fetch: fetchImpl as unknown as ApiClientOptions["fetch"] }
: undefined;
const api = opts.api ?? new Bot(token, client ? { client } : undefined).api; const api = opts.api ?? new Bot(token, client ? { client } : undefined).api;
const request = createTelegramRetryRunner({ const request = createTelegramRetryRunner({
retry: opts.retry, retry: opts.retry,
@@ -105,7 +105,7 @@ describe("web auto-reply", () => {
vi.useRealTimers(); vi.useRealTimers();
}); });
it("sends tool summaries immediately with responsePrefix", async () => { it("skips tool summaries and sends final reply with responsePrefix", async () => {
setLoadConfigMock(() => ({ setLoadConfigMock(() => ({
channels: { whatsapp: { allowFrom: ["*"] } }, channels: { whatsapp: { allowFrom: ["*"] } },
messages: { messages: {
@@ -125,15 +125,7 @@ describe("web auto-reply", () => {
return { close: vi.fn() }; return { close: vi.fn() };
}; };
const resolver = vi const resolver = vi.fn().mockResolvedValue({ text: "final" });
.fn()
.mockImplementation(
async (_ctx, opts?: { onToolResult?: (r: { text: string }) => Promise<void> }) => {
await opts?.onToolResult?.({ text: "🧩 tool1" });
await opts?.onToolResult?.({ text: "🧩 tool2" });
return { text: "final" };
},
);
await monitorWebChannel(false, listenerFactory, false, resolver); await monitorWebChannel(false, listenerFactory, false, resolver);
expect(capturedOnMessage).toBeDefined(); expect(capturedOnMessage).toBeDefined();
@@ -149,7 +141,7 @@ describe("web auto-reply", () => {
}); });
const replies = reply.mock.calls.map((call) => call[0]); const replies = reply.mock.calls.map((call) => call[0]);
expect(replies).toEqual(["🦞 🧩 tool1", "🦞 🧩 tool2", "🦞 final"]); expect(replies).toEqual(["🦞 final"]);
resetLoadConfigMock(); resetLoadConfigMock();
}); });
it("uses identity.name for messagePrefix when set", async () => { it("uses identity.name for messagePrefix when set", async () => {