Compare commits

..
Author SHA1 Message Date
Peter Steinberger f1adf22249 docs: note self-chat config docs update 2026-01-03 02:04:26 +01:00
rafaelreis-randClaude Opus 4.5 ed7962500d docs: clarify routing.allowFrom and self-chat mode for group mentions
- Add new section explaining self-chat mode for group control
- Document routing.allowFrom as the key setting for controlling metadata mentions
- Clarify difference between whatsapp.allowFrom (DM allowlist) and routing.allowFrom (self-chat mode)
- Explain metadata mentions vs text patterns in routing.groupChat
- Add example config for responding only to specific text triggers

When routing.allowFrom contains the bot's own number, WhatsApp native
@-mentions are ignored in groups, and only mentionPatterns trigger responses.
This prevents unwanted responses when users tap-to-mention the bot owner.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 02:04:00 +01:00
Peter Steinberger 72d1fa4da5 fix: dedupe repeated block replies 2026-01-03 01:49:27 +01:00
Peter Steinberger 2042013360 test: cover provider textChunkLimit config 2026-01-03 01:49:27 +01:00
Peter Steinberger f5189cc897 refactor: move text chunk limits to providers 2026-01-03 01:49:27 +01:00
Peter Steinberger 75a9cd83a0 fix(mac): resolve camera type deprecation 2026-01-03 01:49:27 +01:00
Peter Steinberger 5684e2d658 feat: configurable outbound text chunk limits 2026-01-03 01:49:27 +01:00
Peter Steinberger 2d28fa34f5 feat: make block streaming break configurable 2026-01-03 01:49:27 +01:00
Peter Steinberger ea7d967625 Update Discord invite to vanity URL discord.gg/clawd 🦞
Thanks camerondare for the boosts! Level 3 unlocked.
2026-01-03 00:47:22 +00:00
Peter Steinberger 5dfb2b1128 coding-agent: add temp space pattern, never start in ~/clawd
Learnings from tonight:
- Codex reads AGENTS.md/SOUL.md and gets ideas about org hierarchy
- Use mktemp -d for scratch/chat sessions
- Never start in ~/clawd or agent home dirs
- Keep agents in their 'little box' 📦🦞
2026-01-03 00:35:51 +00:00
Peter Steinberger cbc599a5b8 coding-agent: add batch PR review pattern
Tonight's learnings:
- Parallel Codex army for batch PR reviews
- Fetch PR refs: git fetch origin '+refs/pull/*/head:refs/remotes/origin/pr/*'
- Use git diff origin/main...origin/pr/XX (don't checkout)
- Post results with gh pr comment
- Successfully reviewed 13 PRs in parallel! 🦞
2026-01-03 00:24:34 +00:00
Peter Steinberger 1354d0836f coding-agent: comprehensive update from Jan 2 learnings
- workdir 'little box' pattern (don't read unrelated files)
- background mode replaces tmux
- --full-auto for building, vanilla for reviewing
- parallel Codex processes supported
- PR review tips (fetch refs, use git diff, don't checkout)
- patience rules (don't kill slow sessions!)
2026-01-03 00:11:21 +00:00
Peter Steinberger b313250638 coding-agent: switch to native background mode, drop tmux requirement
- Use bash background:true instead of tmux
- Full programmatic control: log/poll/write/kill
- Simpler, no shell escaping issues
- workdir still critical for 'little box' pattern
2026-01-03 00:00:37 +00:00
Peter Steinberger e37c147ea9 coding-agent: unified workdir+tmux pattern for all tools 2026-01-02 23:58:33 +00:00
Peter Steinberger feb4f9028d coding-agent: choose reasoning effort based on task complexity 2026-01-02 23:57:46 +00:00
Peter Steinberger 4804ce5678 coding-agent: simplify to gpt-5.2-codex only, remove old models 2026-01-02 23:57:24 +00:00
Peter Steinberger 001a342f20 coding-agent: workdir pattern + patience rules
- Use bash workdir param so Codex wakes up in a 'little box'
- Prevents reading unrelated files (like my soul.md lol)
- Added rule: NEVER offer to build it yourself when user asks for Codex
- gpt-5.2-codex requires medium reasoning effort
2026-01-02 23:56:10 +00:00
Peter Steinberger fe040b84d9 chore: sync lockfile and bundle hash 2026-01-03 00:40:39 +01:00
Sreekaran Srinath 0ac30afb29 feat: add coding-agent skill and anyBins gating
Co-authored-by: Sreekaran Srinath <ss@sreekaran.com>
2026-01-03 00:40:03 +01:00
Peter Steinberger 59601eb99c fix: preserve newlines in reply tags 2026-01-02 23:36:43 +00:00
Peter Steinberger 9616f4b2b1 feat: stream reply blocks immediately 2026-01-03 00:28:33 +01:00
Peter Steinberger 9dd613edf7 fix(mac): harden remote tunnel recovery 2026-01-03 00:02:27 +01:00
Peter Steinberger 88ed58b3d0 chore: update deps and extend read tool tests 2026-01-02 23:47:28 +01:00
Peter Steinberger fc54e905c0 chore: upgrade pi-mono deps to 0.31.1 2026-01-02 23:37:08 +01:00
Peter Steinberger d1b76cb1b2 test: cover replyToMode behavior 2026-01-02 23:20:52 +01:00
Peter Steinberger 2c92ccd66e feat: add reply tags and replyToMode 2026-01-02 23:18:41 +01:00
Peter Steinberger a9ff03acaf feat: unify group mention defaults 2026-01-02 22:50:58 +01:00
Shadow 281dc10b2f Changelog: mention Discord reply context 2026-01-02 15:41:45 -06:00
Peter Steinberger fd32fc8d8d feat: add discord guild wildcard defaults 2026-01-02 22:33:26 +01:00
Peter Steinberger 47f4f59692 chore: remove stray ds_store files 2026-01-02 22:24:26 +01:00
Peter Steinberger 5cf1a9535e feat: move group mention gating to provider groups 2026-01-02 22:24:26 +01:00
Peter Steinberger e93102b276 chore: bump peekaboo submodule 2026-01-02 22:24:26 +01:00
Shadow da57c314ef Discord: clarify docs and drop legacy guild schema 2026-01-02 15:21:13 -06:00
Shadow 2676636316 Discord: fix reply context formatting 2026-01-02 14:55:07 -06:00
Shadow f3a973dc9e Discord: include reply context 2026-01-02 14:49:16 -06:00
Peter SteinbergerandClawd f4a1190bdd docs: add CONTRIBUTING.md with maintainers and guidelines
- List maintainers with GitHub/X links
- Link to Discord and GitHub Discussions
- AI/vibe-coded PRs welcome with transparency guidelines
- Link from README

Co-authored-by: Clawd <clawdbot@gmail.com>
2026-01-02 20:31:41 +00:00
Peter Steinberger 118a6d7421 fix: align discord config ui 2026-01-02 21:15:59 +01:00
Peter Steinberger 4541bb2716 Merge pull request #108 from thewilloftheshadow/shadow/ui-connection-update
UI: Update connections UIs
2026-01-02 21:04:45 +01:00
Peter Steinberger 505c4262c6 docs: note optional docker setup 2026-01-02 20:59:58 +01:00
Peter Steinberger 3104b088e4 chore(canvas): update a2ui bundle hash 2026-01-02 19:58:46 +00:00
Peter Steinberger f12f814816 docs(whatsapp): add number guidance 2026-01-02 19:58:44 +00:00
Peter Steinberger 3b0ad719c9 chore(discord): add verbose diagnostics 2026-01-02 19:58:42 +00:00
Peter Steinberger e368e56246 chore(gateway): quiet provider startup logs 2026-01-02 19:58:40 +00:00
Peter Steinberger 675420013d fix(macos): resolve gateway launch args 2026-01-02 19:58:38 +00:00
Peter Steinberger eaa69fb6b2 test: silence docker onboarding noise 2026-01-02 20:46:26 +01:00
Peter Steinberger e0795cf18c test: annotate onboarding docker e2e 2026-01-02 20:41:47 +01:00
Peter Steinberger 8ed878e73c test: stabilize docker onboarding e2e 2026-01-02 20:40:33 +01:00
Peter Steinberger 08b95411df chore: add goplaces skill 2026-01-02 20:33:06 +01:00
Peter Steinberger 460fafff7f docs: thank @dan-dr for docker setup 2026-01-02 20:24:44 +01:00
Peter Steinberger 7b4fa9e1a1 test: cover camera list invoke 2026-01-02 20:24:41 +01:00
Peter Steinberger 7e4ebb22a0 Merge pull request #107 from dan-dr/main
Add Docker setup script
2026-01-02 20:24:21 +01:00
Peter Steinberger 8b47315845 fix(macos): improve session preview loading 2026-01-02 19:55:19 +01:00
Shadow 96a5e01878 macOS: swiftformat connections settings 2026-01-02 12:30:59 -06:00
Shadow 5e36390a27 macOS: fix swiftlint param count 2026-01-02 12:25:47 -06:00
Shadow 729a545173 Update connections UIs 2026-01-02 12:06:05 -06:00
Dan 488f5e2dac Merge branch 'steipete:main' into main 2026-01-02 19:53:16 +02:00
Peter Steinberger 49e89cf3f1 fix: satisfy swiftformat for ios build 2026-01-02 18:48:05 +01:00
Peter Steinberger 43f6b9ef32 fix: resolve camera tool handling 2026-01-02 17:44:25 +00:00
Peter Steinberger 8e48cffe3b fix(macos): decode session preview payload 2026-01-02 18:32:03 +01:00
Peter Steinberger 3ed01adabc feat(macos): add session previews in menu 2026-01-02 18:29:47 +01:00
Dan 4239de8060 Merge branch 'steipete:main' into main 2026-01-02 19:26:14 +02:00
Peter Steinberger cba37f99b6 test: cover camera device selection 2026-01-02 18:25:22 +01:00
Peter Steinberger 74db53d939 feat: add camera list and device selection 2026-01-02 18:23:26 +01:00
Peter Steinberger 2b34bf08da fix: resolve mac camera continuation isolation 2026-01-02 18:02:24 +01:00
Dan b92f70c52b Merge branch 'steipete:main' into main 2026-01-02 19:00:21 +02:00
Peter Steinberger 34d2e1e2e8 fix: wait for camera exposure to settle 2026-01-02 17:57:34 +01:00
Peter Steinberger 5f82739e2b test: cover camera snap mime mapping 2026-01-02 17:49:20 +01:00
Peter Steinberger d79dc4d742 fix: correct camera snap mime mapping 2026-01-02 17:43:34 +01:00
Peter Steinberger 1d12a844c2 docs: add WhatsApp disconnect workaround to FAQ
When using macOS app with WhatsApp issues:
1. Run pnpm gateway:watch (Node instead of bun)
2. Enable 'External gateway' in app debug settings

Verified gateway:watch command exists in package.json
2026-01-02 16:34:27 +00:00
Peter Steinberger 2d16450869 feat: add weather skill (wttr.in + Open-Meteo fallback)
No API key required. Two services:
- wttr.in: human-readable, emoji, ASCII art, PNG
- Open-Meteo: JSON API fallback for programmatic use

🌤️🦞
2026-01-02 16:33:31 +00:00
Peter Steinberger 2a6248dad6 fix: add camera entitlement to macOS signing 2026-01-02 17:31:59 +01:00
Peter Steinberger 8b27c03472 docs(skills/local-places): add emoji and tagline
📍 Find places, Go fast

🦞
2026-01-02 16:22:26 +00:00
Peter Steinberger baf3bea574 docs(changelog): note macOS config actor fix 2026-01-02 17:16:49 +01:00
Peter Steinberger 868b438e67 test(gateway): fix nix mode mock toggle 2026-01-02 17:15:26 +01:00
Peter Steinberger 8989bd9fd7 fix(auto-reply): default whatsapp self-only on empty config 2026-01-02 17:15:26 +01:00
Peter Steinberger a4f12babb7 test(macos): cover gateway password whitespace 2026-01-02 17:15:26 +01:00
Peter Steinberger 97e06a8eb4 chore(canvas): regenerate a2ui bundle hash 2026-01-02 17:15:26 +01:00
Peter Steinberger 0de6e38ce9 fix(macos): keep config writes on main actor 2026-01-02 17:15:26 +01:00
Peter Steinberger 314164fb8a chore: fix lint and add gateway auth tests 2026-01-02 17:15:26 +01:00
Peter Steinberger 8d925226cb docs: expand FAQ with Docker, OAuth, bun vs Node, debugging
- Docker/container setup (volumes, pnpm persistence, startup script)
- OAuth vs API key billing differences
- OAuth callback workarounds for headless/containers
- Terminal onboarding vs macOS app (terminal more stable)
- bun binary vs Node runtime (Node more stable for WhatsApp)
- gateway:watch for debugging
- Tailscale link and setup clarification

Based on community questions from Discord #help
2026-01-02 16:04:02 +00:00
Peter Steinberger f2eb2004aa docs: thank @jeffersonwarrior for gateway auth 2026-01-02 16:51:48 +01:00
Peter Steinberger bf37015c23 Merge pull request #85 from jeffersonwarrior/main
feat: add gateway password auth support and fix Swift 6.2 concurrency
2026-01-02 16:50:57 +01:00
Peter Steinberger f489b6e7a5 chore: merge origin/main 2026-01-02 16:50:29 +01:00
Peter Steinberger 921e5be8e6 fix(skills/local-places): copy files instead of submodule
Submodules are pain. Just copy the Python code directly.

🦞
2026-01-02 15:48:24 +00:00
Peter Steinberger a8bc974a2e fix: harden gateway password auth 2026-01-02 16:47:52 +01:00
Peter Steinberger 100a022ab7 feat(skills/local-places): add server as submodule
- Links to Hyaxia/local_places for easy upstream updates
- Updated SKILL.md with {baseDir}/server path

🦞
2026-01-02 15:47:42 +00:00
Peter Steinberger 6b7484a885 feat(skills): add local-places skill for Google Places search
- Wraps Hyaxia/local_places FastAPI server
- Two-step flow: resolve location → search places
- Supports filters: type, rating, price, open_now

🦞
2026-01-02 15:46:08 +00:00
Peter Steinberger 8de40e0c10 feat(macos): add Camera permission to onboarding flow
- Add 'camera' case to Capability enum
- Add UI strings (title, subtitle, icon) in PermissionsSettings
- Add ensureCamera() and camera status check in PermissionManager
- Add CameraPermissionHelper for opening System Settings

🦞 Clawd's first code contribution!
2026-01-02 15:27:54 +00:00
Peter Steinberger 9b3aef3567 fix: show skill descriptions in onboarding list 2026-01-02 16:25:28 +01:00
Peter Steinberger 25e52e19dc fix(macos): return node name 2026-01-02 15:28:34 +01:00
Peter Steinberger 68806902ff fix(macos): show gateway in devices list 2026-01-02 15:27:21 +01:00
Peter Steinberger ebf8649940 feat: add songsee skill 2026-01-02 15:22:23 +01:00
Peter Steinberger c93d02891a test: cover control ui token url 2026-01-02 15:13:05 +01:00
Peter Steinberger 87be5c737c fix(macos): suppress cancelled node refresh 2026-01-02 15:12:57 +01:00
Peter Steinberger ad9d6f616d fix: improve onboarding auth UX 2026-01-02 15:03:38 +01:00
Peter Steinberger f57f892409 fix(macos): clarify gateway error state 2026-01-02 13:48:19 +01:00
Peter Steinberger 5ecb65cbbe fix: persist gateway token for local CLI auth 2026-01-02 13:46:48 +01:00
Peter Steinberger 1e04481aaf style: format discord slash handler 2026-01-02 13:38:36 +01:00
Peter Steinberger 5f103e32bd fix: gate discord slash commands 2026-01-02 13:38:35 +01:00
Shadow fff9efe8a8 Discord: auto-register slash command 2026-01-02 13:38:35 +01:00
Shadow b135b3efb9 Discord: add slash command handling 2026-01-02 13:38:35 +01:00
Peter Steinberger 17e17f85ae docs: note gateway auto-migrate 2026-01-02 13:10:09 +01:00
Peter Steinberger ecef49605b test: cover gateway legacy auto-migrate 2026-01-02 13:09:41 +01:00
Peter Steinberger 7b72b35cca chore: update doctor migration hash 2026-01-02 13:07:26 +01:00
Peter Steinberger 16420e5b31 refactor: auto-migrate legacy config in gateway 2026-01-02 13:07:14 +01:00
Peter Steinberger 55665246bb chore: refresh doctor migration commit 2026-01-02 13:00:44 +01:00
Peter Steinberger b9b862a380 chore: note doctor migration commit 2026-01-02 13:00:29 +01:00
Peter Steinberger 0766c5e3cb refactor: move whatsapp allowFrom config 2026-01-02 13:00:29 +01:00
ddyo 8d4c6d41ab Docker: add root-level setup 2026-01-02 13:53:06 +02:00
Peter Steinberger 58d32d4542 docs: expand FAQ with skills, Tailscale, troubleshooting
- How to add/reload skills (/reset)
- Tailscale for multi-machine setups
- Using Codex to debug
- Handling supervised processes on Linux
- Clean uninstall steps
2026-01-02 11:50:09 +00:00
Peter Steinberger 6bad75827a docs: clarify Signal setup and env-token gating 2026-01-02 11:41:08 +00:00
Peter Steinberger 2b3ddabe90 fix(gateway): gate providers by config presence 2026-01-02 11:41:01 +00:00
Peter Steinberger e92b480629 fix(signal): surface signal-cli failures as errors 2026-01-02 11:40:55 +00:00
Peter Steinberger a53cdbf1b4 docs: clarify Windows is untested in FAQ 2026-01-02 11:30:27 +00:00
Peter Steinberger 21a64a9957 docs: link FAQ and add platforms note 2026-01-02 11:24:41 +00:00
Peter Steinberger d656db4d04 fix: widen discord channel type check 2026-01-02 12:23:35 +01:00
Peter Steinberger 506b66a852 docs: add FAQ with common questions from Discord
Covers:
- Installation & setup (data locations, unauthorized errors, fresh start, doctor)
- Migration & deployment (new machine, VPS, Docker)
- Multi-instance & contexts (one Clawd philosophy, groups for separation)
- Context & memory (200k tokens, autocompaction, workspace location)
- Platforms (supported platforms, multi-platform, WhatsApp numbers)
- Troubleshooting (build errors, WhatsApp logout, gateway issues)
- Chat commands reference

Based on community questions from #help channel.

🦞
2026-01-02 11:22:06 +00:00
Peter Steinberger 95f03d63ad style(ui): refresh dashboard theme 2026-01-02 11:22:06 +00:00
Peter Steinberger 7f8af736dd chore(canvas): regenerate a2ui bundle hash 2026-01-02 11:22:06 +00:00
Peter Steinberger eaacebeecc fix: improve onboarding/imessage errors 2026-01-02 12:20:48 +01:00
Peter Steinberger fd4cff06ca test: fix discord/config test lint 2026-01-02 12:20:43 +01:00
Peter Steinberger b50df6eb1d style: format linted files 2026-01-02 12:20:38 +01:00
Peter Steinberger fa16304e4f docs: note discord ignore-list removal 2026-01-02 11:54:30 +01:00
Peter Steinberger eda74d3a55 test: cover every schedule anchor boundary 2026-01-02 11:33:49 +01:00
Peter Steinberger 25762c0ac6 docs(discord): note from label includes tag/id 2026-01-02 11:32:59 +01:00
Peter Steinberger 2d7289bcad docs: update changelog for cron fix 2026-01-02 11:29:35 +01:00
Peter Steinberger 2d1d5d603d Merge pull request #80 from jamesgroat/fix/cron-every-schedule-infinite-loop
fix(cron): prevent every schedule from firing in infinite loop
2026-01-02 11:29:08 +01:00
Peter Steinberger 94206cf10f Merge pull request #92 from thewilloftheshadow/shadow/discord-id
Discord: pass user id to clawd so it can ping users
2026-01-02 11:27:37 +01:00
Peter Steinberger dc2521a1cf merge main into shadow/discord-id 2026-01-02 11:27:24 +01:00
Peter Steinberger 30b5955f22 fix(discord): add tag/id to from label 2026-01-02 11:26:09 +01:00
Peter Steinberger 4267a1b87d test: cover discord config + slug routing 2026-01-02 11:19:10 +01:00
Peter Steinberger eb44ae76f1 feat: add discord guild map + group dm controls 2026-01-02 11:15:52 +01:00
Peter Steinberger bd3d18f660 fix: unbreak TypeScript build 2026-01-02 11:02:06 +01:00
Peter Steinberger 8bd5f1b9f2 fix: improve onboarding allowlist + Control UI link 2026-01-02 10:57:04 +01:00
Peter Steinberger 71b0dcc922 Merge pull request #100 from steipete/feat/trello-skill
feat(skills): add Trello skill for board/list/card management
2026-01-02 10:47:45 +01:00
Peter Steinberger 1bf7d2f3bd docs: update trello skill requirements 2026-01-02 10:47:31 +01:00
Peter Steinberger 87127fd133 fix: refine web chat session selector 2026-01-02 10:40:24 +01:00
Peter Steinberger e85c15d178 docs: note mac app rebuilds need local 2026-01-02 10:38:18 +01:00
Peter Steinberger 0f56dce748 feat: add discord dm/guild allowlists 2026-01-02 10:32:21 +01:00
Peter Steinberger d2e2077ada fix: add top padding before first chat message 2026-01-02 10:23:40 +01:00
Peter Steinberger 9adbf47773 refactor: normalize group session keys 2026-01-02 10:14:58 +01:00
Peter Steinberger e5ee041d4e feat(skills): add Trello skill for board/list/card management 2026-01-02 08:37:15 +00:00
Shadow 63a46a85f6 feat: pass discord id to clawd so it can ping users 2026-01-01 23:30:03 -06:00
Jefferson Nunn fe87d6d8be feat(macOS): add gateway password auth support and fix Swift 6.2 concurrency
- Add CLAWDIS_GATEWAY_PASSWORD to launchd plist environment
- Read password from gateway.remote.password config in client
- Fix Swift 6.2 sending parameter violations in config save functions
- Add password parameter to GatewayConnection.Config type
- GatewayChannel now sends password in connect auth params
- GatewayEndpointStore and GatewayLaunchAgentManager read password from config
- CLI gateway client reads password from remote config and env
2026-01-01 21:34:46 -06:00
jeffersonwarriorandClaude 9387ecf043 fix(macos): support password auth mode for gateway connections
GatewayChannel now sends both 'token' and 'password' fields in the auth
payload to support both authentication modes. Gateway checks the field
matching its auth.mode configuration ('token' or 'password').

Also adds config file password fallback for remote mode, allowing
gateway password to be configured in ~/.clawdis/clawdis.json without
requiring environment variables.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-01 21:26:37 -06:00
Peter Steinberger 35582cfe8a docs: fix broken clawd.md link in index 2026-01-02 02:45:01 +00:00
Peter Steinberger 76e24653e9 fix(media): preserve GIF animation, skip JPEG optimization
- Skip JPEG optimization for image/gif content type (both local and URL)
- Preserves animation in uploaded GIFs to Discord/other providers
- Added tests for GIF preservation from local files and URLs
- Updated changelog
2026-01-02 00:56:04 +00:00
Peter Steinberger 4c2812b429 fix: refine HEARTBEAT_OK handling 2026-01-02 01:42:27 +01:00
James Groat 7154bc6857 fix(cron): prevent every schedule from firing in infinite loop
When anchorMs is not provided (always in production), the schedule
computed nextRunAtMs as nowMs, causing jobs to fire immediately and
repeatedly instead of at the configured interval.

- Change nowMs <= anchor to nowMs < anchor to prevent early return
- Add Math.max(1, ...) to ensure steps is always at least 1
- Add test for anchorMs not provided case
2026-01-01 17:30:05 -07:00
Peter Steinberger c31070db24 style: apply biome formatting 2026-01-02 01:29:05 +01:00
Peter Steinberger 336048441c docs: add imessage rpc and groups docs 2026-01-02 01:19:40 +01:00
Peter Steinberger cbac34347b feat: add imessage rpc adapter 2026-01-02 01:19:40 +01:00
Peter Steinberger 3ee27a00c7 docs(changelog): note log prefix cleanup 2026-01-02 00:15:03 +00:00
Peter Steinberger 4ec020a86d fix(logging): trim provider log prefixes 2026-01-02 00:15:01 +00:00
Peter Steinberger 464dabdc16 docs: default discord reactions to on 2026-01-02 01:11:04 +01:00
Peter Steinberger c0976ec099 fix(gateway): stream chat events for agent runs 2026-01-02 01:04:59 +01:00
Peter Steinberger 7f3113b8d4 feat: add discord reaction tool 2026-01-02 00:29:32 +01:00
Peter Steinberger 9180cbe821 fix: keep chat scrolled to bottom on session switch 2026-01-02 00:21:48 +01:00
Peter Steinberger c5daa754ff chore: refresh a2ui bundle hash 2026-01-02 00:17:59 +01:00
Peter Steinberger 23a29216d3 fix: allow remote gateway password config 2026-01-02 00:17:54 +01:00
Peter Steinberger 8a2168ecf6 style: fix swiftlint warnings 2026-01-02 00:17:49 +01:00
Peter Steinberger 38d8a669b4 fix: add discord mention context history 2026-01-01 23:58:35 +01:00
Peter Steinberger 06e379a239 fix: suppress stray HEARTBEAT_OK replies 2026-01-01 23:53:29 +01:00
Peter Steinberger 7c0379ce05 feat: add recent session switchers 2026-01-01 23:50:26 +01:00
Peter Steinberger c7c13f2d5e fix(gateway): read CLAWDIS_GATEWAY_PASSWORD from env
The CLI client (callGateway) now reads password from:
1. opts.password (explicit parameter)
2. CLAWDIS_GATEWAY_PASSWORD env var (NEW)
3. remote.password from config

This allows CLI commands like doctor/health to authenticate
without needing a --password flag when the env var is set.

Fixes auth issues for users with password-protected gateways.
2026-01-01 22:40:36 +00:00
Peter Steinberger 6df9b3f38c docs: update changelog 2026-01-01 23:30:02 +01:00
Peter Steinberger ca81d94b90 feat(cli): hint gateway reachability for local/remote 2026-01-01 23:30:02 +01:00
Peter Steinberger a39ef7181d feat(cli): add provider setup primers 2026-01-01 23:22:52 +01:00
Peter Steinberger 93b7e3431b docs: update changelog 2026-01-01 23:22:52 +01:00
Peter Steinberger dd02cc0747 docs: update changelog 2026-01-01 23:19:30 +01:00
Peter Steinberger 867883453e fix(cli): allow skipping skill dependency installs 2026-01-01 23:19:26 +01:00
Peter Steinberger a68784c319 docs: update changelog 2026-01-01 23:16:42 +01:00
Peter Steinberger 46c763410f fix(cli): colorize provider status note 2026-01-01 23:16:36 +01:00
Peter Steinberger 815d4572f6 feat(cli): explain Tailscale exposure options 2026-01-01 23:16:28 +01:00
Peter Steinberger 279a191b86 fix(macos): colorize provider status subtitles 2026-01-01 23:16:22 +01:00
Peter Steinberger f0da42917b feat(macos): verify Claude OAuth in onboarding 2026-01-01 23:16:15 +01:00
Peter Steinberger 6e87fd2d4c docs: update changelog 2026-01-01 22:55:25 +01:00
Peter Steinberger fbf5efb570 feat(process): support env overrides in exec 2026-01-01 22:55:21 +01:00
Peter Steinberger 1a3323a261 fix(cli): improve skill install failure output 2026-01-01 22:55:15 +01:00
Peter Steinberger b858fdd755 feat(macos): show skills in onboarding 2026-01-01 22:55:10 +01:00
Peter Steinberger 0aff827414 fix: preserve webchat run ordering 2026-01-01 22:46:43 +01:00
Peter Steinberger bd8a0a9f8f feat: add remote CDP browser support 2026-01-01 22:44:52 +01:00
Peter Steinberger 73d0e2cb81 fix: gate skills by OS 2026-01-01 22:25:37 +01:00
Peter Steinberger 47f816696c fix: refine A2UI status HUD styling 2026-01-01 20:47:51 +00:00
Peter Steinberger 1cf455e91c fix: use brew installer for imsg skill 2026-01-01 21:41:39 +01:00
Peter Steinberger 952c8c2d64 fix: improve canvas debug status in remote mode 2026-01-01 20:41:13 +00:00
Peter Steinberger dce3bf01fd build: refresh a2ui bundle hash 2026-01-01 20:41:09 +00:00
Peter Steinberger 7b1687d7e5 fix: resolve macOS config store concurrency 2026-01-01 21:31:37 +01:00
Peter Steinberger 9ad6863567 docs: trim changelog 2026-01-01 21:31:13 +01:00
Peter Steinberger 4c1424bb83 chore: fix lint warnings 2026-01-01 21:25:29 +01:00
Peter Steinberger c7364de2f0 fix: align telegram token resolution 2026-01-01 21:22:59 +01:00
Peter Steinberger e0043906be docs: add Discord badge 2026-01-01 21:22:00 +01:00
Peter Steinberger eda9fb5522 feat(skills): add things-mac 2026-01-01 21:12:37 +01:00
Peter Steinberger 8a775144bf docs: update changelog 2026-01-01 21:09:36 +01:00
Peter Steinberger 9b65534561 test: harden wizard e2e flow 2026-01-01 21:09:32 +01:00
Peter Steinberger f6c0618596 fix: improve web chat scroll and text 2026-01-01 21:09:28 +01:00
Peter Steinberger 15fd030fa4 docs: refresh onboarding wizard docs 2026-01-01 21:09:24 +01:00
Peter Steinberger 693be03dcc test: cover remote config routing 2026-01-01 20:29:53 +01:00
Peter Steinberger 6e3cb34024 chore: pin ElevenLabsKit + wizard note 2026-01-01 20:19:00 +01:00
Peter Steinberger bd7cd33b02 feat: add remote gateway client config 2026-01-01 20:10:50 +01:00
Peter Steinberger a72fdf7c26 feat: expand wizard setup flow 2026-01-01 19:14:14 +01:00
Peter Steinberger 850cbfe369 fix: route macOS remote config via gateway 2026-01-01 18:58:41 +01:00
Peter Steinberger 351db0632d fix(signal): map stderr INFO to log 2026-01-01 17:30:51 +00:00
Peter Steinberger d642e90cdd style: format onboarding commands 2026-01-01 17:30:51 +00:00
Peter Steinberger c454f7ac0d fix: detect bun relay assets 2026-01-01 18:30:16 +01:00
Peter Steinberger b5b47d7273 docs: update changelog 2026-01-01 17:24:42 +00:00
Peter Steinberger 7c2c541729 feat: expand onboarding wizard 2026-01-01 18:23:59 +01:00
Peter Steinberger f10abc8ee0 fix: narrow onboarding prompt types 2026-01-01 17:14:02 +00:00
Peter Steinberger 8ea27968d7 style: apply biome formatting 2026-01-01 17:06:47 +00:00
Peter Steinberger 956db9c182 fix: keep pi-ai tool types for published sdk 2026-01-01 17:02:02 +00:00
Peter Steinberger 3eb3f38adf test: add onboarding e2e harness 2026-01-01 18:01:42 +01:00
Peter Steinberger 35b66e5ad1 feat: add onboarding wizard 2026-01-01 17:58:07 +01:00
Peter Steinberger d83ea305b5 fix: satisfy bun biome formatting 2026-01-01 16:54:46 +00:00
Peter Steinberger c1d8508748 fix: clean up pi-agent-core lint 2026-01-01 16:51:08 +00:00
Peter Steinberger dc8f2bda2a fix: restart via systemd on linux 2026-01-01 17:48:28 +01:00
Peter Steinberger f0f5acfa42 fix: update pi-agent-core integration 2026-01-01 16:46:40 +00:00
Peter Steinberger 4e00edf8a7 docs: update changelog for macOS rpath fix 2026-01-01 17:44:53 +01:00
Petter Blomberg 02d5c00873 macOS: move rpath configuration to build step for reliability 2026-01-01 17:44:39 +01:00
Petter Blomberg 17009d28cf build: fix hardcoded dependency path for ElevenLabsKit 2026-01-01 17:43:27 +01:00
Peter Steinberger 325a6a4e02 docs: update changelog for chat duplicate fix 2026-01-01 17:42:30 +01:00
Marc Beaupre b51b24955c fix(chat): clear input immediately after send to prevent duplicate messages
Two issues were causing the input field to retain text after sending:

1. ChatComposer's NSViewRepresentable was skipping all updates while the
   text view was first responder. Now it allows clearing (empty binding)
   even during editing, only skipping other updates to avoid cursor jumps.

2. ChatViewModel cleared input after awaiting the network response, leaving
   text visible during the round trip. Now clears immediately after capturing
   the message content, before the async send.

Together these prevent users from accidentally re-sending messages when
the input appeared unchanged after pressing Enter.
2026-01-01 17:42:05 +01:00
Peter Steinberger a954aaa507 docs: thank contributor for macOS device resource fix 2026-01-01 17:39:54 +01:00
Petter Blomberg ad475239a5 fix(macos): prioritize main bundle for device resources to prevent crash 2026-01-01 17:39:33 +01:00
Peter Steinberger 5e280674f9 docs: require Xcode 26.2+ 2026-01-01 17:38:16 +01:00
Petter Blomberg 6cdfd143b0 docs: add macOS developer setup and troubleshooting guides 2026-01-01 17:37:19 +01:00
Petter Blomberg da454fa376 build: update A2UI bundle hash 2026-01-01 17:37:19 +01:00
Peter Steinberger 358dd4f791 merge: fix/codesign-adhoc 2026-01-01 17:34:46 +01:00
Peter Steinberger 2401abe17e docs: update changelog for codesign fix 2026-01-01 17:30:22 +01:00
Peter Steinberger 56ea6b6e43 fix: align tool schemas and health snapshot 2026-01-01 17:30:19 +01:00
Peter Steinberger 04691ed598 chore: apply biome formatting 2026-01-01 17:30:15 +01:00
William StockandClaude Opus 4.5 7366b55b14 docs: Add manual OAuth setup for remote/headless deployments
Expand "Remote mode note" section with:
- Exact oauth.json format required (access, refresh, expires)
- Note that auto-import doesn't work with Claude Code credentials
- jq script to convert Claude Code credentials to Clawdis format

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 17:21:27 +01:00
Peter Steinberger a248bea50f chore(browser): format CDP helpers 2026-01-01 16:19:37 +00:00
Peter Steinberger c8c84bc419 test(browser): fix chrome reachability mock 2026-01-01 16:16:55 +00:00
Peter Steinberger 5f990fb3a2 docs: note browser resiliency and reset 2026-01-01 16:15:17 +00:00
Peter Steinberger 538c1eb660 fix(browser): harden CDP readiness 2026-01-01 16:15:12 +00:00
Peter Steinberger 9f704d7aa7 docs: note macos app logging menu icon 2026-01-01 17:12:49 +01:00
Peter Steinberger a5777300d8 fix(macos): add icon to app logging menu 2026-01-01 16:48:17 +01:00
Peter Steinberger 57e1362344 docs(signal): explain bot-number setup 2026-01-01 15:37:45 +00:00
Peter Steinberger c1ccbd58f5 fix(signal): stabilize daemon + add signal delivery 2026-01-01 15:31:41 +00:00
Peter Steinberger 09a2ab420b style: biome formatting 2026-01-01 15:31:36 +00:00
Peter Steinberger 596770942a feat: add Signal provider support 2026-01-01 15:43:15 +01:00
Petter Blomberg fe5e58af91 scripts: fix ad-hoc signing crashes and bash unbound variable error 2026-01-01 15:29:01 +01:00
Peter Steinberger 0a4c2f91f5 fix: add bottom padding to macos web chat 2026-01-01 13:20:27 +01:00
Peter Steinberger 5b33a7dcbe fix: polish macos web chat composer 2026-01-01 12:49:05 +01:00
Peter Steinberger c7e2b1230c fix: make composer pill full-width 2026-01-01 12:18:18 +01:00
Peter Steinberger bdf6a23de9 fix: polish web chat empty/error state 2026-01-01 11:40:11 +01:00
Peter Steinberger 1a539b9830 fix(macos): restore swift test build 2026-01-01 11:05:14 +01:00
Peter Steinberger 3addd3420b fix: tidy web chat composer layout 2026-01-01 11:05:14 +01:00
Peter Steinberger 6ea10dd153 fix: allow direct file input uploads 2026-01-01 09:44:29 +00:00
Peter Steinberger bf0bee58b3 fix: improve browser upload triggering 2026-01-01 09:35:20 +00:00
Peter Steinberger fbcbc60e85 feat: unify skills config 2026-01-01 10:07:31 +01:00
Peter Steinberger 0a9f06d60f docs: annotate nix path resolution 2026-01-01 09:30:12 +01:00
Peter Steinberger f6956320f9 feat: centralize config paths and expose in snapshot 2026-01-01 09:22:37 +01:00
Peter Steinberger 20bc323963 docs: note nix support 2026-01-01 09:17:24 +01:00
Peter Steinberger bcead5f0f4 fix: honor nix config overrides in mac app 2026-01-01 09:17:21 +01:00
Peter Steinberger cf3049ae34 Merge pull request #40 from joshp123/upstream-preview-nix-2025-12-20
Nix mode support + macOS Info.plist template
2026-01-01 09:15:41 +01:00
Peter Steinberger ad9a9d8d35 Merge remote-tracking branch 'origin/main' into upstream-preview-nix-2025-12-20 2026-01-01 09:15:28 +01:00
Peter Steinberger 14e9077584 chore: add bench-model script 2026-01-01 08:59:31 +01:00
Peter Steinberger 43cf526b5f docs: thank contributor for PR #64 2026-01-01 08:59:24 +01:00
Peter Steinberger 2d5c401d11 fix: prefer module bundle for device models 2026-01-01 08:58:54 +01:00
Peter Steinberger 78cf68549f Merge pull request #64 from mbelinky/fix-instances-crash
Fix Instances crash by bundling device model resources
2026-01-01 08:58:35 +01:00
Peter Steinberger dececccd8e docs: thank contributor for PR #65 2026-01-01 08:55:51 +01:00
Mariano Belinky 941ad27551 Bundle Control UI in Mac app 2026-01-01 08:55:09 +01:00
Peter Steinberger 24e95ab38e docs: update changelog for PR #66 2026-01-01 08:37:49 +01:00
Mariano Belinky c4de0b8255 Use user home for pnpm path 2026-01-01 08:35:54 +01:00
Peter Steinberger 7baaca4a76 docs: add model latency bench notes 2025-12-31 22:39:42 +01:00
Mariano Belinky ea248f6743 Fix device model resources for Instances 2025-12-31 16:45:35 +01:00
Peter Steinberger f03605d8ae test: add minimax live test 2025-12-31 16:31:23 +01:00
Peter Steinberger 0babf08926 chore: add mac app logging coverage 2025-12-31 16:28:51 +01:00
Peter Steinberger 6517b05abe feat: add swift-log app logging controls 2025-12-31 16:03:18 +01:00
Peter Steinberger fa91b5fd03 docs: update changelog for Android chat bubble 2025-12-31 12:50:34 +01:00
Manuel Jiménez Torres f831ccfc63 fix(android): wrong text color in user chat bubbles 2025-12-31 12:48:59 +01:00
Peter Steinberger 12084fc4f9 test: extend Z.AI live test timeout 2025-12-31 12:43:34 +01:00
Peter Steinberger 21237dae98 feat: add Z.AI env support and live test 2025-12-31 11:36:57 +01:00
Peter Steinberger 4bdc25d072 docs: link Anthropic OAuth setup 2025-12-31 11:35:42 +01:00
Peter Steinberger 2f55abace2 fix: add brew installer for ordercli skill 2025-12-31 04:52:40 +01:00
Peter Steinberger 3213e5df2d feat: add gifgrep skill 2025-12-31 04:52:37 +01:00
Peter Steinberger 7e40147aa3 fix: gate web chat/talk on mobile nodes 2025-12-30 22:05:17 +01:00
Peter Steinberger a2a26b26fb fix: satisfy swiftformat in chat view 2025-12-30 20:41:12 +01:00
Peter Steinberger b3cf07d6cb feat: add ui theme toggle 2025-12-30 20:25:58 +01:00
Peter Steinberger ed76cd7574 fix: restore talk orb hit testing 2025-12-30 20:25:52 +01:00
Peter Steinberger 01b8a71ee6 docs: clarify browser wait guidance 2025-12-30 19:22:38 +00:00
Peter Steinberger cc86bbf27d feat: add food-order skill 2025-12-30 15:43:13 +01:00
Peter Steinberger 42cbb11de8 build: update a2ui bundle 2025-12-30 14:43:34 +01:00
Peter Steinberger 52303e8eda docs: update changelog for status pill 2025-12-30 14:39:33 +01:00
Peter Steinberger cf903be4a7 fix: avoid duplicate gateway reconnecting pill 2025-12-30 14:37:59 +01:00
Peter Steinberger 6306786645 fix: allow mp3 fallback result 2025-12-30 14:35:53 +01:00
Peter Steinberger d7b267843e fix: fallback mp3 when pcm blocked 2025-12-30 14:32:47 +01:00
Peter Steinberger 3aefe375c1 chore: update deps and add control ui routing tests 2025-12-30 14:30:46 +01:00
Peter Steinberger 3d6cc435ef fix: hop audio to main actor 2025-12-30 14:22:03 +01:00
Peter Steinberger 973bd3a427 fix: improve talk overlay input + drag 2025-12-30 14:18:51 +01:00
Peter Steinberger 7d1ec51df5 fix: modernize chat scroll position 2025-12-30 13:52:12 +01:00
Peter Steinberger 9fb74399c8 refactor: inject audio players 2025-12-30 13:46:14 +01:00
Peter Steinberger bc0a6fffd1 fix: tighten macOS menu device rows 2025-12-30 13:31:11 +01:00
Peter Steinberger fa85dd6527 docs: note macOS menu layout 2025-12-30 12:57:10 +01:00
Peter Steinberger 73d595eecc chore: sync local changes 2025-12-30 12:53:17 +01:00
Peter Steinberger 3bf8b9ccf4 fix: default android talk pcm_24000 2025-12-30 12:52:56 +01:00
Peter Steinberger 83262a67b1 refactor: extract elevenlabs kit 2025-12-30 12:48:09 +01:00
Peter Steinberger 66952a682d test: add pcm streaming smoke 2025-12-30 12:27:06 +01:00
Peter Steinberger 9df22c0093 fix: address talk streaming build 2025-12-30 12:20:32 +01:00
Peter Steinberger 27adfb76fa fix: stream elevenlabs tts playback 2025-12-30 12:17:40 +01:00
Peter Steinberger 9c532eac07 feat(talk): pause + drag overlay orb 2025-12-30 11:37:52 +01:00
Peter Steinberger 2814815312 feat: add talk voice alias map 2025-12-30 11:35:29 +01:00
Peter Steinberger ab27586674 test: cover external chat completion 2025-12-30 11:23:45 +01:00
Peter Steinberger 2749c5cac3 fix: clear external streaming bubbles 2025-12-30 11:21:57 +01:00
Peter Steinberger 715cf311df fix(ui): move mac talk orb to corner 2025-12-30 11:20:14 +01:00
Peter Steinberger 312443235d fix(ios): unblock device builds 2025-12-30 11:16:15 +01:00
Peter Steinberger 0d95d63258 fix(macos): await-safe session key selection 2025-12-30 11:07:34 +01:00
Peter Steinberger f86772f26c fix(talk): harden TTS + add system fallback 2025-12-30 07:40:02 +01:00
Peter Steinberger a7617e4d79 fix(ui): refine talk overlays 2025-12-30 06:47:35 +01:00
Peter Steinberger 7612a83fa2 fix(talk): align sessions and chat UI 2025-12-30 06:47:19 +01:00
Peter Steinberger afbd18e8df fix(talk): harden playback, interrupts, and timeouts 2025-12-30 06:05:43 +01:00
Peter Steinberger be2bc61d38 fix(talk): hard-timeout ElevenLabs synthesis 2025-12-30 05:46:47 +01:00
Peter Steinberger dcee8beb99 style: biome format gateway server tests 2025-12-30 05:34:53 +01:00
Peter Steinberger fb8f72d5a9 feat(ui): add centered talk orb 2025-12-30 05:27:29 +01:00
Peter Steinberger b3f2416a09 test: reduce flaky timeouts 2025-12-30 05:27:18 +01:00
Peter Steinberger b5ae2ccc3c fix(voice): sync talk mode chat events 2025-12-30 05:27:11 +01:00
Peter Steinberger 05efc3eace fix: avoid iOS talk mode audio tap crash 2025-12-30 04:52:57 +01:00
Peter Steinberger 24f8ff7548 chore(protocol): regenerate Swift gateway models 2025-12-30 04:42:08 +01:00
Peter Steinberger c0c6782a17 fix(android): stabilize BridgeSession shutdown 2025-12-30 04:42:02 +01:00
Peter Steinberger d2ac672f47 feat: add ui.seamColor accent 2025-12-30 04:14:36 +01:00
Peter Steinberger e3d8d5f300 fix(macos): prevent Talk Mode audio hang 2025-12-30 04:14:16 +01:00
Peter Steinberger c5d5c9fcb5 fix: make android canvas background visible 2025-12-30 04:02:52 +01:00
Peter Steinberger 2e040ee07a fix: brighten android canvas 2025-12-30 03:58:18 +01:00
Peter Steinberger 9846c46434 fix: tag A2UI platform and boost Android canvas 2025-12-30 03:49:24 +01:00
Peter Steinberger 5c7c1af44e fix: android talk timestamp parsing 2025-12-30 02:05:14 +01:00
Peter Steinberger e119a82334 feat: talk mode key distribution and tts polling 2025-12-30 01:57:58 +01:00
Peter Steinberger 02db68aa67 fix(macos): hide Restart Gateway when remote 2025-12-30 01:57:58 +01:00
Peter Steinberger 10e1e7fd44 chore: apply biome formatting 2025-12-30 00:16:07 +00:00
Peter Steinberger 7aabe73521 chore: sync pending changes 2025-12-30 00:59:30 +01:00
Peter Steinberger 37f85bb2d1 fix: expand talk overlay bounds 2025-12-30 00:58:58 +01:00
Peter Steinberger 39fccc3699 fix: talk overlay + elevenlabs defaults 2025-12-30 00:51:17 +01:00
Peter Steinberger 53eccc1c1e fix: wire talk menu + mac build 2025-12-30 00:17:10 +01:00
Peter Steinberger c56292a6ec feat: move talk mode to overlay button 2025-12-30 00:01:21 +01:00
Peter Steinberger 857cd6a28a fix: align ios lint and android build 2025-12-29 23:45:58 +01:00
Peter Steinberger 303954ae8c feat: extend status activity indicators 2025-12-29 23:42:22 +01:00
Peter Steinberger 3c338d1858 fix: adjust android talk parser for kotlin json 2025-12-29 23:26:38 +01:00
Peter Steinberger 20d7882033 feat: add talk mode across nodes 2025-12-29 23:21:05 +01:00
Peter Steinberger 6927b0fb8d fix: align camera payload caps 2025-12-29 23:20:55 +01:00
Peter Steinberger 6e83f95c83 fix: clamp tool images to 5MB 2025-12-29 22:13:39 +00:00
Peter Steinberger 8f0c8a6561 fix: cap camera snap payload size 2025-12-29 23:12:20 +01:00
Peter Steinberger a61b7056d5 feat: surface camera activity in status pill 2025-12-29 23:12:03 +01:00
Peter Steinberger f41ade9417 feat(skills): add obsidian skill 2025-12-29 22:51:42 +01:00
Peter Steinberger b0396e196f fix: refresh bridge tokens and enrich node settings 2025-12-29 22:11:12 +01:00
Peter Steinberger cf42fabfd8 test: add ios swift testing + android kotest 2025-12-29 21:10:44 +01:00
Peter Steinberger 52263bd5a3 fix: avoid cli gateway close race 2025-12-29 20:45:50 +01:00
Peter Steinberger 24151a2028 fix: mark screen recorder sendable 2025-12-29 20:28:06 +01:00
Peter Steinberger c11e2d9e5e fix: avoid self capture in ReplayKit start 2025-12-29 20:26:49 +01:00
Peter Steinberger a8c9b2810b fix: align ReplayKit stopCapture call 2025-12-29 20:25:44 +01:00
Peter Steinberger 7a849ab7d1 fix: isolate ReplayKit capture state 2025-12-29 20:24:34 +01:00
Peter Steinberger c14d738d37 fix: avoid screen recorder data races 2025-12-29 20:22:26 +01:00
Peter Steinberger 65478a6ff3 fix: avoid main-actor stopCapture error 2025-12-29 20:20:14 +01:00
Peter Steinberger 41be9232fe fix: prevent iOS screen capture crash 2025-12-29 20:10:36 +01:00
Peter Steinberger 653932e50d fix: show connected nodes only 2025-12-29 18:35:52 +01:00
Peter Steinberger 09ef991e1a chore: harden restart script 2025-12-29 18:09:27 +01:00
Josh Palmer 0f7029583c macOS: load device models from bundle resources 2025-12-29 17:49:13 +01:00
Josh Palmer 10eced9971 fix: use telegram token file for sends and guard console EPIPE 2025-12-29 17:49:13 +01:00
Josh Palmer 1d8b47785c feat(macos): add current TeamID to Peekaboo allowlist
Problem: The bridge only accepts the upstream TeamID, so packaged builds signed locally (Nix/CI) can’t use the bridge even though they are the same app.

Fix: Include the running app’s TeamID (from its code signature) in the allowlist.

Safety: TeamID gating remains; this just adds the app’s own TeamID to preserve permissions/automation in reproducible installs.
2025-12-29 17:49:13 +01:00
Josh Palmer ced271bec1 chore(macos): harden mktemp templates in codesign 2025-12-29 17:49:13 +01:00
Josh Palmer 5d19afd422 feat: improve health checks (telegram tokenFile + hints) 2025-12-29 17:49:13 +01:00
Josh Palmer b7363f7c18 feat: Nix mode config, UX, onboarding, SwiftPM plist, docs 2025-12-29 17:49:13 +01:00
Peter Steinberger aa2700ffa7 chore: set ios signing team for device builds 2025-12-29 17:38:21 +01:00
Peter Steinberger 510e2a1d17 fix: menu devices list 2025-12-29 17:31:23 +01:00
Peter Steinberger ebfe55f909 fix: enable canvas webview scrolling on mobile nodes 2025-12-29 17:13:31 +01:00
Peter Steinberger 26fa9dea97 chore: bump version to 2.0.0-beta5 2025-12-28 14:38:48 +00:00
Peter Steinberger 3bb4c0c237 fix: report macos product version in presence 2025-12-28 14:34:07 +00:00
Peter Steinberger 255a875a2a chore: refresh a2ui bundle hash 2025-12-28 12:06:48 +00:00
Peter Steinberger 2b5f3f1361 docs: clarify watchdog reconnect note 2025-12-28 12:05:06 +00:00
Peter Steinberger eb158545fc fix: force web reconnect on stalled close 2025-12-28 12:04:20 +00:00
Peter Steinberger cade7b1132 docs: clarify gateway readiness in changelog 2025-12-28 10:30:40 +00:00
Peter Steinberger d529736597 fix(macos): fully stop Voice Wake runtime when disabled 2025-12-28 10:17:30 +00:00
Peter Steinberger 8dfc031c4d fix: start gateway before control channel 2025-12-28 09:24:43 +00:00
Peter Steinberger 91c9859000 fix: harden heartbeat acks + gateway reconnect 2025-12-27 20:02:27 +00:00
Peter Steinberger 3a485a14a4 fix: skip whatsapp heartbeat when provider inactive 2025-12-27 19:34:10 +00:00
Peter Steinberger a61c27c4d0 fix: correct beta3 appcast URL 2025-12-27 20:00:08 +01:00
431 changed files with 35467 additions and 3849 deletions

No files matched your search

+48
View File
@@ -0,0 +1,48 @@
.git
.worktrees
.bun-cache
.bun
.tmp
**/.tmp
.DS_Store
**/.DS_Store
*.png
*.jpg
*.jpeg
*.webp
*.gif
*.mp4
*.mov
*.wav
*.mp3
node_modules
**/node_modules
.pnpm-store
**/.pnpm-store
.turbo
**/.turbo
.cache
**/.cache
.next
**/.next
coverage
**/coverage
*.log
tmp
**/tmp
# build artifacts
dist
**/dist
apps/macos/.build
apps/ios/build
**/*.trace
# large app trees not needed for CLI build
apps/
assets/
Peekaboo/
Swabble/
Core/
Users/
vendor/
+4
View File
@@ -8,6 +8,9 @@ coverage
.worktrees/
.DS_Store
**/.DS_Store
ui/src/ui/__screenshots__/
ui/playwright-report/
ui/test-results/
# Bun build artifacts
*.bun-build
@@ -42,3 +45,4 @@ apps/ios/*.dSYM.zip
# provisioning profiles (local)
apps/ios/*.mobileprovision
.env
+6
View File
@@ -23,6 +23,7 @@
- Naming: match source names with `*.test.ts`; e2e in `*.e2e.test.ts`.
- Run `pnpm test` (or `pnpm test:coverage`) before pushing when you touch logic.
- Pure test additions/fixes generally do **not** need a changelog entry unless they alter user-facing behavior or the user asks for one.
- Mobile: before using a simulator, check for connected real devices (iOS + Android) and prefer them when available.
## Commit & Pull Request Guidelines
- Create commits with `scripts/committer "<msg>" <file...>`; avoid manual `git add`/`git commit` so staging stays scoped.
@@ -40,7 +41,11 @@
- macOS logs: use `./scripts/clawlog.sh` (aka `vtlog`) to query unified logs for subsystem `com.steipete.clawdis`; it supports follow/tail/category filters and expects passwordless sudo for `/usr/bin/log`.
- Also read the shared guardrails at `~/Projects/oracle/AGENTS.md` and `~/Projects/agent-scripts/AGENTS.MD` before making changes; align with any cross-repo rules noted there.
- SwiftUI state management (iOS/macOS): prefer the `Observation` framework (`@Observable`, `@Bindable`) over `ObservableObject`/`@StateObject`; don’t introduce new `ObservableObject` unless required for compatibility, and migrate existing usages when touching related code.
- Connection providers: when adding a new connection, update every UI surface and docs (macOS app, web UI, mobile if applicable, onboarding/overview docs) and add matching status + configuration forms so provider lists and settings stay in sync.
- **Restart apps:** “restart iOS/Android apps” means rebuild (recompile/install) and relaunch, not just kill/launch.
- **Device checks:** before testing, verify connected real devices (iOS/Android) before reaching for simulators/emulators.
- iOS Team ID lookup: `security find-identity -p codesigning -v` → use Apple Development (…) TEAMID. Fallback: `defaults read com.apple.dt.Xcode IDEProvisioningTeamIdentifiers`.
- A2UI bundle hash: `src/canvas-host/a2ui/.bundle.hash` is auto-generated; regenerate via `pnpm canvas:a2ui:bundle` (or `scripts/bundle-a2ui.sh`) instead of manual conflict resolution.
- Notary key file lives at `~/Library/CloudStorage/Dropbox/Backup/AppStore/AuthKey_NJF3NFGTS3.p8` (Sparkle keys live under `~/Library/CloudStorage/Dropbox/Backup/Sparkle`).
- **Multi-agent safety:** do **not** create/apply/drop `git stash` entries unless Peter explicitly asks (this includes `git pull --rebase --autostash`). Assume other agents may be working; keep unrelated WIP untouched and avoid cross-cutting state changes.
- **Multi-agent safety:** when Peter says "push", you may `git pull --rebase` to integrate latest changes (never discard other agents' work). When Peter says "commit", scope to your changes only. When Peter says "commit all", commit everything in grouped chunks.
@@ -49,6 +54,7 @@
- **Multi-agent safety:** running multiple agents is OK as long as each agent has its own session.
- When asked to open a “session” file, open the Pi session logs under `~/.clawdis/sessions/*.jsonl` (newest unless a specific ID is given), not the default `sessions.json`. If logs are needed from Mac Studio, SSH via Tailscale and read the same path there.
- Menubar dimming + restart flow mirrors Trimmy: use `scripts/restart-mac.sh` (kills all Clawdis variants, runs `swift build`, packages, relaunches). Icon dimming depends on MenuBarExtraAccess wiring in AppMain; keep `appearsDisabled` updates intact when touching the status item.
- Do not rebuild the macOS app over SSH; rebuilds must be run directly on the Mac.
- Never send streaming/partial replies to external messaging surfaces (WhatsApp, Telegram); only final replies should be delivered there. Streaming/tool events may still go to internal UIs/control channel.
- Voice wake forwarding tips:
- Command template should stay `clawdis-mac agent --message "${text}" --thinking low`; `VoiceWakeForwarder` already shell-escapes `${text}`. Don’t add extra quotes.
+183
View File
@@ -1,9 +1,191 @@
# Changelog
## 2.0.0-beta5 — Unreleased
### Fixed
- Media: preserve GIF animation when uploading to Discord/other providers (skip JPEG optimization for image/gif).
- Agent runtime: update pi-mono dependencies to 0.31.1 (agent-core split).
- Dependencies: bump to latest compatible versions (TypeBox, grammY, Zod, Rolldown, oxlint-tsgolint).
- Tests: cover read tool image metadata + text output.
### Breaking
- Skills config schema moved under `skills.*`:
- `skillsLoad.extraDirs` → `skills.load.extraDirs`
- `skillsInstall.*` → `skills.install.*`
- per-skill config map moved to `skills.entries` (e.g. `skills.peekaboo.enabled` → `skills.entries.peekaboo.enabled`)
- new optional bundled allowlist: `skills.allowBundled` (only affects bundled skills)
- Sessions: group keys now use `surface:group:<id>` / `surface:channel:<id>`; legacy `group:*` keys migrate on next message; `groupdm` keys are no longer recognized.
- Discord: remove legacy `discord.allowFrom`, `discord.guildAllowFrom`, and `discord.requireMention`; use `discord.dm` + `discord.guilds`.
- Providers: Discord/Telegram no longer auto-start from env tokens alone; add `discord: { enabled: true }` / `telegram: { enabled: true }` to your config when using `DISCORD_BOT_TOKEN` / `TELEGRAM_BOT_TOKEN`.
- Config: remove `routing.allowFrom`; use `whatsapp.allowFrom` instead (run `clawdis doctor` to migrate).
- Config: remove `routing.groupChat.requireMention` + `telegram.requireMention`; use `whatsapp.groups`, `imessage.groups`, and `telegram.groups` defaults instead (run `clawdis doctor` to migrate).
### Features
- Discord/Telegram: add reply tags (`[[reply_to_current]]`, `[[reply_to:<id>]]`) with per-provider `replyToMode` (off|first|all) for native threaded replies.
- Talk mode: continuous speech conversations (macOS/iOS/Android) with ElevenLabs TTS, reply directives, and optional interrupt-on-speech.
- UI: add optional `ui.seamColor` accent to tint the Talk Mode side bubble (macOS/iOS/Android).
- Nix mode: opt-in declarative config + read-only settings UI when `CLAWDIS_NIX_MODE=1` (thanks @joshp123 for the persistence — earned my trust; I'll merge these going forward).
- Agent runtime: accept legacy `Z_AI_API_KEY` for Z.AI provider auth (maps to `ZAI_API_KEY`).
- Groups: add per-group mention gating defaults/overrides for Telegram/WhatsApp/iMessage via `*.groups` with `"*"` defaults; Discord now supports `discord.guilds."*"` as a default.
- Discord: add user-installed slash command handling with per-user sessions and auto-registration (#94) — thanks @thewilloftheshadow.
- Discord: add DM enable/allowlist plus guild channel/user/guild allowlists with id/name matching.
- Signal: add `signal-cli` JSON-RPC support for send/receive via the Signal provider.
- iMessage: add imsg JSON-RPC integration (stdio), chat_id routing, and group chat support.
- Chat UI: add recent-session dropdown switcher (main first) in macOS/iOS/Android + Control UI.
- UI: add Discord/Signal/iMessage connection panels in macOS + Control UI (thanks @thewilloftheshadow).
- Discord: allow agent-triggered reactions via `clawdis_discord` when enabled, and surface message ids in context.
- Discord: revamp guild routing config with per-guild/channel rules and slugged display names; add optional group DM support (default off).
- Discord: remove legacy guild/channel ignore lists in favor of per-guild allowlists (and proposed per-guild ignore lists).
- Skills: add Trello skill for board/list/card management (thanks @clawd).
- Docker: add containerized gateway/CLI setup via Dockerfile, compose, and setup script (thanks @dan-dr).
- Tests: add a Z.AI live test gate for smoke validation when keys are present.
- macOS Debug: add app log verbosity and rolling file log toggle for swift-log-backed app logs.
- CLI: add onboarding wizard (gateway + workspace + skills) with daemon installers and Anthropic/Minimax setup paths.
- CLI: add ASCII banner header to wizard entry points.
- CLI: add `configure`, `doctor`, and `update` wizards for ongoing setup, health checks, and modernization.
- CLI: add Signal CLI auto-install from GitHub releases in the wizard and persist wizard run metadata in config.
- CLI: add remote gateway client config (gateway.remote.*) with Bonjour-assisted discovery.
- Skills: allow `bun` as a node manager for skill installs.
- Skills: add `things-mac` (Things 3 CLI) for read/search plus add/update via URL scheme.
- Tests: add a Docker-based onboarding E2E harness.
- Tests: harden wizard E2E flows for reset, providers, skills, and remote non-interactive runs.
- Browser tools: add remote CDP URL support, Linux launcher options (`executablePath`, `noSandbox`), and surface `cdpUrl` in status.
- Skills: add tmux-first coding-agent skill + `requires.anyBins` gate for multi-CLI setup (thanks @sreekaransrinath).
### Fixes
- Chat UI: keep the chat scrolled to the latest message after switching sessions.
- Auto-reply: stream completed reply blocks as soon as they finish (configurable default + break); skip empty tool-only blocks unless verbose.
- Providers: make outbound text chunk limits configurable via `*.textChunkLimit` (defaults remain 4000/Discord 2000).
- CLI onboarding: persist gateway token in config so local CLI auth works; recommend auth Off unless you need multi-machine access.
- Control UI: accept a `?token=` URL param to auto-fill Gateway auth; onboarding now opens the dashboard with token auth when configured.
- Agent prompt: remove hardcoded user name in system prompt example.
- Chat UI: add extra top padding before the first message bubble in Web Chat (macOS/iOS/Android).
- Control UI: refine Web Chat session selector styling (chevron spacing + background).
- WebChat: stream live updates for sessions even when runs start outside the chat UI.
- Gateway CLI: read `CLAWDIS_GATEWAY_PASSWORD` from environment in `callGateway()` — allows `doctor`/`health` commands to auth without explicit `--password` flag.
- Gateway: add password auth support for remote gateway connections (thanks @jeffersonwarrior).
- Auto-reply: strip stray leading/trailing `HEARTBEAT_OK` from normal replies; drop short (≤ 30 chars) heartbeat acks.
- WhatsApp auto-reply: default to self-only when no config is present.
- Logging: trim provider prefix duplication in Discord/Signal/Telegram runtime log lines.
- Logging/Signal: treat signal-cli "Failed …" lines as errors in gateway logs.
- Discord: include recent guild context when replying to mentions and add `discord.historyLimit` to tune how many messages are captured.
- Discord: include author tag + id in group context `[from:]` lines for ping-ready replies (thanks @thewilloftheshadow).
- Discord: include replied-to message context when a Discord message references another message (thanks @thewilloftheshadow).
- Discord: preserve newlines when stripping reply tags from agent output.
- Gateway: fix TypeScript build by aligning hook mapping `channel` types and removing a dead Group DM branch in Discord monitor.
- Skills: switch imsg installer to brew tap formula.
- Skills: gate macOS-only skills by OS and surface block reasons in the Skills UI.
- Onboarding: show skill descriptions in the macOS setup flow and surface clearer Gateway/skills error messages.
- Onboarding: auto-verify Claude OAuth tokens, show “verified” when detected working, and avoid re-auth prompts unless verification fails.
- CLI onboarding: include exit code + a useful one-line summary when skill dependency installs fail.
- CLI onboarding: explain Tailscale exposure options (Off/Serve/Funnel) and colorize provider status (linked/configured/needs setup).
- CLI onboarding: add provider primers (WhatsApp/Telegram/Discord/Signal) incl. Discord bot token setup steps.
- CLI onboarding: allow skipping the “install missing skill dependencies” selection without canceling the wizard.
- CLI onboarding: always prompt for WhatsApp `whatsapp.allowFrom` and print (optionally open) the Control UI URL when done.
- CLI onboarding: detect gateway reachability and annotate Local/Remote choices (helps pick the right mode).
- macOS settings: colorize provider status subtitles to distinguish healthy vs degraded states.
- macOS: keep config writes on the main actor to satisfy Swift concurrency rules.
- macOS menu: show multi-line gateway error details, add an always-visible gateway row, avoid duplicate gateway status rows, suppress transient `cancelled` device refresh errors, and auto-recover the control channel on disconnect.
- macOS menu: show session last-used timestamps in the list and add recent-message previews in session submenus.
- macOS menu: tighten session row padding and time out session preview loading with cached fallback.
- macOS: log health refresh failures and recovery to make gateway issues easier to diagnose.
- macOS codesign: skip hardened runtime for ad-hoc signing and avoid empty options args (#70) — thanks @petter-b
- macOS codesign: include camera entitlement so permission prompts work in the menu bar app.
- Agent tools: map `camera.snap` JPEG payloads to `image/jpeg` to avoid MIME mismatch errors.
- Tests: cover `camera.snap` MIME mapping to prevent image/png vs image/jpeg mismatches.
- macOS camera: wait for exposure/white balance to settle before capturing a snap to avoid dark images.
- Camera snap: add `delayMs` parameter (default 2000ms on macOS) to improve exposure reliability.
- Camera: add `camera.list` and optional `deviceId` selection for snaps/clips.
- Tests: cover camera device selection params in CLI + agent tools.
- macOS packaging: move rpath config into swift build for reliability (#69) — thanks @petter-b
- macOS: prioritize main bundle for device resources to prevent crash (#73) — thanks @petter-b
- macOS remote: route settings through gateway config and avoid local config reads in remote mode.
- Telegram: align token resolution for cron/agent/CLI sends (env/config/tokenFile) to prevent isolated delivery failures (#76).
- Chat UI: clear composer input immediately and allow clear while editing to prevent duplicate sends (#72) — thanks @hrdwdmrbl
- Restart: use systemd on Linux (and report actual restart method) instead of always launchctl.
- Gateway relay: detect Bun binaries via execPath to resolve packaged assets on macOS.
- Cron: prevent `every` schedules without an anchor from firing in a tight loop (thanks @jamesgroat).
- Docs: add manual OAuth setup for remote/headless deployments (#67) — thanks @wstock
- Docs/agent tools: clarify that browser `wait` should be avoided by default and used only in exceptional cases.
- Docs: clarify self-chat mode and group mention gating config (#111) — thanks @rafaelreis-r.
- Browser tools: `upload` supports auto-click refs, direct `inputRef`/`element` file inputs, and emits input/change after `setFiles` so JS-heavy sites pick up attachments.
- Browser tools: harden CDP readiness (HTTP + WS), retry CDP connects, and auto-restart the clawd browser when the socket handshake stalls.
- Browser CLI: add `clawdis browser reset-profile` to move the clawd profile to Trash when it gets wedged.
- Signal: fix daemon startup race (wait for `/api/v1/check`) and normalize JSON-RPC `version` probe parsing.
- Docs/Signal: clarify bot-number vs personal-account setup (self-chat loop protection) and add a quickstart config snippet.
- Docs: refresh the CLI wizard guide and highlight onboarding in the README.
- CLI: tighten onboarding prompt typing to keep bun builds green.
- macOS: Voice Wake now fully tears down the Speech pipeline when disabled (cancel pending restarts, drop stale callbacks) to avoid high CPU in the background.
- macOS menu: add a Talk Mode action alongside the Open Dashboard/Chat/Canvas entries.
- macOS Debug: hide “Restart Gateway” when the app won’t start a local gateway (remote mode / attach-only).
- macOS Debug: add an icon for the App Logging submenu.
- macOS Talk Mode: orb overlay refresh, ElevenLabs request logging, API key status in settings, and auto-select first voice when none is configured.
- macOS Talk Mode: add hard timeout around ElevenLabs TTS synthesis to avoid getting stuck “speaking” forever on hung requests.
- macOS Talk Mode: avoid stuck playback when the audio player never starts (fail-fast + watchdog).
- macOS Talk Mode: fix audio stop ordering so disabling Talk Mode always stops in-flight playback.
- macOS Talk Mode: throttle audio-level updates (avoid per-buffer task creation) to reduce CPU/task churn.
- macOS Talk Mode: increase overlay window size so wave rings don’t clip; close button is hover-only and closer to the orb.
- WebChat: preserve chat run ordering per session so concurrent runs don’t strand the typing indicator.
- Talk Mode: fall back to system TTS when ElevenLabs is unavailable, returns non-audio, or playback fails (macOS/iOS/Android).
- Talk Mode: stream PCM on macOS/iOS for lower latency (incremental playback); Android continues MP3 streaming.
- Talk Mode: validate ElevenLabs v3 stability and latency tier directives before sending requests.
- iOS/Android Talk Mode: auto-select the first ElevenLabs voice when none is configured.
- ElevenLabs: add retry/backoff for 429/5xx and include content-type in errors for debugging.
- Talk Mode: align to the gateway’s main session key and fall back to history polling when chat events drop (prevents stuck “thinking” / missing messages).
- Talk Mode: treat history timestamps as seconds or milliseconds to avoid stale assistant picks (macOS/iOS/Android).
- Chat UI: clear streaming/tool bubbles when external runs finish, preventing duplicate assistant bubbles.
- Chat UI: user bubbles use `ui.seamColor` (fallback to a calmer default blue).
- Android Chat UI: use `onPrimary` for user bubble text to preserve contrast (thanks @Syhids).
- Control UI: sync sidebar navigation with the URL for deep-linking, and auto-scroll chat to the latest message.
- Control UI: disable Web Chat + Talk when no iOS/Android node is connected; refreshed Web Chat styling and keyboard send.
- Control UI: keep chat pinned to the latest message while typing/sending and restore drafts on send failures.
- Control UI: soften chat bubble text opacity for calmer readability.
- macOS Web Chat: improve empty/error states, focus message field on open, keep pill/send inside the input field, and make the composer pill edge-to-edge with square top corners.
- macOS: bundle Control UI assets into the app relay so the packaged app can serve them (thanks @mbelinky).
- Talk Mode: wait for chat history to surface the assistant reply before starting TTS (macOS/iOS/Android).
- iOS Talk Mode: fix chat completion wait to time out even if no events arrive (prevents “Thinking…” hangs).
- iOS Talk Mode: keep recognition running during playback to support interrupt-on-speech.
- iOS Talk Mode: preserve directive voice/model overrides across config reloads and add ElevenLabs request timeouts.
- iOS/Android Talk Mode: explicitly `chat.subscribe` when Talk Mode is active, so completion events arrive even if the Chat UI isn’t open.
- Chat UI: refresh history when another client finishes a run in the same session, so Talk Mode + Voice Wake transcripts appear consistently.
- Gateway: `voice.transcript` now also maps agent bus output to `chat` events, ensuring chat UIs refresh for voice-triggered runs.
- Gateway: auto-migrate legacy config on startup (non-Nix); Nix mode hard-fails with a clear error when legacy keys are present.
- iOS/Android: show a centered Talk Mode orb overlay while Talk Mode is enabled.
- Gateway config: inject `talk.apiKey` from `ELEVENLABS_API_KEY`/shell profile so nodes can fetch it on demand.
- Canvas A2UI: tag requests with `platform=android|ios|macos` and boost Android canvas background contrast.
- iOS/Android nodes: enable scrolling for loaded web pages in the Canvas WebView (default scaffold stays touch-first).
- macOS menu: device list now uses `node.list` (devices only; no agent/tool presence entries).
- macOS menu: device list now shows connected nodes only.
- macOS menu: device rows now pack platform/version on the first line, and command lists wrap in submenus.
- macOS menu: split device platform/version across first and second rows for better fit.
- macOS Canvas: show remote control status in the debug overlay and log A2UI auto-nav decisions.
- Canvas A2UI: polish the debug status HUD styling.
- iOS node: fix ReplayKit screen recording crash caused by queue isolation assertions during capture.
- iOS Talk Mode: avoid audio tap queue assertions when starting recognition.
- macOS: use $HOME/Library/pnpm for SSH PATH exports (thanks @mbelinky).
- macOS remote: harden SSH tunnel recovery/logging, honor `gateway.remote.url` port when forwarding, clarify gateway disconnect status, and add Debug menu tunnel reset.
- iOS/Android nodes: bridge auto-connect refreshes stale tokens and settings now show richer bridge/device details.
- macOS: bundle device model resources to prevent Instances crashes (thanks @mbelinky).
- iOS/Android nodes: status pill now surfaces camera activity instead of overlay toasts.
- iOS/Android/macOS nodes: camera snaps recompress to keep base64 payloads under 5 MB.
- iOS/Android nodes: status pill now surfaces pairing, screen recording, voice wake, and foreground-required states.
- iOS/Android nodes: avoid duplicating “Gateway reconnecting…” when the bridge is already connecting.
- iOS/Android nodes: Talk Mode now lives on a side bubble (with an iOS toggle to hide it), and Android settings no longer show the Talk Mode switch.
- macOS menu: top status line now shows pending node pairing approvals (incl. repairs).
- CLI: avoid spurious gateway close errors after successful request/response cycles.
- Agent runtime: clamp tool-result images to the 5MB Anthropic limit to avoid hard request rejections.
- Agent runtime: write v2 session headers so Pi session branching stays in the Clawdis sessions dir.
- Tests: add Swift Testing coverage for camera errors and Kotest coverage for Android bridge endpoints.
## 2.0.0-beta4 — 2025-12-27
### Fixes
- Package contents: include Discord/hooks build outputs in the npm tarball to avoid missing module errors.
- Heartbeat replies now drop any output containing `HEARTBEAT_OK`, preventing stray emoji/text from being delivered.
- macOS menu now refreshes the control channel after the gateway starts and shows “Connecting to gateway…” while the gateway is coming up.
- macOS local mode now waits for the gateway to be ready before configuring the control channel, avoiding false “no connection” flashes.
- WhatsApp watchdog now forces a reconnect even if the socket close event stalls (force-close to unblock reconnect loop).
- Gateway presence now reports macOS product version (via `sw_vers`) instead of Darwin kernel version.
## 2.0.0-beta3 — 2025-12-27
@@ -27,6 +209,7 @@
### Fixes
- Heartbeat replies now strip repeated `HEARTBEAT_OK` tails to avoid accidental “OK OK” spam.
- Heartbeat delivery now uses the last non-empty payload, preventing tool preambles from swallowing the final reply.
- Heartbeats now skip WhatsApp delivery when the web provider is inactive or unlinked (instead of logging “no active gateway listener”).
- Heartbeat failure logs now include the error reason instead of `[object Object]`.
- Duration strings now accept `h` (hours) where durations are parsed (e.g., heartbeat intervals).
- WhatsApp inbound now normalizes more wrapper types so quoted reply bodies are extracted reliably.
+42
View File
@@ -0,0 +1,42 @@
# Contributing to Clawdis
Welcome to the lobster tank! 🦞
## Quick Links
- **GitHub:** https://github.com/steipete/clawdis
- **Discord:** https://discord.gg/qkhbAGHRBT
- **X/Twitter:** [@steipete](https://x.com/steipete) / [@clawdbot](https://x.com/clawdbot)
## Maintainers
- **Peter Steinberger** - Benevolent Dictator
- GitHub: [@steipete](https://github.com/steipete) · X: [@steipete](https://x.com/steipete)
- **Shadow** - Discord subsystem
- GitHub: [@4shadowed](https://github.com/4shadowed) · X: [@4shad0wed](https://x.com/4shad0wed)
- **Jos** - Telegram, API, Nix mode
- GitHub: [@joshp123](https://github.com/joshp123) · X: [@jjpcodes](https://x.com/jjpcodes)
## How to Contribute
1. **Bugs & small fixes** → Open a PR!
2. **New features / architecture** → Start a [GitHub Discussion](https://github.com/steipete/clawdis/discussions) or ask in Discord first
3. **Questions** → Discord #setup-help
## Before You PR
- Test locally with your Clawdis instance
- Run linter: `npm run lint`
- Keep PRs focused (one thing per PR)
- Describe what & why
## AI/Vibe-Coded PRs Welcome! 🤖
Built with Codex, Claude, or other AI tools? **Awesome - just mark it!**
Please include in your PR:
- [ ] Mark as AI-assisted in the PR title or description
- [ ] Note the degree of testing (untested / lightly tested / fully tested)
- [ ] Include prompts or session logs if possible (super helpful!)
- [ ] Confirm you understand what the code does
AI PRs are first-class citizens here. We just want transparency so reviewers know what to look for.
+16
View File
@@ -0,0 +1,16 @@
FROM node:22-bookworm
RUN corepack enable
WORKDIR /app
COPY . .
RUN pnpm install --frozen-lockfile
RUN pnpm build
RUN pnpm ui:install
RUN pnpm ui:build
ENV NODE_ENV=production
CMD ["node", "dist/index.js"]
+24 -6
View File
@@ -11,14 +11,21 @@
<p align="center">
<a href="https://github.com/steipete/clawdis/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/steipete/clawdis/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://github.com/steipete/clawdis/releases"><img src="https://img.shields.io/github/v/release/steipete/clawdis?include_prereleases&style=for-the-badge" alt="GitHub release"></a>
<a href="https://discord.gg/clawd"><img src="https://img.shields.io/discord/1456350064065904867?label=Discord&logo=discord&logoColor=white&color=5865F2&style=for-the-badge" alt="Discord"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
</p>
**Clawdis** is a *personal AI assistant* you run on your own devices.
It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.
It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, iMessage, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.
If you want a private, single-user assistant that feels local, fast, and always-on, this is it.
Website: https://clawd.me · Docs: [`docs/index.md`](docs/index.md) · FAQ: [`docs/faq.md`](docs/faq.md) · Wizard: [`docs/wizard.md`](docs/wizard.md) · Docker (optional): [`docs/docker.md`](docs/docker.md) · Discord: https://discord.gg/clawd
Preferred setup: run the onboarding wizard (`clawdis onboard`). It walks through gateway, workspace, providers, and skills. The CLI wizard is the recommended path and works on **macOS, Windows, and Linux**.
Using Claude Pro/Max subscription? See `docs/onboarding.md` for the Anthropic OAuth setup.
```
Your surfaces
│
@@ -38,12 +45,13 @@ Your surfaces
## What Clawdis does
- **Personal assistant** — one user, one identity, one memory surface.
- **Multi-surface inbox** — WhatsApp, Telegram, Discord, WebChat, macOS, iOS.
- **Multi-surface inbox** — WhatsApp, Telegram, Discord, iMessage, WebChat, macOS, iOS. Signal support via `signal-cli` (see `docs/signal.md`). iMessage uses `imsg` (see `docs/imessage.md`).
- **Voice wake + push-to-talk** — local speech recognition on macOS/iOS.
- **Canvas** — a live visual workspace you can drive from the agent.
- **Automation-ready** — browser control, media handling, and tool streaming.
- **Local-first control plane** — the Gateway owns state, everything else connects.
- **Group chats** — mention-based by default, `/activation always|mention` per group (owner-only).
- **Nix mode** — opt-in declarative config + read-only UI when `CLAWDIS_NIX_MODE=1`.
## How it works (short)
@@ -61,6 +69,9 @@ pnpm install
pnpm build
pnpm ui:build
# Recommended: run the onboarding wizard
pnpm clawdis onboard
# Link WhatsApp (stores creds in ~/.clawdis/credentials)
pnpm clawdis login
@@ -146,7 +157,7 @@ Minimal `~/.clawdis/clawdis.json`:
```json5
{
routing: {
whatsapp: {
allowFrom: ["+1234567890"]
}
}
@@ -155,12 +166,12 @@ Minimal `~/.clawdis/clawdis.json`:
### WhatsApp
- Link the device: `pnpm clawdis login` (stores creds in `~/.clawdis/credentials`).
- Allowlist who can talk to the assistant via `routing.allowFrom`.
- Allowlist who can talk to the assistant via `whatsapp.allowFrom`.
### Telegram
- Set `TELEGRAM_BOT_TOKEN` or `telegram.botToken` (env wins).
- Optional: set `telegram.requireMention`, `telegram.allowFrom`, or `telegram.webhookUrl` as needed.
- Optional: set `telegram.groups` (with `telegram.groups."*".requireMention`), `telegram.allowFrom`, or `telegram.webhookUrl` as needed.
```json5
{
@@ -173,7 +184,7 @@ Minimal `~/.clawdis/clawdis.json`:
### Discord
- Set `DISCORD_BOT_TOKEN` or `discord.token` (env wins).
- Optional: set `discord.requireMention`, `discord.allowFrom`, or `discord.mediaMaxMb` as needed.
- Optional: set `discord.slashCommand`, `discord.dm.allowFrom`, `discord.guilds`, or `discord.mediaMaxMb` as needed.
```json5
{
@@ -205,6 +216,7 @@ Browser control (optional):
- [`docs/discovery.md`](docs/discovery.md)
- [`docs/agent.md`](docs/agent.md)
- [`docs/discord.md`](docs/discord.md)
- [`docs/wizard.md`](docs/wizard.md)
- Webhooks + external triggers: [`docs/webhook.md`](docs/webhook.md)
- Gmail hooks (email → wake): [`docs/gmail-pubsub.md`](docs/gmail-pubsub.md)
@@ -219,6 +231,12 @@ clawdis hooks gmail run
- [`docs/ios/connect.md`](docs/ios/connect.md)
- [`docs/clawdis-mac.md`](docs/clawdis-mac.md)
## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines, maintainers, and how to submit PRs.
AI/vibe-coded PRs welcome! 🤖
## Clawd
Clawdis was built for **Clawd**, a space lobster AI assistant.
+2 -2
View File
@@ -9,7 +9,7 @@
<sparkle:version>2.0.0-beta3</sparkle:version>
<sparkle:shortVersionString>2.0.0-beta3</sparkle:shortVersionString>
<sparkle:minimumSystemVersion>15.0</sparkle:minimumSystemVersion>
<enclosure url="https://github.com/steipete/clawdis/releases/download/v2.0.0-beta4/Clawdis-2.0.0-beta3.zip" length="70407960" type="application/octet-stream" sparkle:edSignature="A8ySMmbLRrpIkqkrmc9QrC+6om8Iyqray/6x/YNiJxDoJeXjp2T5t8XT0CKJeNBUlDkzIj/fwiK53v0qQ59cDQ=="/>
<enclosure url="https://github.com/steipete/clawdis/releases/download/v2.0.0-beta3/Clawdis-2.0.0-beta3.zip" length="70407960" type="application/octet-stream" sparkle:edSignature="A8ySMmbLRrpIkqkrmc9QrC+6om8Iyqray/6x/YNiJxDoJeXjp2T5t8XT0CKJeNBUlDkzIj/fwiK53v0qQ59cDQ=="/>
</item>
<item>
<title>2.0.0-beta4</title>
@@ -28,4 +28,4 @@
<enclosure url="https://github.com/steipete/clawdis/releases/download/v2.0.0-beta4/Clawdis-2.0.0-beta4.zip" length="70407894" type="application/octet-stream" sparkle:edSignature="HmuiC7TnUn80ZApnKfb6w+JGSrjc3uUOndMrtbTp42bkBSVifbttNVazqvJueGBo4MgoJV8CP+zQNzVmtVihAQ=="/>
</item>
</channel>
</rss>
</rss>
+8
View File
@@ -64,6 +64,7 @@ dependencies {
implementation("androidx.core:core-ktx:1.17.0")
implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.10.0")
implementation("androidx.activity:activity-compose:1.12.2")
implementation("androidx.webkit:webkit:1.14.0")
implementation("androidx.compose.ui:ui")
implementation("androidx.compose.ui:ui-tooling-preview")
@@ -93,4 +94,11 @@ dependencies {
testImplementation("junit:junit:4.13.2")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.10.2")
testImplementation("io.kotest:kotest-runner-junit5-jvm:6.0.7")
testImplementation("io.kotest:kotest-assertions-core-jvm:6.0.7")
testRuntimeOnly("org.junit.vintage:junit-vintage-engine:5.13.3")
}
tasks.withType<Test>().configureEach {
useJUnitPlatform()
}
@@ -23,9 +23,12 @@ class MainViewModel(app: Application) : AndroidViewModel(app) {
val statusText: StateFlow<String> = runtime.statusText
val serverName: StateFlow<String?> = runtime.serverName
val remoteAddress: StateFlow<String?> = runtime.remoteAddress
val isForeground: StateFlow<Boolean> = runtime.isForeground
val seamColorArgb: StateFlow<Long> = runtime.seamColorArgb
val cameraHud: StateFlow<CameraHudState?> = runtime.cameraHud
val cameraFlashToken: StateFlow<Long> = runtime.cameraFlashToken
val screenRecordActive: StateFlow<Boolean> = runtime.screenRecordActive
val instanceId: StateFlow<String> = runtime.instanceId
val displayName: StateFlow<String> = runtime.displayName
@@ -35,6 +38,10 @@ class MainViewModel(app: Application) : AndroidViewModel(app) {
val voiceWakeMode: StateFlow<VoiceWakeMode> = runtime.voiceWakeMode
val voiceWakeStatusText: StateFlow<String> = runtime.voiceWakeStatusText
val voiceWakeIsListening: StateFlow<Boolean> = runtime.voiceWakeIsListening
val talkEnabled: StateFlow<Boolean> = runtime.talkEnabled
val talkStatusText: StateFlow<String> = runtime.talkStatusText
val talkIsListening: StateFlow<Boolean> = runtime.talkIsListening
val talkIsSpeaking: StateFlow<Boolean> = runtime.talkIsSpeaking
val manualEnabled: StateFlow<Boolean> = runtime.manualEnabled
val manualHost: StateFlow<String> = runtime.manualHost
val manualPort: StateFlow<Int> = runtime.manualPort
@@ -95,6 +102,10 @@ class MainViewModel(app: Application) : AndroidViewModel(app) {
runtime.setVoiceWakeMode(mode)
}
fun setTalkEnabled(enabled: Boolean) {
runtime.setTalkEnabled(enabled)
}
fun connect(endpoint: BridgeEndpoint) {
runtime.connect(endpoint)
}
@@ -25,6 +25,7 @@ import com.steipete.clawdis.node.protocol.ClawdisCanvasA2UIAction
import com.steipete.clawdis.node.protocol.ClawdisCanvasA2UICommand
import com.steipete.clawdis.node.protocol.ClawdisCanvasCommand
import com.steipete.clawdis.node.protocol.ClawdisScreenCommand
import com.steipete.clawdis.node.voice.TalkModeManager
import com.steipete.clawdis.node.voice.VoiceWakeManager
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
@@ -69,7 +70,7 @@ class NodeRuntime(context: Context) {
payloadJson =
buildJsonObject {
put("message", JsonPrimitive(command))
put("sessionKey", JsonPrimitive("main"))
put("sessionKey", JsonPrimitive(mainSessionKey.value))
put("thinking", JsonPrimitive(chatThinkingLevel.value))
put("deliver", JsonPrimitive(false))
}.toString(),
@@ -84,6 +85,15 @@ class NodeRuntime(context: Context) {
val voiceWakeStatusText: StateFlow<String>
get() = voiceWake.statusText
val talkStatusText: StateFlow<String>
get() = talkMode.statusText
val talkIsListening: StateFlow<Boolean>
get() = talkMode.isListening
val talkIsSpeaking: StateFlow<Boolean>
get() = talkMode.isSpeaking
private val discovery = BridgeDiscovery(appContext, scope = scope)
val bridges: StateFlow<List<BridgeEndpoint>> = discovery.bridges
val discoveryStatusText: StateFlow<String> = discovery.statusText
@@ -94,6 +104,9 @@ class NodeRuntime(context: Context) {
private val _statusText = MutableStateFlow("Offline")
val statusText: StateFlow<String> = _statusText.asStateFlow()
private val _mainSessionKey = MutableStateFlow("main")
val mainSessionKey: StateFlow<String> = _mainSessionKey.asStateFlow()
private val cameraHudSeq = AtomicLong(0)
private val _cameraHud = MutableStateFlow<CameraHudState?>(null)
val cameraHud: StateFlow<CameraHudState?> = _cameraHud.asStateFlow()
@@ -101,12 +114,18 @@ class NodeRuntime(context: Context) {
private val _cameraFlashToken = MutableStateFlow(0L)
val cameraFlashToken: StateFlow<Long> = _cameraFlashToken.asStateFlow()
private val _screenRecordActive = MutableStateFlow(false)
val screenRecordActive: StateFlow<Boolean> = _screenRecordActive.asStateFlow()
private val _serverName = MutableStateFlow<String?>(null)
val serverName: StateFlow<String?> = _serverName.asStateFlow()
private val _remoteAddress = MutableStateFlow<String?>(null)
val remoteAddress: StateFlow<String?> = _remoteAddress.asStateFlow()
private val _seamColorArgb = MutableStateFlow(DEFAULT_SEAM_COLOR_ARGB)
val seamColorArgb: StateFlow<Long> = _seamColorArgb.asStateFlow()
private val _isForeground = MutableStateFlow(true)
val isForeground: StateFlow<Boolean> = _isForeground.asStateFlow()
@@ -120,6 +139,8 @@ class NodeRuntime(context: Context) {
_serverName.value = name
_remoteAddress.value = remote
_isConnected.value = true
_seamColorArgb.value = DEFAULT_SEAM_COLOR_ARGB
scope.launch { refreshBrandingFromGateway() }
scope.launch { refreshWakeWordsFromGateway() }
maybeNavigateToA2uiOnConnect()
},
@@ -133,12 +154,17 @@ class NodeRuntime(context: Context) {
)
private val chat = ChatController(scope = scope, session = session, json = json)
private val talkMode: TalkModeManager by lazy {
TalkModeManager(context = appContext, scope = scope).also { it.attachSession(session) }
}
private fun handleSessionDisconnected(message: String) {
_statusText.value = message
_serverName.value = null
_remoteAddress.value = null
_isConnected.value = false
_seamColorArgb.value = DEFAULT_SEAM_COLOR_ARGB
_mainSessionKey.value = "main"
chat.onDisconnected(message)
showLocalCanvasOnDisconnect()
}
@@ -163,6 +189,7 @@ class NodeRuntime(context: Context) {
val preventSleep: StateFlow<Boolean> = prefs.preventSleep
val wakeWords: StateFlow<List<String>> = prefs.wakeWords
val voiceWakeMode: StateFlow<VoiceWakeMode> = prefs.voiceWakeMode
val talkEnabled: StateFlow<Boolean> = prefs.talkEnabled
val manualEnabled: StateFlow<Boolean> = prefs.manualEnabled
val manualHost: StateFlow<String> = prefs.manualHost
val manualPort: StateFlow<Int> = prefs.manualPort
@@ -218,6 +245,13 @@ class NodeRuntime(context: Context) {
}
}
scope.launch {
talkEnabled.collect { enabled ->
talkMode.setEnabled(enabled)
externalAudioCaptureActive.value = enabled
}
}
scope.launch(Dispatchers.Default) {
bridges.collect { list ->
if (list.isNotEmpty()) {
@@ -311,6 +345,10 @@ class NodeRuntime(context: Context) {
prefs.setVoiceWakeMode(mode)
}
fun setTalkEnabled(value: Boolean) {
prefs.setTalkEnabled(value)
}
fun connect(endpoint: BridgeEndpoint) {
scope.launch {
_statusText.value = "Connecting…"
@@ -548,6 +586,7 @@ class NodeRuntime(context: Context) {
return
}
talkMode.handleBridgeEvent(event, payloadJson)
chat.handleBridgeEvent(event, payloadJson)
}
@@ -589,6 +628,25 @@ class NodeRuntime(context: Context) {
}
}
private suspend fun refreshBrandingFromGateway() {
if (!_isConnected.value) return
try {
val res = session.request("config.get", "{}")
val root = json.parseToJsonElement(res).asObjectOrNull()
val config = root?.get("config").asObjectOrNull()
val ui = config?.get("ui").asObjectOrNull()
val raw = ui?.get("seamColor").asStringOrNull()?.trim()
val sessionCfg = config?.get("session").asObjectOrNull()
val rawMainKey = sessionCfg?.get("mainKey").asStringOrNull()?.trim()
_mainSessionKey.value = rawMainKey?.takeIf { it.isNotEmpty() } ?: "main"
val parsed = parseHexColorArgb(raw)
_seamColorArgb.value = parsed ?: DEFAULT_SEAM_COLOR_ARGB
} catch (_: Throwable) {
// ignore
}
}
private suspend fun handleInvoke(command: String, paramsJson: String?): BridgeSession.InvokeResult {
if (
command.startsWith(ClawdisCanvasCommand.NamespacePrefix) ||
@@ -730,14 +788,20 @@ class NodeRuntime(context: Context) {
}
}
ClawdisScreenCommand.Record.rawValue -> {
val res =
try {
screenRecorder.record(paramsJson)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
return BridgeSession.InvokeResult.error(code = code, message = message)
}
BridgeSession.InvokeResult.ok(res.payloadJson)
// Status pill mirrors screen recording state so it stays visible without overlay stacking.
_screenRecordActive.value = true
try {
val res =
try {
screenRecorder.record(paramsJson)
} catch (err: Throwable) {
val (code, message) = invokeErrorFromThrowable(err)
return BridgeSession.InvokeResult.error(code = code, message = message)
}
BridgeSession.InvokeResult.ok(res.payloadJson)
} finally {
_screenRecordActive.value = false
}
}
else ->
BridgeSession.InvokeResult.error(
@@ -780,7 +844,7 @@ class NodeRuntime(context: Context) {
val raw = session.currentCanvasHostUrl()?.trim().orEmpty()
if (raw.isBlank()) return null
val base = raw.trimEnd('/')
return "${base}/__clawdis__/a2ui/"
return "${base}/__clawdis__/a2ui/?platform=android"
}
private suspend fun ensureA2uiReady(a2uiUrl: String): Boolean {
@@ -866,6 +930,8 @@ class NodeRuntime(context: Context) {
private data class Quad<A, B, C, D>(val first: A, val second: B, val third: C, val fourth: D)
private const val DEFAULT_SEAM_COLOR_ARGB: Long = 0xFF4F7A9A
private const val a2uiReadyCheckJS: String =
"""
(() => {
@@ -920,3 +986,12 @@ private fun JsonElement?.asStringOrNull(): String? =
is JsonPrimitive -> content
else -> null
}
private fun parseHexColorArgb(raw: String?): Long? {
val trimmed = raw?.trim().orEmpty()
if (trimmed.isEmpty()) return null
val hex = if (trimmed.startsWith("#")) trimmed.drop(1) else trimmed
if (hex.length != 6) return null
val rgb = hex.toLongOrNull(16) ?: return null
return 0xFF000000L or rgb
}
@@ -73,6 +73,9 @@ class SecurePrefs(context: Context) {
private val _voiceWakeMode = MutableStateFlow(loadVoiceWakeMode())
val voiceWakeMode: StateFlow<VoiceWakeMode> = _voiceWakeMode
private val _talkEnabled = MutableStateFlow(prefs.getBoolean("talk.enabled", false))
val talkEnabled: StateFlow<Boolean> = _talkEnabled
fun setLastDiscoveredStableId(value: String) {
val trimmed = value.trim()
prefs.edit { putString("bridge.lastDiscoveredStableId", trimmed) }
@@ -158,6 +161,11 @@ class SecurePrefs(context: Context) {
_voiceWakeMode.value = mode
}
fun setTalkEnabled(value: Boolean) {
prefs.edit { putBoolean("talk.enabled", value) }
_talkEnabled.value = value
}
private fun loadVoiceWakeMode(): VoiceWakeMode {
val raw = prefs.getString(voiceWakeModeKey, null)
val resolved = VoiceWakeMode.fromRawValue(raw)
@@ -130,20 +130,36 @@ class BridgeDiscovery(
object : NsdManager.ResolveListener {
override fun onResolveFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {}
override fun onServiceResolved(resolved: NsdServiceInfo) {
val host = resolved.host?.hostAddress ?: return
val port = resolved.port
if (port <= 0) return
override fun onServiceResolved(resolved: NsdServiceInfo) {
val host = resolved.host?.hostAddress ?: return
val port = resolved.port
if (port <= 0) return
val rawServiceName = resolved.serviceName
val serviceName = BonjourEscapes.decode(rawServiceName)
val displayName = BonjourEscapes.decode(txt(resolved, "displayName") ?: serviceName)
val id = stableId(serviceName, "local.")
localById[id] = BridgeEndpoint(stableId = id, name = displayName, host = host, port = port)
publish()
}
},
)
val rawServiceName = resolved.serviceName
val serviceName = BonjourEscapes.decode(rawServiceName)
val displayName = BonjourEscapes.decode(txt(resolved, "displayName") ?: serviceName)
val lanHost = txt(resolved, "lanHost")
val tailnetDns = txt(resolved, "tailnetDns")
val gatewayPort = txtInt(resolved, "gatewayPort")
val bridgePort = txtInt(resolved, "bridgePort")
val canvasPort = txtInt(resolved, "canvasPort")
val id = stableId(serviceName, "local.")
localById[id] =
BridgeEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
bridgePort = bridgePort,
canvasPort = canvasPort,
)
publish()
}
},
)
}
private fun publish() {
@@ -189,6 +205,10 @@ class BridgeDiscovery(
}
}
private fun txtInt(info: NsdServiceInfo, key: String): Int? {
return txt(info, key)?.toIntOrNull()
}
private suspend fun refreshUnicast(domain: String) {
val ptrName = "${serviceType}${domain}"
val ptrMsg = lookupUnicastMessage(ptrName, Type.PTR) ?: return
@@ -227,8 +247,24 @@ class BridgeDiscovery(
}
val instanceName = BonjourEscapes.decode(decodeInstanceName(instanceFqdn, domain))
val displayName = BonjourEscapes.decode(txtValue(txt, "displayName") ?: instanceName)
val lanHost = txtValue(txt, "lanHost")
val tailnetDns = txtValue(txt, "tailnetDns")
val gatewayPort = txtIntValue(txt, "gatewayPort")
val bridgePort = txtIntValue(txt, "bridgePort")
val canvasPort = txtIntValue(txt, "canvasPort")
val id = stableId(instanceName, domain)
next[id] = BridgeEndpoint(stableId = id, name = displayName, host = host, port = port)
next[id] =
BridgeEndpoint(
stableId = id,
name = displayName,
host = host,
port = port,
lanHost = lanHost,
tailnetDns = tailnetDns,
gatewayPort = gatewayPort,
bridgePort = bridgePort,
canvasPort = canvasPort,
)
}
unicastById.clear()
@@ -434,6 +470,10 @@ class BridgeDiscovery(
return null
}
private fun txtIntValue(records: List<TXTRecord>, key: String): Int? {
return txtValue(records, key)?.toIntOrNull()
}
private fun decodeDnsTxtString(raw: String): String {
// dnsjava treats TXT as opaque bytes and decodes as ISO-8859-1 to preserve bytes.
// Our TXT payload is UTF-8 (written by the gateway), so re-decode when possible.
@@ -5,6 +5,11 @@ data class BridgeEndpoint(
val name: String,
val host: String,
val port: Int,
val lanHost: String? = null,
val tailnetDns: String? = null,
val gatewayPort: Int? = null,
val bridgePort: Int? = null,
val canvasPort: Int? = null,
) {
companion object {
fun manual(host: String, port: Int): BridgeEndpoint =
@@ -16,4 +21,3 @@ data class BridgeEndpoint(
)
}
}
@@ -11,6 +11,7 @@ import kotlinx.coroutines.launch
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import com.steipete.clawdis.node.BuildConfig
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
@@ -23,6 +24,7 @@ import java.io.BufferedWriter
import java.io.InputStreamReader
import java.io.OutputStreamWriter
import java.net.InetSocketAddress
import java.net.URI
import java.net.Socket
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
@@ -75,6 +77,8 @@ class BridgeSession(
fun disconnect() {
desired = null
// Unblock connectOnce() read loop. Coroutine cancellation alone won't interrupt BufferedReader.readLine().
currentConnection?.closeQuietly()
scope.launch(Dispatchers.IO) {
job?.cancelAndJoin()
job = null
@@ -213,7 +217,17 @@ class BridgeSession(
when (first["type"].asStringOrNull()) {
"hello-ok" -> {
val name = first["serverName"].asStringOrNull() ?: "Bridge"
canvasHostUrl = first["canvasHostUrl"].asStringOrNull()?.trim()?.ifEmpty { null }
val rawCanvasUrl = first["canvasHostUrl"].asStringOrNull()?.trim()?.ifEmpty { null }
canvasHostUrl = normalizeCanvasHostUrl(rawCanvasUrl, endpoint)
if (BuildConfig.DEBUG) {
// Local JVM unit tests use android.jar stubs; Log.d can throw "not mocked".
runCatching {
android.util.Log.d(
"ClawdisBridge",
"canvasHostUrl resolved=${canvasHostUrl ?: "none"} (raw=${rawCanvasUrl ?: "none"})",
)
}
}
onConnected(name, conn.remoteAddress)
}
"error" -> {
@@ -292,6 +306,37 @@ class BridgeSession(
conn.closeQuietly()
}
}
private fun normalizeCanvasHostUrl(raw: String?, endpoint: BridgeEndpoint): String? {
val trimmed = raw?.trim().orEmpty()
val parsed = trimmed.takeIf { it.isNotBlank() }?.let { runCatching { URI(it) }.getOrNull() }
val host = parsed?.host?.trim().orEmpty()
val port = parsed?.port ?: -1
val scheme = parsed?.scheme?.trim().orEmpty().ifBlank { "http" }
if (trimmed.isNotBlank() && !isLoopbackHost(host)) {
return trimmed
}
val fallbackHost =
endpoint.tailnetDns?.trim().takeIf { !it.isNullOrEmpty() }
?: endpoint.lanHost?.trim().takeIf { !it.isNullOrEmpty() }
?: endpoint.host.trim()
if (fallbackHost.isEmpty()) return trimmed.ifBlank { null }
val fallbackPort = endpoint.canvasPort ?: if (port > 0) port else 18793
val formattedHost = if (fallbackHost.contains(":")) "[${fallbackHost}]" else fallbackHost
return "$scheme://$formattedHost:$fallbackPort"
}
private fun isLoopbackHost(raw: String?): Boolean {
val host = raw?.trim()?.lowercase().orEmpty()
if (host.isEmpty()) return false
if (host == "localhost") return true
if (host == "::1") return true
if (host == "0.0.0.0" || host == "::") return true
return host.startsWith("127.")
}
}
private fun JsonElement?.asObjectOrNull(): JsonObject? = this as? JsonObject
@@ -469,7 +469,8 @@ class ChatController(
val key = obj["key"].asStringOrNull()?.trim().orEmpty()
if (key.isEmpty()) return@mapNotNull null
val updatedAt = obj["updatedAt"].asLongOrNull()
ChatSessionEntry(key = key, updatedAtMs = updatedAt)
val displayName = obj["displayName"].asStringOrNull()?.trim()
ChatSessionEntry(key = key, updatedAtMs = updatedAt, displayName = displayName)
}
}
@@ -25,6 +25,7 @@ data class ChatPendingToolCall(
data class ChatSessionEntry(
val key: String,
val updatedAtMs: Long?,
val displayName: String? = null,
)
data class ChatHistory(
@@ -28,6 +28,7 @@ import kotlinx.coroutines.withContext
import java.io.ByteArrayOutputStream
import java.io.File
import java.util.concurrent.Executor
import kotlin.math.roundToInt
import kotlin.coroutines.resume
import kotlin.coroutines.resumeWithException
@@ -99,14 +100,36 @@ class CameraCaptureManager(private val context: Context) {
decoded
}
val out = ByteArrayOutputStream()
val jpegQuality = (quality * 100.0).toInt().coerceIn(10, 100)
if (!scaled.compress(Bitmap.CompressFormat.JPEG, jpegQuality, out)) {
throw IllegalStateException("UNAVAILABLE: failed to encode JPEG")
}
val base64 = Base64.encodeToString(out.toByteArray(), Base64.NO_WRAP)
val maxPayloadBytes = 5 * 1024 * 1024
// Base64 inflates payloads by ~4/3; cap encoded bytes so the payload stays under 5MB (API limit).
val maxEncodedBytes = (maxPayloadBytes / 4) * 3
val result =
JpegSizeLimiter.compressToLimit(
initialWidth = scaled.width,
initialHeight = scaled.height,
startQuality = (quality * 100.0).roundToInt().coerceIn(10, 100),
maxBytes = maxEncodedBytes,
encode = { width, height, q ->
val bitmap =
if (width == scaled.width && height == scaled.height) {
scaled
} else {
scaled.scale(width, height)
}
val out = ByteArrayOutputStream()
if (!bitmap.compress(Bitmap.CompressFormat.JPEG, q, out)) {
if (bitmap !== scaled) bitmap.recycle()
throw IllegalStateException("UNAVAILABLE: failed to encode JPEG")
}
if (bitmap !== scaled) {
bitmap.recycle()
}
out.toByteArray()
},
)
val base64 = Base64.encodeToString(result.bytes, Base64.NO_WRAP)
Payload(
"""{"format":"jpg","base64":"$base64","width":${scaled.width},"height":${scaled.height}}""",
"""{"format":"jpg","base64":"$base64","width":${result.width},"height":${result.height}}""",
)
}
@@ -3,6 +3,7 @@ package com.steipete.clawdis.node.node
import android.graphics.Bitmap
import android.graphics.Canvas
import android.os.Looper
import android.util.Log
import android.webkit.WebView
import androidx.core.graphics.createBitmap
import androidx.core.graphics.scale
@@ -16,6 +17,7 @@ import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import com.steipete.clawdis.node.BuildConfig
import kotlin.coroutines.resume
class CanvasController {
@@ -81,8 +83,14 @@ class CanvasController {
val currentUrl = url
withWebViewOnMain { wv ->
if (currentUrl == null) {
if (BuildConfig.DEBUG) {
Log.d("ClawdisCanvas", "load scaffold: $scaffoldAssetUrl")
}
wv.loadUrl(scaffoldAssetUrl)
} else {
if (BuildConfig.DEBUG) {
Log.d("ClawdisCanvas", "load url: $currentUrl")
}
wv.loadUrl(currentUrl)
}
}
@@ -0,0 +1,61 @@
package com.steipete.clawdis.node.node
import kotlin.math.max
import kotlin.math.min
import kotlin.math.roundToInt
internal data class JpegSizeLimiterResult(
val bytes: ByteArray,
val width: Int,
val height: Int,
val quality: Int,
)
internal object JpegSizeLimiter {
fun compressToLimit(
initialWidth: Int,
initialHeight: Int,
startQuality: Int,
maxBytes: Int,
minQuality: Int = 20,
minSize: Int = 256,
scaleStep: Double = 0.85,
maxScaleAttempts: Int = 6,
maxQualityAttempts: Int = 6,
encode: (width: Int, height: Int, quality: Int) -> ByteArray,
): JpegSizeLimiterResult {
require(initialWidth > 0 && initialHeight > 0) { "Invalid image size" }
require(maxBytes > 0) { "Invalid maxBytes" }
var width = initialWidth
var height = initialHeight
val clampedStartQuality = startQuality.coerceIn(minQuality, 100)
var best = JpegSizeLimiterResult(bytes = encode(width, height, clampedStartQuality), width = width, height = height, quality = clampedStartQuality)
if (best.bytes.size <= maxBytes) return best
repeat(maxScaleAttempts) {
var quality = clampedStartQuality
repeat(maxQualityAttempts) {
val bytes = encode(width, height, quality)
best = JpegSizeLimiterResult(bytes = bytes, width = width, height = height, quality = quality)
if (bytes.size <= maxBytes) return best
if (quality <= minQuality) return@repeat
quality = max(minQuality, (quality * 0.75).roundToInt())
}
val minScale = (minSize.toDouble() / min(width, height).toDouble()).coerceAtMost(1.0)
val nextScale = max(scaleStep, minScale)
val nextWidth = max(minSize, (width * nextScale).roundToInt())
val nextHeight = max(minSize, (height * nextScale).roundToInt())
if (nextWidth == width && nextHeight == height) return@repeat
width = min(nextWidth, width)
height = min(nextHeight, height)
}
if (best.bytes.size > maxBytes) {
throw IllegalStateException("CAMERA_TOO_LARGE: ${best.bytes.size} bytes > $maxBytes bytes")
}
return best
}
}
@@ -1,64 +1,26 @@
package com.steipete.clawdis.node.ui
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.slideInVertically
import androidx.compose.animation.slideOutVertically
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.statusBarsPadding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Error
import androidx.compose.material.icons.filled.FiberManualRecord
import androidx.compose.material.icons.filled.PhotoCamera
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableFloatStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.CameraHudKind
import com.steipete.clawdis.node.CameraHudState
import kotlinx.coroutines.delay
@Composable
fun CameraHudOverlay(
hud: CameraHudState?,
flashToken: Long,
fun CameraFlashOverlay(
token: Long,
modifier: Modifier = Modifier,
) {
Box(modifier = modifier.fillMaxSize()) {
CameraFlash(token = flashToken)
AnimatedVisibility(
visible = hud != null,
enter = slideInVertically(initialOffsetY = { -it / 2 }) + fadeIn(),
exit = slideOutVertically(targetOffsetY = { -it / 2 }) + fadeOut(),
modifier = Modifier.align(Alignment.TopStart).statusBarsPadding().padding(start = 12.dp, top = 58.dp),
) {
if (hud != null) {
Toast(hud = hud)
}
}
CameraFlash(token = token)
}
}
@@ -80,44 +42,3 @@ private fun CameraFlash(token: Long) {
.background(Color.White),
)
}
@Composable
private fun Toast(hud: CameraHudState) {
Surface(
shape = RoundedCornerShape(14.dp),
color = MaterialTheme.colorScheme.surface.copy(alpha = 0.85f),
tonalElevation = 2.dp,
shadowElevation = 8.dp,
) {
Row(
modifier = Modifier.padding(vertical = 10.dp, horizontal = 12.dp),
verticalAlignment = Alignment.CenterVertically,
) {
when (hud.kind) {
CameraHudKind.Photo -> {
Icon(Icons.Default.PhotoCamera, contentDescription = null)
Spacer(Modifier.size(10.dp))
CircularProgressIndicator(modifier = Modifier.size(14.dp), strokeWidth = 2.dp)
}
CameraHudKind.Recording -> {
Icon(Icons.Default.FiberManualRecord, contentDescription = null, tint = Color.Red)
}
CameraHudKind.Success -> {
Icon(Icons.Default.CheckCircle, contentDescription = null)
}
CameraHudKind.Error -> {
Icon(Icons.Default.Error, contentDescription = null)
}
}
Spacer(Modifier.size(10.dp))
Text(
text = hud.message,
style = MaterialTheme.typography.bodyMedium,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
}
}
}
@@ -7,12 +7,18 @@ import android.graphics.Color
import android.util.Log
import android.view.View
import android.webkit.JavascriptInterface
import android.webkit.ConsoleMessage
import android.webkit.WebChromeClient
import android.webkit.WebView
import android.webkit.WebSettings
import android.webkit.WebResourceError
import android.webkit.WebResourceRequest
import android.webkit.WebResourceResponse
import android.webkit.WebViewClient
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.webkit.WebSettingsCompat
import androidx.webkit.WebViewFeature
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
@@ -28,10 +34,20 @@ import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.FilledTonalIconButton
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.LocalContentColor
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ChatBubble
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Error
import androidx.compose.material.icons.filled.FiberManualRecord
import androidx.compose.material.icons.filled.PhotoCamera
import androidx.compose.material.icons.filled.RecordVoiceOver
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.Report
import androidx.compose.material.icons.filled.ScreenShare
import androidx.compose.material.icons.filled.Settings
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
@@ -41,12 +57,15 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color as ComposeColor
import androidx.compose.ui.graphics.lerp
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.compose.ui.window.Popup
import androidx.compose.ui.window.PopupProperties
import androidx.core.content.ContextCompat
import com.steipete.clawdis.node.CameraHudKind
import com.steipete.clawdis.node.MainViewModel
@OptIn(ExperimentalMaterial3Api::class)
@@ -60,6 +79,105 @@ fun RootScreen(viewModel: MainViewModel) {
val statusText by viewModel.statusText.collectAsState()
val cameraHud by viewModel.cameraHud.collectAsState()
val cameraFlashToken by viewModel.cameraFlashToken.collectAsState()
val screenRecordActive by viewModel.screenRecordActive.collectAsState()
val isForeground by viewModel.isForeground.collectAsState()
val voiceWakeStatusText by viewModel.voiceWakeStatusText.collectAsState()
val talkEnabled by viewModel.talkEnabled.collectAsState()
val talkStatusText by viewModel.talkStatusText.collectAsState()
val talkIsListening by viewModel.talkIsListening.collectAsState()
val talkIsSpeaking by viewModel.talkIsSpeaking.collectAsState()
val seamColorArgb by viewModel.seamColorArgb.collectAsState()
val seamColor = remember(seamColorArgb) { ComposeColor(seamColorArgb) }
val audioPermissionLauncher =
rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
if (granted) viewModel.setTalkEnabled(true)
}
val activity =
remember(cameraHud, screenRecordActive, isForeground, statusText, voiceWakeStatusText) {
// Status pill owns transient activity state so it doesn't overlap the connection indicator.
if (!isForeground) {
return@remember StatusActivity(
title = "Foreground required",
icon = Icons.Default.Report,
contentDescription = "Foreground required",
)
}
val lowerStatus = statusText.lowercase()
if (lowerStatus.contains("repair")) {
return@remember StatusActivity(
title = "Repairing…",
icon = Icons.Default.Refresh,
contentDescription = "Repairing",
)
}
if (lowerStatus.contains("pairing") || lowerStatus.contains("approval")) {
return@remember StatusActivity(
title = "Approval pending",
icon = Icons.Default.RecordVoiceOver,
contentDescription = "Approval pending",
)
}
// Avoid duplicating the primary bridge status ("Connecting…") in the activity slot.
if (screenRecordActive) {
return@remember StatusActivity(
title = "Recording screen…",
icon = Icons.Default.ScreenShare,
contentDescription = "Recording screen",
tint = androidx.compose.ui.graphics.Color.Red,
)
}
cameraHud?.let { hud ->
return@remember when (hud.kind) {
CameraHudKind.Photo ->
StatusActivity(
title = hud.message,
icon = Icons.Default.PhotoCamera,
contentDescription = "Taking photo",
)
CameraHudKind.Recording ->
StatusActivity(
title = hud.message,
icon = Icons.Default.FiberManualRecord,
contentDescription = "Recording",
tint = androidx.compose.ui.graphics.Color.Red,
)
CameraHudKind.Success ->
StatusActivity(
title = hud.message,
icon = Icons.Default.CheckCircle,
contentDescription = "Capture finished",
)
CameraHudKind.Error ->
StatusActivity(
title = hud.message,
icon = Icons.Default.Error,
contentDescription = "Capture failed",
tint = androidx.compose.ui.graphics.Color.Red,
)
}
}
if (voiceWakeStatusText.contains("Microphone permission", ignoreCase = true)) {
return@remember StatusActivity(
title = "Mic permission",
icon = Icons.Default.Error,
contentDescription = "Mic permission required",
)
}
if (voiceWakeStatusText == "Paused") {
val suffix = if (!isForeground) " (background)" else ""
return@remember StatusActivity(
title = "Voice Wake paused$suffix",
icon = Icons.Default.RecordVoiceOver,
contentDescription = "Voice Wake paused",
)
}
null
}
val bridgeState =
remember(serverName, statusText) {
@@ -80,9 +198,9 @@ fun RootScreen(viewModel: MainViewModel) {
CanvasView(viewModel = viewModel, modifier = Modifier.fillMaxSize())
}
// Camera HUD (flash + toast) must be in a Popup to render above the WebView.
// Camera flash must be in a Popup to render above the WebView.
Popup(alignment = Alignment.Center, properties = PopupProperties(focusable = false)) {
CameraHudOverlay(hud = cameraHud, flashToken = cameraFlashToken, modifier = Modifier.fillMaxSize())
CameraFlashOverlay(token = cameraFlashToken, modifier = Modifier.fillMaxSize())
}
// Keep the overlay buttons above the WebView canvas (AndroidView), otherwise they may not receive touches.
@@ -90,6 +208,7 @@ fun RootScreen(viewModel: MainViewModel) {
StatusPill(
bridge = bridgeState,
voiceEnabled = voiceEnabled,
activity = activity,
onClick = { sheet = Sheet.Settings },
modifier = Modifier.windowInsetsPadding(safeOverlayInsets).padding(start = 12.dp, top = 12.dp),
)
@@ -106,6 +225,38 @@ fun RootScreen(viewModel: MainViewModel) {
icon = { Icon(Icons.Default.ChatBubble, contentDescription = "Chat") },
)
// Talk mode gets a dedicated side bubble instead of burying it in settings.
val baseOverlay = overlayContainerColor()
val talkContainer =
lerp(
baseOverlay,
seamColor.copy(alpha = baseOverlay.alpha),
if (talkEnabled) 0.35f else 0.22f,
)
val talkContent = if (talkEnabled) seamColor else overlayIconColor()
OverlayIconButton(
onClick = {
val next = !talkEnabled
if (next) {
val micOk =
ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) ==
PackageManager.PERMISSION_GRANTED
if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
viewModel.setTalkEnabled(true)
} else {
viewModel.setTalkEnabled(false)
}
},
containerColor = talkContainer,
contentColor = talkContent,
icon = {
Icon(
Icons.Default.RecordVoiceOver,
contentDescription = "Talk Mode",
)
},
)
OverlayIconButton(
onClick = { sheet = Sheet.Settings },
icon = { Icon(Icons.Default.Settings, contentDescription = "Settings") },
@@ -113,6 +264,17 @@ fun RootScreen(viewModel: MainViewModel) {
}
}
if (talkEnabled) {
Popup(alignment = Alignment.Center, properties = PopupProperties(focusable = false)) {
TalkOrbOverlay(
seamColor = seamColor,
statusText = talkStatusText,
isListening = talkIsListening,
isSpeaking = talkIsSpeaking,
)
}
}
val currentSheet = sheet
if (currentSheet != null) {
ModalBottomSheet(
@@ -136,14 +298,16 @@ private enum class Sheet {
private fun OverlayIconButton(
onClick: () -> Unit,
icon: @Composable () -> Unit,
containerColor: ComposeColor? = null,
contentColor: ComposeColor? = null,
) {
FilledTonalIconButton(
onClick = onClick,
modifier = Modifier.size(44.dp),
colors =
IconButtonDefaults.filledTonalIconButtonColors(
containerColor = overlayContainerColor(),
contentColor = overlayIconColor(),
containerColor = containerColor ?: overlayContainerColor(),
contentColor = contentColor ?: overlayIconColor(),
),
) {
icon()
@@ -163,6 +327,19 @@ private fun CanvasView(viewModel: MainViewModel, modifier: Modifier = Modifier)
// Some embedded web UIs (incl. the "background website") use localStorage/sessionStorage.
settings.domStorageEnabled = true
settings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE
if (WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK)) {
WebSettingsCompat.setForceDark(settings, WebSettingsCompat.FORCE_DARK_OFF)
}
if (WebViewFeature.isFeatureSupported(WebViewFeature.ALGORITHMIC_DARKENING)) {
WebSettingsCompat.setAlgorithmicDarkeningAllowed(settings, false)
}
if (isDebuggable) {
Log.d("ClawdisWebView", "userAgent: ${settings.userAgentString}")
}
isScrollContainer = true
overScrollMode = View.OVER_SCROLL_IF_CONTENT_SCROLLS
isVerticalScrollBarEnabled = true
isHorizontalScrollBarEnabled = true
webViewClient =
object : WebViewClient() {
override fun onReceivedError(
@@ -189,11 +366,38 @@ private fun CanvasView(viewModel: MainViewModel, modifier: Modifier = Modifier)
}
override fun onPageFinished(view: WebView, url: String?) {
if (isDebuggable) {
Log.d("ClawdisWebView", "onPageFinished: $url")
}
viewModel.canvas.onPageFinished()
}
override fun onRenderProcessGone(
view: WebView,
detail: android.webkit.RenderProcessGoneDetail,
): Boolean {
if (isDebuggable) {
Log.e(
"ClawdisWebView",
"onRenderProcessGone didCrash=${detail.didCrash()} priorityAtExit=${detail.rendererPriorityAtExit()}",
)
}
return true
}
}
setBackgroundColor(Color.BLACK)
setLayerType(View.LAYER_TYPE_HARDWARE, null)
webChromeClient =
object : WebChromeClient() {
override fun onConsoleMessage(consoleMessage: ConsoleMessage?): Boolean {
if (!isDebuggable) return false
val msg = consoleMessage ?: return false
Log.d(
"ClawdisWebView",
"console ${msg.messageLevel()} @ ${msg.sourceId()}:${msg.lineNumber()} ${msg.message()}",
)
return false
}
}
// Use default layer/background; avoid forcing a black fill over WebView content.
val a2uiBridge =
CanvasA2UIActionBridge { payload ->
@@ -2,6 +2,7 @@ package com.steipete.clawdis.node.ui
import android.Manifest
import android.content.pm.PackageManager
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.animation.AnimatedVisibility
@@ -46,6 +47,7 @@ import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import androidx.core.content.ContextCompat
import com.steipete.clawdis.node.BuildConfig
import com.steipete.clawdis.node.MainViewModel
import com.steipete.clawdis.node.NodeForegroundService
import com.steipete.clawdis.node.VoiceWakeMode
@@ -74,6 +76,22 @@ fun SettingsSheet(viewModel: MainViewModel) {
val listState = rememberLazyListState()
val (wakeWordsText, setWakeWordsText) = remember { mutableStateOf("") }
val (advancedExpanded, setAdvancedExpanded) = remember { mutableStateOf(false) }
val deviceModel =
remember {
listOfNotNull(Build.MANUFACTURER, Build.MODEL)
.joinToString(" ")
.trim()
.ifEmpty { "Android" }
}
val appVersion =
remember {
val versionName = BuildConfig.VERSION_NAME.trim().ifEmpty { "dev" }
if (BuildConfig.DEBUG && !versionName.contains("dev", ignoreCase = true)) {
"$versionName-dev"
} else {
versionName
}
}
LaunchedEffect(wakeWords) { setWakeWordsText(wakeWords.joinToString(", ")) }
@@ -142,6 +160,8 @@ fun SettingsSheet(viewModel: MainViewModel) {
)
}
item { Text("Instance ID: $instanceId", color = MaterialTheme.colorScheme.onSurfaceVariant) }
item { Text("Device: $deviceModel", color = MaterialTheme.colorScheme.onSurfaceVariant) }
item { Text("Version: $appVersion", color = MaterialTheme.colorScheme.onSurfaceVariant) }
item { HorizontalDivider() }
@@ -181,9 +201,27 @@ fun SettingsSheet(viewModel: MainViewModel) {
item { Text("No bridges found yet.", color = MaterialTheme.colorScheme.onSurfaceVariant) }
} else {
items(items = visibleBridges, key = { it.stableId }) { bridge ->
val detailLines =
buildList {
add("IP: ${bridge.host}:${bridge.port}")
bridge.lanHost?.let { add("LAN: $it") }
bridge.tailnetDns?.let { add("Tailnet: $it") }
if (bridge.gatewayPort != null || bridge.bridgePort != null || bridge.canvasPort != null) {
val gw = bridge.gatewayPort?.toString() ?: "—"
val br = (bridge.bridgePort ?: bridge.port).toString()
val canvas = bridge.canvasPort?.toString() ?: "—"
add("Ports: gw $gw · bridge $br · canvas $canvas")
}
}
ListItem(
headlineContent = { Text(bridge.name) },
supportingContent = { Text("${bridge.host}:${bridge.port}") },
supportingContent = {
Column(verticalArrangement = Arrangement.spacedBy(2.dp)) {
detailLines.forEach { line ->
Text(line, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
},
trailingContent = {
Button(
onClick = {
@@ -28,6 +28,7 @@ import androidx.compose.ui.unit.dp
fun StatusPill(
bridge: BridgeState,
voiceEnabled: Boolean,
activity: StatusActivity? = null,
onClick: () -> Unit,
modifier: Modifier = Modifier,
) {
@@ -62,23 +63,49 @@ fun StatusPill(
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Icon(
imageVector = if (voiceEnabled) Icons.Default.Mic else Icons.Default.MicOff,
contentDescription = if (voiceEnabled) "Voice enabled" else "Voice disabled",
tint =
if (voiceEnabled) {
overlayIconColor()
} else {
MaterialTheme.colorScheme.onSurfaceVariant
},
modifier = Modifier.size(18.dp),
)
if (activity != null) {
Row(
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Icon(
imageVector = activity.icon,
contentDescription = activity.contentDescription,
tint = activity.tint ?: overlayIconColor(),
modifier = Modifier.size(18.dp),
)
Text(
text = activity.title,
style = MaterialTheme.typography.labelLarge,
maxLines = 1,
)
}
} else {
Icon(
imageVector = if (voiceEnabled) Icons.Default.Mic else Icons.Default.MicOff,
contentDescription = if (voiceEnabled) "Voice enabled" else "Voice disabled",
tint =
if (voiceEnabled) {
overlayIconColor()
} else {
MaterialTheme.colorScheme.onSurfaceVariant
},
modifier = Modifier.size(18.dp),
)
}
Spacer(modifier = Modifier.width(2.dp))
}
}
}
data class StatusActivity(
val title: String,
val icon: androidx.compose.ui.graphics.vector.ImageVector,
val contentDescription: String,
val tint: Color? = null,
)
enum class BridgeState(val title: String, val color: Color) {
Connected("Connected", Color(0xFF2ECC71)),
Connecting("Connecting…", Color(0xFFF1C40F)),
@@ -0,0 +1,134 @@
package com.steipete.clawdis.node.ui
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
@Composable
fun TalkOrbOverlay(
seamColor: Color,
statusText: String,
isListening: Boolean,
isSpeaking: Boolean,
modifier: Modifier = Modifier,
) {
val transition = rememberInfiniteTransition(label = "talk-orb")
val t by
transition.animateFloat(
initialValue = 0f,
targetValue = 1f,
animationSpec =
infiniteRepeatable(
animation = tween(durationMillis = 1500, easing = LinearEasing),
repeatMode = RepeatMode.Restart,
),
label = "pulse",
)
val trimmed = statusText.trim()
val showStatus = trimmed.isNotEmpty() && trimmed != "Off"
val phase =
when {
isSpeaking -> "Speaking"
isListening -> "Listening"
else -> "Thinking"
}
Column(
modifier = modifier.padding(24.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Box(contentAlignment = Alignment.Center) {
Canvas(modifier = Modifier.size(360.dp)) {
val center = this.center
val baseRadius = size.minDimension * 0.30f
val ring1 = 1.05f + (t * 0.25f)
val ring2 = 1.20f + (t * 0.55f)
val ringAlpha1 = (1f - t) * 0.34f
val ringAlpha2 = (1f - t) * 0.22f
drawCircle(
color = seamColor.copy(alpha = ringAlpha1),
radius = baseRadius * ring1,
center = center,
style = Stroke(width = 3.dp.toPx()),
)
drawCircle(
color = seamColor.copy(alpha = ringAlpha2),
radius = baseRadius * ring2,
center = center,
style = Stroke(width = 3.dp.toPx()),
)
drawCircle(
brush =
Brush.radialGradient(
colors =
listOf(
seamColor.copy(alpha = 0.92f),
seamColor.copy(alpha = 0.40f),
Color.Black.copy(alpha = 0.56f),
),
center = center,
radius = baseRadius * 1.35f,
),
radius = baseRadius,
center = center,
)
drawCircle(
color = seamColor.copy(alpha = 0.34f),
radius = baseRadius,
center = center,
style = Stroke(width = 1.dp.toPx()),
)
}
}
if (showStatus) {
Surface(
color = Color.Black.copy(alpha = 0.40f),
shape = CircleShape,
) {
Text(
text = trimmed,
modifier = Modifier.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White.copy(alpha = 0.92f),
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
)
}
} else {
Text(
text = phase,
color = Color.White.copy(alpha = 0.80f),
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
)
}
}
}
@@ -15,7 +15,6 @@ import androidx.compose.foundation.horizontalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ArrowUpward
import androidx.compose.material.icons.filled.AttachFile
import androidx.compose.material.icons.filled.FolderOpen
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material3.ButtonDefaults
@@ -39,10 +38,12 @@ import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.unit.dp
import com.steipete.clawdis.node.chat.ChatSessionEntry
@Composable
fun ChatComposer(
sessionKey: String,
sessions: List<ChatSessionEntry>,
healthOk: Boolean,
thinkingLevel: String,
pendingRunCount: Int,
@@ -51,13 +52,18 @@ fun ChatComposer(
onPickImages: () -> Unit,
onRemoveAttachment: (id: String) -> Unit,
onSetThinkingLevel: (level: String) -> Unit,
onShowSessions: () -> Unit,
onSelectSession: (sessionKey: String) -> Unit,
onRefresh: () -> Unit,
onAbort: () -> Unit,
onSend: (text: String) -> Unit,
) {
var input by rememberSaveable { mutableStateOf("") }
var showThinkingMenu by remember { mutableStateOf(false) }
var showSessionMenu by remember { mutableStateOf(false) }
val sessionOptions = resolveSessionChoices(sessionKey, sessions)
val currentSessionLabel =
sessionOptions.firstOrNull { it.key == sessionKey }?.displayName ?: sessionKey
val canSend = pendingRunCount == 0 && (input.trim().isNotEmpty() || attachments.isNotEmpty()) && healthOk
@@ -73,6 +79,34 @@ fun ChatComposer(
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Box {
FilledTonalButton(
onClick = { showSessionMenu = true },
contentPadding = ButtonDefaults.ContentPadding,
) {
Text("Session: $currentSessionLabel")
}
DropdownMenu(expanded = showSessionMenu, onDismissRequest = { showSessionMenu = false }) {
for (entry in sessionOptions) {
DropdownMenuItem(
text = { Text(entry.displayName ?: entry.key) },
onClick = {
onSelectSession(entry.key)
showSessionMenu = false
},
trailingIcon = {
if (entry.key == sessionKey) {
Text("✓")
} else {
Spacer(modifier = Modifier.width(10.dp))
}
},
)
}
}
}
Box {
FilledTonalButton(
onClick = { showThinkingMenu = true },
@@ -91,10 +125,6 @@ fun ChatComposer(
Spacer(modifier = Modifier.weight(1f))
FilledTonalIconButton(onClick = onShowSessions, modifier = Modifier.size(42.dp)) {
Icon(Icons.Default.FolderOpen, contentDescription = "Sessions")
}
FilledTonalIconButton(onClick = onRefresh, modifier = Modifier.size(42.dp)) {
Icon(Icons.Default.Refresh, contentDescription = "Refresh")
}
@@ -17,6 +17,7 @@ import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.text.AnnotatedString
@@ -31,7 +32,7 @@ import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
@Composable
fun ChatMarkdown(text: String) {
fun ChatMarkdown(text: String, textColor: Color) {
val blocks = remember(text) { splitMarkdown(text) }
val inlineCodeBg = MaterialTheme.colorScheme.surfaceContainerLow
@@ -44,7 +45,7 @@ fun ChatMarkdown(text: String) {
Text(
text = parseInlineMarkdown(trimmed, inlineCodeBg = inlineCodeBg),
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
color = textColor,
)
}
is ChatMarkdownBlock.Code -> {
@@ -7,11 +7,9 @@ import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
@@ -60,20 +58,21 @@ fun ChatMessageBubble(message: ChatMessage) {
.background(bubbleBackground(isUser))
.padding(horizontal = 12.dp, vertical = 10.dp),
) {
ChatMessageBody(content = message.content)
val textColor = textColorOverBubble(isUser)
ChatMessageBody(content = message.content, textColor = textColor)
}
}
}
}
@Composable
private fun ChatMessageBody(content: List<ChatMessageContent>) {
private fun ChatMessageBody(content: List<ChatMessageContent>, textColor: Color) {
Column(verticalArrangement = Arrangement.spacedBy(10.dp)) {
for (part in content) {
when (part.type) {
"text" -> {
val text = part.text ?: continue
ChatMarkdown(text = text)
ChatMarkdown(text = text, textColor = textColor)
}
else -> {
val b64 = part.base64 ?: continue
@@ -131,7 +130,7 @@ fun ChatStreamingAssistantBubble(text: String) {
color = MaterialTheme.colorScheme.surfaceContainer,
) {
Box(modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp)) {
ChatMarkdown(text = text)
ChatMarkdown(text = text, textColor = MaterialTheme.colorScheme.onSurface)
}
}
}
@@ -150,6 +149,15 @@ private fun bubbleBackground(isUser: Boolean): Brush {
}
}
@Composable
private fun textColorOverBubble(isUser: Boolean): Color {
return if (isUser) {
MaterialTheme.colorScheme.onPrimary
} else {
MaterialTheme.colorScheme.onSurface
}
}
@Composable
private fun ChatBase64Image(base64: String, mimeType: String?) {
var image by remember(base64) { mutableStateOf<androidx.compose.ui.graphics.ImageBitmap?>(null) }
@@ -82,7 +82,7 @@ private fun SessionRow(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp),
) {
Text(entry.key, style = MaterialTheme.typography.bodyMedium)
Text(entry.displayName ?: entry.key, style = MaterialTheme.typography.bodyMedium)
Spacer(modifier = Modifier.weight(1f))
if (isCurrent) {
Text("Current", style = MaterialTheme.typography.labelSmall, color = MaterialTheme.colorScheme.onSurfaceVariant)
@@ -90,4 +90,3 @@ private fun SessionRow(
}
}
}
@@ -14,10 +14,8 @@ import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateListOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
@@ -40,10 +38,9 @@ fun ChatSheetContent(viewModel: MainViewModel) {
val pendingToolCalls by viewModel.chatPendingToolCalls.collectAsState()
val sessions by viewModel.chatSessions.collectAsState()
var showSessions by remember { mutableStateOf(false) }
LaunchedEffect(Unit) {
viewModel.loadChat("main")
viewModel.refreshChatSessions(limit = 200)
}
val context = LocalContext.current
@@ -87,6 +84,7 @@ fun ChatSheetContent(viewModel: MainViewModel) {
ChatComposer(
sessionKey = sessionKey,
sessions = sessions,
healthOk = healthOk,
thinkingLevel = thinkingLevel,
pendingRunCount = pendingRunCount,
@@ -95,8 +93,11 @@ fun ChatSheetContent(viewModel: MainViewModel) {
onPickImages = { pickImages.launch("image/*") },
onRemoveAttachment = { id -> attachments.removeAll { it.id == id } },
onSetThinkingLevel = { level -> viewModel.setChatThinkingLevel(level) },
onShowSessions = { showSessions = true },
onRefresh = { viewModel.refreshChat() },
onSelectSession = { key -> viewModel.switchChatSession(key) },
onRefresh = {
viewModel.refreshChat()
viewModel.refreshChatSessions(limit = 200)
},
onAbort = { viewModel.abortChat() },
onSend = { text ->
val outgoing =
@@ -113,19 +114,6 @@ fun ChatSheetContent(viewModel: MainViewModel) {
},
)
}
if (showSessions) {
ChatSessionsDialog(
currentSessionKey = sessionKey,
sessions = sessions,
onDismiss = { showSessions = false },
onRefresh = { viewModel.refreshChatSessions(limit = 50) },
onSelect = { key ->
viewModel.switchChatSession(key)
showSessions = false
},
)
}
}
data class PendingImageAttachment(
@@ -0,0 +1,46 @@
package com.steipete.clawdis.node.ui.chat
import com.steipete.clawdis.node.chat.ChatSessionEntry
private const val MAIN_SESSION_KEY = "main"
private const val RECENT_WINDOW_MS = 24 * 60 * 60 * 1000L
fun resolveSessionChoices(
currentSessionKey: String,
sessions: List<ChatSessionEntry>,
nowMs: Long = System.currentTimeMillis(),
): List<ChatSessionEntry> {
val current = currentSessionKey.trim()
val cutoff = nowMs - RECENT_WINDOW_MS
val sorted = sessions.sortedByDescending { it.updatedAtMs ?: 0L }
val recent = mutableListOf<ChatSessionEntry>()
val seen = mutableSetOf<String>()
for (entry in sorted) {
if (!seen.add(entry.key)) continue
if ((entry.updatedAtMs ?: 0L) < cutoff) continue
recent.add(entry)
}
val result = mutableListOf<ChatSessionEntry>()
val included = mutableSetOf<String>()
val mainEntry = sorted.firstOrNull { it.key == MAIN_SESSION_KEY }
if (mainEntry != null) {
result.add(mainEntry)
included.add(MAIN_SESSION_KEY)
} else if (current == MAIN_SESSION_KEY) {
result.add(ChatSessionEntry(key = MAIN_SESSION_KEY, updatedAtMs = null))
included.add(MAIN_SESSION_KEY)
}
for (entry in recent) {
if (included.add(entry.key)) {
result.add(entry)
}
}
if (current.isNotEmpty() && !included.contains(current)) {
result.add(ChatSessionEntry(key = current, updatedAtMs = null))
}
return result
}
@@ -0,0 +1,98 @@
package com.steipete.clawdis.node.voice
import android.media.MediaDataSource
import kotlin.math.min
internal class StreamingMediaDataSource : MediaDataSource() {
private data class Chunk(val start: Long, val data: ByteArray)
private val lock = Object()
private val chunks = ArrayList<Chunk>()
private var totalSize: Long = 0
private var closed = false
private var finished = false
private var lastReadIndex = 0
fun append(data: ByteArray) {
if (data.isEmpty()) return
synchronized(lock) {
if (closed || finished) return
val chunk = Chunk(totalSize, data)
chunks.add(chunk)
totalSize += data.size.toLong()
lock.notifyAll()
}
}
fun finish() {
synchronized(lock) {
if (closed) return
finished = true
lock.notifyAll()
}
}
fun fail() {
synchronized(lock) {
closed = true
lock.notifyAll()
}
}
override fun readAt(position: Long, buffer: ByteArray, offset: Int, size: Int): Int {
if (position < 0) return -1
synchronized(lock) {
while (!closed && !finished && position >= totalSize) {
lock.wait()
}
if (closed) return -1
if (position >= totalSize && finished) return -1
val available = (totalSize - position).toInt()
val toRead = min(size, available)
var remaining = toRead
var destOffset = offset
var pos = position
var index = findChunkIndex(pos)
while (remaining > 0 && index < chunks.size) {
val chunk = chunks[index]
val inChunkOffset = (pos - chunk.start).toInt()
if (inChunkOffset >= chunk.data.size) {
index++
continue
}
val copyLen = min(remaining, chunk.data.size - inChunkOffset)
System.arraycopy(chunk.data, inChunkOffset, buffer, destOffset, copyLen)
remaining -= copyLen
destOffset += copyLen
pos += copyLen
if (inChunkOffset + copyLen >= chunk.data.size) {
index++
}
}
return toRead - remaining
}
}
override fun getSize(): Long = -1
override fun close() {
synchronized(lock) {
closed = true
lock.notifyAll()
}
}
private fun findChunkIndex(position: Long): Int {
var index = lastReadIndex
while (index < chunks.size) {
val chunk = chunks[index]
if (position < chunk.start + chunk.data.size) break
index++
}
lastReadIndex = index
return index
}
}
@@ -0,0 +1,191 @@
package com.steipete.clawdis.node.voice
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
private val directiveJson = Json { ignoreUnknownKeys = true }
data class TalkDirective(
val voiceId: String? = null,
val modelId: String? = null,
val speed: Double? = null,
val rateWpm: Int? = null,
val stability: Double? = null,
val similarity: Double? = null,
val style: Double? = null,
val speakerBoost: Boolean? = null,
val seed: Long? = null,
val normalize: String? = null,
val language: String? = null,
val outputFormat: String? = null,
val latencyTier: Int? = null,
val once: Boolean? = null,
)
data class TalkDirectiveParseResult(
val directive: TalkDirective?,
val stripped: String,
val unknownKeys: List<String>,
)
object TalkDirectiveParser {
fun parse(text: String): TalkDirectiveParseResult {
val normalized = text.replace("\r\n", "\n")
val lines = normalized.split("\n").toMutableList()
if (lines.isEmpty()) return TalkDirectiveParseResult(null, text, emptyList())
val firstNonEmpty = lines.indexOfFirst { it.trim().isNotEmpty() }
if (firstNonEmpty == -1) return TalkDirectiveParseResult(null, text, emptyList())
val head = lines[firstNonEmpty].trim()
if (!head.startsWith("{") || !head.endsWith("}")) {
return TalkDirectiveParseResult(null, text, emptyList())
}
val obj = parseJsonObject(head) ?: return TalkDirectiveParseResult(null, text, emptyList())
val speakerBoost =
boolValue(obj, listOf("speaker_boost", "speakerBoost"))
?: boolValue(obj, listOf("no_speaker_boost", "noSpeakerBoost"))?.not()
val directive = TalkDirective(
voiceId = stringValue(obj, listOf("voice", "voice_id", "voiceId")),
modelId = stringValue(obj, listOf("model", "model_id", "modelId")),
speed = doubleValue(obj, listOf("speed")),
rateWpm = intValue(obj, listOf("rate", "wpm")),
stability = doubleValue(obj, listOf("stability")),
similarity = doubleValue(obj, listOf("similarity", "similarity_boost", "similarityBoost")),
style = doubleValue(obj, listOf("style")),
speakerBoost = speakerBoost,
seed = longValue(obj, listOf("seed")),
normalize = stringValue(obj, listOf("normalize", "apply_text_normalization")),
language = stringValue(obj, listOf("lang", "language_code", "language")),
outputFormat = stringValue(obj, listOf("output_format", "format")),
latencyTier = intValue(obj, listOf("latency", "latency_tier", "latencyTier")),
once = boolValue(obj, listOf("once")),
)
val hasDirective = listOf(
directive.voiceId,
directive.modelId,
directive.speed,
directive.rateWpm,
directive.stability,
directive.similarity,
directive.style,
directive.speakerBoost,
directive.seed,
directive.normalize,
directive.language,
directive.outputFormat,
directive.latencyTier,
directive.once,
).any { it != null }
if (!hasDirective) return TalkDirectiveParseResult(null, text, emptyList())
val knownKeys = setOf(
"voice", "voice_id", "voiceid",
"model", "model_id", "modelid",
"speed", "rate", "wpm",
"stability", "similarity", "similarity_boost", "similarityboost",
"style",
"speaker_boost", "speakerboost",
"no_speaker_boost", "nospeakerboost",
"seed",
"normalize", "apply_text_normalization",
"lang", "language_code", "language",
"output_format", "format",
"latency", "latency_tier", "latencytier",
"once",
)
val unknownKeys = obj.keys.filter { !knownKeys.contains(it.lowercase()) }.sorted()
lines.removeAt(firstNonEmpty)
if (firstNonEmpty < lines.size) {
if (lines[firstNonEmpty].trim().isEmpty()) {
lines.removeAt(firstNonEmpty)
}
}
return TalkDirectiveParseResult(directive, lines.joinToString("\n"), unknownKeys)
}
private fun parseJsonObject(line: String): JsonObject? {
return try {
directiveJson.parseToJsonElement(line) as? JsonObject
} catch (_: Throwable) {
null
}
}
private fun stringValue(obj: JsonObject, keys: List<String>): String? {
for (key in keys) {
val value = obj[key].asStringOrNull()?.trim()
if (!value.isNullOrEmpty()) return value
}
return null
}
private fun doubleValue(obj: JsonObject, keys: List<String>): Double? {
for (key in keys) {
val value = obj[key].asDoubleOrNull()
if (value != null) return value
}
return null
}
private fun intValue(obj: JsonObject, keys: List<String>): Int? {
for (key in keys) {
val value = obj[key].asIntOrNull()
if (value != null) return value
}
return null
}
private fun longValue(obj: JsonObject, keys: List<String>): Long? {
for (key in keys) {
val value = obj[key].asLongOrNull()
if (value != null) return value
}
return null
}
private fun boolValue(obj: JsonObject, keys: List<String>): Boolean? {
for (key in keys) {
val value = obj[key].asBooleanOrNull()
if (value != null) return value
}
return null
}
}
private fun JsonElement?.asStringOrNull(): String? =
(this as? JsonPrimitive)?.takeIf { it.isString }?.content
private fun JsonElement?.asDoubleOrNull(): Double? {
val primitive = this as? JsonPrimitive ?: return null
return primitive.content.toDoubleOrNull()
}
private fun JsonElement?.asIntOrNull(): Int? {
val primitive = this as? JsonPrimitive ?: return null
return primitive.content.toIntOrNull()
}
private fun JsonElement?.asLongOrNull(): Long? {
val primitive = this as? JsonPrimitive ?: return null
return primitive.content.toLongOrNull()
}
private fun JsonElement?.asBooleanOrNull(): Boolean? {
val primitive = this as? JsonPrimitive ?: return null
val content = primitive.content.trim().lowercase()
return when (content) {
"true", "yes", "1" -> true
"false", "no", "0" -> false
else -> null
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,14 @@
package com.steipete.clawdis.node.bridge
import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe
class BridgeEndpointKotestTest : StringSpec({
"manual endpoint builds stable id + name" {
val endpoint = BridgeEndpoint.manual("10.0.0.5", 18790)
endpoint.stableId shouldBe "manual|10.0.0.5|18790"
endpoint.name shouldBe "10.0.0.5:18790"
endpoint.host shouldBe "10.0.0.5"
endpoint.port shouldBe 18790
}
})
@@ -0,0 +1,47 @@
package com.steipete.clawdis.node.node
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
import kotlin.math.min
class JpegSizeLimiterTest {
@Test
fun compressesLargePayloadsUnderLimit() {
val maxBytes = 5 * 1024 * 1024
val result =
JpegSizeLimiter.compressToLimit(
initialWidth = 4000,
initialHeight = 3000,
startQuality = 95,
maxBytes = maxBytes,
encode = { width, height, quality ->
val estimated = (width.toLong() * height.toLong() * quality.toLong()) / 100
val size = min(maxBytes.toLong() * 2, estimated).toInt()
ByteArray(size)
},
)
assertTrue(result.bytes.size <= maxBytes)
assertTrue(result.width <= 4000)
assertTrue(result.height <= 3000)
assertTrue(result.quality <= 95)
}
@Test
fun keepsSmallPayloadsAsIs() {
val maxBytes = 5 * 1024 * 1024
val result =
JpegSizeLimiter.compressToLimit(
initialWidth = 800,
initialHeight = 600,
startQuality = 90,
maxBytes = maxBytes,
encode = { _, _, _ -> ByteArray(120_000) },
)
assertEquals(800, result.width)
assertEquals(600, result.height)
assertEquals(90, result.quality)
}
}
@@ -0,0 +1,35 @@
package com.steipete.clawdis.node.ui.chat
import com.steipete.clawdis.node.chat.ChatSessionEntry
import org.junit.Assert.assertEquals
import org.junit.Test
class SessionFiltersTest {
@Test
fun sessionChoicesPreferMainAndRecent() {
val now = 1_700_000_000_000L
val recent1 = now - 2 * 60 * 60 * 1000L
val recent2 = now - 5 * 60 * 60 * 1000L
val stale = now - 26 * 60 * 60 * 1000L
val sessions =
listOf(
ChatSessionEntry(key = "recent-1", updatedAtMs = recent1),
ChatSessionEntry(key = "main", updatedAtMs = stale),
ChatSessionEntry(key = "old-1", updatedAtMs = stale),
ChatSessionEntry(key = "recent-2", updatedAtMs = recent2),
)
val result = resolveSessionChoices("main", sessions, nowMs = now).map { it.key }
assertEquals(listOf("main", "recent-1", "recent-2"), result)
}
@Test
fun sessionChoicesIncludeCurrentWhenMissing() {
val now = 1_700_000_000_000L
val recent = now - 10 * 60 * 1000L
val sessions = listOf(ChatSessionEntry(key = "main", updatedAtMs = recent))
val result = resolveSessionChoices("custom", sessions, nowMs = now).map { it.key }
assertEquals(listOf("main", "custom"), result)
}
}
@@ -0,0 +1,55 @@
package com.steipete.clawdis.node.voice
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
class TalkDirectiveParserTest {
@Test
fun parsesDirectiveAndStripsHeader() {
val input = """
{"voice":"voice-123","once":true}
Hello from talk mode.
""".trimIndent()
val result = TalkDirectiveParser.parse(input)
assertEquals("voice-123", result.directive?.voiceId)
assertEquals(true, result.directive?.once)
assertEquals("Hello from talk mode.", result.stripped.trim())
}
@Test
fun ignoresUnknownKeysButReportsThem() {
val input = """
{"voice":"abc","foo":1,"bar":"baz"}
Hi there.
""".trimIndent()
val result = TalkDirectiveParser.parse(input)
assertEquals("abc", result.directive?.voiceId)
assertTrue(result.unknownKeys.containsAll(listOf("bar", "foo")))
}
@Test
fun parsesAlternateKeys() {
val input = """
{"model_id":"eleven_v3","similarity_boost":0.4,"no_speaker_boost":true,"rate":200}
Speak.
""".trimIndent()
val result = TalkDirectiveParser.parse(input)
assertEquals("eleven_v3", result.directive?.modelId)
assertEquals(0.4, result.directive?.similarity)
assertEquals(false, result.directive?.speakerBoost)
assertEquals(200, result.directive?.rateWpm)
}
@Test
fun returnsNullWhenNoDirectivePresent() {
val input = """
{}
Hello.
""".trimIndent()
val result = TalkDirectiveParser.parse(input)
assertNull(result.directive)
assertEquals(input, result.stripped)
}
}
@@ -6,6 +6,15 @@ import Observation
import SwiftUI
import UIKit
protocol BridgePairingClient: Sendable {
func pairAndHello(
endpoint: NWEndpoint,
hello: BridgeHello,
onStatus: (@Sendable (String) -> Void)?) async throws -> String
}
extension BridgeClient: BridgePairingClient {}
@MainActor
@Observable
final class BridgeConnectionController {
@@ -16,10 +25,16 @@ final class BridgeConnectionController {
private let discovery = BridgeDiscoveryModel()
private weak var appModel: NodeAppModel?
private var didAutoConnect = false
private var seenStableIDs = Set<String>()
init(appModel: NodeAppModel, startDiscovery: Bool = true) {
private let bridgeClientFactory: @Sendable () -> any BridgePairingClient
init(
appModel: NodeAppModel,
startDiscovery: Bool = true,
bridgeClientFactory: @escaping @Sendable () -> any BridgePairingClient = { BridgeClient() })
{
self.appModel = appModel
self.bridgeClientFactory = bridgeClientFactory
BridgeSettingsStore.bootstrapPersistence()
let defaults = UserDefaults.standard
@@ -85,7 +100,7 @@ final class BridgeConnectionController {
let token = KeychainStore.loadString(
service: "com.steipete.clawdis.bridge",
account: "bridge-token.\(instanceId)")?
account: self.keychainAccount(instanceId: instanceId))?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !token.isEmpty else { return }
@@ -99,28 +114,40 @@ final class BridgeConnectionController {
guard let port = NWEndpoint.Port(rawValue: UInt16(resolvedPort)) else { return }
self.didAutoConnect = true
appModel.connectToBridge(
endpoint: .hostPort(host: NWEndpoint.Host(manualHost), port: port),
hello: self.makeHello(token: token))
let endpoint = NWEndpoint.hostPort(host: NWEndpoint.Host(manualHost), port: port)
self.startAutoConnect(endpoint: endpoint, token: token, instanceId: instanceId)
return
}
let targetStableID = defaults.string(forKey: "bridge.lastDiscoveredStableID")?
let preferredStableID = defaults.string(forKey: "bridge.preferredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !targetStableID.isEmpty else { return }
let lastDiscoveredStableID = defaults.string(forKey: "bridge.lastDiscoveredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let candidates = [preferredStableID, lastDiscoveredStableID].filter { !$0.isEmpty }
guard let targetStableID = candidates.first(where: { id in
self.bridges.contains(where: { $0.stableID == id })
}) else { return }
guard let target = self.bridges.first(where: { $0.stableID == targetStableID }) else { return }
self.didAutoConnect = true
appModel.connectToBridge(endpoint: target.endpoint, hello: self.makeHello(token: token))
self.startAutoConnect(endpoint: target.endpoint, token: token, instanceId: instanceId)
}
private func updateLastDiscoveredBridge(from bridges: [BridgeDiscoveryModel.DiscoveredBridge]) {
let newlyDiscovered = bridges.filter { self.seenStableIDs.insert($0.stableID).inserted }
guard let last = newlyDiscovered.last else { return }
let defaults = UserDefaults.standard
let preferred = defaults.string(forKey: "bridge.preferredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let existingLast = defaults.string(forKey: "bridge.lastDiscoveredStableID")?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
UserDefaults.standard.set(last.stableID, forKey: "bridge.lastDiscoveredStableID")
BridgeSettingsStore.saveLastDiscoveredBridgeStableID(last.stableID)
// Avoid overriding user intent (preferred/lastDiscovered are also set on manual Connect).
guard preferred.isEmpty, existingLast.isEmpty else { return }
guard let first = bridges.first else { return }
defaults.set(first.stableID, forKey: "bridge.lastDiscoveredStableID")
BridgeSettingsStore.saveLastDiscoveredBridgeStableID(first.stableID)
}
private func makeHello(token: String) -> BridgeHello {
@@ -140,6 +167,40 @@ final class BridgeConnectionController {
commands: self.currentCommands())
}
private func keychainAccount(instanceId: String) -> String {
"bridge-token.\(instanceId)"
}
private func startAutoConnect(endpoint: NWEndpoint, token: String, instanceId: String) {
guard let appModel else { return }
Task { [weak self] in
guard let self else { return }
do {
let hello = self.makeHello(token: token)
let refreshed = try await self.bridgeClientFactory().pairAndHello(
endpoint: endpoint,
hello: hello,
onStatus: { status in
Task { @MainActor in
appModel.bridgeStatusText = status
}
})
let resolvedToken = refreshed.isEmpty ? token : refreshed
if !refreshed.isEmpty, refreshed != token {
_ = KeychainStore.saveString(
refreshed,
service: "com.steipete.clawdis.bridge",
account: self.keychainAccount(instanceId: instanceId))
}
appModel.connectToBridge(endpoint: endpoint, hello: self.makeHello(token: resolvedToken))
} catch {
await MainActor.run {
appModel.bridgeStatusText = "Bridge error: \(error.localizedDescription)"
}
}
}
}
private func resolvedDisplayName(defaults: UserDefaults) -> String {
let key = "node.displayName"
let existing = defaults.string(forKey: key)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
@@ -186,6 +247,7 @@ final class BridgeConnectionController {
let caps = Set(self.currentCaps())
if caps.contains(ClawdisCapability.camera.rawValue) {
commands.append(ClawdisCameraCommand.list.rawValue)
commands.append(ClawdisCameraCommand.snap.rawValue)
commands.append(ClawdisCameraCommand.clip.rawValue)
}
@@ -265,5 +327,13 @@ extension BridgeConnectionController {
func _test_appVersion() -> String {
self.appVersion()
}
func _test_setBridges(_ bridges: [BridgeDiscoveryModel.DiscoveredBridge]) {
self.bridges = bridges
}
func _test_triggerAutoConnect() {
self.maybeAutoConnect()
}
}
#endif
@@ -18,6 +18,12 @@ final class BridgeDiscoveryModel {
var endpoint: NWEndpoint
var stableID: String
var debugID: String
var lanHost: String?
var tailnetDns: String?
var gatewayPort: Int?
var bridgePort: Int?
var canvasPort: Int?
var cliPath: String?
}
var bridges: [DiscoveredBridge] = []
@@ -68,7 +74,8 @@ final class BridgeDiscoveryModel {
switch result.endpoint {
case let .service(name, _, _, _):
let decodedName = BonjourEscapes.decode(name)
let advertisedName = result.endpoint.txtRecord?.dictionary["displayName"]
let txt = result.endpoint.txtRecord?.dictionary ?? [:]
let advertisedName = txt["displayName"]
let prettyAdvertised = advertisedName
.map(Self.prettifyInstanceName)
.flatMap { $0.isEmpty ? nil : $0 }
@@ -77,7 +84,13 @@ final class BridgeDiscoveryModel {
name: prettyName,
endpoint: result.endpoint,
stableID: BridgeEndpointID.stableID(result.endpoint),
debugID: BridgeEndpointID.prettyDescription(result.endpoint))
debugID: BridgeEndpointID.prettyDescription(result.endpoint),
lanHost: Self.txtValue(txt, key: "lanHost"),
tailnetDns: Self.txtValue(txt, key: "tailnetDns"),
gatewayPort: Self.txtIntValue(txt, key: "gatewayPort"),
bridgePort: Self.txtIntValue(txt, key: "bridgePort"),
canvasPort: Self.txtIntValue(txt, key: "canvasPort"),
cliPath: Self.txtValue(txt, key: "cliPath"))
default:
return nil
}
@@ -191,4 +204,14 @@ final class BridgeDiscoveryModel {
.replacingOccurrences(of: #"\s+\(\d+\)$"#, with: "", options: .regularExpression)
return stripped.trimmingCharacters(in: .whitespacesAndNewlines)
}
private static func txtValue(_ dict: [String: String], key: String) -> String? {
let raw = dict[key]?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
return raw.isEmpty ? nil : raw
}
private static func txtIntValue(_ dict: [String: String], key: String) -> Int? {
guard let raw = self.txtValue(dict, key: key) else { return nil }
return Int(raw)
}
}
+57 -4
View File
@@ -3,6 +3,13 @@ import ClawdisKit
import Foundation
actor CameraController {
struct CameraDeviceInfo: Codable, Sendable {
var id: String
var name: String
var position: String
var deviceType: String
}
enum CameraError: LocalizedError, Sendable {
case cameraUnavailable
case microphoneUnavailable
@@ -41,13 +48,14 @@ actor CameraController {
// If you need the full-res photo, explicitly request a larger maxWidth.
let maxWidth = params.maxWidth.flatMap { $0 > 0 ? $0 : nil } ?? 1600
let quality = Self.clampQuality(params.quality)
let delayMs = max(0, params.delayMs ?? 0)
try await self.ensureAccess(for: .video)
let session = AVCaptureSession()
session.sessionPreset = .photo
guard let device = Self.pickCamera(facing: facing) else {
guard let device = Self.pickCamera(facing: facing, deviceId: params.deviceId) else {
throw CameraError.cameraUnavailable
}
@@ -67,6 +75,7 @@ actor CameraController {
session.startRunning()
defer { session.stopRunning() }
await Self.warmUpCaptureSession()
await Self.sleepDelayMs(delayMs)
let settings: AVCapturePhotoSettings = {
if output.availablePhotoCodecTypes.contains(.jpeg) {
@@ -84,10 +93,14 @@ actor CameraController {
}
withExtendedLifetime(delegate) {}
let maxPayloadBytes = 5 * 1024 * 1024
// Base64 inflates payloads by ~4/3; cap encoded bytes so the payload stays under 5MB (API limit).
let maxEncodedBytes = (maxPayloadBytes / 4) * 3
let res = try JPEGTranscoder.transcodeToJPEG(
imageData: rawData,
maxWidthPx: maxWidth,
quality: quality)
quality: quality,
maxBytes: maxEncodedBytes)
return (
format: format.rawValue,
@@ -115,7 +128,7 @@ actor CameraController {
let session = AVCaptureSession()
session.sessionPreset = .high
guard let camera = Self.pickCamera(facing: facing) else {
guard let camera = Self.pickCamera(facing: facing, deviceId: params.deviceId) else {
throw CameraError.cameraUnavailable
}
let cameraInput = try AVCaptureDeviceInput(device: camera)
@@ -176,6 +189,23 @@ actor CameraController {
hasAudio: includeAudio)
}
func listDevices() -> [CameraDeviceInfo] {
let types: [AVCaptureDevice.DeviceType] = [
.builtInWideAngleCamera,
]
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: types,
mediaType: .video,
position: .unspecified)
return session.devices.map { device in
CameraDeviceInfo(
id: device.uniqueID,
name: device.localizedName,
position: Self.positionLabel(device.position),
deviceType: device.deviceType.rawValue)
}
}
private func ensureAccess(for mediaType: AVMediaType) async throws {
let status = AVCaptureDevice.authorizationStatus(for: mediaType)
switch status {
@@ -197,7 +227,15 @@ actor CameraController {
}
}
private nonisolated static func pickCamera(facing: ClawdisCameraFacing) -> AVCaptureDevice? {
private nonisolated static func pickCamera(
facing: ClawdisCameraFacing,
deviceId: String?) -> AVCaptureDevice?
{
if let deviceId, !deviceId.isEmpty {
if let match = AVCaptureDevice.devices(for: .video).first(where: { $0.uniqueID == deviceId }) {
return match
}
}
let position: AVCaptureDevice.Position = (facing == .front) ? .front : .back
if let device = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: position) {
return device
@@ -206,6 +244,14 @@ actor CameraController {
return AVCaptureDevice.default(for: .video)
}
private nonisolated static func positionLabel(_ position: AVCaptureDevice.Position) -> String {
switch position {
case .front: "front"
case .back: "back"
default: "unspecified"
}
}
nonisolated static func clampQuality(_ quality: Double?) -> Double {
let q = quality ?? 0.9
return min(1.0, max(0.05, q))
@@ -258,6 +304,13 @@ actor CameraController {
// A short delay after `startRunning()` significantly reduces "blank first frame" captures on some devices.
try? await Task.sleep(nanoseconds: 150_000_000) // 150ms
}
private nonisolated static func sleepDelayMs(_ delayMs: Int) async {
guard delayMs > 0 else { return }
let maxDelayMs = 10 * 1000
let ns = UInt64(min(delayMs, maxDelayMs)) * UInt64(NSEC_PER_MSEC)
try? await Task.sleep(nanoseconds: ns)
}
}
private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegate {
+7 -2
View File
@@ -4,18 +4,23 @@ import SwiftUI
struct ChatSheet: View {
@Environment(\.dismiss) private var dismiss
@State private var viewModel: ClawdisChatViewModel
private let userAccent: Color?
init(bridge: BridgeSession, sessionKey: String = "main") {
init(bridge: BridgeSession, sessionKey: String = "main", userAccent: Color? = nil) {
let transport = IOSBridgeChatTransport(bridge: bridge)
self._viewModel = State(
initialValue: ClawdisChatViewModel(
sessionKey: sessionKey,
transport: transport))
self.userAccent = userAccent
}
var body: some View {
NavigationStack {
ClawdisChatView(viewModel: self.viewModel)
ClawdisChatView(
viewModel: self.viewModel,
showsSessionSwitcher: true,
userAccent: self.userAccent)
.navigationTitle("Chat")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
+65 -2
View File
@@ -22,12 +22,15 @@ final class NodeAppModel {
var bridgeServerName: String?
var bridgeRemoteAddress: String?
var connectedBridgeID: String?
var seamColorHex: String?
var mainSessionKey: String = "main"
private let bridge = BridgeSession()
private var bridgeTask: Task<Void, Never>?
private var voiceWakeSyncTask: Task<Void, Never>?
@ObservationIgnored private var cameraHUDDismissTask: Task<Void, Never>?
let voiceWake = VoiceWakeManager()
let talkMode = TalkModeManager()
private var lastAutoA2uiURL: String?
var bridgeSession: BridgeSession { self.bridge }
@@ -35,11 +38,12 @@ final class NodeAppModel {
var cameraHUDText: String?
var cameraHUDKind: CameraHUDKind?
var cameraFlashNonce: Int = 0
var screenRecordActive: Bool = false
init() {
self.voiceWake.configure { [weak self] cmd in
guard let self else { return }
let sessionKey = "main"
let sessionKey = await MainActor.run { self.mainSessionKey }
do {
try await self.sendVoiceTranscript(text: cmd, sessionKey: sessionKey)
} catch {
@@ -49,6 +53,9 @@ final class NodeAppModel {
let enabled = UserDefaults.standard.bool(forKey: "voiceWake.enabled")
self.voiceWake.setEnabled(enabled)
self.talkMode.attachBridge(self.bridge)
let talkEnabled = UserDefaults.standard.bool(forKey: "talk.enabled")
self.talkMode.setEnabled(talkEnabled)
// Wire up deep links from canvas taps
self.screen.onDeepLink = { [weak self] url in
@@ -145,7 +152,7 @@ final class NodeAppModel {
guard let raw = await self.bridge.currentCanvasHostUrl() else { return nil }
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty, let base = URL(string: trimmed) else { return nil }
return base.appendingPathComponent("__clawdis__/a2ui/").absoluteString
return base.appendingPathComponent("__clawdis__/a2ui/").absoluteString + "?platform=ios"
}
private func showA2UIOnConnectIfNeeded() async {
@@ -177,6 +184,10 @@ final class NodeAppModel {
self.voiceWake.setEnabled(enabled)
}
func setTalkEnabled(_ enabled: Bool) {
self.talkMode.setEnabled(enabled)
}
func connectToBridge(
endpoint: NWEndpoint,
hello: BridgeHello)
@@ -216,6 +227,7 @@ final class NodeAppModel {
self.bridgeRemoteAddress = addr
}
}
await self.refreshBrandingFromGateway()
await self.startVoiceWakeSync()
await self.showA2UIOnConnectIfNeeded()
},
@@ -255,6 +267,8 @@ final class NodeAppModel {
self.bridgeServerName = nil
self.bridgeRemoteAddress = nil
self.connectedBridgeID = nil
self.seamColorHex = nil
self.mainSessionKey = "main"
self.showLocalCanvasOnDisconnect()
}
}
@@ -270,9 +284,47 @@ final class NodeAppModel {
self.bridgeServerName = nil
self.bridgeRemoteAddress = nil
self.connectedBridgeID = nil
self.seamColorHex = nil
self.mainSessionKey = "main"
self.showLocalCanvasOnDisconnect()
}
var seamColor: Color {
Self.color(fromHex: self.seamColorHex) ?? Self.defaultSeamColor
}
private static let defaultSeamColor = Color(red: 79 / 255.0, green: 122 / 255.0, blue: 154 / 255.0)
private static func color(fromHex raw: String?) -> Color? {
let trimmed = (raw ?? "").trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return nil }
let hex = trimmed.hasPrefix("#") ? String(trimmed.dropFirst()) : trimmed
guard hex.count == 6, let value = Int(hex, radix: 16) else { return nil }
let r = Double((value >> 16) & 0xFF) / 255.0
let g = Double((value >> 8) & 0xFF) / 255.0
let b = Double(value & 0xFF) / 255.0
return Color(red: r, green: g, blue: b)
}
private func refreshBrandingFromGateway() async {
do {
let res = try await self.bridge.request(method: "config.get", paramsJSON: "{}", timeoutSeconds: 8)
guard let json = try JSONSerialization.jsonObject(with: res) as? [String: Any] else { return }
guard let config = json["config"] as? [String: Any] else { return }
let ui = config["ui"] as? [String: Any]
let raw = (ui?["seamColor"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let session = config["session"] as? [String: Any]
let rawMainKey = (session?["mainKey"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
let mainKey = rawMainKey.isEmpty ? "main" : rawMainKey
await MainActor.run {
self.seamColorHex = raw.isEmpty ? nil : raw
self.mainSessionKey = mainKey
}
} catch {
// ignore
}
}
func setGlobalWakeWords(_ words: [String]) async {
let sanitized = VoiceWakePreferences.sanitizeTriggerWords(words)
@@ -537,6 +589,14 @@ final class NodeAppModel {
let resultJSON = try await self.screen.eval(javaScript: js)
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: resultJSON)
case ClawdisCameraCommand.list.rawValue:
let devices = await self.camera.listDevices()
struct Payload: Codable {
var devices: [CameraController.CameraDeviceInfo]
}
let payload = try Self.encodePayload(Payload(devices: devices))
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload)
case ClawdisCameraCommand.snap.rawValue:
self.showCameraHUD(text: "Taking photo…", kind: .photo)
self.triggerCameraFlash()
@@ -590,6 +650,9 @@ final class NodeAppModel {
NSLocalizedDescriptionKey: "INVALID_REQUEST: screen format must be mp4",
])
}
// Status pill mirrors screen recording state so it stays visible without overlay stacking.
self.screenRecordActive = true
defer { self.screenRecordActive = false }
let path = try await self.screenRecorder.record(
screenIndex: params.screenIndex,
durationMs: params.durationMs,
+104 -82
View File
@@ -51,7 +51,10 @@ struct RootCanvas: View {
case .settings:
SettingsTab()
case .chat:
ChatSheet(bridge: self.appModel.bridgeSession)
ChatSheet(
bridge: self.appModel.bridgeSession,
sessionKey: self.appModel.mainSessionKey,
userAccent: self.appModel.seamColor)
}
}
.onAppear { self.updateIdleTimer() }
@@ -119,6 +122,9 @@ struct RootCanvas: View {
}
private struct CanvasContent: View {
@Environment(NodeAppModel.self) private var appModel
@AppStorage("talk.enabled") private var talkEnabled: Bool = false
@AppStorage("talk.button.enabled") private var talkButtonEnabled: Bool = true
var systemColorScheme: ColorScheme
var bridgeStatus: StatusPill.BridgeState
var voiceWakeEnabled: Bool
@@ -140,6 +146,21 @@ private struct CanvasContent: View {
}
.accessibilityLabel("Chat")
if self.talkButtonEnabled {
// Talk mode lives on a side bubble so it doesn't get buried in settings.
OverlayButton(
systemImage: self.appModel.talkMode.isEnabled ? "waveform.circle.fill" : "waveform.circle",
brighten: self.brightenButtons,
tint: self.appModel.seamColor,
isActive: self.appModel.talkMode.isEnabled)
{
let next = !self.appModel.talkMode.isEnabled
self.talkEnabled = next
self.appModel.setTalkEnabled(next)
}
.accessibilityLabel("Talk Mode")
}
OverlayButton(systemImage: "gearshape.fill", brighten: self.brightenButtons) {
self.openSettings()
}
@@ -148,10 +169,17 @@ private struct CanvasContent: View {
.padding(.top, 10)
.padding(.trailing, 10)
}
.overlay(alignment: .center) {
if self.appModel.talkMode.isEnabled {
TalkOrbOverlay()
.transition(.opacity)
}
}
.overlay(alignment: .topLeading) {
StatusPill(
bridge: self.bridgeStatus,
voiceWakeEnabled: self.voiceWakeEnabled,
activity: self.statusActivity,
brighten: self.brightenButtons,
onTap: {
self.openSettings()
@@ -169,45 +197,78 @@ private struct CanvasContent: View {
.transition(.move(edge: .top).combined(with: .opacity))
}
}
.overlay(alignment: .topLeading) {
if let cameraHUDText, !cameraHUDText.isEmpty, let cameraHUDKind {
CameraCaptureToast(
text: cameraHUDText,
kind: self.mapCameraKind(cameraHUDKind),
brighten: self.brightenButtons)
.padding(SwiftUI.Edge.Set.leading, 10)
.safeAreaPadding(SwiftUI.Edge.Set.top, 106)
.transition(
AnyTransition.move(edge: SwiftUI.Edge.top)
.combined(with: AnyTransition.opacity))
}
}
}
private func mapCameraKind(_ kind: NodeAppModel.CameraHUDKind) -> CameraCaptureToast.Kind {
switch kind {
case .photo:
.photo
case .recording:
.recording
case .success:
.success
case .error:
.error
private var statusActivity: StatusPill.Activity? {
// Status pill owns transient activity state so it doesn't overlap the connection indicator.
if self.appModel.isBackgrounded {
return StatusPill.Activity(
title: "Foreground required",
systemImage: "exclamationmark.triangle.fill",
tint: .orange)
}
let bridgeStatus = self.appModel.bridgeStatusText.trimmingCharacters(in: .whitespacesAndNewlines)
let bridgeLower = bridgeStatus.lowercased()
if bridgeLower.contains("repair") {
return StatusPill.Activity(title: "Repairing…", systemImage: "wrench.and.screwdriver", tint: .orange)
}
if bridgeLower.contains("approval") || bridgeLower.contains("pairing") {
return StatusPill.Activity(title: "Approval pending", systemImage: "person.crop.circle.badge.clock")
}
// Avoid duplicating the primary bridge status ("Connecting…") in the activity slot.
if self.appModel.screenRecordActive {
return StatusPill.Activity(title: "Recording screen…", systemImage: "record.circle.fill", tint: .red)
}
if let cameraHUDText, !cameraHUDText.isEmpty, let cameraHUDKind {
let systemImage: String
let tint: Color?
switch cameraHUDKind {
case .photo:
systemImage = "camera.fill"
tint = nil
case .recording:
systemImage = "video.fill"
tint = .red
case .success:
systemImage = "checkmark.circle.fill"
tint = .green
case .error:
systemImage = "exclamationmark.triangle.fill"
tint = .red
}
return StatusPill.Activity(title: cameraHUDText, systemImage: systemImage, tint: tint)
}
if self.voiceWakeEnabled {
let voiceStatus = self.appModel.voiceWake.statusText
if voiceStatus.localizedCaseInsensitiveContains("microphone permission") {
return StatusPill.Activity(title: "Mic permission", systemImage: "mic.slash", tint: .orange)
}
if voiceStatus == "Paused" {
let suffix = self.appModel.isBackgrounded ? " (background)" : ""
return StatusPill.Activity(title: "Voice Wake paused\(suffix)", systemImage: "pause.circle.fill")
}
}
return nil
}
}
private struct OverlayButton: View {
let systemImage: String
let brighten: Bool
var tint: Color?
var isActive: Bool = false
let action: () -> Void
var body: some View {
Button(action: self.action) {
Image(systemName: self.systemImage)
.font(.system(size: 16, weight: .semibold))
.foregroundStyle(.primary)
.foregroundStyle(self.isActive ? (self.tint ?? .primary) : .primary)
.padding(10)
.background {
RoundedRectangle(cornerRadius: 12, style: .continuous)
@@ -225,9 +286,26 @@ private struct OverlayButton: View {
endPoint: .bottomTrailing))
.blendMode(.overlay)
}
.overlay {
if let tint {
RoundedRectangle(cornerRadius: 12, style: .continuous)
.fill(
LinearGradient(
colors: [
tint.opacity(self.isActive ? 0.22 : 0.14),
tint.opacity(self.isActive ? 0.10 : 0.06),
.clear,
],
startPoint: .topLeading,
endPoint: .bottomTrailing))
.blendMode(.overlay)
}
}
.overlay {
RoundedRectangle(cornerRadius: 12, style: .continuous)
.strokeBorder(.white.opacity(self.brighten ? 0.24 : 0.18), lineWidth: 0.5)
.strokeBorder(
(self.tint ?? .white).opacity(self.isActive ? 0.34 : (self.brighten ? 0.24 : 0.18)),
lineWidth: self.isActive ? 0.7 : 0.5)
}
.shadow(color: .black.opacity(0.35), radius: 12, y: 6)
}
@@ -261,59 +339,3 @@ private struct CameraFlashOverlay: View {
}
}
}
private struct CameraCaptureToast: View {
enum Kind {
case photo
case recording
case success
case error
}
var text: String
var kind: Kind
var brighten: Bool = false
var body: some View {
HStack(spacing: 10) {
self.icon
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.primary)
Text(self.text)
.font(.system(size: 14, weight: .semibold))
.foregroundStyle(.primary)
.lineLimit(1)
.truncationMode(.tail)
}
.padding(.vertical, 10)
.padding(.horizontal, 12)
.background {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.fill(.ultraThinMaterial)
.overlay {
RoundedRectangle(cornerRadius: 14, style: .continuous)
.strokeBorder(.white.opacity(self.brighten ? 0.24 : 0.18), lineWidth: 0.5)
}
.shadow(color: .black.opacity(0.25), radius: 12, y: 6)
}
.accessibilityLabel("Camera")
.accessibilityValue(self.text)
}
@ViewBuilder
private var icon: some View {
switch self.kind {
case .photo:
Image(systemName: "camera.fill")
case .recording:
Image(systemName: "record.circle.fill")
.symbolRenderingMode(.palette)
.foregroundStyle(.red, .primary)
case .success:
Image(systemName: "checkmark.circle.fill")
case .error:
Image(systemName: "exclamationmark.triangle.fill")
}
}
}
+61
View File
@@ -26,6 +26,7 @@ struct RootTabs: View {
StatusPill(
bridge: self.bridgeStatus,
voiceWakeEnabled: self.voiceWakeEnabled,
activity: self.statusActivity,
onTap: { self.selectedTab = 2 })
.padding(.leading, 10)
.safeAreaPadding(.top, 10)
@@ -79,4 +80,64 @@ struct RootTabs: View {
return .disconnected
}
private var statusActivity: StatusPill.Activity? {
// Keep the top pill consistent across tabs (camera + voice wake + pairing states).
if self.appModel.isBackgrounded {
return StatusPill.Activity(
title: "Foreground required",
systemImage: "exclamationmark.triangle.fill",
tint: .orange)
}
let bridgeStatus = self.appModel.bridgeStatusText.trimmingCharacters(in: .whitespacesAndNewlines)
let bridgeLower = bridgeStatus.lowercased()
if bridgeLower.contains("repair") {
return StatusPill.Activity(title: "Repairing…", systemImage: "wrench.and.screwdriver", tint: .orange)
}
if bridgeLower.contains("approval") || bridgeLower.contains("pairing") {
return StatusPill.Activity(title: "Approval pending", systemImage: "person.crop.circle.badge.clock")
}
// Avoid duplicating the primary bridge status ("Connecting…") in the activity slot.
if self.appModel.screenRecordActive {
return StatusPill.Activity(title: "Recording screen…", systemImage: "record.circle.fill", tint: .red)
}
if let cameraHUDText = self.appModel.cameraHUDText,
let cameraHUDKind = self.appModel.cameraHUDKind,
!cameraHUDText.isEmpty
{
let systemImage: String
let tint: Color?
switch cameraHUDKind {
case .photo:
systemImage = "camera.fill"
tint = nil
case .recording:
systemImage = "video.fill"
tint = .red
case .success:
systemImage = "checkmark.circle.fill"
tint = .green
case .error:
systemImage = "exclamationmark.triangle.fill"
tint = .red
}
return StatusPill.Activity(title: cameraHUDText, systemImage: systemImage, tint: tint)
}
if self.voiceWakeEnabled {
let voiceStatus = self.appModel.voiceWake.statusText
if voiceStatus.localizedCaseInsensitiveContains("microphone permission") {
return StatusPill.Activity(title: "Mic permission", systemImage: "mic.slash", tint: .orange)
}
if voiceStatus == "Paused" {
let suffix = self.appModel.isBackgrounded ? " (background)" : ""
return StatusPill.Activity(title: "Voice Wake paused\(suffix)", systemImage: "pause.circle.fill")
}
}
return nil
}
}
+11 -3
View File
@@ -43,9 +43,7 @@ final class ScreenController {
self.webView.scrollView.contentInset = .zero
self.webView.scrollView.scrollIndicatorInsets = .zero
self.webView.scrollView.automaticallyAdjustsScrollIndicatorInsets = false
// Disable scroll to allow touch events to pass through to canvas
self.webView.scrollView.isScrollEnabled = false
self.webView.scrollView.bounces = false
self.applyScrollBehavior()
self.webView.navigationDelegate = self.navigationDelegate
self.navigationDelegate.controller = self
a2uiActionHandler.controller = self
@@ -60,6 +58,7 @@ final class ScreenController {
func reload() {
let trimmed = self.urlString.trimmingCharacters(in: .whitespacesAndNewlines)
self.applyScrollBehavior()
if trimmed.isEmpty {
guard let url = Self.canvasScaffoldURL else { return }
self.errorText = nil
@@ -250,6 +249,15 @@ final class ScreenController {
return false
}
private func applyScrollBehavior() {
let trimmed = self.urlString.trimmingCharacters(in: .whitespacesAndNewlines)
let allowScroll = !trimmed.isEmpty
let scrollView = self.webView.scrollView
// Default canvas needs raw touch events; external pages should scroll.
scrollView.isScrollEnabled = allowScroll
scrollView.bounces = allowScroll
}
private static func jsValue(_ value: String?) -> String {
guard let value else { return "null" }
if let data = try? JSONSerialization.data(withJSONObject: [value]),
+161 -97
View File
@@ -1,12 +1,28 @@
import AVFoundation
import ReplayKit
@MainActor
final class ScreenRecordService {
final class ScreenRecordService: @unchecked Sendable {
private struct UncheckedSendableBox<T>: @unchecked Sendable {
let value: T
}
private final class CaptureState: @unchecked Sendable {
private let lock = NSLock()
var writer: AVAssetWriter?
var videoInput: AVAssetWriterInput?
var audioInput: AVAssetWriterInput?
var started = false
var sawVideo = false
var lastVideoTime: CMTime?
var handlerError: Error?
func withLock<T>(_ body: (CaptureState) -> T) -> T {
self.lock.lock()
defer { lock.unlock() }
return body(self)
}
}
enum ScreenRecordError: LocalizedError {
case invalidScreenIndex(Int)
case captureFailed(String)
@@ -51,126 +67,158 @@ final class ScreenRecordService {
}()
try? FileManager.default.removeItem(at: outURL)
let recorder = RPScreenRecorder.shared()
recorder.isMicrophoneEnabled = includeAudio
var writer: AVAssetWriter?
var videoInput: AVAssetWriterInput?
var audioInput: AVAssetWriterInput?
var started = false
var sawVideo = false
var lastVideoTime: CMTime?
var handlerError: Error?
let lock = NSLock()
func setHandlerError(_ error: Error) {
lock.lock()
defer { lock.unlock() }
if handlerError == nil { handlerError = error }
}
let state = CaptureState()
let recordQueue = DispatchQueue(label: "com.steipete.clawdis.screenrecord")
try await withCheckedThrowingContinuation { (cont: CheckedContinuation<Void, Error>) in
recorder.startCapture(handler: { sample, type, error in
if let error {
setHandlerError(error)
return
}
guard CMSampleBufferDataIsReady(sample) else { return }
switch type {
case .video:
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
if let lastVideoTime {
let delta = CMTimeSubtract(pts, lastVideoTime)
if delta.seconds < (1.0 / fpsValue) { return }
}
if writer == nil {
guard let imageBuffer = CMSampleBufferGetImageBuffer(sample) else {
setHandlerError(ScreenRecordError.captureFailed("Missing image buffer"))
return
let handler: @Sendable (CMSampleBuffer, RPSampleBufferType, Error?) -> Void = { sample, type, error in
// ReplayKit can call the capture handler on a background queue.
// Serialize writes to avoid queue asserts.
recordQueue.async {
if let error {
state.withLock { state in
if state.handlerError == nil { state.handlerError = error }
}
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
do {
let w = try AVAssetWriter(outputURL: outURL, fileType: .mp4)
let settings: [String: Any] = [
AVVideoCodecKey: AVVideoCodecType.h264,
AVVideoWidthKey: width,
AVVideoHeightKey: height,
]
let vInput = AVAssetWriterInput(mediaType: .video, outputSettings: settings)
vInput.expectsMediaDataInRealTime = true
guard w.canAdd(vInput) else {
throw ScreenRecordError.writeFailed("Cannot add video input")
}
w.add(vInput)
return
}
guard CMSampleBufferDataIsReady(sample) else { return }
if includeAudio {
let aInput = AVAssetWriterInput(mediaType: .audio, outputSettings: nil)
aInput.expectsMediaDataInRealTime = true
if w.canAdd(aInput) {
w.add(aInput)
audioInput = aInput
switch type {
case .video:
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
let shouldSkip = state.withLock { state in
if let lastVideoTime = state.lastVideoTime {
let delta = CMTimeSubtract(pts, lastVideoTime)
return delta.seconds < (1.0 / fpsValue)
}
return false
}
if shouldSkip { return }
if state.withLock({ $0.writer == nil }) {
guard let imageBuffer = CMSampleBufferGetImageBuffer(sample) else {
state.withLock { state in
if state.handlerError == nil {
state.handlerError = ScreenRecordError.captureFailed("Missing image buffer")
}
}
return
}
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
do {
let w = try AVAssetWriter(outputURL: outURL, fileType: .mp4)
let settings: [String: Any] = [
AVVideoCodecKey: AVVideoCodecType.h264,
AVVideoWidthKey: width,
AVVideoHeightKey: height,
]
let vInput = AVAssetWriterInput(mediaType: .video, outputSettings: settings)
vInput.expectsMediaDataInRealTime = true
guard w.canAdd(vInput) else {
throw ScreenRecordError.writeFailed("Cannot add video input")
}
w.add(vInput)
if includeAudio {
let aInput = AVAssetWriterInput(mediaType: .audio, outputSettings: nil)
aInput.expectsMediaDataInRealTime = true
if w.canAdd(aInput) {
w.add(aInput)
state.withLock { state in
state.audioInput = aInput
}
}
}
guard w.startWriting() else {
throw ScreenRecordError
.writeFailed(w.error?.localizedDescription ?? "Failed to start writer")
}
w.startSession(atSourceTime: pts)
state.withLock { state in
state.writer = w
state.videoInput = vInput
state.started = true
}
} catch {
state.withLock { state in
if state.handlerError == nil { state.handlerError = error }
}
return
}
}
let vInput = state.withLock { $0.videoInput }
let isStarted = state.withLock { $0.started }
guard let vInput, isStarted else { return }
if vInput.isReadyForMoreMediaData {
if vInput.append(sample) {
state.withLock { state in
state.sawVideo = true
state.lastVideoTime = pts
}
} else {
let err = state.withLock { $0.writer?.error }
if let err {
state.withLock { state in
if state.handlerError == nil {
state.handlerError = ScreenRecordError.writeFailed(err.localizedDescription)
}
}
}
}
guard w.startWriting() else {
throw ScreenRecordError
.writeFailed(w.error?.localizedDescription ?? "Failed to start writer")
}
w.startSession(atSourceTime: pts)
writer = w
videoInput = vInput
started = true
} catch {
setHandlerError(error)
return
}
}
guard let vInput = videoInput, started else { return }
if vInput.isReadyForMoreMediaData {
if vInput.append(sample) {
sawVideo = true
lastVideoTime = pts
} else {
if let err = writer?.error {
setHandlerError(ScreenRecordError.writeFailed(err.localizedDescription))
}
case .audioApp, .audioMic:
let aInput = state.withLock { $0.audioInput }
let isStarted = state.withLock { $0.started }
guard includeAudio, let aInput, isStarted else { return }
if aInput.isReadyForMoreMediaData {
_ = aInput.append(sample)
}
}
case .audioApp, .audioMic:
guard includeAudio, let aInput = audioInput, started else { return }
if aInput.isReadyForMoreMediaData {
_ = aInput.append(sample)
@unknown default:
break
}
@unknown default:
break
}
}, completionHandler: { error in
}
let completion: @Sendable (Error?) -> Void = { error in
if let error { cont.resume(throwing: error) } else { cont.resume() }
})
}
Task { @MainActor in
startReplayKitCapture(
includeAudio: includeAudio,
handler: handler,
completion: completion)
}
}
try await Task.sleep(nanoseconds: UInt64(durationMs) * 1_000_000)
let stopError = await withCheckedContinuation { cont in
recorder.stopCapture { error in cont.resume(returning: error) }
Task { @MainActor in
stopReplayKitCapture { error in cont.resume(returning: error) }
}
}
if let stopError { throw stopError }
if let handlerError { throw handlerError }
guard let writer, let videoInput, sawVideo else {
let handlerErrorSnapshot = state.withLock { $0.handlerError }
if let handlerErrorSnapshot { throw handlerErrorSnapshot }
let writerSnapshot = state.withLock { $0.writer }
let videoInputSnapshot = state.withLock { $0.videoInput }
let audioInputSnapshot = state.withLock { $0.audioInput }
let sawVideoSnapshot = state.withLock { $0.sawVideo }
guard let writerSnapshot, let videoInputSnapshot, sawVideoSnapshot else {
throw ScreenRecordError.captureFailed("No frames captured")
}
videoInput.markAsFinished()
audioInput?.markAsFinished()
videoInputSnapshot.markAsFinished()
audioInputSnapshot?.markAsFinished()
let writerBox = UncheckedSendableBox(value: writer)
let writerBox = UncheckedSendableBox(value: writerSnapshot)
try await withCheckedThrowingContinuation { (cont: CheckedContinuation<Void, Error>) in
writerBox.value.finishWriting {
let writer = writerBox.value
@@ -199,6 +247,22 @@ final class ScreenRecordService {
}
}
@MainActor
private func startReplayKitCapture(
includeAudio: Bool,
handler: @escaping @Sendable (CMSampleBuffer, RPSampleBufferType, Error?) -> Void,
completion: @escaping @Sendable (Error?) -> Void)
{
let recorder = RPScreenRecorder.shared()
recorder.isMicrophoneEnabled = includeAudio
recorder.startCapture(handler: handler, completionHandler: completion)
}
@MainActor
private func stopReplayKitCapture(_ completion: @escaping @Sendable (Error?) -> Void) {
RPScreenRecorder.shared().stopCapture { error in completion(error) }
}
#if DEBUG
extension ScreenRecordService {
nonisolated static func _test_clampDurationMs(_ ms: Int?) -> Int {
@@ -20,6 +20,8 @@ struct SettingsTab: View {
@AppStorage("node.displayName") private var displayName: String = "iOS Node"
@AppStorage("node.instanceId") private var instanceId: String = UUID().uuidString
@AppStorage("voiceWake.enabled") private var voiceWakeEnabled: Bool = false
@AppStorage("talk.enabled") private var talkEnabled: Bool = false
@AppStorage("talk.button.enabled") private var talkButtonEnabled: Bool = true
@AppStorage("camera.enabled") private var cameraEnabled: Bool = true
@AppStorage("screen.preventSleep") private var preventSleep: Bool = true
@AppStorage("bridge.preferredStableID") private var preferredBridgeStableID: String = ""
@@ -51,6 +53,9 @@ struct SettingsTab: View {
}
}
}
LabeledContent("Platform", value: self.platformString())
LabeledContent("Version", value: self.appVersion())
LabeledContent("Model", value: self.modelIdentifier())
}
Section("Bridge") {
@@ -153,6 +158,12 @@ struct SettingsTab: View {
.onChange(of: self.voiceWakeEnabled) { _, newValue in
self.appModel.setVoiceWakeEnabled(newValue)
}
Toggle("Talk Mode", isOn: self.$talkEnabled)
.onChange(of: self.talkEnabled) { _, newValue in
self.appModel.setTalkEnabled(newValue)
}
// Keep this separate so users can hide the side bubble without disabling Talk Mode.
Toggle("Show Talk Button", isOn: self.$talkButtonEnabled)
NavigationLink {
VoiceWakeWordsSettingsView()
@@ -227,6 +238,12 @@ struct SettingsTab: View {
HStack {
VStack(alignment: .leading, spacing: 2) {
Text(bridge.name)
let detailLines = self.bridgeDetailLines(bridge)
ForEach(detailLines, id: \.self) { line in
Text(line)
.font(.footnote)
.foregroundStyle(.secondary)
}
}
Spacer()
@@ -316,6 +333,7 @@ struct SettingsTab: View {
let caps = Set(self.currentCaps())
if caps.contains(ClawdisCapability.camera.rawValue) {
commands.append(ClawdisCameraCommand.list.rawValue)
commands.append(ClawdisCameraCommand.snap.rawValue)
commands.append(ClawdisCameraCommand.clip.rawValue)
}
@@ -504,4 +522,26 @@ struct SettingsTab: View {
private static func httpURLString(host: String?, port: Int?, fallback: String) -> String {
SettingsNetworkingHelpers.httpURLString(host: host, port: port, fallback: fallback)
}
private func bridgeDetailLines(_ bridge: BridgeDiscoveryModel.DiscoveredBridge) -> [String] {
var lines: [String] = []
if let lanHost = bridge.lanHost { lines.append("LAN: \(lanHost)") }
if let tailnet = bridge.tailnetDns { lines.append("Tailnet: \(tailnet)") }
let gatewayPort = bridge.gatewayPort
let bridgePort = bridge.bridgePort
let canvasPort = bridge.canvasPort
if gatewayPort != nil || bridgePort != nil || canvasPort != nil {
let gw = gatewayPort.map(String.init) ?? "—"
let br = bridgePort.map(String.init) ?? "—"
let canvas = canvasPort.map(String.init) ?? "—"
lines.append("Ports: gw \(gw) · bridge \(br) · canvas \(canvas)")
}
if lines.isEmpty {
lines.append(bridge.debugID)
}
return lines
}
}
+34 -5
View File
@@ -28,8 +28,15 @@ struct StatusPill: View {
}
}
struct Activity: Equatable {
var title: String
var systemImage: String
var tint: Color?
}
var bridge: BridgeState
var voiceWakeEnabled: Bool
var activity: Activity?
var brighten: Bool = false
var onTap: () -> Void
@@ -54,10 +61,24 @@ struct StatusPill: View {
.frame(height: 14)
.opacity(0.35)
Image(systemName: self.voiceWakeEnabled ? "mic.fill" : "mic.slash")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(self.voiceWakeEnabled ? .primary : .secondary)
.accessibilityLabel(self.voiceWakeEnabled ? "Voice Wake enabled" : "Voice Wake disabled")
if let activity {
HStack(spacing: 6) {
Image(systemName: activity.systemImage)
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(activity.tint ?? .primary)
Text(activity.title)
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(.primary)
.lineLimit(1)
}
.transition(.opacity.combined(with: .move(edge: .top)))
} else {
Image(systemName: self.voiceWakeEnabled ? "mic.fill" : "mic.slash")
.font(.system(size: 13, weight: .semibold))
.foregroundStyle(self.voiceWakeEnabled ? .primary : .secondary)
.accessibilityLabel(self.voiceWakeEnabled ? "Voice Wake enabled" : "Voice Wake disabled")
.transition(.opacity.combined(with: .move(edge: .top)))
}
}
.padding(.vertical, 8)
.padding(.horizontal, 12)
@@ -73,7 +94,7 @@ struct StatusPill: View {
}
.buttonStyle(.plain)
.accessibilityLabel("Status")
.accessibilityValue("\(self.bridge.title), Voice Wake \(self.voiceWakeEnabled ? "enabled" : "disabled")")
.accessibilityValue(self.accessibilityValue)
.onAppear { self.updatePulse(for: self.bridge, scenePhase: self.scenePhase) }
.onDisappear { self.pulse = false }
.onChange(of: self.bridge) { _, newValue in
@@ -82,6 +103,14 @@ struct StatusPill: View {
.onChange(of: self.scenePhase) { _, newValue in
self.updatePulse(for: self.bridge, scenePhase: newValue)
}
.animation(.easeInOut(duration: 0.18), value: self.activity?.title)
}
private var accessibilityValue: String {
if let activity {
return "\(self.bridge.title), \(activity.title)"
}
return "\(self.bridge.title), Voice Wake \(self.voiceWakeEnabled ? "enabled" : "disabled")"
}
private func updatePulse(for bridge: BridgeState, scenePhase: ScenePhase) {
@@ -0,0 +1,713 @@
import AVFAudio
import ClawdisKit
import Foundation
import Observation
import OSLog
import Speech
@MainActor
@Observable
final class TalkModeManager: NSObject {
private typealias SpeechRequest = SFSpeechAudioBufferRecognitionRequest
private static let defaultModelIdFallback = "eleven_v3"
var isEnabled: Bool = false
var isListening: Bool = false
var isSpeaking: Bool = false
var statusText: String = "Off"
private let audioEngine = AVAudioEngine()
private var speechRecognizer: SFSpeechRecognizer?
private var recognitionRequest: SFSpeechAudioBufferRecognitionRequest?
private var recognitionTask: SFSpeechRecognitionTask?
private var silenceTask: Task<Void, Never>?
private var lastHeard: Date?
private var lastTranscript: String = ""
private var lastSpokenText: String?
private var lastInterruptedAtSeconds: Double?
private var defaultVoiceId: String?
private var currentVoiceId: String?
private var defaultModelId: String?
private var currentModelId: String?
private var voiceOverrideActive = false
private var modelOverrideActive = false
private var defaultOutputFormat: String?
private var apiKey: String?
private var voiceAliases: [String: String] = [:]
private var interruptOnSpeech: Bool = true
private var mainSessionKey: String = "main"
private var fallbackVoiceId: String?
private var lastPlaybackWasPCM: Bool = false
var pcmPlayer: PCMStreamingAudioPlaying = PCMStreamingAudioPlayer.shared
var mp3Player: StreamingAudioPlaying = StreamingAudioPlayer.shared
private var bridge: BridgeSession?
private let silenceWindow: TimeInterval = 0.7
private var chatSubscribedSessionKeys = Set<String>()
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "TalkMode")
func attachBridge(_ bridge: BridgeSession) {
self.bridge = bridge
}
func setEnabled(_ enabled: Bool) {
self.isEnabled = enabled
if enabled {
self.logger.info("enabled")
Task { await self.start() }
} else {
self.logger.info("disabled")
self.stop()
}
}
func start() async {
guard self.isEnabled else { return }
if self.isListening { return }
self.logger.info("start")
self.statusText = "Requesting permissions…"
let micOk = await Self.requestMicrophonePermission()
guard micOk else {
self.logger.warning("start blocked: microphone permission denied")
self.statusText = "Microphone permission denied"
return
}
let speechOk = await Self.requestSpeechPermission()
guard speechOk else {
self.logger.warning("start blocked: speech permission denied")
self.statusText = "Speech recognition permission denied"
return
}
await self.reloadConfig()
do {
try Self.configureAudioSession()
try self.startRecognition()
self.isListening = true
self.statusText = "Listening"
self.startSilenceMonitor()
await self.subscribeChatIfNeeded(sessionKey: self.mainSessionKey)
self.logger.info("listening")
} catch {
self.isListening = false
self.statusText = "Start failed: \(error.localizedDescription)"
self.logger.error("start failed: \(error.localizedDescription, privacy: .public)")
}
}
func stop() {
self.isEnabled = false
self.isListening = false
self.statusText = "Off"
self.lastTranscript = ""
self.lastHeard = nil
self.silenceTask?.cancel()
self.silenceTask = nil
self.stopRecognition()
self.stopSpeaking()
self.lastInterruptedAtSeconds = nil
TalkSystemSpeechSynthesizer.shared.stop()
do {
try AVAudioSession.sharedInstance().setActive(false, options: [.notifyOthersOnDeactivation])
} catch {
self.logger.warning("audio session deactivate failed: \(error.localizedDescription, privacy: .public)")
}
Task { await self.unsubscribeAllChats() }
}
func userTappedOrb() {
self.stopSpeaking()
}
private func startRecognition() throws {
self.stopRecognition()
self.speechRecognizer = SFSpeechRecognizer()
guard let recognizer = self.speechRecognizer else {
throw NSError(domain: "TalkMode", code: 1, userInfo: [
NSLocalizedDescriptionKey: "Speech recognizer unavailable",
])
}
self.recognitionRequest = SFSpeechAudioBufferRecognitionRequest()
self.recognitionRequest?.shouldReportPartialResults = true
guard let request = self.recognitionRequest else { return }
let input = self.audioEngine.inputNode
let format = input.outputFormat(forBus: 0)
input.removeTap(onBus: 0)
let tapBlock = Self.makeAudioTapAppendCallback(request: request)
input.installTap(onBus: 0, bufferSize: 2048, format: format, block: tapBlock)
self.audioEngine.prepare()
try self.audioEngine.start()
self.recognitionTask = recognizer.recognitionTask(with: request) { [weak self] result, error in
guard let self else { return }
if let error {
if !self.isSpeaking {
self.statusText = "Speech error: \(error.localizedDescription)"
}
self.logger.debug("speech recognition error: \(error.localizedDescription, privacy: .public)")
}
guard let result else { return }
let transcript = result.bestTranscription.formattedString
Task { @MainActor in
await self.handleTranscript(transcript: transcript, isFinal: result.isFinal)
}
}
}
private func stopRecognition() {
self.recognitionTask?.cancel()
self.recognitionTask = nil
self.recognitionRequest?.endAudio()
self.recognitionRequest = nil
self.audioEngine.inputNode.removeTap(onBus: 0)
self.audioEngine.stop()
self.speechRecognizer = nil
}
private nonisolated static func makeAudioTapAppendCallback(request: SpeechRequest) -> AVAudioNodeTapBlock {
{ buffer, _ in
request.append(buffer)
}
}
private func handleTranscript(transcript: String, isFinal: Bool) async {
let trimmed = transcript.trimmingCharacters(in: .whitespacesAndNewlines)
if self.isSpeaking, self.interruptOnSpeech {
if self.shouldInterrupt(with: trimmed) {
self.stopSpeaking()
}
return
}
guard self.isListening else { return }
if !trimmed.isEmpty {
self.lastTranscript = trimmed
self.lastHeard = Date()
}
if isFinal {
self.lastTranscript = trimmed
}
}
private func startSilenceMonitor() {
self.silenceTask?.cancel()
self.silenceTask = Task { [weak self] in
guard let self else { return }
while self.isEnabled {
try? await Task.sleep(nanoseconds: 200_000_000)
await self.checkSilence()
}
}
}
private func checkSilence() async {
guard self.isListening, !self.isSpeaking else { return }
let transcript = self.lastTranscript.trimmingCharacters(in: .whitespacesAndNewlines)
guard !transcript.isEmpty else { return }
guard let lastHeard else { return }
if Date().timeIntervalSince(lastHeard) < self.silenceWindow { return }
await self.finalizeTranscript(transcript)
}
private func finalizeTranscript(_ transcript: String) async {
self.isListening = false
self.statusText = "Thinking…"
self.lastTranscript = ""
self.lastHeard = nil
self.stopRecognition()
await self.reloadConfig()
let prompt = self.buildPrompt(transcript: transcript)
guard let bridge else {
self.statusText = "Bridge not connected"
self.logger.warning("finalize: bridge not connected")
await self.start()
return
}
do {
let startedAt = Date().timeIntervalSince1970
let sessionKey = self.mainSessionKey
await self.subscribeChatIfNeeded(sessionKey: sessionKey)
self.logger.info(
"chat.send start sessionKey=\(sessionKey, privacy: .public) chars=\(prompt.count, privacy: .public)")
let runId = try await self.sendChat(prompt, bridge: bridge)
self.logger.info("chat.send ok runId=\(runId, privacy: .public)")
let completion = await self.waitForChatCompletion(runId: runId, bridge: bridge, timeoutSeconds: 120)
if completion == .timeout {
self.logger.warning(
"chat completion timeout runId=\(runId, privacy: .public); attempting history fallback")
} else if completion == .aborted {
self.statusText = "Aborted"
self.logger.warning("chat completion aborted runId=\(runId, privacy: .public)")
await self.start()
return
} else if completion == .error {
self.statusText = "Chat error"
self.logger.warning("chat completion error runId=\(runId, privacy: .public)")
await self.start()
return
}
guard let assistantText = try await self.waitForAssistantText(
bridge: bridge,
since: startedAt,
timeoutSeconds: completion == .final ? 12 : 25)
else {
self.statusText = "No reply"
self.logger.warning("assistant text timeout runId=\(runId, privacy: .public)")
await self.start()
return
}
self.logger.info("assistant text ok chars=\(assistantText.count, privacy: .public)")
await self.playAssistant(text: assistantText)
} catch {
self.statusText = "Talk failed: \(error.localizedDescription)"
self.logger.error("finalize failed: \(error.localizedDescription, privacy: .public)")
}
await self.start()
}
private func subscribeChatIfNeeded(sessionKey: String) async {
let key = sessionKey.trimmingCharacters(in: .whitespacesAndNewlines)
guard !key.isEmpty else { return }
guard let bridge else { return }
guard !self.chatSubscribedSessionKeys.contains(key) else { return }
do {
let payload = "{\"sessionKey\":\"\(key)\"}"
try await bridge.sendEvent(event: "chat.subscribe", payloadJSON: payload)
self.chatSubscribedSessionKeys.insert(key)
self.logger.info("chat.subscribe ok sessionKey=\(key, privacy: .public)")
} catch {
self.logger
.warning(
"chat.subscribe failed sessionKey=\(key, privacy: .public) err=\(error.localizedDescription, privacy: .public)")
}
}
private func unsubscribeAllChats() async {
guard let bridge else { return }
let keys = self.chatSubscribedSessionKeys
self.chatSubscribedSessionKeys.removeAll()
for key in keys {
do {
let payload = "{\"sessionKey\":\"\(key)\"}"
try await bridge.sendEvent(event: "chat.unsubscribe", payloadJSON: payload)
} catch {
// ignore
}
}
}
private func buildPrompt(transcript: String) -> String {
let interrupted = self.lastInterruptedAtSeconds
self.lastInterruptedAtSeconds = nil
return TalkPromptBuilder.build(transcript: transcript, interruptedAtSeconds: interrupted)
}
private enum ChatCompletionState: CustomStringConvertible {
case final
case aborted
case error
case timeout
var description: String {
switch self {
case .final: "final"
case .aborted: "aborted"
case .error: "error"
case .timeout: "timeout"
}
}
}
private func sendChat(_ message: String, bridge: BridgeSession) async throws -> String {
struct SendResponse: Decodable { let runId: String }
let payload: [String: Any] = [
"sessionKey": self.mainSessionKey,
"message": message,
"thinking": "low",
"timeoutMs": 30000,
"idempotencyKey": UUID().uuidString,
]
let data = try JSONSerialization.data(withJSONObject: payload)
let json = String(decoding: data, as: UTF8.self)
let res = try await bridge.request(method: "chat.send", paramsJSON: json, timeoutSeconds: 30)
let decoded = try JSONDecoder().decode(SendResponse.self, from: res)
return decoded.runId
}
private func waitForChatCompletion(
runId: String,
bridge: BridgeSession,
timeoutSeconds: Int = 120) async -> ChatCompletionState
{
let stream = await bridge.subscribeServerEvents(bufferingNewest: 200)
return await withTaskGroup(of: ChatCompletionState.self) { group in
group.addTask { [runId] in
for await evt in stream {
if Task.isCancelled { return .timeout }
guard evt.event == "chat", let payload = evt.payloadJSON else { continue }
guard let data = payload.data(using: .utf8) else { continue }
guard let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { continue }
if (json["runId"] as? String) != runId { continue }
if let state = json["state"] as? String {
switch state {
case "final": return .final
case "aborted": return .aborted
case "error": return .error
default: break
}
}
}
return .timeout
}
group.addTask {
try? await Task.sleep(nanoseconds: UInt64(timeoutSeconds) * 1_000_000_000)
return .timeout
}
let result = await group.next() ?? .timeout
group.cancelAll()
return result
}
}
private func waitForAssistantText(
bridge: BridgeSession,
since: Double,
timeoutSeconds: Int) async throws -> String?
{
let deadline = Date().addingTimeInterval(TimeInterval(timeoutSeconds))
while Date() < deadline {
if let text = try await self.fetchLatestAssistantText(bridge: bridge, since: since) {
return text
}
try? await Task.sleep(nanoseconds: 300_000_000)
}
return nil
}
private func fetchLatestAssistantText(bridge: BridgeSession, since: Double? = nil) async throws -> String? {
let res = try await bridge.request(
method: "chat.history",
paramsJSON: "{\"sessionKey\":\"\(self.mainSessionKey)\"}",
timeoutSeconds: 15)
guard let json = try JSONSerialization.jsonObject(with: res) as? [String: Any] else { return nil }
guard let messages = json["messages"] as? [[String: Any]] else { return nil }
for msg in messages.reversed() {
guard (msg["role"] as? String) == "assistant" else { continue }
if let since, let timestamp = msg["timestamp"] as? Double,
TalkHistoryTimestamp.isAfter(timestamp, sinceSeconds: since) == false
{
continue
}
guard let content = msg["content"] as? [[String: Any]] else { continue }
let text = content.compactMap { $0["text"] as? String }.joined(separator: "\n")
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty { return trimmed }
}
return nil
}
private func playAssistant(text: String) async {
let parsed = TalkDirectiveParser.parse(text)
let directive = parsed.directive
let cleaned = parsed.stripped.trimmingCharacters(in: .whitespacesAndNewlines)
guard !cleaned.isEmpty else { return }
let requestedVoice = directive?.voiceId?.trimmingCharacters(in: .whitespacesAndNewlines)
let resolvedVoice = self.resolveVoiceAlias(requestedVoice)
if requestedVoice?.isEmpty == false, resolvedVoice == nil {
self.logger.warning("unknown voice alias \(requestedVoice ?? "?", privacy: .public)")
}
if let voice = resolvedVoice {
if directive?.once != true {
self.currentVoiceId = voice
self.voiceOverrideActive = true
}
}
if let model = directive?.modelId {
if directive?.once != true {
self.currentModelId = model
self.modelOverrideActive = true
}
}
self.statusText = "Generating voice…"
self.isSpeaking = true
self.lastSpokenText = cleaned
do {
let started = Date()
let language = ElevenLabsTTSClient.validatedLanguage(directive?.language)
let resolvedKey =
(self.apiKey?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false ? self.apiKey : nil) ??
ProcessInfo.processInfo.environment["ELEVENLABS_API_KEY"]
let apiKey = resolvedKey?.trimmingCharacters(in: .whitespacesAndNewlines)
let preferredVoice = resolvedVoice ?? self.currentVoiceId ?? self.defaultVoiceId
let voiceId: String? = if let apiKey, !apiKey.isEmpty {
await self.resolveVoiceId(preferred: preferredVoice, apiKey: apiKey)
} else {
nil
}
let canUseElevenLabs = (voiceId?.isEmpty == false) && (apiKey?.isEmpty == false)
if canUseElevenLabs, let voiceId, let apiKey {
let desiredOutputFormat = (directive?.outputFormat ?? self.defaultOutputFormat)?
.trimmingCharacters(in: .whitespacesAndNewlines)
let requestedOutputFormat = (desiredOutputFormat?.isEmpty == false) ? desiredOutputFormat : nil
let outputFormat = ElevenLabsTTSClient.validatedOutputFormat(requestedOutputFormat ?? "pcm_44100")
if outputFormat == nil, let requestedOutputFormat {
self.logger.warning(
"talk output_format unsupported for local playback: \(requestedOutputFormat, privacy: .public)")
}
let modelId = directive?.modelId ?? self.currentModelId ?? self.defaultModelId
func makeRequest(outputFormat: String?) -> ElevenLabsTTSRequest {
ElevenLabsTTSRequest(
text: cleaned,
modelId: modelId,
outputFormat: outputFormat,
speed: TalkTTSValidation.resolveSpeed(speed: directive?.speed, rateWPM: directive?.rateWPM),
stability: TalkTTSValidation.validatedStability(directive?.stability, modelId: modelId),
similarity: TalkTTSValidation.validatedUnit(directive?.similarity),
style: TalkTTSValidation.validatedUnit(directive?.style),
speakerBoost: directive?.speakerBoost,
seed: TalkTTSValidation.validatedSeed(directive?.seed),
normalize: ElevenLabsTTSClient.validatedNormalize(directive?.normalize),
language: language,
latencyTier: TalkTTSValidation.validatedLatencyTier(directive?.latencyTier))
}
let request = makeRequest(outputFormat: outputFormat)
let client = ElevenLabsTTSClient(apiKey: apiKey)
let stream = client.streamSynthesize(voiceId: voiceId, request: request)
if self.interruptOnSpeech {
do {
try self.startRecognition()
} catch {
self.logger.warning(
"startRecognition during speak failed: \(error.localizedDescription, privacy: .public)")
}
}
self.statusText = "Speaking…"
let sampleRate = TalkTTSValidation.pcmSampleRate(from: outputFormat)
let result: StreamingPlaybackResult
if let sampleRate {
self.lastPlaybackWasPCM = true
var playback = await self.pcmPlayer.play(stream: stream, sampleRate: sampleRate)
if !playback.finished, playback.interruptedAt == nil {
let mp3Format = ElevenLabsTTSClient.validatedOutputFormat("mp3_44100")
self.logger.warning("pcm playback failed; retrying mp3")
self.lastPlaybackWasPCM = false
let mp3Stream = client.streamSynthesize(
voiceId: voiceId,
request: makeRequest(outputFormat: mp3Format))
playback = await self.mp3Player.play(stream: mp3Stream)
}
result = playback
} else {
self.lastPlaybackWasPCM = false
result = await self.mp3Player.play(stream: stream)
}
self.logger
.info(
"elevenlabs stream finished=\(result.finished, privacy: .public) dur=\(Date().timeIntervalSince(started), privacy: .public)s")
if !result.finished, let interruptedAt = result.interruptedAt {
self.lastInterruptedAtSeconds = interruptedAt
}
} else {
self.logger.warning("tts unavailable; falling back to system voice (missing key or voiceId)")
if self.interruptOnSpeech {
do {
try self.startRecognition()
} catch {
self.logger.warning(
"startRecognition during speak failed: \(error.localizedDescription, privacy: .public)")
}
}
self.statusText = "Speaking (System)…"
try await TalkSystemSpeechSynthesizer.shared.speak(text: cleaned, language: language)
}
} catch {
self.logger.error(
"tts failed: \(error.localizedDescription, privacy: .public); falling back to system voice")
do {
if self.interruptOnSpeech {
do {
try self.startRecognition()
} catch {
self.logger.warning(
"startRecognition during speak failed: \(error.localizedDescription, privacy: .public)")
}
}
self.statusText = "Speaking (System)…"
let language = ElevenLabsTTSClient.validatedLanguage(directive?.language)
try await TalkSystemSpeechSynthesizer.shared.speak(text: cleaned, language: language)
} catch {
self.statusText = "Speak failed: \(error.localizedDescription)"
self.logger.error("system voice failed: \(error.localizedDescription, privacy: .public)")
}
}
self.stopRecognition()
self.isSpeaking = false
}
private func stopSpeaking(storeInterruption: Bool = true) {
guard self.isSpeaking else { return }
let interruptedAt = self.lastPlaybackWasPCM
? self.pcmPlayer.stop()
: self.mp3Player.stop()
if storeInterruption {
self.lastInterruptedAtSeconds = interruptedAt
}
_ = self.lastPlaybackWasPCM
? self.mp3Player.stop()
: self.pcmPlayer.stop()
TalkSystemSpeechSynthesizer.shared.stop()
self.isSpeaking = false
}
private func shouldInterrupt(with transcript: String) -> Bool {
let trimmed = transcript.trimmingCharacters(in: .whitespacesAndNewlines)
guard trimmed.count >= 3 else { return false }
if let spoken = self.lastSpokenText?.lowercased(), spoken.contains(trimmed.lowercased()) {
return false
}
return true
}
private func resolveVoiceAlias(_ value: String?) -> String? {
let trimmed = (value ?? "").trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return nil }
let normalized = trimmed.lowercased()
if let mapped = self.voiceAliases[normalized] { return mapped }
if self.voiceAliases.values.contains(where: { $0.caseInsensitiveCompare(trimmed) == .orderedSame }) {
return trimmed
}
return Self.isLikelyVoiceId(trimmed) ? trimmed : nil
}
private func resolveVoiceId(preferred: String?, apiKey: String) async -> String? {
let trimmed = preferred?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if !trimmed.isEmpty {
if let resolved = self.resolveVoiceAlias(trimmed) { return resolved }
self.logger.warning("unknown voice alias \(trimmed, privacy: .public)")
}
if let fallbackVoiceId { return fallbackVoiceId }
do {
let voices = try await ElevenLabsTTSClient(apiKey: apiKey).listVoices()
guard let first = voices.first else {
self.logger.warning("elevenlabs voices list empty")
return nil
}
self.fallbackVoiceId = first.voiceId
if self.defaultVoiceId == nil {
self.defaultVoiceId = first.voiceId
}
if !self.voiceOverrideActive {
self.currentVoiceId = first.voiceId
}
let name = first.name ?? "unknown"
self.logger
.info("default voice selected \(name, privacy: .public) (\(first.voiceId, privacy: .public))")
return first.voiceId
} catch {
self.logger.error("elevenlabs list voices failed: \(error.localizedDescription, privacy: .public)")
return nil
}
}
private static func isLikelyVoiceId(_ value: String) -> Bool {
guard value.count >= 10 else { return false }
return value.allSatisfy { $0.isLetter || $0.isNumber || $0 == "-" || $0 == "_" }
}
private func reloadConfig() async {
guard let bridge else { return }
do {
let res = try await bridge.request(method: "config.get", paramsJSON: "{}", timeoutSeconds: 8)
guard let json = try JSONSerialization.jsonObject(with: res) as? [String: Any] else { return }
guard let config = json["config"] as? [String: Any] else { return }
let talk = config["talk"] as? [String: Any]
let session = config["session"] as? [String: Any]
let rawMainKey = (session?["mainKey"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
self.mainSessionKey = rawMainKey.isEmpty ? "main" : rawMainKey
self.defaultVoiceId = (talk?["voiceId"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines)
if let aliases = talk?["voiceAliases"] as? [String: Any] {
var resolved: [String: String] = [:]
for (key, value) in aliases {
guard let id = value as? String else { continue }
let normalizedKey = key.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
let trimmedId = id.trimmingCharacters(in: .whitespacesAndNewlines)
guard !normalizedKey.isEmpty, !trimmedId.isEmpty else { continue }
resolved[normalizedKey] = trimmedId
}
self.voiceAliases = resolved
} else {
self.voiceAliases = [:]
}
if !self.voiceOverrideActive {
self.currentVoiceId = self.defaultVoiceId
}
let model = (talk?["modelId"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines)
self.defaultModelId = (model?.isEmpty == false) ? model : Self.defaultModelIdFallback
if !self.modelOverrideActive {
self.currentModelId = self.defaultModelId
}
self.defaultOutputFormat = (talk?["outputFormat"] as? String)?
.trimmingCharacters(in: .whitespacesAndNewlines)
self.apiKey = (talk?["apiKey"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines)
if let interrupt = talk?["interruptOnSpeech"] as? Bool {
self.interruptOnSpeech = interrupt
}
} catch {
self.defaultModelId = Self.defaultModelIdFallback
if !self.modelOverrideActive {
self.currentModelId = self.defaultModelId
}
}
}
private static func configureAudioSession() throws {
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playAndRecord, mode: .voiceChat, options: [
.duckOthers,
.mixWithOthers,
.allowBluetoothHFP,
.defaultToSpeaker,
])
try session.setActive(true, options: [])
}
private nonisolated static func requestMicrophonePermission() async -> Bool {
await withCheckedContinuation(isolation: nil) { cont in
AVAudioApplication.requestRecordPermission { ok in
cont.resume(returning: ok)
}
}
}
private nonisolated static func requestSpeechPermission() async -> Bool {
await withCheckedContinuation(isolation: nil) { cont in
SFSpeechRecognizer.requestAuthorization { status in
cont.resume(returning: status == .authorized)
}
}
}
}
@@ -0,0 +1,70 @@
import SwiftUI
struct TalkOrbOverlay: View {
@Environment(NodeAppModel.self) private var appModel
@State private var pulse: Bool = false
var body: some View {
let seam = self.appModel.seamColor
let status = self.appModel.talkMode.statusText.trimmingCharacters(in: .whitespacesAndNewlines)
VStack(spacing: 14) {
ZStack {
Circle()
.stroke(seam.opacity(0.26), lineWidth: 2)
.frame(width: 320, height: 320)
.scaleEffect(self.pulse ? 1.15 : 0.96)
.opacity(self.pulse ? 0.0 : 1.0)
.animation(.easeOut(duration: 1.3).repeatForever(autoreverses: false), value: self.pulse)
Circle()
.stroke(seam.opacity(0.18), lineWidth: 2)
.frame(width: 320, height: 320)
.scaleEffect(self.pulse ? 1.45 : 1.02)
.opacity(self.pulse ? 0.0 : 0.9)
.animation(.easeOut(duration: 1.9).repeatForever(autoreverses: false).delay(0.2), value: self.pulse)
Circle()
.fill(
RadialGradient(
colors: [
seam.opacity(0.95),
seam.opacity(0.40),
Color.black.opacity(0.55),
],
center: .center,
startRadius: 1,
endRadius: 112))
.frame(width: 190, height: 190)
.overlay(
Circle()
.stroke(seam.opacity(0.35), lineWidth: 1))
.shadow(color: seam.opacity(0.32), radius: 26, x: 0, y: 0)
.shadow(color: Color.black.opacity(0.50), radius: 22, x: 0, y: 10)
}
.contentShape(Circle())
.onTapGesture {
self.appModel.talkMode.userTappedOrb()
}
if !status.isEmpty, status != "Off" {
Text(status)
.font(.system(.footnote, design: .rounded).weight(.semibold))
.foregroundStyle(Color.white.opacity(0.92))
.padding(.horizontal, 12)
.padding(.vertical, 8)
.background(
Capsule()
.fill(Color.black.opacity(0.40))
.overlay(
Capsule().stroke(seam.opacity(0.22), lineWidth: 1)))
}
}
.padding(28)
.onAppear {
self.pulse = true
}
.accessibilityElement(children: .combine)
.accessibilityLabel("Talk Mode \(status)")
}
}
+5
View File
@@ -4,6 +4,7 @@ struct VoiceTab: View {
@Environment(NodeAppModel.self) private var appModel
@Environment(VoiceWakeManager.self) private var voiceWake
@AppStorage("voiceWake.enabled") private var voiceWakeEnabled: Bool = false
@AppStorage("talk.enabled") private var talkEnabled: Bool = false
var body: some View {
NavigationStack {
@@ -14,6 +15,7 @@ struct VoiceTab: View {
Text(self.voiceWake.statusText)
.font(.footnote)
.foregroundStyle(.secondary)
LabeledContent("Talk Mode", value: self.talkEnabled ? "Enabled" : "Disabled")
}
Section("Notes") {
@@ -36,6 +38,9 @@ struct VoiceTab: View {
.onChange(of: self.voiceWakeEnabled) { _, newValue in
self.appModel.setVoiceWakeEnabled(newValue)
}
.onChange(of: self.talkEnabled) { _, newValue in
self.appModel.setTalkEnabled(newValue)
}
}
}
}
+3
View File
@@ -54,4 +54,7 @@ Sources/Voice/VoiceWakePreferences.swift
../shared/ClawdisKit/Sources/ClawdisKit/ScreenCommands.swift
../shared/ClawdisKit/Sources/ClawdisKit/StoragePaths.swift
../shared/ClawdisKit/Sources/ClawdisKit/SystemCommands.swift
../shared/ClawdisKit/Sources/ClawdisKit/TalkDirective.swift
../../Swabble/Sources/SwabbleKit/WakeWordGate.swift
Sources/Voice/TalkModeManager.swift
Sources/Voice/TalkOrbOverlay.swift
@@ -1,5 +1,6 @@
import ClawdisKit
import Foundation
import Network
import Testing
import UIKit
@testable import Clawdis
@@ -15,6 +16,25 @@ private let instanceIdEntry = KeychainEntry(service: nodeService, account: "inst
private let preferredBridgeEntry = KeychainEntry(service: bridgeService, account: "preferredStableID")
private let lastBridgeEntry = KeychainEntry(service: bridgeService, account: "lastDiscoveredStableID")
private actor MockBridgePairingClient: BridgePairingClient {
private(set) var lastToken: String?
private let resultToken: String
init(resultToken: String) {
self.resultToken = resultToken
}
func pairAndHello(
endpoint: NWEndpoint,
hello: BridgeHello,
onStatus: (@Sendable (String) -> Void)?) async throws -> String
{
self.lastToken = hello.token
onStatus?("Testing…")
return self.resultToken
}
}
private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws -> T) rethrows -> T {
let defaults = UserDefaults.standard
var snapshot: [String: Any?] = [:]
@@ -40,6 +60,35 @@ private func withUserDefaults<T>(_ updates: [String: Any?], _ body: () throws ->
return try body()
}
@MainActor
private func withUserDefaults<T>(
_ updates: [String: Any?],
_ body: () async throws -> T) async rethrows -> T
{
let defaults = UserDefaults.standard
var snapshot: [String: Any?] = [:]
for key in updates.keys {
snapshot[key] = defaults.object(forKey: key)
}
for (key, value) in updates {
if let value {
defaults.set(value, forKey: key)
} else {
defaults.removeObject(forKey: key)
}
}
defer {
for (key, value) in snapshot {
if let value {
defaults.set(value, forKey: key)
} else {
defaults.removeObject(forKey: key)
}
}
}
return try await body()
}
private func withKeychainValues<T>(_ updates: [KeychainEntry: String?], _ body: () throws -> T) rethrows -> T {
var snapshot: [KeychainEntry: String?] = [:]
for entry in updates.keys {
@@ -64,6 +113,34 @@ private func withKeychainValues<T>(_ updates: [KeychainEntry: String?], _ body:
return try body()
}
@MainActor
private func withKeychainValues<T>(
_ updates: [KeychainEntry: String?],
_ body: () async throws -> T) async rethrows -> T
{
var snapshot: [KeychainEntry: String?] = [:]
for entry in updates.keys {
snapshot[entry] = KeychainStore.loadString(service: entry.service, account: entry.account)
}
for (entry, value) in updates {
if let value {
_ = KeychainStore.saveString(value, service: entry.service, account: entry.account)
} else {
_ = KeychainStore.delete(service: entry.service, account: entry.account)
}
}
defer {
for (entry, value) in snapshot {
if let value {
_ = KeychainStore.saveString(value, service: entry.service, account: entry.account)
} else {
_ = KeychainStore.delete(service: entry.service, account: entry.account)
}
}
}
return try await body()
}
@Suite(.serialized) struct BridgeConnectionControllerTests {
@Test @MainActor func resolvedDisplayNameSetsDefaultWhenMissing() {
let defaults = UserDefaults.standard
@@ -156,4 +233,109 @@ private func withKeychainValues<T>(_ updates: [KeychainEntry: String?], _ body:
}
}
}
@Test @MainActor func autoConnectRefreshesTokenOnUnauthorized() async {
let bridge = BridgeDiscoveryModel.DiscoveredBridge(
name: "Gateway",
endpoint: .hostPort(host: NWEndpoint.Host("127.0.0.1"), port: 18790),
stableID: "bridge-1",
debugID: "bridge-debug",
lanHost: "Mac.local",
tailnetDns: nil,
gatewayPort: 18789,
bridgePort: 18790,
canvasPort: 18793,
cliPath: nil)
let mock = MockBridgePairingClient(resultToken: "new-token")
let account = "bridge-token.ios-test"
await withKeychainValues([
instanceIdEntry: nil,
preferredBridgeEntry: nil,
lastBridgeEntry: nil,
KeychainEntry(service: bridgeService, account: account): "old-token",
]) {
await withUserDefaults([
"node.instanceId": "ios-test",
"bridge.lastDiscoveredStableID": "bridge-1",
"bridge.manual.enabled": false,
]) {
let appModel = NodeAppModel()
let controller = BridgeConnectionController(
appModel: appModel,
startDiscovery: false,
bridgeClientFactory: { mock })
controller._test_setBridges([bridge])
controller._test_triggerAutoConnect()
for _ in 0..<20 {
if appModel.connectedBridgeID == bridge.stableID { break }
try? await Task.sleep(nanoseconds: 50_000_000)
}
#expect(appModel.connectedBridgeID == bridge.stableID)
let stored = KeychainStore.loadString(service: bridgeService, account: account)
#expect(stored == "new-token")
let lastToken = await mock.lastToken
#expect(lastToken == "old-token")
}
}
}
@Test @MainActor func autoConnectPrefersPreferredBridgeOverLastDiscovered() async {
let bridgeA = BridgeDiscoveryModel.DiscoveredBridge(
name: "Gateway A",
endpoint: .hostPort(host: NWEndpoint.Host("127.0.0.1"), port: 18790),
stableID: "bridge-1",
debugID: "bridge-a",
lanHost: "MacA.local",
tailnetDns: nil,
gatewayPort: 18789,
bridgePort: 18790,
canvasPort: 18793,
cliPath: nil)
let bridgeB = BridgeDiscoveryModel.DiscoveredBridge(
name: "Gateway B",
endpoint: .hostPort(host: NWEndpoint.Host("127.0.0.1"), port: 28790),
stableID: "bridge-2",
debugID: "bridge-b",
lanHost: "MacB.local",
tailnetDns: nil,
gatewayPort: 28789,
bridgePort: 28790,
canvasPort: 28793,
cliPath: nil)
let mock = MockBridgePairingClient(resultToken: "token-ok")
let account = "bridge-token.ios-test"
await withKeychainValues([
instanceIdEntry: nil,
preferredBridgeEntry: nil,
lastBridgeEntry: nil,
KeychainEntry(service: bridgeService, account: account): "old-token",
]) {
await withUserDefaults([
"node.instanceId": "ios-test",
"bridge.preferredStableID": "bridge-2",
"bridge.lastDiscoveredStableID": "bridge-1",
"bridge.manual.enabled": false,
]) {
let appModel = NodeAppModel()
let controller = BridgeConnectionController(
appModel: appModel,
startDiscovery: false,
bridgeClientFactory: { mock })
controller._test_setBridges([bridgeA, bridgeB])
controller._test_triggerAutoConnect()
for _ in 0..<20 {
if appModel.connectedBridgeID == bridgeB.stableID { break }
try? await Task.sleep(nanoseconds: 50_000_000)
}
#expect(appModel.connectedBridgeID == bridgeB.stableID)
}
}
}
}
@@ -0,0 +1,13 @@
import Testing
@testable import Clawdis
@Suite struct CameraControllerErrorTests {
@Test func errorDescriptionsAreStable() {
#expect(CameraController.CameraError.cameraUnavailable.errorDescription == "Camera unavailable")
#expect(CameraController.CameraError.microphoneUnavailable.errorDescription == "Microphone unavailable")
#expect(CameraController.CameraError.permissionDenied(kind: "Camera").errorDescription == "Camera permission denied")
#expect(CameraController.CameraError.invalidParams("bad").errorDescription == "bad")
#expect(CameraController.CameraError.captureFailed("nope").errorDescription == "nope")
#expect(CameraController.CameraError.exportFailed("export").errorDescription == "export")
}
}
@@ -16,6 +16,15 @@ import WebKit
#expect(scrollView.bounces == false)
}
@Test @MainActor func navigateEnablesScrollForWebPages() {
let screen = ScreenController()
screen.navigate(to: "https://example.com")
let scrollView = screen.webView.scrollView
#expect(scrollView.isScrollEnabled == true)
#expect(scrollView.bounces == true)
}
@Test @MainActor func navigateSlashShowsDefaultCanvas() {
let screen = ScreenController()
screen.navigate(to: "/")
+4
View File
@@ -62,7 +62,11 @@ targets:
swiftlint lint --config "$SRCROOT/.swiftlint.yml" --use-script-input-file-lists
settings:
base:
CODE_SIGN_IDENTITY: "Apple Development"
CODE_SIGN_STYLE: Manual
DEVELOPMENT_TEAM: Y5PE65HELJ
PRODUCT_BUNDLE_IDENTIFIER: com.steipete.clawdis.ios
PROVISIONING_PROFILE_SPECIFIER: "com.steipete.clawdis.ios Development"
SWIFT_VERSION: "6.0"
info:
path: Sources/Info.plist
+10 -1
View File
@@ -1,6 +1,15 @@
{
"originHash" : "d8a19a95c479a3c7cb20aded07bd18cfeda5d85b95284983da83dbee7c941e5c",
"originHash" : "9de32b5fc115432dadd84c3ab4d67d2fed22ffaf5675a77033d69ea194ac3862",
"pins" : [
{
"identity" : "elevenlabskit",
"kind" : "remoteSourceControl",
"location" : "https://github.com/steipete/ElevenLabsKit",
"state" : {
"revision" : "c8679fbd37416a8780fe43be88a497ff16209e2d",
"version" : "0.1.0"
}
},
{
"identity" : "eventsource",
"kind" : "remoteSourceControl",
+5
View File
@@ -15,6 +15,7 @@ let package = Package(
dependencies: [
.package(url: "https://github.com/orchetect/MenuBarExtraAccess", exact: "1.2.2"),
.package(url: "https://github.com/swiftlang/swift-subprocess.git", from: "0.1.0"),
.package(url: "https://github.com/apple/swift-log.git", from: "1.8.0"),
.package(url: "https://github.com/sparkle-project/Sparkle", from: "2.8.1"),
.package(path: "../shared/ClawdisKit"),
.package(path: "../../Swabble"),
@@ -45,10 +46,14 @@ let package = Package(
.product(name: "SwabbleKit", package: "swabble"),
.product(name: "MenuBarExtraAccess", package: "MenuBarExtraAccess"),
.product(name: "Subprocess", package: "swift-subprocess"),
.product(name: "Logging", package: "swift-log"),
.product(name: "Sparkle", package: "Sparkle"),
.product(name: "PeekabooBridge", package: "PeekabooCore"),
.product(name: "PeekabooAutomationKit", package: "PeekabooAutomationKit"),
],
exclude: [
"Resources/Info.plist",
],
resources: [
.copy("Resources/Clawdis.icns"),
.copy("Resources/DeviceModels"),
@@ -149,6 +149,49 @@ enum AnthropicOAuth {
self.logger.info("Anthropic OAuth exchange ok; expiresAtMs=\(expiresAtMs, privacy: .public)")
return AnthropicOAuthCredentials(type: "oauth", refresh: refresh, access: access, expires: expiresAtMs)
}
static func refresh(refreshToken: String) async throws -> AnthropicOAuthCredentials {
let payload: [String: Any] = [
"grant_type": "refresh_token",
"client_id": self.clientId,
"refresh_token": refreshToken,
]
let body = try JSONSerialization.data(withJSONObject: payload, options: [])
var request = URLRequest(url: self.tokenURL)
request.httpMethod = "POST"
request.httpBody = body
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
let (data, response) = try await URLSession.shared.data(for: request)
guard let http = response as? HTTPURLResponse else {
throw URLError(.badServerResponse)
}
guard (200..<300).contains(http.statusCode) else {
let text = String(data: data, encoding: .utf8) ?? "<non-utf8>"
throw NSError(
domain: "AnthropicOAuth",
code: http.statusCode,
userInfo: [NSLocalizedDescriptionKey: "Token refresh failed: \(text)"])
}
let decoded = try JSONSerialization.jsonObject(with: data) as? [String: Any]
let access = decoded?["access_token"] as? String
let refresh = (decoded?["refresh_token"] as? String) ?? refreshToken
let expiresIn = decoded?["expires_in"] as? Double
guard let access, let expiresIn else {
throw NSError(domain: "AnthropicOAuth", code: 0, userInfo: [
NSLocalizedDescriptionKey: "Unexpected token response.",
])
}
let expiresAtMs = Int64(Date().timeIntervalSince1970 * 1000)
+ Int64(expiresIn * 1000)
- Int64(5 * 60 * 1000)
self.logger.info("Anthropic OAuth refresh ok; expiresAtMs=\(expiresAtMs, privacy: .public)")
return AnthropicOAuthCredentials(type: "oauth", refresh: refresh, access: access, expires: expiresAtMs)
}
}
enum ClawdisOAuthStore {
@@ -279,6 +322,14 @@ enum ClawdisOAuthStore {
return .connected(expiresAtMs: expiresAtMs)
}
static func loadAnthropicOAuthRefreshToken() -> String? {
let url = self.oauthURL()
guard let storage = self.loadStorage(at: url) else { return nil }
guard let rawEntry = storage[self.providerKey] as? [String: Any] else { return nil }
let refresh = self.firstString(in: rawEntry, keys: ["refresh", "refresh_token", "refreshToken"])
return refresh?.trimmingCharacters(in: .whitespacesAndNewlines)
}
private static func firstString(in dict: [String: Any], keys: [String]) -> String? {
for key in keys {
if let value = dict[key] as? String { return value }
+44
View File
@@ -121,6 +121,18 @@ final class AppState {
forKey: voicePushToTalkEnabledKey) } }
}
var talkEnabled: Bool {
didSet {
self.ifNotPreview {
UserDefaults.standard.set(self.talkEnabled, forKey: talkEnabledKey)
Task { await TalkModeController.shared.setEnabled(self.talkEnabled) }
}
}
}
/// Gateway-provided UI accent color (hex). Optional; clients provide a default.
var seamColorHex: String?
var iconOverride: IconOverrideSelection {
didSet { self.ifNotPreview { UserDefaults.standard.set(self.iconOverride.rawValue, forKey: iconOverrideKey) } }
}
@@ -216,6 +228,8 @@ final class AppState {
.stringArray(forKey: voiceWakeAdditionalLocalesKey) ?? []
self.voicePushToTalkEnabled = UserDefaults.standard
.object(forKey: voicePushToTalkEnabledKey) as? Bool ?? false
self.talkEnabled = UserDefaults.standard.bool(forKey: talkEnabledKey)
self.seamColorHex = nil
if let storedHeartbeats = UserDefaults.standard.object(forKey: heartbeatsEnabledKey) as? Bool {
self.heartbeatsEnabled = storedHeartbeats
} else {
@@ -256,9 +270,13 @@ final class AppState {
if self.swabbleEnabled, !PermissionManager.voiceWakePermissionsGranted() {
self.swabbleEnabled = false
}
if self.talkEnabled, !PermissionManager.voiceWakePermissionsGranted() {
self.talkEnabled = false
}
if !self.isPreview {
Task { await VoiceWakeRuntime.shared.refresh(state: self) }
Task { await TalkModeController.shared.setEnabled(self.talkEnabled) }
}
}
@@ -312,6 +330,31 @@ final class AppState {
Task { await VoiceWakeRuntime.shared.refresh(state: self) }
}
func setTalkEnabled(_ enabled: Bool) async {
guard voiceWakeSupported else {
self.talkEnabled = false
await GatewayConnection.shared.talkMode(enabled: false, phase: "disabled")
return
}
self.talkEnabled = enabled
guard !self.isPreview else { return }
if !enabled {
await GatewayConnection.shared.talkMode(enabled: false, phase: "disabled")
return
}
if PermissionManager.voiceWakePermissionsGranted() {
await GatewayConnection.shared.talkMode(enabled: true, phase: "enabled")
return
}
let granted = await PermissionManager.ensureVoiceWakePermissions(interactive: true)
self.talkEnabled = granted
await GatewayConnection.shared.talkMode(enabled: granted, phase: granted ? "enabled" : "denied")
}
// MARK: - Global wake words sync (Gateway-owned)
func applyGlobalVoiceWakeTriggers(_ triggers: [String]) {
@@ -367,6 +410,7 @@ extension AppState {
state.voiceWakeLocaleID = Locale.current.identifier
state.voiceWakeAdditionalLocaleIDs = ["en-US", "de-DE"]
state.voicePushToTalkEnabled = false
state.talkEnabled = false
state.iconOverride = .system
state.heartbeatsEnabled = true
state.connectionMode = .local
@@ -6,6 +6,13 @@ import Foundation
import OSLog
actor CameraCaptureService {
struct CameraDeviceInfo: Encodable, Sendable {
let id: String
let name: String
let position: String
let deviceType: String
}
enum CameraError: LocalizedError, Sendable {
case cameraUnavailable
case microphoneUnavailable
@@ -31,18 +38,36 @@ actor CameraCaptureService {
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "camera")
func snap(facing: CameraFacing?, maxWidth: Int?, quality: Double?) async throws -> (data: Data, size: CGSize) {
func listDevices() -> [CameraDeviceInfo] {
Self.availableCameras().map { device in
CameraDeviceInfo(
id: device.uniqueID,
name: device.localizedName,
position: Self.positionLabel(device.position),
deviceType: device.deviceType.rawValue)
}
}
func snap(
facing: CameraFacing?,
maxWidth: Int?,
quality: Double?,
deviceId: String?,
delayMs: Int) async throws -> (data: Data, size: CGSize)
{
let facing = facing ?? .front
let normalized = Self.normalizeSnap(maxWidth: maxWidth, quality: quality)
let maxWidth = normalized.maxWidth
let quality = normalized.quality
let delayMs = max(0, delayMs)
let deviceId = deviceId?.trimmingCharacters(in: .whitespacesAndNewlines)
try await self.ensureAccess(for: .video)
let session = AVCaptureSession()
session.sessionPreset = .photo
guard let device = Self.pickCamera(facing: facing) else {
guard let device = Self.pickCamera(facing: facing, deviceId: deviceId) else {
throw CameraError.cameraUnavailable
}
@@ -62,6 +87,8 @@ actor CameraCaptureService {
session.startRunning()
defer { session.stopRunning() }
await Self.warmUpCaptureSession()
await self.waitForExposureAndWhiteBalance(device: device)
await self.sleepDelayMs(delayMs)
let settings: AVCapturePhotoSettings = {
if output.availablePhotoCodecTypes.contains(.jpeg) {
@@ -72,14 +99,21 @@ actor CameraCaptureService {
settings.photoQualityPrioritization = .quality
var delegate: PhotoCaptureDelegate?
let rawData: Data = try await withCheckedThrowingContinuation(isolation: nil) { cont in
let rawData: Data = try await withCheckedThrowingContinuation { cont in
let d = PhotoCaptureDelegate(cont)
delegate = d
output.capturePhoto(with: settings, delegate: d)
}
withExtendedLifetime(delegate) {}
let res = try JPEGTranscoder.transcodeToJPEG(imageData: rawData, maxWidthPx: maxWidth, quality: quality)
let maxPayloadBytes = 5 * 1024 * 1024
// Base64 inflates payloads by ~4/3; cap encoded bytes so the payload stays under 5MB (API limit).
let maxEncodedBytes = (maxPayloadBytes / 4) * 3
let res = try JPEGTranscoder.transcodeToJPEG(
imageData: rawData,
maxWidthPx: maxWidth,
quality: quality,
maxBytes: maxEncodedBytes)
return (data: res.data, size: CGSize(width: res.widthPx, height: res.heightPx))
}
@@ -87,10 +121,12 @@ actor CameraCaptureService {
facing: CameraFacing?,
durationMs: Int?,
includeAudio: Bool,
deviceId: String?,
outPath: String?) async throws -> (path: String, durationMs: Int, hasAudio: Bool)
{
let facing = facing ?? .front
let durationMs = Self.clampDurationMs(durationMs)
let deviceId = deviceId?.trimmingCharacters(in: .whitespacesAndNewlines)
try await self.ensureAccess(for: .video)
if includeAudio {
@@ -100,7 +136,7 @@ actor CameraCaptureService {
let session = AVCaptureSession()
session.sessionPreset = .high
guard let camera = Self.pickCamera(facing: facing) else {
guard let camera = Self.pickCamera(facing: facing, deviceId: deviceId) else {
throw CameraError.cameraUnavailable
}
let cameraInput = try AVCaptureDeviceInput(device: camera)
@@ -148,7 +184,7 @@ actor CameraCaptureService {
let logger = self.logger
var delegate: MovieFileDelegate?
let recordedURL: URL = try await withCheckedThrowingContinuation(isolation: nil) { cont in
let recordedURL: URL = try await withCheckedThrowingContinuation { cont in
let d = MovieFileDelegate(cont, logger: logger)
delegate = d
output.startRecording(to: tmpMovURL, recordingDelegate: d)
@@ -180,7 +216,38 @@ actor CameraCaptureService {
}
}
private nonisolated static func pickCamera(facing: CameraFacing) -> AVCaptureDevice? {
private nonisolated static func availableCameras() -> [AVCaptureDevice] {
var types: [AVCaptureDevice.DeviceType] = [
.builtInWideAngleCamera,
.continuityCamera,
]
if let external = externalDeviceType() {
types.append(external)
}
let session = AVCaptureDevice.DiscoverySession(
deviceTypes: types,
mediaType: .video,
position: .unspecified)
return session.devices
}
private nonisolated static func externalDeviceType() -> AVCaptureDevice.DeviceType? {
if #available(macOS 14.0, *) {
return .external
}
// Use raw value to avoid deprecated symbol in the SDK.
return AVCaptureDevice.DeviceType(rawValue: "AVCaptureDeviceTypeExternalUnknown")
}
private nonisolated static func pickCamera(
facing: CameraFacing,
deviceId: String?) -> AVCaptureDevice?
{
if let deviceId, !deviceId.isEmpty {
if let match = Self.availableCameras().first(where: { $0.uniqueID == deviceId }) {
return match
}
}
let position: AVCaptureDevice.Position = (facing == .front) ? .front : .back
if let device = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: position) {
@@ -250,6 +317,31 @@ actor CameraCaptureService {
// A short delay after `startRunning()` significantly reduces "blank first frame" captures on some devices.
try? await Task.sleep(nanoseconds: 150_000_000) // 150ms
}
private func waitForExposureAndWhiteBalance(device: AVCaptureDevice) async {
let stepNs: UInt64 = 50_000_000
let maxSteps = 30 // ~1.5s
for _ in 0..<maxSteps {
if !(device.isAdjustingExposure || device.isAdjustingWhiteBalance) {
return
}
try? await Task.sleep(nanoseconds: stepNs)
}
}
private func sleepDelayMs(_ delayMs: Int) async {
guard delayMs > 0 else { return }
let ns = UInt64(min(delayMs, 10_000)) * 1_000_000
try? await Task.sleep(nanoseconds: ns)
}
private nonisolated static func positionLabel(_ position: AVCaptureDevice.Position) -> String {
switch position {
case .front: "front"
case .back: "back"
default: "unspecified"
}
}
}
private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegate {
+46 -6
View File
@@ -151,8 +151,22 @@ final class CanvasManager {
private func handleGatewayPush(_ push: GatewayPush) {
guard case let .snapshot(snapshot) = push else { return }
let a2uiUrl = Self.resolveA2UIHostUrl(from: snapshot.canvashosturl)
guard let controller = self.panelController else { return }
let raw = snapshot.canvashosturl?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
if raw.isEmpty {
Self.logger.debug("canvas host url missing in gateway snapshot")
} else {
Self.logger.debug("canvas host url snapshot=\(raw, privacy: .public)")
}
let a2uiUrl = Self.resolveA2UIHostUrl(from: raw)
if a2uiUrl == nil, !raw.isEmpty {
Self.logger.debug("canvas host url invalid; cannot resolve A2UI")
}
guard let controller = self.panelController else {
if a2uiUrl != nil {
Self.logger.debug("canvas panel not visible; skipping auto-nav")
}
return
}
self.maybeAutoNavigateToA2UI(controller: controller, a2uiUrl: a2uiUrl)
}
@@ -169,7 +183,12 @@ final class CanvasManager {
private func maybeAutoNavigateToA2UI(controller: CanvasWindowController, a2uiUrl: String?) {
guard let a2uiUrl else { return }
guard controller.shouldAutoNavigateToA2UI(lastAutoTarget: self.lastAutoA2UIUrl) else { return }
let shouldNavigate = controller.shouldAutoNavigateToA2UI(lastAutoTarget: self.lastAutoA2UIUrl)
guard shouldNavigate else {
Self.logger.debug("canvas auto-nav skipped; target unchanged")
return
}
Self.logger.debug("canvas auto-nav -> \(a2uiUrl, privacy: .public)")
controller.load(target: a2uiUrl)
self.lastAutoA2UIUrl = a2uiUrl
}
@@ -182,15 +201,36 @@ final class CanvasManager {
func refreshDebugStatus() {
guard let controller = self.panelController else { return }
let enabled = AppStateStore.shared.debugPaneEnabled
let title = GatewayProcessManager.shared.status.label
let subtitle = AppStateStore.shared.connectionMode.rawValue
let mode = AppStateStore.shared.connectionMode
let title: String?
let subtitle: String?
switch mode {
case .remote:
title = "Remote control"
switch ControlChannel.shared.state {
case .connected:
subtitle = "Connected"
case .connecting:
subtitle = "Connecting…"
case .disconnected:
subtitle = "Disconnected"
case let .degraded(message):
subtitle = message.isEmpty ? "Degraded" : message
}
case .local:
title = GatewayProcessManager.shared.status.label
subtitle = mode.rawValue
case .unconfigured:
title = "Unconfigured"
subtitle = mode.rawValue
}
controller.updateDebugStatus(enabled: enabled, title: title, subtitle: subtitle)
}
private static func resolveA2UIHostUrl(from raw: String?) -> String? {
let trimmed = raw?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
guard !trimmed.isEmpty, let base = URL(string: trimmed) else { return nil }
return base.appendingPathComponent("__clawdis__/a2ui/").absoluteString
return base.appendingPathComponent("__clawdis__/a2ui/").absoluteString + "?platform=macos"
}
// MARK: - Anchoring
@@ -1,5 +1,4 @@
import AppKit
import OSLog
let canvasWindowLogger = Logger(subsystem: "com.steipete.clawdis", category: "Canvas")
@@ -1,25 +1,38 @@
import Foundation
enum ClawdisConfigFile {
private static let logger = Logger(subsystem: "com.steipete.clawdis", category: "config")
static func url() -> URL {
FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent(".clawdis")
.appendingPathComponent("clawdis.json")
ClawdisPaths.configURL
}
static func stateDirURL() -> URL {
ClawdisPaths.stateDirURL
}
static func defaultWorkspaceURL() -> URL {
FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent(".clawdis")
.appendingPathComponent("workspace", isDirectory: true)
ClawdisPaths.workspaceURL
}
static func loadDict() -> [String: Any] {
let url = self.url()
guard let data = try? Data(contentsOf: url) else { return [:] }
return (try? JSONSerialization.jsonObject(with: data) as? [String: Any]) ?? [:]
guard FileManager.default.fileExists(atPath: url.path) else { return [:] }
do {
let data = try Data(contentsOf: url)
guard let root = try JSONSerialization.jsonObject(with: data) as? [String: Any] else {
self.logger.warning("config JSON root invalid")
return [:]
}
return root
} catch {
self.logger.warning("config read failed: \(error.localizedDescription)")
return [:]
}
}
static func saveDict(_ dict: [String: Any]) {
if ProcessInfo.processInfo.isNixMode { return }
do {
let data = try JSONSerialization.data(withJSONObject: dict, options: [.prettyPrinted, .sortedKeys])
let url = self.url()
@@ -27,7 +40,9 @@ enum ClawdisConfigFile {
at: url.deletingLastPathComponent(),
withIntermediateDirectories: true)
try data.write(to: url, options: [.atomic])
} catch {}
} catch {
self.logger.error("config save failed: \(error.localizedDescription)")
}
}
static func loadGatewayDict() -> [String: Any] {
@@ -59,6 +74,7 @@ enum ClawdisConfigFile {
browser["enabled"] = enabled
root["browser"] = browser
self.saveDict(root)
self.logger.debug("browser control updated enabled=\(enabled)")
}
static func agentWorkspace() -> String? {
@@ -78,5 +94,16 @@ enum ClawdisConfigFile {
}
root["agent"] = agent
self.saveDict(root)
self.logger.debug("agent workspace updated set=\(!trimmed.isEmpty)")
}
static func gatewayPassword() -> String? {
let root = self.loadDict()
guard let gateway = root["gateway"] as? [String: Any],
let remote = gateway["remote"] as? [String: Any] else {
return nil
}
return remote["password"] as? String
}
}
@@ -0,0 +1,38 @@
import Foundation
enum ClawdisEnv {
static func path(_ key: String) -> String? {
// Normalize env overrides once so UI + file IO stay consistent.
guard let value = ProcessInfo.processInfo.environment[key]?
.trimmingCharacters(in: .whitespacesAndNewlines),
!value.isEmpty
else {
return nil
}
return value
}
}
enum ClawdisPaths {
private static let configPathEnv = "CLAWDIS_CONFIG_PATH"
private static let stateDirEnv = "CLAWDIS_STATE_DIR"
static var stateDirURL: URL {
if let override = ClawdisEnv.path(self.stateDirEnv) {
return URL(fileURLWithPath: override, isDirectory: true)
}
return FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent(".clawdis", isDirectory: true)
}
static var configURL: URL {
if let override = ClawdisEnv.path(self.configPathEnv) {
return URL(fileURLWithPath: override)
}
return self.stateDirURL.appendingPathComponent("clawdis.json")
}
static var workspaceURL: URL {
self.stateDirURL.appendingPathComponent("workspace", isDirectory: true)
}
}
@@ -16,6 +16,10 @@ enum CommandResolver {
RuntimeLocator.resolve(searchPaths: self.preferredPaths())
}
static func runtimeResolution(searchPaths: [String]?) -> Result<RuntimeResolution, RuntimeResolutionError> {
RuntimeLocator.resolve(searchPaths: searchPaths ?? self.preferredPaths())
}
static func makeRuntimeCommand(
runtime: RuntimeResolution,
entrypoint: String,
@@ -152,8 +156,8 @@ enum CommandResolver {
return paths
}
static func findExecutable(named name: String) -> String? {
for dir in self.preferredPaths() {
static func findExecutable(named name: String, searchPaths: [String]? = nil) -> String? {
for dir in (searchPaths ?? self.preferredPaths()) {
let candidate = (dir as NSString).appendingPathComponent(name)
if FileManager.default.isExecutableFile(atPath: candidate) {
return candidate
@@ -162,8 +166,14 @@ enum CommandResolver {
return nil
}
static func clawdisExecutable() -> String? {
self.findExecutable(named: self.helperName)
static func clawdisExecutable(searchPaths: [String]? = nil) -> String? {
self.findExecutable(named: self.helperName, searchPaths: searchPaths)
}
static func projectClawdisExecutable(projectRoot: URL? = nil) -> String? {
let root = projectRoot ?? self.projectRoot()
let candidate = root.appendingPathComponent("node_modules/.bin").appendingPathComponent(self.helperName).path
return FileManager.default.isExecutableFile(atPath: candidate) ? candidate : nil
}
static func nodeCliPath() -> String? {
@@ -171,17 +181,22 @@ enum CommandResolver {
return FileManager.default.isReadableFile(atPath: candidate) ? candidate : nil
}
static func hasAnyClawdisInvoker() -> Bool {
if self.clawdisExecutable() != nil { return true }
if self.findExecutable(named: "pnpm") != nil { return true }
if self.findExecutable(named: "node") != nil, self.nodeCliPath() != nil { return true }
static func hasAnyClawdisInvoker(searchPaths: [String]? = nil) -> Bool {
if self.clawdisExecutable(searchPaths: searchPaths) != nil { return true }
if self.findExecutable(named: "pnpm", searchPaths: searchPaths) != nil { return true }
if self.findExecutable(named: "node", searchPaths: searchPaths) != nil,
self.nodeCliPath() != nil
{
return true
}
return false
}
static func clawdisNodeCommand(
subcommand: String,
extraArgs: [String] = [],
defaults: UserDefaults = .standard) -> [String]
defaults: UserDefaults = .standard,
searchPaths: [String]? = nil) -> [String]
{
let settings = self.connectionSettings(defaults: defaults)
if settings.mode == .remote, let ssh = self.sshNodeCommand(
@@ -192,25 +207,29 @@ enum CommandResolver {
return ssh
}
let runtimeResult = self.runtimeResolution()
let runtimeResult = self.runtimeResolution(searchPaths: searchPaths)
switch runtimeResult {
case let .success(runtime):
if let clawdisPath = self.clawdisExecutable() {
let root = self.projectRoot()
if let clawdisPath = self.projectClawdisExecutable(projectRoot: root) {
return [clawdisPath, subcommand] + extraArgs
}
if let entry = self.gatewayEntrypoint(in: self.projectRoot()) {
if let entry = self.gatewayEntrypoint(in: root) {
return self.makeRuntimeCommand(
runtime: runtime,
entrypoint: entry,
subcommand: subcommand,
extraArgs: extraArgs)
}
if let pnpm = self.findExecutable(named: "pnpm") {
if let pnpm = self.findExecutable(named: "pnpm", searchPaths: searchPaths) {
// Use --silent to avoid pnpm lifecycle banners that would corrupt JSON outputs.
return [pnpm, "--silent", "clawdis", subcommand] + extraArgs
}
if let clawdisPath = self.clawdisExecutable(searchPaths: searchPaths) {
return [clawdisPath, subcommand] + extraArgs
}
let missingEntry = """
clawdis entrypoint missing (looked for dist/index.js or bin/clawdis.js); run pnpm build.
@@ -226,9 +245,14 @@ enum CommandResolver {
static func clawdisCommand(
subcommand: String,
extraArgs: [String] = [],
defaults: UserDefaults = .standard) -> [String]
defaults: UserDefaults = .standard,
searchPaths: [String]? = nil) -> [String]
{
self.clawdisNodeCommand(subcommand: subcommand, extraArgs: extraArgs, defaults: defaults)
self.clawdisNodeCommand(
subcommand: subcommand,
extraArgs: extraArgs,
defaults: defaults,
searchPaths: searchPaths)
}
// MARK: - SSH helpers
@@ -258,7 +282,7 @@ enum CommandResolver {
"/bin",
"/usr/sbin",
"/sbin",
"/Users/steipete/Library/pnpm",
"$HOME/Library/pnpm",
"$PATH",
].joined(separator: ":")
let quotedArgs = ([subcommand] + extraArgs).map(self.shellQuote).joined(separator: " ")
+239 -17
View File
@@ -3,6 +3,7 @@ import SwiftUI
@MainActor
struct ConfigSettings: View {
private let isPreview = ProcessInfo.processInfo.isPreview
private let isNixMode = ProcessInfo.processInfo.isNixMode
private let state = AppStateStore.shared
private let labelColumnWidth: CGFloat = 120
private static let browserAttachOnlyHelp =
@@ -30,6 +31,26 @@ struct ConfigSettings: View {
@State private var browserColorHex: String = "#FF4500"
@State private var browserAttachOnly: Bool = false
// Talk mode settings (stored in ~/.clawdis/clawdis.json under "talk")
@State private var talkVoiceId: String = ""
@State private var talkInterruptOnSpeech: Bool = true
@State private var talkApiKey: String = ""
@State private var gatewayApiKeyFound = false
private struct ConfigDraft {
let configModel: String
let customModel: String
let heartbeatMinutes: Int?
let heartbeatBody: String
let browserEnabled: Bool
let browserControlUrl: String
let browserColorHex: String
let browserAttachOnly: Bool
let talkVoiceId: String
let talkApiKey: String
let talkInterruptOnSpeech: Bool
}
var body: some View {
ScrollView { self.content }
.onChange(of: self.modelCatalogPath) { _, _ in
@@ -42,8 +63,9 @@ struct ConfigSettings: View {
guard !self.hasLoaded else { return }
guard !self.isPreview else { return }
self.hasLoaded = true
self.loadConfig()
await self.loadConfig()
await self.loadModels()
await self.refreshGatewayTalkApiKey()
self.allowAutosave = true
}
}
@@ -52,8 +74,13 @@ struct ConfigSettings: View {
VStack(alignment: .leading, spacing: 14) {
self.header
self.agentSection
.disabled(self.isNixMode)
self.heartbeatSection
.disabled(self.isNixMode)
self.talkSection
.disabled(self.isNixMode)
self.browserSection
.disabled(self.isNixMode)
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, alignment: .leading)
@@ -66,7 +93,9 @@ struct ConfigSettings: View {
private var header: some View {
Text("Clawdis CLI config")
.font(.title3.weight(.semibold))
Text("Edit ~/.clawdis/clawdis.json (agent / session / routing / messages).")
Text(self.isNixMode
? "This tab is read-only in Nix mode. Edit config via Nix and rebuild."
: "Edit ~/.clawdis/clawdis.json (agent / session / routing / messages).")
.font(.callout)
.foregroundStyle(.secondary)
}
@@ -266,18 +295,103 @@ struct ConfigSettings: View {
.frame(maxWidth: .infinity, alignment: .leading)
}
private var talkSection: some View {
GroupBox("Talk Mode") {
Grid(alignment: .leadingFirstTextBaseline, horizontalSpacing: 14, verticalSpacing: 10) {
GridRow {
self.gridLabel("Voice ID")
VStack(alignment: .leading, spacing: 6) {
HStack(spacing: 8) {
TextField("ElevenLabs voice ID", text: self.$talkVoiceId)
.textFieldStyle(.roundedBorder)
.frame(maxWidth: .infinity)
.onChange(of: self.talkVoiceId) { _, _ in self.autosaveConfig() }
if !self.talkVoiceSuggestions.isEmpty {
Menu {
ForEach(self.talkVoiceSuggestions, id: \.self) { value in
Button(value) {
self.talkVoiceId = value
self.autosaveConfig()
}
}
} label: {
Label("Suggestions", systemImage: "chevron.up.chevron.down")
}
.fixedSize()
}
}
Text("Defaults to ELEVENLABS_VOICE_ID / SAG_VOICE_ID if unset.")
.font(.footnote)
.foregroundStyle(.secondary)
}
}
GridRow {
self.gridLabel("API key")
VStack(alignment: .leading, spacing: 6) {
HStack(spacing: 8) {
SecureField("ELEVENLABS_API_KEY", text: self.$talkApiKey)
.textFieldStyle(.roundedBorder)
.frame(maxWidth: .infinity)
.disabled(self.hasEnvApiKey)
.onChange(of: self.talkApiKey) { _, _ in self.autosaveConfig() }
if !self.hasEnvApiKey, !self.talkApiKey.isEmpty {
Button("Clear") {
self.talkApiKey = ""
self.autosaveConfig()
}
}
}
self.statusLine(label: self.apiKeyStatusLabel, color: self.apiKeyStatusColor)
if self.hasEnvApiKey {
Text("Using ELEVENLABS_API_KEY from the environment.")
.font(.footnote)
.foregroundStyle(.secondary)
} else if self.gatewayApiKeyFound,
self.talkApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
{
Text("Using API key from the gateway profile.")
.font(.footnote)
.foregroundStyle(.secondary)
}
}
}
GridRow {
self.gridLabel("Interrupt")
Toggle("Stop speaking when you start talking", isOn: self.$talkInterruptOnSpeech)
.labelsHidden()
.toggleStyle(.checkbox)
.onChange(of: self.talkInterruptOnSpeech) { _, _ in self.autosaveConfig() }
}
}
}
.frame(maxWidth: .infinity, alignment: .leading)
}
private func gridLabel(_ text: String) -> some View {
Text(text)
.foregroundStyle(.secondary)
.frame(width: self.labelColumnWidth, alignment: .leading)
}
private func loadConfig() {
let parsed = self.loadConfigDict()
private func statusLine(label: String, color: Color) -> some View {
HStack(spacing: 6) {
Circle()
.fill(color)
.frame(width: 6, height: 6)
Text(label)
.font(.footnote)
.foregroundStyle(.secondary)
}
.padding(.top, 2)
}
private func loadConfig() async {
let parsed = await ConfigStore.load()
let agent = parsed["agent"] as? [String: Any]
let heartbeatMinutes = agent?["heartbeatMinutes"] as? Int
let heartbeatBody = agent?["heartbeatBody"] as? String
let browser = parsed["browser"] as? [String: Any]
let talk = parsed["talk"] as? [String: Any]
let loadedModel = (agent?["model"] as? String) ?? ""
if !loadedModel.isEmpty {
@@ -297,10 +411,32 @@ struct ConfigSettings: View {
if let color = browser["color"] as? String, !color.isEmpty { self.browserColorHex = color }
if let attachOnly = browser["attachOnly"] as? Bool { self.browserAttachOnly = attachOnly }
}
if let talk {
if let voice = talk["voiceId"] as? String { self.talkVoiceId = voice }
if let apiKey = talk["apiKey"] as? String { self.talkApiKey = apiKey }
if let interrupt = talk["interruptOnSpeech"] as? Bool {
self.talkInterruptOnSpeech = interrupt
}
}
}
private func refreshGatewayTalkApiKey() async {
do {
let snap: ConfigSnapshot = try await GatewayConnection.shared.requestDecoded(
method: .configGet,
params: nil,
timeoutMs: 8000)
let talk = snap.config?["talk"]?.dictionaryValue
let apiKey = talk?["apiKey"]?.stringValue?.trimmingCharacters(in: .whitespacesAndNewlines)
self.gatewayApiKeyFound = !(apiKey ?? "").isEmpty
} catch {
self.gatewayApiKeyFound = false
}
}
private func autosaveConfig() {
guard self.allowAutosave else { return }
guard self.allowAutosave, !self.isNixMode else { return }
Task { await self.saveConfig() }
}
@@ -309,39 +445,90 @@ struct ConfigSettings: View {
self.configSaving = true
defer { self.configSaving = false }
var root = self.loadConfigDict()
let configModel = self.configModel
let customModel = self.customModel
let heartbeatMinutes = self.heartbeatMinutes
let heartbeatBody = self.heartbeatBody
let browserEnabled = self.browserEnabled
let browserControlUrl = self.browserControlUrl
let browserColorHex = self.browserColorHex
let browserAttachOnly = self.browserAttachOnly
let talkVoiceId = self.talkVoiceId
let talkApiKey = self.talkApiKey
let talkInterruptOnSpeech = self.talkInterruptOnSpeech
let draft = ConfigDraft(
configModel: configModel,
customModel: customModel,
heartbeatMinutes: heartbeatMinutes,
heartbeatBody: heartbeatBody,
browserEnabled: browserEnabled,
browserControlUrl: browserControlUrl,
browserColorHex: browserColorHex,
browserAttachOnly: browserAttachOnly,
talkVoiceId: talkVoiceId,
talkApiKey: talkApiKey,
talkInterruptOnSpeech: talkInterruptOnSpeech)
let errorMessage = await ConfigSettings.buildAndSaveConfig(draft)
if let errorMessage {
self.modelError = errorMessage
}
}
@MainActor
private static func buildAndSaveConfig(_ draft: ConfigDraft) async -> String? {
var root = await ConfigStore.load()
var agent = root["agent"] as? [String: Any] ?? [:]
var browser = root["browser"] as? [String: Any] ?? [:]
var talk = root["talk"] as? [String: Any] ?? [:]
let chosenModel = (self.configModel == "__custom__" ? self.customModel : self.configModel)
let chosenModel = (draft.configModel == "__custom__" ? draft.customModel : draft.configModel)
.trimmingCharacters(in: .whitespacesAndNewlines)
let trimmedModel = chosenModel
if !trimmedModel.isEmpty { agent["model"] = trimmedModel }
if let heartbeatMinutes {
if let heartbeatMinutes = draft.heartbeatMinutes {
agent["heartbeatMinutes"] = heartbeatMinutes
}
let trimmedBody = self.heartbeatBody.trimmingCharacters(in: .whitespacesAndNewlines)
let trimmedBody = draft.heartbeatBody.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmedBody.isEmpty {
agent["heartbeatBody"] = trimmedBody
}
root["agent"] = agent
browser["enabled"] = self.browserEnabled
let trimmedUrl = self.browserControlUrl.trimmingCharacters(in: .whitespacesAndNewlines)
browser["enabled"] = draft.browserEnabled
let trimmedUrl = draft.browserControlUrl.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmedUrl.isEmpty { browser["controlUrl"] = trimmedUrl }
let trimmedColor = self.browserColorHex.trimmingCharacters(in: .whitespacesAndNewlines)
let trimmedColor = draft.browserColorHex.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmedColor.isEmpty { browser["color"] = trimmedColor }
browser["attachOnly"] = self.browserAttachOnly
browser["attachOnly"] = draft.browserAttachOnly
root["browser"] = browser
ClawdisConfigFile.saveDict(root)
}
let trimmedVoice = draft.talkVoiceId.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmedVoice.isEmpty {
talk.removeValue(forKey: "voiceId")
} else {
talk["voiceId"] = trimmedVoice
}
let trimmedApiKey = draft.talkApiKey.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmedApiKey.isEmpty {
talk.removeValue(forKey: "apiKey")
} else {
talk["apiKey"] = trimmedApiKey
}
talk["interruptOnSpeech"] = draft.talkInterruptOnSpeech
root["talk"] = talk
private func loadConfigDict() -> [String: Any] {
ClawdisConfigFile.loadDict()
do {
try await ConfigStore.save(root)
return nil
} catch {
return error.localizedDescription
}
}
private var browserColor: Color {
@@ -354,6 +541,41 @@ struct ConfigSettings: View {
return Color(red: r, green: g, blue: b)
}
private var talkVoiceSuggestions: [String] {
let env = ProcessInfo.processInfo.environment
let candidates = [
self.talkVoiceId,
env["ELEVENLABS_VOICE_ID"] ?? "",
env["SAG_VOICE_ID"] ?? "",
]
var seen = Set<String>()
return candidates
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
.filter { seen.insert($0).inserted }
}
private var hasEnvApiKey: Bool {
let raw = ProcessInfo.processInfo.environment["ELEVENLABS_API_KEY"] ?? ""
return !raw.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
}
private var apiKeyStatusLabel: String {
if self.hasEnvApiKey { return "ElevenLabs API key: found (environment)" }
if !self.talkApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return "ElevenLabs API key: stored in config"
}
if self.gatewayApiKeyFound { return "ElevenLabs API key: found (gateway)" }
return "ElevenLabs API key: missing"
}
private var apiKeyStatusColor: Color {
if self.hasEnvApiKey { return .green }
if !self.talkApiKey.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { return .green }
if self.gatewayApiKeyFound { return .green }
return .red
}
private var browserPathLabel: String? {
guard self.browserEnabled else { return nil }
@@ -0,0 +1,100 @@
import Foundation
enum ConfigStore {
struct Overrides: Sendable {
var isRemoteMode: (@Sendable () async -> Bool)?
var loadLocal: (@MainActor @Sendable () -> [String: Any])?
var saveLocal: (@MainActor @Sendable ([String: Any]) -> Void)?
var loadRemote: (@MainActor @Sendable () async -> [String: Any])?
var saveRemote: (@MainActor @Sendable ([String: Any]) async throws -> Void)?
}
private actor OverrideStore {
var overrides = Overrides()
func setOverride(_ overrides: Overrides) {
self.overrides = overrides
}
}
private static let overrideStore = OverrideStore()
private static func isRemoteMode() async -> Bool {
let overrides = await self.overrideStore.overrides
if let override = overrides.isRemoteMode {
return await override()
}
return await MainActor.run { AppStateStore.shared.connectionMode == .remote }
}
@MainActor
static func load() async -> [String: Any] {
let overrides = await self.overrideStore.overrides
if await self.isRemoteMode() {
if let override = overrides.loadRemote {
return await override()
}
return await self.loadFromGateway()
}
if let override = overrides.loadLocal {
return override()
}
return ClawdisConfigFile.loadDict()
}
@MainActor
static func save(_ root: sending [String: Any]) async throws {
let overrides = await self.overrideStore.overrides
if await self.isRemoteMode() {
if let override = overrides.saveRemote {
try await override(root)
} else {
try await self.saveToGateway(root)
}
} else {
if let override = overrides.saveLocal {
override(root)
} else {
ClawdisConfigFile.saveDict(root)
}
}
}
@MainActor
private static func loadFromGateway() async -> [String: Any] {
do {
let snap: ConfigSnapshot = try await GatewayConnection.shared.requestDecoded(
method: .configGet,
params: nil,
timeoutMs: 8000)
return snap.config?.mapValues { $0.foundationValue } ?? [:]
} catch {
return [:]
}
}
@MainActor
private static func saveToGateway(_ root: [String: Any]) async throws {
let data = try JSONSerialization.data(withJSONObject: root, options: [.prettyPrinted, .sortedKeys])
guard let raw = String(data: data, encoding: .utf8) else {
throw NSError(domain: "ConfigStore", code: 1, userInfo: [
NSLocalizedDescriptionKey: "Failed to encode config."
])
}
let params: [String: AnyCodable] = ["raw": AnyCodable(raw)]
_ = try await GatewayConnection.shared.requestRaw(
method: .configSet,
params: params,
timeoutMs: 10000)
}
#if DEBUG
static func _testSetOverrides(_ overrides: Overrides) async {
await self.overrideStore.setOverride(overrides)
}
static func _testClearOverrides() async {
await self.overrideStore.setOverride(.init())
}
#endif
}
@@ -22,13 +22,6 @@ final class ConnectionModeCoordinator {
case .local:
await RemoteTunnelManager.shared.stopAll()
WebChatManager.shared.resetTunnels()
do {
try await ControlChannel.shared.configure(mode: .local)
} catch {
// Control channel will mark itself degraded; nothing else to do here.
self.logger.error(
"control channel local configure failed: \(error.localizedDescription, privacy: .public)")
}
let shouldStart = GatewayAutostartPolicy.shouldStartGateway(mode: .local, paused: paused)
if shouldStart {
GatewayProcessManager.shared.setActive(true)
@@ -39,9 +32,17 @@ final class ConnectionModeCoordinator {
{
Task { await GatewayProcessManager.shared.ensureLaunchAgentEnabledIfNeeded() }
}
_ = await GatewayProcessManager.shared.waitForGatewayReady()
} else {
GatewayProcessManager.shared.stop()
}
do {
try await ControlChannel.shared.configure(mode: .local)
} catch {
// Control channel will mark itself degraded; nothing else to do here.
self.logger.error(
"control channel local configure failed: \(error.localizedDescription, privacy: .public)")
}
Task.detached { await PortGuardian.shared.sweep(mode: .local) }
case .remote:
@@ -2,6 +2,26 @@ import AppKit
import SwiftUI
struct ConnectionsSettings: View {
private enum ConnectionProvider: String, CaseIterable, Identifiable {
case whatsapp
case telegram
case discord
case signal
case imessage
var id: String { self.rawValue }
var sortOrder: Int {
switch self {
case .whatsapp: 0
case .telegram: 1
case .discord: 2
case .signal: 3
case .imessage: 4
}
}
}
@Bindable var store: ConnectionsStore
@State private var showTelegramToken = false
@State private var showDiscordToken = false
@@ -14,9 +34,9 @@ struct ConnectionsSettings: View {
ScrollView(.vertical) {
VStack(alignment: .leading, spacing: 14) {
self.header
self.whatsAppSection
self.telegramSection
self.discordSection
ForEach(self.orderedProviders) { provider in
self.providerSection(provider)
}
Spacer(minLength: 0)
}
.frame(maxWidth: .infinity, alignment: .leading)
@@ -31,7 +51,7 @@ struct ConnectionsSettings: View {
VStack(alignment: .leading, spacing: 6) {
Text("Connections")
.font(.title3.weight(.semibold))
Text("Link and monitor WhatsApp, Telegram, and Discord providers.")
Text("Link and monitor messaging providers.")
.font(.callout)
.foregroundStyle(.secondary)
}
@@ -243,6 +263,12 @@ struct ConnectionsSettings: View {
Divider().padding(.vertical, 2)
Grid(alignment: .leadingFirstTextBaseline, horizontalSpacing: 14, verticalSpacing: 10) {
GridRow {
self.gridLabel("Enabled")
Toggle("", isOn: self.$store.discordEnabled)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Bot token")
if self.showDiscordToken {
@@ -259,24 +285,19 @@ struct ConnectionsSettings: View {
.disabled(self.isDiscordTokenLocked)
}
GridRow {
self.gridLabel("Require mention")
Toggle("", isOn: self.$store.discordRequireMention)
self.gridLabel("Allow DMs from")
TextField("123456789, username#1234", text: self.$store.discordAllowFrom)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Group DMs")
Toggle("", isOn: self.$store.discordGroupEnabled)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Allow from")
TextField("discord:123, user:456", text: self.$store.discordAllowFrom)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Allow guilds")
TextField("guildId1, guildId2", text: self.$store.discordGuildAllowFrom)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Allow guild users")
TextField("userId1, userId2", text: self.$store.discordGuildUsersAllowFrom)
self.gridLabel("Group channels")
TextField("channelId1, channelId2", text: self.$store.discordGroupChannels)
.textFieldStyle(.roundedBorder)
}
GridRow {
@@ -284,6 +305,39 @@ struct ConnectionsSettings: View {
TextField("8", text: self.$store.discordMediaMaxMb)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("History limit")
TextField("20", text: self.$store.discordHistoryLimit)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Reactions")
Toggle("", isOn: self.$store.discordEnableReactions)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Slash command")
Toggle("", isOn: self.$store.discordSlashEnabled)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Slash name")
TextField("clawd", text: self.$store.discordSlashName)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Slash session prefix")
TextField("discord:slash", text: self.$store.discordSlashSessionPrefix)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Slash ephemeral")
Toggle("", isOn: self.$store.discordSlashEphemeral)
.labelsHidden()
.toggleStyle(.checkbox)
}
}
if self.isDiscordTokenLocked {
@@ -319,28 +373,280 @@ struct ConnectionsSettings: View {
}
}
private var signalSection: some View {
GroupBox("Signal") {
VStack(alignment: .leading, spacing: 10) {
self.providerHeader(
title: "Signal REST",
color: self.signalTint,
subtitle: self.signalSummary)
if let details = self.signalDetails {
Text(details)
.font(.caption)
.foregroundStyle(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
if let status = self.store.configStatus {
Text(status)
.font(.caption)
.foregroundStyle(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
Divider().padding(.vertical, 2)
Grid(alignment: .leadingFirstTextBaseline, horizontalSpacing: 14, verticalSpacing: 10) {
GridRow {
self.gridLabel("Enabled")
Toggle("", isOn: self.$store.signalEnabled)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Account")
TextField("+15551234567", text: self.$store.signalAccount)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("HTTP URL")
TextField("http://127.0.0.1:8080", text: self.$store.signalHttpUrl)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("HTTP host")
TextField("127.0.0.1", text: self.$store.signalHttpHost)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("HTTP port")
TextField("8080", text: self.$store.signalHttpPort)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("CLI path")
TextField("signal-cli", text: self.$store.signalCliPath)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Auto start")
Toggle("", isOn: self.$store.signalAutoStart)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Receive mode")
Picker("", selection: self.$store.signalReceiveMode) {
Text("Default").tag("")
Text("on-start").tag("on-start")
Text("manual").tag("manual")
}
.labelsHidden()
.pickerStyle(.menu)
}
GridRow {
self.gridLabel("Ignore attachments")
Toggle("", isOn: self.$store.signalIgnoreAttachments)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Ignore stories")
Toggle("", isOn: self.$store.signalIgnoreStories)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Read receipts")
Toggle("", isOn: self.$store.signalSendReadReceipts)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Allow from")
TextField("12345, +1555", text: self.$store.signalAllowFrom)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Media max MB")
TextField("8", text: self.$store.signalMediaMaxMb)
.textFieldStyle(.roundedBorder)
}
}
HStack(spacing: 12) {
Button {
Task { await self.store.saveSignalConfig() }
} label: {
if self.store.isSavingConfig {
ProgressView().controlSize(.small)
} else {
Text("Save")
}
}
.buttonStyle(.borderedProminent)
.disabled(self.store.isSavingConfig)
Spacer()
Button("Refresh") {
Task { await self.store.refresh(probe: true) }
}
.buttonStyle(.bordered)
.disabled(self.store.isRefreshing)
}
.font(.caption)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
}
private var imessageSection: some View {
GroupBox("iMessage") {
VStack(alignment: .leading, spacing: 10) {
self.providerHeader(
title: "iMessage (imsg)",
color: self.imessageTint,
subtitle: self.imessageSummary)
if let details = self.imessageDetails {
Text(details)
.font(.caption)
.foregroundStyle(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
if let status = self.store.configStatus {
Text(status)
.font(.caption)
.foregroundStyle(.secondary)
.fixedSize(horizontal: false, vertical: true)
}
Divider().padding(.vertical, 2)
Grid(alignment: .leadingFirstTextBaseline, horizontalSpacing: 14, verticalSpacing: 10) {
GridRow {
self.gridLabel("Enabled")
Toggle("", isOn: self.$store.imessageEnabled)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("CLI path")
TextField("imsg", text: self.$store.imessageCliPath)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("DB path")
TextField("~/Library/Messages/chat.db", text: self.$store.imessageDbPath)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Service")
Picker("", selection: self.$store.imessageService) {
Text("auto").tag("auto")
Text("imessage").tag("imessage")
Text("sms").tag("sms")
}
.labelsHidden()
.pickerStyle(.menu)
}
GridRow {
self.gridLabel("Region")
TextField("US", text: self.$store.imessageRegion)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Allow from")
TextField("chat_id:101, +1555", text: self.$store.imessageAllowFrom)
.textFieldStyle(.roundedBorder)
}
GridRow {
self.gridLabel("Attachments")
Toggle("", isOn: self.$store.imessageIncludeAttachments)
.labelsHidden()
.toggleStyle(.checkbox)
}
GridRow {
self.gridLabel("Media max MB")
TextField("16", text: self.$store.imessageMediaMaxMb)
.textFieldStyle(.roundedBorder)
}
}
HStack(spacing: 12) {
Button {
Task { await self.store.saveIMessageConfig() }
} label: {
if self.store.isSavingConfig {
ProgressView().controlSize(.small)
} else {
Text("Save")
}
}
.buttonStyle(.borderedProminent)
.disabled(self.store.isSavingConfig)
Spacer()
Button("Refresh") {
Task { await self.store.refresh(probe: true) }
}
.buttonStyle(.bordered)
.disabled(self.store.isRefreshing)
}
.font(.caption)
}
.frame(maxWidth: .infinity, alignment: .leading)
}
}
private var whatsAppTint: Color {
guard let status = self.store.snapshot?.whatsapp else { return .secondary }
if !status.configured { return .secondary }
if !status.linked { return .red }
if status.connected { return .green }
if status.lastError != nil { return .orange }
return .green
if status.connected { return .green }
if status.running { return .orange }
return .orange
}
private var telegramTint: Color {
guard let status = self.store.snapshot?.telegram else { return .secondary }
if !status.configured { return .secondary }
if status.running { return .green }
if status.lastError != nil { return .orange }
return .secondary
if status.probe?.ok == false { return .orange }
if status.running { return .green }
return .orange
}
private var discordTint: Color {
guard let status = self.store.snapshot?.discord else { return .secondary }
if !status.configured { return .secondary }
if status.running { return .green }
if status.lastError != nil { return .orange }
return .secondary
if status.probe?.ok == false { return .orange }
if status.running { return .green }
return .orange
}
private var signalTint: Color {
guard let status = self.store.snapshot?.signal else { return .secondary }
if !status.configured { return .secondary }
if status.lastError != nil { return .orange }
if status.probe?.ok == false { return .orange }
if status.running { return .green }
return .orange
}
private var imessageTint: Color {
guard let status = self.store.snapshot?.imessage else { return .secondary }
if !status.configured { return .secondary }
if status.lastError != nil { return .orange }
if status.probe?.ok == false { return .orange }
if status.running { return .green }
return .orange
}
private var whatsAppSummary: String {
@@ -365,6 +671,20 @@ struct ConnectionsSettings: View {
return "Configured"
}
private var signalSummary: String {
guard let status = self.store.snapshot?.signal else { return "Checking…" }
if !status.configured { return "Not configured" }
if status.running { return "Running" }
return "Configured"
}
private var imessageSummary: String {
guard let status = self.store.snapshot?.imessage else { return "Checking…" }
if !status.configured { return "Not configured" }
if status.running { return "Running" }
return "Configured"
}
private var whatsAppDetails: String? {
guard let status = self.store.snapshot?.whatsapp else { return nil }
var lines: [String] = []
@@ -454,6 +774,54 @@ struct ConnectionsSettings: View {
return lines.isEmpty ? nil : lines.joined(separator: " · ")
}
private var signalDetails: String? {
guard let status = self.store.snapshot?.signal else { return nil }
var lines: [String] = []
lines.append("Base URL: \(status.baseUrl)")
if let probe = status.probe {
if probe.ok {
if let version = probe.version, !version.isEmpty {
lines.append("Version \(version)")
}
if let elapsed = probe.elapsedMs {
lines.append("Probe \(Int(elapsed))ms")
}
} else {
let code = probe.status.map { String($0) } ?? "unknown"
lines.append("Probe failed (\(code))")
}
}
if let last = self.date(fromMs: status.lastProbeAt) {
lines.append("Last probe \(relativeAge(from: last))")
}
if let err = status.lastError, !err.isEmpty {
lines.append("Error: \(err)")
}
return lines.isEmpty ? nil : lines.joined(separator: " · ")
}
private var imessageDetails: String? {
guard let status = self.store.snapshot?.imessage else { return nil }
var lines: [String] = []
if let cliPath = status.cliPath, !cliPath.isEmpty {
lines.append("CLI: \(cliPath)")
}
if let dbPath = status.dbPath, !dbPath.isEmpty {
lines.append("DB: \(dbPath)")
}
if let probe = status.probe, !probe.ok {
let err = probe.error ?? "probe failed"
lines.append("Probe error: \(err)")
}
if let last = self.date(fromMs: status.lastProbeAt) {
lines.append("Last probe \(relativeAge(from: last))")
}
if let err = status.lastError, !err.isEmpty {
lines.append("Error: \(err)")
}
return lines.isEmpty ? nil : lines.joined(separator: " · ")
}
private var isTelegramTokenLocked: Bool {
self.store.snapshot?.telegram.tokenSource == "env"
}
@@ -462,6 +830,51 @@ struct ConnectionsSettings: View {
self.store.snapshot?.discord?.tokenSource == "env"
}
private var orderedProviders: [ConnectionProvider] {
ConnectionProvider.allCases.sorted { lhs, rhs in
let lhsEnabled = self.providerEnabled(lhs)
let rhsEnabled = self.providerEnabled(rhs)
if lhsEnabled != rhsEnabled { return lhsEnabled && !rhsEnabled }
return lhs.sortOrder < rhs.sortOrder
}
}
private func providerEnabled(_ provider: ConnectionProvider) -> Bool {
switch provider {
case .whatsapp:
guard let status = self.store.snapshot?.whatsapp else { return false }
return status.configured || status.linked || status.running
case .telegram:
guard let status = self.store.snapshot?.telegram else { return false }
return status.configured || status.running
case .discord:
guard let status = self.store.snapshot?.discord else { return false }
return status.configured || status.running
case .signal:
guard let status = self.store.snapshot?.signal else { return false }
return status.configured || status.running
case .imessage:
guard let status = self.store.snapshot?.imessage else { return false }
return status.configured || status.running
}
}
@ViewBuilder
private func providerSection(_ provider: ConnectionProvider) -> some View {
switch provider {
case .whatsapp:
self.whatsAppSection
case .telegram:
self.telegramSection
case .discord:
self.discordSection
case .signal:
self.signalSection
case .imessage:
self.imessageSection
}
}
private func providerHeader(title: String, color: Color, subtitle: String) -> some View {
HStack(spacing: 10) {
Circle()
@@ -472,7 +885,7 @@ struct ConnectionsSettings: View {
.font(.headline)
Text(subtitle)
.font(.caption)
.foregroundStyle(.secondary)
.foregroundStyle(color)
}
Spacer()
}
+432 -51
View File
@@ -85,10 +85,48 @@ struct ProvidersStatusSnapshot: Codable {
let lastProbeAt: Double?
}
struct SignalProbe: Codable {
let ok: Bool
let status: Int?
let error: String?
let elapsedMs: Double?
let version: String?
}
struct SignalStatus: Codable {
let configured: Bool
let baseUrl: String
let running: Bool
let lastStartAt: Double?
let lastStopAt: Double?
let lastError: String?
let probe: SignalProbe?
let lastProbeAt: Double?
}
struct IMessageProbe: Codable {
let ok: Bool
let error: String?
}
struct IMessageStatus: Codable {
let configured: Bool
let running: Bool
let lastStartAt: Double?
let lastStopAt: Double?
let lastError: String?
let cliPath: String?
let dbPath: String?
let probe: IMessageProbe?
let lastProbeAt: Double?
}
let ts: Double
let whatsapp: WhatsAppStatus
let telegram: TelegramStatus
let discord: DiscordStatus?
let signal: SignalStatus?
let imessage: IMessageStatus?
}
struct ConfigSnapshot: Codable {
@@ -129,12 +167,39 @@ final class ConnectionsStore {
var telegramWebhookSecret: String = ""
var telegramWebhookPath: String = ""
var telegramBusy = false
var discordEnabled = true
var discordToken: String = ""
var discordRequireMention = true
var discordAllowFrom: String = ""
var discordGuildAllowFrom: String = ""
var discordGuildUsersAllowFrom: String = ""
var discordGroupEnabled = false
var discordGroupChannels: String = ""
var discordMediaMaxMb: String = ""
var discordHistoryLimit: String = ""
var discordEnableReactions = true
var discordSlashEnabled = false
var discordSlashName: String = ""
var discordSlashSessionPrefix: String = ""
var discordSlashEphemeral = true
var signalEnabled = true
var signalAccount: String = ""
var signalHttpUrl: String = ""
var signalHttpHost: String = ""
var signalHttpPort: String = ""
var signalCliPath: String = ""
var signalAutoStart = true
var signalReceiveMode: String = ""
var signalIgnoreAttachments = false
var signalIgnoreStories = false
var signalSendReadReceipts = false
var signalAllowFrom: String = ""
var signalMediaMaxMb: String = ""
var imessageEnabled = true
var imessageCliPath: String = ""
var imessageDbPath: String = ""
var imessageService: String = "auto"
var imessageRegion: String = ""
var imessageAllowFrom: String = ""
var imessageIncludeAttachments = false
var imessageMediaMaxMb: String = ""
var configStatus: String?
var isSavingConfig = false
@@ -294,6 +359,11 @@ final class ConnectionsStore {
: nil
self.configRoot = snap.config?.mapValues { $0.foundationValue } ?? [:]
self.configLoaded = true
let ui = snap.config?["ui"]?.dictionaryValue
let rawSeam = ui?["seamColor"]?.stringValue?.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
AppStateStore.shared.seamColorHex = rawSeam.isEmpty ? nil : rawSeam
let telegram = snap.config?["telegram"]?.dictionaryValue
self.telegramToken = telegram?["botToken"]?.stringValue ?? ""
self.telegramRequireMention = telegram?["requireMention"]?.boolValue ?? true
@@ -314,9 +384,10 @@ final class ConnectionsStore {
self.telegramWebhookPath = telegram?["webhookPath"]?.stringValue ?? ""
let discord = snap.config?["discord"]?.dictionaryValue
self.discordEnabled = discord?["enabled"]?.boolValue ?? true
self.discordToken = discord?["token"]?.stringValue ?? ""
self.discordRequireMention = discord?["requireMention"]?.boolValue ?? true
if let allow = discord?["allowFrom"]?.arrayValue {
let discordDm = discord?["dm"]?.dictionaryValue
if let allow = discordDm?["allowFrom"]?.arrayValue {
let strings = allow.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
@@ -327,38 +398,91 @@ final class ConnectionsStore {
} else {
self.discordAllowFrom = ""
}
if let guildAllow = discord?["guildAllowFrom"]?.dictionaryValue {
if let guilds = guildAllow["guilds"]?.arrayValue {
let strings = guilds.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
if let doubleVal = entry.doubleValue { return String(Int(doubleVal)) }
return nil
}
self.discordGuildAllowFrom = strings.joined(separator: ", ")
} else {
self.discordGuildAllowFrom = ""
}
if let users = guildAllow["users"]?.arrayValue {
let strings = users.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
if let doubleVal = entry.doubleValue { return String(Int(doubleVal)) }
return nil
}
self.discordGuildUsersAllowFrom = strings.joined(separator: ", ")
} else {
self.discordGuildUsersAllowFrom = ""
self.discordGroupEnabled = discordDm?["groupEnabled"]?.boolValue ?? false
if let channels = discordDm?["groupChannels"]?.arrayValue {
let strings = channels.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
if let doubleVal = entry.doubleValue { return String(Int(doubleVal)) }
return nil
}
self.discordGroupChannels = strings.joined(separator: ", ")
} else {
self.discordGuildAllowFrom = ""
self.discordGuildUsersAllowFrom = ""
self.discordGroupChannels = ""
}
if let media = discord?["mediaMaxMb"]?.doubleValue ?? discord?["mediaMaxMb"]?.intValue.map(Double.init) {
self.discordMediaMaxMb = String(Int(media))
} else {
self.discordMediaMaxMb = ""
}
if let history = discord?["historyLimit"]?.doubleValue ?? discord?["historyLimit"]?.intValue.map(Double.init) {
self.discordHistoryLimit = String(Int(history))
} else {
self.discordHistoryLimit = ""
}
self.discordEnableReactions = discord?["enableReactions"]?.boolValue ?? true
let slash = discord?["slashCommand"]?.dictionaryValue
self.discordSlashEnabled = slash?["enabled"]?.boolValue ?? false
self.discordSlashName = slash?["name"]?.stringValue ?? ""
self.discordSlashSessionPrefix = slash?["sessionPrefix"]?.stringValue ?? ""
self.discordSlashEphemeral = slash?["ephemeral"]?.boolValue ?? true
let signal = snap.config?["signal"]?.dictionaryValue
self.signalEnabled = signal?["enabled"]?.boolValue ?? true
self.signalAccount = signal?["account"]?.stringValue ?? ""
self.signalHttpUrl = signal?["httpUrl"]?.stringValue ?? ""
self.signalHttpHost = signal?["httpHost"]?.stringValue ?? ""
if let port = signal?["httpPort"]?.doubleValue ?? signal?["httpPort"]?.intValue.map(Double.init) {
self.signalHttpPort = String(Int(port))
} else {
self.signalHttpPort = ""
}
self.signalCliPath = signal?["cliPath"]?.stringValue ?? ""
self.signalAutoStart = signal?["autoStart"]?.boolValue ?? true
self.signalReceiveMode = signal?["receiveMode"]?.stringValue ?? ""
self.signalIgnoreAttachments = signal?["ignoreAttachments"]?.boolValue ?? false
self.signalIgnoreStories = signal?["ignoreStories"]?.boolValue ?? false
self.signalSendReadReceipts = signal?["sendReadReceipts"]?.boolValue ?? false
if let allow = signal?["allowFrom"]?.arrayValue {
let strings = allow.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
if let doubleVal = entry.doubleValue { return String(Int(doubleVal)) }
return nil
}
self.signalAllowFrom = strings.joined(separator: ", ")
} else {
self.signalAllowFrom = ""
}
if let media = signal?["mediaMaxMb"]?.doubleValue ?? signal?["mediaMaxMb"]?.intValue.map(Double.init) {
self.signalMediaMaxMb = String(Int(media))
} else {
self.signalMediaMaxMb = ""
}
let imessage = snap.config?["imessage"]?.dictionaryValue
self.imessageEnabled = imessage?["enabled"]?.boolValue ?? true
self.imessageCliPath = imessage?["cliPath"]?.stringValue ?? ""
self.imessageDbPath = imessage?["dbPath"]?.stringValue ?? ""
self.imessageService = imessage?["service"]?.stringValue ?? "auto"
self.imessageRegion = imessage?["region"]?.stringValue ?? ""
if let allow = imessage?["allowFrom"]?.arrayValue {
let strings = allow.compactMap { entry -> String? in
if let str = entry.stringValue { return str }
if let intVal = entry.intValue { return String(intVal) }
if let doubleVal = entry.doubleValue { return String(Int(doubleVal)) }
return nil
}
self.imessageAllowFrom = strings.joined(separator: ", ")
} else {
self.imessageAllowFrom = ""
}
self.imessageIncludeAttachments = imessage?["includeAttachments"]?.boolValue ?? false
if let media = imessage?["mediaMaxMb"]?.doubleValue ?? imessage?["mediaMaxMb"]?.intValue.map(Double.init) {
self.imessageMediaMaxMb = String(Int(media))
} else {
self.imessageMediaMaxMb = ""
}
} catch {
self.configStatus = error.localizedDescription
}
@@ -459,6 +583,11 @@ final class ConnectionsStore {
}
var discord: [String: Any] = (self.configRoot["discord"] as? [String: Any]) ?? [:]
if self.discordEnabled {
discord.removeValue(forKey: "enabled")
} else {
discord["enabled"] = false
}
let token = self.discordToken.trimmingCharacters(in: .whitespacesAndNewlines)
if token.isEmpty {
discord.removeValue(forKey: "token")
@@ -466,43 +595,37 @@ final class ConnectionsStore {
discord["token"] = token
}
discord["requireMention"] = self.discordRequireMention
var dm: [String: Any] = (discord["dm"] as? [String: Any]) ?? [:]
let allow = self.discordAllowFrom
.split(separator: ",")
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if allow.isEmpty {
discord.removeValue(forKey: "allowFrom")
dm.removeValue(forKey: "allowFrom")
} else {
discord["allowFrom"] = allow
dm["allowFrom"] = allow
}
var guildAllow: [String: Any] = (discord["guildAllowFrom"] as? [String: Any]) ?? [:]
let guilds = self.discordGuildAllowFrom
if self.discordGroupEnabled {
dm["groupEnabled"] = true
} else {
dm.removeValue(forKey: "groupEnabled")
}
let groupChannels = self.discordGroupChannels
.split(separator: ",")
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if guilds.isEmpty {
guildAllow.removeValue(forKey: "guilds")
if groupChannels.isEmpty {
dm.removeValue(forKey: "groupChannels")
} else {
guildAllow["guilds"] = guilds
dm["groupChannels"] = groupChannels
}
let users = self.discordGuildUsersAllowFrom
.split(separator: ",")
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if users.isEmpty {
guildAllow.removeValue(forKey: "users")
if dm.isEmpty {
discord.removeValue(forKey: "dm")
} else {
guildAllow["users"] = users
}
if guildAllow.isEmpty {
discord.removeValue(forKey: "guildAllowFrom")
} else {
discord["guildAllowFrom"] = guildAllow
discord["dm"] = dm
}
let media = self.discordMediaMaxMb.trimmingCharacters(in: .whitespacesAndNewlines)
@@ -512,6 +635,50 @@ final class ConnectionsStore {
discord["mediaMaxMb"] = value
}
let history = self.discordHistoryLimit.trimmingCharacters(in: .whitespacesAndNewlines)
if history.isEmpty {
discord.removeValue(forKey: "historyLimit")
} else if let value = Int(history), value >= 0 {
discord["historyLimit"] = value
} else {
discord.removeValue(forKey: "historyLimit")
}
if self.discordEnableReactions {
discord.removeValue(forKey: "enableReactions")
} else {
discord["enableReactions"] = false
}
var slash: [String: Any] = (discord["slashCommand"] as? [String: Any]) ?? [:]
if self.discordSlashEnabled {
slash["enabled"] = true
} else {
slash.removeValue(forKey: "enabled")
}
let slashName = self.discordSlashName.trimmingCharacters(in: .whitespacesAndNewlines)
if slashName.isEmpty {
slash.removeValue(forKey: "name")
} else {
slash["name"] = slashName
}
let slashPrefix = self.discordSlashSessionPrefix.trimmingCharacters(in: .whitespacesAndNewlines)
if slashPrefix.isEmpty {
slash.removeValue(forKey: "sessionPrefix")
} else {
slash["sessionPrefix"] = slashPrefix
}
if self.discordSlashEphemeral {
slash.removeValue(forKey: "ephemeral")
} else {
slash["ephemeral"] = false
}
if slash.isEmpty {
discord.removeValue(forKey: "slashCommand")
} else {
discord["slashCommand"] = slash
}
if discord.isEmpty {
self.configRoot.removeValue(forKey: "discord")
} else {
@@ -537,6 +704,220 @@ final class ConnectionsStore {
self.configStatus = error.localizedDescription
}
}
func saveSignalConfig() async {
guard !self.isSavingConfig else { return }
self.isSavingConfig = true
defer { self.isSavingConfig = false }
if !self.configLoaded {
await self.loadConfig()
}
var signal: [String: Any] = (self.configRoot["signal"] as? [String: Any]) ?? [:]
if self.signalEnabled {
signal.removeValue(forKey: "enabled")
} else {
signal["enabled"] = false
}
let account = self.signalAccount.trimmingCharacters(in: .whitespacesAndNewlines)
if account.isEmpty {
signal.removeValue(forKey: "account")
} else {
signal["account"] = account
}
let httpUrl = self.signalHttpUrl.trimmingCharacters(in: .whitespacesAndNewlines)
if httpUrl.isEmpty {
signal.removeValue(forKey: "httpUrl")
} else {
signal["httpUrl"] = httpUrl
}
let httpHost = self.signalHttpHost.trimmingCharacters(in: .whitespacesAndNewlines)
if httpHost.isEmpty {
signal.removeValue(forKey: "httpHost")
} else {
signal["httpHost"] = httpHost
}
let httpPort = self.signalHttpPort.trimmingCharacters(in: .whitespacesAndNewlines)
if httpPort.isEmpty {
signal.removeValue(forKey: "httpPort")
} else if let value = Double(httpPort) {
signal["httpPort"] = value
}
let cliPath = self.signalCliPath.trimmingCharacters(in: .whitespacesAndNewlines)
if cliPath.isEmpty {
signal.removeValue(forKey: "cliPath")
} else {
signal["cliPath"] = cliPath
}
if self.signalAutoStart {
signal.removeValue(forKey: "autoStart")
} else {
signal["autoStart"] = false
}
let receiveMode = self.signalReceiveMode.trimmingCharacters(in: .whitespacesAndNewlines)
if receiveMode.isEmpty {
signal.removeValue(forKey: "receiveMode")
} else {
signal["receiveMode"] = receiveMode
}
if self.signalIgnoreAttachments {
signal["ignoreAttachments"] = true
} else {
signal.removeValue(forKey: "ignoreAttachments")
}
if self.signalIgnoreStories {
signal["ignoreStories"] = true
} else {
signal.removeValue(forKey: "ignoreStories")
}
if self.signalSendReadReceipts {
signal["sendReadReceipts"] = true
} else {
signal.removeValue(forKey: "sendReadReceipts")
}
let allow = self.signalAllowFrom
.split(separator: ",")
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if allow.isEmpty {
signal.removeValue(forKey: "allowFrom")
} else {
signal["allowFrom"] = allow
}
let media = self.signalMediaMaxMb.trimmingCharacters(in: .whitespacesAndNewlines)
if media.isEmpty {
signal.removeValue(forKey: "mediaMaxMb")
} else if let value = Double(media) {
signal["mediaMaxMb"] = value
}
if signal.isEmpty {
self.configRoot.removeValue(forKey: "signal")
} else {
self.configRoot["signal"] = signal
}
do {
let data = try JSONSerialization.data(
withJSONObject: self.configRoot,
options: [.prettyPrinted, .sortedKeys])
guard let raw = String(data: data, encoding: .utf8) else {
self.configStatus = "Failed to encode config."
return
}
let params: [String: AnyCodable] = ["raw": AnyCodable(raw)]
_ = try await GatewayConnection.shared.requestRaw(
method: .configSet,
params: params,
timeoutMs: 10000)
self.configStatus = "Saved to ~/.clawdis/clawdis.json."
await self.refresh(probe: true)
} catch {
self.configStatus = error.localizedDescription
}
}
func saveIMessageConfig() async {
guard !self.isSavingConfig else { return }
self.isSavingConfig = true
defer { self.isSavingConfig = false }
if !self.configLoaded {
await self.loadConfig()
}
var imessage: [String: Any] = (self.configRoot["imessage"] as? [String: Any]) ?? [:]
if self.imessageEnabled {
imessage.removeValue(forKey: "enabled")
} else {
imessage["enabled"] = false
}
let cliPath = self.imessageCliPath.trimmingCharacters(in: .whitespacesAndNewlines)
if cliPath.isEmpty {
imessage.removeValue(forKey: "cliPath")
} else {
imessage["cliPath"] = cliPath
}
let dbPath = self.imessageDbPath.trimmingCharacters(in: .whitespacesAndNewlines)
if dbPath.isEmpty {
imessage.removeValue(forKey: "dbPath")
} else {
imessage["dbPath"] = dbPath
}
let service = self.imessageService.trimmingCharacters(in: .whitespacesAndNewlines)
if service.isEmpty || service == "auto" {
imessage.removeValue(forKey: "service")
} else {
imessage["service"] = service
}
let region = self.imessageRegion.trimmingCharacters(in: .whitespacesAndNewlines)
if region.isEmpty {
imessage.removeValue(forKey: "region")
} else {
imessage["region"] = region
}
let allow = self.imessageAllowFrom
.split(separator: ",")
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
if allow.isEmpty {
imessage.removeValue(forKey: "allowFrom")
} else {
imessage["allowFrom"] = allow
}
if self.imessageIncludeAttachments {
imessage["includeAttachments"] = true
} else {
imessage.removeValue(forKey: "includeAttachments")
}
let media = self.imessageMediaMaxMb.trimmingCharacters(in: .whitespacesAndNewlines)
if media.isEmpty {
imessage.removeValue(forKey: "mediaMaxMb")
} else if let value = Double(media) {
imessage["mediaMaxMb"] = value
}
if imessage.isEmpty {
self.configRoot.removeValue(forKey: "imessage")
} else {
self.configRoot["imessage"] = imessage
}
do {
let data = try JSONSerialization.data(
withJSONObject: self.configRoot,
options: [.prettyPrinted, .sortedKeys])
guard let raw = String(data: data, encoding: .utf8) else {
self.configStatus = "Failed to encode config."
return
}
let params: [String: AnyCodable] = ["raw": AnyCodable(raw)]
_ = try await GatewayConnection.shared.requestRaw(
method: .configSet,
params: params,
timeoutMs: 10000)
self.configStatus = "Saved to ~/.clawdis/clawdis.json."
await self.refresh(probe: true)
} catch {
self.configStatus = error.localizedDescription
}
}
}
private struct WhatsAppLoginStartResult: Codable {
@@ -16,6 +16,7 @@ let voiceWakeMicKey = "clawdis.voiceWakeMicID"
let voiceWakeLocaleKey = "clawdis.voiceWakeLocaleID"
let voiceWakeAdditionalLocalesKey = "clawdis.voiceWakeAdditionalLocaleIDs"
let voicePushToTalkEnabledKey = "clawdis.voicePushToTalkEnabled"
let talkEnabledKey = "clawdis.talkEnabled"
let iconOverrideKey = "clawdis.iconOverride"
let connectionModeKey = "clawdis.connectionMode"
let remoteTargetKey = "clawdis.remoteTarget"
@@ -31,5 +32,6 @@ let modelCatalogReloadKey = "clawdis.modelCatalogReload"
let attachExistingGatewayOnlyKey = "clawdis.gateway.attachExistingOnly"
let heartbeatsEnabledKey = "clawdis.heartbeatsEnabled"
let debugFileLogEnabledKey = "clawdis.debug.fileLogEnabled"
let appLogLevelKey = "clawdis.debug.appLogLevel"
let voiceWakeSupported: Bool = ProcessInfo.processInfo.operatingSystemVersion.majorVersion >= 26
let cliHelperSearchPaths = ["/usr/local/bin", "/opt/homebrew/bin"]
@@ -79,7 +79,7 @@ struct ContextMenuCardView: View {
height: self.barHeight)
HStack(alignment: .firstTextBaseline, spacing: 8) {
Text(row.key)
Text(row.label)
.font(.caption.weight(row.key == "main" ? .semibold : .regular))
.lineLimit(1)
.truncationMode(.middle)
@@ -1,7 +1,6 @@
import ClawdisProtocol
import Foundation
import Observation
import OSLog
import SwiftUI
struct ControlHeartbeatEvent: Codable {
@@ -53,18 +52,39 @@ final class ControlChannel {
case degraded(String)
}
private(set) var state: ConnectionState = .disconnected
private(set) var state: ConnectionState = .disconnected {
didSet {
CanvasManager.shared.refreshDebugStatus()
guard oldValue != state else { return }
switch state {
case .connected:
self.logger.info("control channel state -> connected")
case .connecting:
self.logger.info("control channel state -> connecting")
case .disconnected:
self.logger.info("control channel state -> disconnected")
self.scheduleRecovery(reason: "disconnected")
case let .degraded(message):
let detail = message.isEmpty ? "degraded" : "degraded: \(message)"
self.logger.info("control channel state -> \(detail, privacy: .public)")
self.scheduleRecovery(reason: message)
}
}
}
private(set) var lastPingMs: Double?
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "control")
private var eventTask: Task<Void, Never>?
private var recoveryTask: Task<Void, Never>?
private var lastRecoveryAt: Date?
private init() {
self.startEventStream()
}
func configure() async {
self.logger.info("control channel configure mode=local")
self.state = .connecting
do {
try await GatewayConnection.shared.refresh()
@@ -83,6 +103,9 @@ final class ControlChannel {
case let .remote(target, identity):
do {
_ = (target, identity)
let idSet = !identity.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
self.logger.info(
"control channel configure mode=remote target=\(target, privacy: .public) identitySet=\(idSet, privacy: .public)")
_ = try await GatewayEndpointStore.shared.ensureRemoteControlTunnel()
await self.configure()
} catch {
@@ -216,7 +239,52 @@ final class ControlChannel {
}
let detail = nsError.localizedDescription.isEmpty ? "unknown gateway error" : nsError.localizedDescription
return "Gateway error: \(detail)"
let trimmed = detail.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmed.lowercased().hasPrefix("gateway error:") { return trimmed }
return "Gateway error: \(trimmed)"
}
private func scheduleRecovery(reason: String) {
let now = Date()
if let last = self.lastRecoveryAt, now.timeIntervalSince(last) < 10 { return }
guard self.recoveryTask == nil else { return }
self.lastRecoveryAt = now
self.recoveryTask = Task { [weak self] in
guard let self else { return }
let mode = await MainActor.run { AppStateStore.shared.connectionMode }
guard mode != .unconfigured else {
self.recoveryTask = nil
return
}
let trimmedReason = reason.trimmingCharacters(in: .whitespacesAndNewlines)
let reasonText = trimmedReason.isEmpty ? "unknown" : trimmedReason
self.logger.info(
"control channel recovery starting mode=\(String(describing: mode), privacy: .public) reason=\(reasonText, privacy: .public)")
if mode == .local {
GatewayProcessManager.shared.setActive(true)
}
if mode == .remote {
do {
let port = try await GatewayEndpointStore.shared.ensureRemoteControlTunnel()
self.logger.info("control channel recovery ensured SSH tunnel port=\(port, privacy: .public)")
} catch {
self.logger.error(
"control channel recovery tunnel failed \(error.localizedDescription, privacy: .public)")
}
}
do {
try await GatewayConnection.shared.refresh()
self.logger.info("control channel recovery finished")
} catch {
self.logger.error(
"control channel recovery failed \(error.localizedDescription, privacy: .public)")
}
self.recoveryTask = nil
}
}
func sendSystemEvent(_ text: String, params: [String: AnyHashable] = [:]) async throws {
@@ -45,6 +45,13 @@ enum DebugActions {
@MainActor
static func openSessionStore() {
if AppStateStore.shared.connectionMode == .remote {
let alert = NSAlert()
alert.messageText = "Remote mode"
alert.informativeText = "Session store lives on the gateway host in remote mode."
alert.runModal()
return
}
let path = self.resolveSessionStorePath()
let url = URL(fileURLWithPath: path)
if FileManager.default.fileExists(atPath: path) {
+29 -6
View File
@@ -1,8 +1,10 @@
import AppKit
import Observation
import SwiftUI
import UniformTypeIdentifiers
struct DebugSettings: View {
@Bindable var state: AppState
private let isPreview = ProcessInfo.processInfo.isPreview
private let labelColumnWidth: CGFloat = 140
@AppStorage(modelCatalogPathKey) private var modelCatalogPath: String = ModelCatalogLoader.defaultPath
@@ -28,6 +30,7 @@ struct DebugSettings: View {
@State private var pendingKill: DebugActions.PortListener?
@AppStorage(attachExistingGatewayOnlyKey) private var attachExistingGatewayOnly: Bool = false
@AppStorage(debugFileLogEnabledKey) private var diagnosticsFileLogEnabled: Bool = false
@AppStorage(appLogLevelKey) private var appLogLevelRaw: String = AppLogLevel.default.rawValue
@State private var canvasSessionKey: String = "main"
@State private var canvasStatus: String?
@@ -36,6 +39,10 @@ struct DebugSettings: View {
@State private var canvasEvalResult: String?
@State private var canvasSnapshotPath: String?
init(state: AppState = AppStateStore.shared) {
self.state = state
}
var body: some View {
ScrollView(.vertical) {
VStack(alignment: .leading, spacing: 14) {
@@ -194,7 +201,9 @@ struct DebugSettings: View {
.overlay(RoundedRectangle(cornerRadius: 6).stroke(Color.secondary.opacity(0.2)))
HStack(spacing: 8) {
Button("Restart Gateway") { DebugActions.restartGateway() }
if self.canRestartGateway {
Button("Restart Gateway") { DebugActions.restartGateway() }
}
Button("Clear log") { GatewayProcessManager.shared.clearLog() }
Spacer(minLength: 0)
}
@@ -224,13 +233,23 @@ struct DebugSettings: View {
}
GridRow {
self.gridLabel("Diagnostics")
VStack(alignment: .leading, spacing: 6) {
self.gridLabel("App logging")
VStack(alignment: .leading, spacing: 8) {
Picker("Verbosity", selection: self.$appLogLevelRaw) {
ForEach(AppLogLevel.allCases) { level in
Text(level.title).tag(level.rawValue)
}
}
.pickerStyle(.menu)
.labelsHidden()
.help("Controls the macOS app log verbosity.")
Toggle("Write rolling diagnostics log (JSONL)", isOn: self.$diagnosticsFileLogEnabled)
.toggleStyle(.checkbox)
.help(
"Writes a rotating, local-only diagnostics log under ~/Library/Logs/Clawdis/. " +
"Writes a rotating, local-only log under ~/Library/Logs/Clawdis/. " +
"Enable only while actively debugging.")
HStack(spacing: 8) {
Button("Open folder") {
NSWorkspace.shared.open(DiagnosticsFileLog.logDirectoryURL())
@@ -762,6 +781,10 @@ struct DebugSettings: View {
CommandResolver.connectionSettings().mode == .remote
}
private var canRestartGateway: Bool {
self.state.connectionMode == .local && !self.attachExistingGatewayOnly
}
private func configURL() -> URL {
FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent(".clawdis")
@@ -902,7 +925,7 @@ private struct PlainSettingsGroupBoxStyle: GroupBoxStyle {
#if DEBUG
struct DebugSettings_Previews: PreviewProvider {
static var previews: some View {
DebugSettings()
DebugSettings(state: .preview)
.frame(width: SettingsTab.windowWidth, height: SettingsTab.windowHeight)
}
}
@@ -910,7 +933,7 @@ struct DebugSettings_Previews: PreviewProvider {
@MainActor
extension DebugSettings {
static func exerciseForTesting() async {
let view = DebugSettings()
let view = DebugSettings(state: .preview)
view.modelsCount = 3
view.modelsLoading = false
view.modelsError = "Failed to load models"
@@ -7,6 +7,8 @@ struct DevicePresentation: Sendable {
enum DeviceModelCatalog {
private static let modelIdentifierToName: [String: String] = loadModelIdentifierToName()
private static let resourceBundle: Bundle? = locateResourceBundle()
private static let resourceSubdirectory = "DeviceModels"
static func presentation(deviceFamily: String?, modelIdentifier: String?) -> DevicePresentation? {
let family = (deviceFamily ?? "").trimmingCharacters(in: .whitespacesAndNewlines)
@@ -104,13 +106,11 @@ enum DeviceModelCatalog {
}
private static func loadMapping(resourceName: String) -> [String: String] {
guard let url = Bundle.module.url(
guard let url = self.resourceBundle?.url(
forResource: resourceName,
withExtension: "json",
subdirectory: "DeviceModels")
else {
return [:]
}
subdirectory: self.resourceSubdirectory)
else { return [:] }
do {
let data = try Data(contentsOf: url)
@@ -121,6 +121,34 @@ enum DeviceModelCatalog {
}
}
private static func locateResourceBundle() -> Bundle? {
// Prefer main bundle (packaged app), then module bundle (SwiftPM/tests).
// Accessing Bundle.module in the packaged app can crash if the bundle isn't where SwiftPM expects it.
if let bundle = self.bundleIfContainsDeviceModels(Bundle.main) {
return bundle
}
if let bundle = self.bundleIfContainsDeviceModels(Bundle.module) {
return bundle
}
return nil
}
private static func bundleIfContainsDeviceModels(_ bundle: Bundle) -> Bundle? {
if bundle.url(
forResource: "ios-device-identifiers",
withExtension: "json",
subdirectory: self.resourceSubdirectory) != nil {
return bundle
}
if bundle.url(
forResource: "mac-device-identifiers",
withExtension: "json",
subdirectory: self.resourceSubdirectory) != nil {
return bundle
}
return nil
}
private enum NameValue: Decodable {
case string(String)
case stringArray([String])
@@ -1,5 +1,4 @@
import AppKit
import OSLog
/// Central manager for Dock icon visibility.
/// Shows the Dock icon while any windows are visible, regardless of user preference.
@@ -66,6 +66,7 @@ actor GatewayChannelActor {
private var connectWaiters: [CheckedContinuation<Void, Error>] = []
private var url: URL
private var token: String?
private var password: String?
private let session: WebSocketSessioning
private var backoffMs: Double = 500
private var shouldReconnect = true
@@ -82,11 +83,13 @@ actor GatewayChannelActor {
init(
url: URL,
token: String?,
password: String? = nil,
session: WebSocketSessionBox? = nil,
pushHandler: (@Sendable (GatewayPush) async -> Void)? = nil)
{
self.url = url
self.token = token
self.password = password
self.session = session?.session ?? URLSession(configuration: .default)
self.pushHandler = pushHandler
Task { [weak self] in
@@ -214,6 +217,8 @@ actor GatewayChannelActor {
]
if let token = self.token {
params["auth"] = ProtoAnyCodable(["token": ProtoAnyCodable(token)])
} else if let password = self.password {
params["auth"] = ProtoAnyCodable(["password": ProtoAnyCodable(password)])
}
let frame = RequestFrame(
@@ -40,7 +40,7 @@ struct GatewayAgentInvocation: Sendable {
actor GatewayConnection {
static let shared = GatewayConnection()
typealias Config = (url: URL, token: String?)
typealias Config = (url: URL, token: String?, password: String?)
enum Method: String, Sendable {
case agent
@@ -51,6 +51,7 @@ actor GatewayConnection {
case providersStatus = "providers.status"
case configGet = "config.get"
case configSet = "config.set"
case talkMode = "talk.mode"
case webLoginStart = "web.login.start"
case webLoginWait = "web.login.wait"
case webLogout = "web.logout"
@@ -82,6 +83,7 @@ actor GatewayConnection {
private var client: GatewayChannelActor?
private var configuredURL: URL?
private var configuredToken: String?
private var configuredPassword: String?
private var subscribers: [UUID: AsyncStream<GatewayPush>.Continuation] = [:]
private var lastSnapshot: HelloOk?
@@ -102,7 +104,7 @@ actor GatewayConnection {
timeoutMs: Double? = nil) async throws -> Data
{
let cfg = try await self.configProvider()
await self.configure(url: cfg.url, token: cfg.token)
await self.configure(url: cfg.url, token: cfg.token, password: cfg.password)
guard let client else {
throw NSError(domain: "Gateway", code: 0, userInfo: [NSLocalizedDescriptionKey: "gateway not configured"])
}
@@ -148,7 +150,7 @@ actor GatewayConnection {
try await Task.sleep(nanoseconds: UInt64(delayMs) * 1_000_000)
do {
let cfg = try await self.configProvider()
await self.configure(url: cfg.url, token: cfg.token)
await self.configure(url: cfg.url, token: cfg.token, password: cfg.password)
guard let client = self.client else {
throw NSError(
domain: "Gateway",
@@ -208,7 +210,7 @@ actor GatewayConnection {
/// Ensure the underlying socket is configured (and replaced if config changed).
func refresh() async throws {
let cfg = try await self.configProvider()
await self.configure(url: cfg.url, token: cfg.token)
await self.configure(url: cfg.url, token: cfg.token, password: cfg.password)
}
func shutdown() async {
@@ -227,6 +229,14 @@ actor GatewayConnection {
return trimmed.isEmpty ? nil : trimmed
}
func snapshotPaths() -> (configPath: String?, stateDir: String?) {
guard let snapshot = self.lastSnapshot else { return (nil, nil) }
let configPath = snapshot.snapshot.configpath?.trimmingCharacters(in: .whitespacesAndNewlines)
let stateDir = snapshot.snapshot.statedir?.trimmingCharacters(in: .whitespacesAndNewlines)
return (configPath?.isEmpty == false ? configPath : nil,
stateDir?.isEmpty == false ? stateDir : nil)
}
func subscribe(bufferingNewest: Int = 100) -> AsyncStream<GatewayPush> {
let id = UUID()
let snapshot = self.lastSnapshot
@@ -255,8 +265,8 @@ actor GatewayConnection {
}
}
private func configure(url: URL, token: String?) async {
if self.client != nil, self.configuredURL == url, self.configuredToken == token {
private func configure(url: URL, token: String?, password: String?) async {
if self.client != nil, self.configuredURL == url, self.configuredToken == token, self.configuredPassword == password {
return
}
if let client {
@@ -266,12 +276,14 @@ actor GatewayConnection {
self.client = GatewayChannelActor(
url: url,
token: token,
password: password,
session: self.sessionBox,
pushHandler: { [weak self] push in
await self?.handle(push: push)
})
self.configuredURL = url
self.configuredToken = token
self.configuredPassword = password
}
private func handle(push: GatewayPush) {
@@ -438,10 +450,18 @@ extension GatewayConnection {
// MARK: - Chat
func chatHistory(sessionKey: String) async throws -> ClawdisChatHistoryPayload {
try await self.requestDecoded(
func chatHistory(
sessionKey: String,
limit: Int? = nil,
timeoutMs: Int? = nil) async throws -> ClawdisChatHistoryPayload
{
var params: [String: AnyCodable] = ["sessionKey": AnyCodable(sessionKey)]
if let limit { params["limit"] = AnyCodable(limit) }
let timeout = timeoutMs.map { Double($0) }
return try await self.requestDecoded(
method: .chatHistory,
params: ["sessionKey": AnyCodable(sessionKey)])
params: params,
timeoutMs: timeout)
}
func chatSend(
@@ -472,7 +492,10 @@ extension GatewayConnection {
params["attachments"] = AnyCodable(encoded)
}
return try await self.requestDecoded(method: .chatSend, params: params)
return try await self.requestDecoded(
method: .chatSend,
params: params,
timeoutMs: Double(timeoutMs))
}
func chatAbort(sessionKey: String, runId: String) async throws -> Bool {
@@ -483,6 +506,12 @@ extension GatewayConnection {
return res.aborted ?? false
}
func talkMode(enabled: Bool, phase: String? = nil) async {
var params: [String: AnyCodable] = ["enabled": AnyCodable(enabled)]
if let phase { params["phase"] = AnyCodable(phase) }
try? await self.requestVoid(method: .talkMode, params: params)
}
// MARK: - VoiceWake
func voiceWakeGetTriggers() async throws -> [String] {
@@ -2,7 +2,7 @@ import Foundation
import OSLog
enum GatewayEndpointState: Sendable, Equatable {
case ready(mode: AppState.ConnectionMode, url: URL, token: String?)
case ready(mode: AppState.ConnectionMode, url: URL, token: String?, password: String?)
case unavailable(mode: AppState.ConnectionMode, reason: String)
}
@@ -17,6 +17,7 @@ actor GatewayEndpointStore {
struct Deps: Sendable {
let mode: @Sendable () async -> AppState.ConnectionMode
let token: @Sendable () -> String?
let password: @Sendable () -> String?
let localPort: @Sendable () -> Int
let remotePortIfRunning: @Sendable () async -> UInt16?
let ensureRemoteTunnel: @Sendable () async throws -> UInt16
@@ -24,11 +25,52 @@ actor GatewayEndpointStore {
static let live = Deps(
mode: { await MainActor.run { AppStateStore.shared.connectionMode } },
token: { ProcessInfo.processInfo.environment["CLAWDIS_GATEWAY_TOKEN"] },
password: {
let root = ClawdisConfigFile.loadDict()
return GatewayEndpointStore.resolveGatewayPassword(
isRemote: CommandResolver.connectionModeIsRemote(),
root: root,
env: ProcessInfo.processInfo.environment)
},
localPort: { GatewayEnvironment.gatewayPort() },
remotePortIfRunning: { await RemoteTunnelManager.shared.controlTunnelPortIfRunning() },
ensureRemoteTunnel: { try await RemoteTunnelManager.shared.ensureControlTunnel() })
}
private static func resolveGatewayPassword(
isRemote: Bool,
root: [String: Any],
env: [String: String]
) -> String? {
let raw = env["CLAWDIS_GATEWAY_PASSWORD"] ?? ""
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty {
return trimmed
}
if isRemote {
if let gateway = root["gateway"] as? [String: Any],
let remote = gateway["remote"] as? [String: Any],
let password = remote["password"] as? String
{
let pw = password.trimmingCharacters(in: .whitespacesAndNewlines)
if !pw.isEmpty {
return pw
}
}
return nil
}
if let gateway = root["gateway"] as? [String: Any],
let auth = gateway["auth"] as? [String: Any],
let password = auth["password"] as? String
{
let pw = password.trimmingCharacters(in: .whitespacesAndNewlines)
if !pw.isEmpty {
return pw
}
}
return nil
}
private let deps: Deps
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "gateway-endpoint")
@@ -47,9 +89,11 @@ actor GatewayEndpointStore {
}
let port = deps.localPort()
let token = deps.token()
let password = deps.password()
switch initialMode {
case .local:
self.state = .ready(mode: .local, url: URL(string: "ws://127.0.0.1:\(port)")!, token: deps.token())
self.state = .ready(mode: .local, url: URL(string: "ws://127.0.0.1:\(port)")!, token: token, password: password)
case .remote:
self.state = .unavailable(mode: .remote, reason: "Remote mode enabled but no active control tunnel")
case .unconfigured:
@@ -77,17 +121,18 @@ actor GatewayEndpointStore {
func setMode(_ mode: AppState.ConnectionMode) async {
let token = self.deps.token()
let password = self.deps.password()
switch mode {
case .local:
let port = self.deps.localPort()
self.setState(.ready(mode: .local, url: URL(string: "ws://127.0.0.1:\(port)")!, token: token))
self.setState(.ready(mode: .local, url: URL(string: "ws://127.0.0.1:\(port)")!, token: token, password: password))
case .remote:
let port = await self.deps.remotePortIfRunning()
guard let port else {
self.setState(.unavailable(mode: .remote, reason: "Remote mode enabled but no active control tunnel"))
return
}
self.setState(.ready(mode: .remote, url: URL(string: "ws://127.0.0.1:\(Int(port))")!, token: token))
self.setState(.ready(mode: .remote, url: URL(string: "ws://127.0.0.1:\(Int(port))")!, token: token, password: password))
case .unconfigured:
self.setState(.unavailable(mode: .unconfigured, reason: "Gateway not configured"))
}
@@ -110,8 +155,8 @@ actor GatewayEndpointStore {
func requireConfig() async throws -> GatewayConnection.Config {
await self.refresh()
switch self.state {
case let .ready(_, url, token):
return (url, token)
case let .ready(_, url, token, password):
return (url, token, password)
case let .unavailable(mode, reason):
guard mode == .remote else {
throw NSError(domain: "GatewayEndpoint", code: 1, userInfo: [NSLocalizedDescriptionKey: reason])
@@ -120,14 +165,18 @@ actor GatewayEndpointStore {
// Auto-recover for remote mode: if the SSH control tunnel died (or hasn't been created yet),
// recreate it on demand so callers can recover without a manual reconnect.
do {
self.logger.info(
"endpoint unavailable; ensuring remote control tunnel reason=\(reason, privacy: .public)")
let forwarded = try await self.deps.ensureRemoteTunnel()
let token = self.deps.token()
let password = self.deps.password()
let url = URL(string: "ws://127.0.0.1:\(Int(forwarded))")!
self.setState(.ready(mode: .remote, url: url, token: token))
return (url, token)
self.setState(.ready(mode: .remote, url: url, token: token, password: password))
return (url, token, password)
} catch {
let msg = "\(reason) (\(error.localizedDescription))"
self.setState(.unavailable(mode: .remote, reason: msg))
self.logger.error("remote control tunnel ensure failed \(msg, privacy: .public)")
throw NSError(domain: "GatewayEndpoint", code: 1, userInfo: [NSLocalizedDescriptionKey: msg])
}
}
@@ -144,7 +193,7 @@ actor GatewayEndpointStore {
continuation.yield(next)
}
switch next {
case let .ready(mode, url, _):
case let .ready(mode, url, _, _):
let modeDesc = String(describing: mode)
let urlDesc = url.absoluteString
self.logger
@@ -158,3 +207,15 @@ actor GatewayEndpointStore {
}
}
}
#if DEBUG
extension GatewayEndpointStore {
static func _testResolveGatewayPassword(
isRemote: Bool,
root: [String: Any],
env: [String: String]
) -> String? {
self.resolveGatewayPassword(isRemote: isRemote, root: root, env: env)
}
}
#endif
@@ -219,6 +219,9 @@ enum GatewayEnvironment {
}
private static func preferredGatewayBind() -> String? {
if CommandResolver.connectionModeIsRemote() {
return nil
}
if let env = ProcessInfo.processInfo.environment["CLAWDIS_GATEWAY_BIND"] {
let trimmed = env.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if self.supportedBindModes.contains(trimmed) {
@@ -257,12 +260,33 @@ enum GatewayEnvironment {
}
statusHandler("Installing clawdis@\(target) via \(label)…")
let response = await ShellExecutor.run(command: cmd, cwd: nil, env: ["PATH": preferred], timeout: 300)
if response.ok {
func summarize(_ text: String) -> String? {
let lines = text
.split(whereSeparator: \.isNewline)
.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) }
.filter { !$0.isEmpty }
guard let last = lines.last else { return nil }
let normalized = last.replacingOccurrences(of: "\\s+", with: " ", options: .regularExpression)
return normalized.count > 200 ? String(normalized.prefix(199)) + "…" : normalized
}
let response = await ShellExecutor.runDetailed(command: cmd, cwd: nil, env: ["PATH": preferred], timeout: 300)
if response.success {
statusHandler("Installed clawdis@\(target)")
} else {
let detail = response.message ?? "install failed"
statusHandler("Install failed: \(detail)")
if response.timedOut {
statusHandler("Install failed: timed out. Check your internet connection and try again.")
return
}
let exit = response.exitCode.map { "exit \($0)" } ?? (response.errorMessage ?? "failed")
let detail = summarize(response.stderr) ?? summarize(response.stdout)
if let detail {
statusHandler("Install failed (\(exit)): \(detail)")
} else {
statusHandler("Install failed (\(exit))")
}
}
}
@@ -1,6 +1,7 @@
import Foundation
enum GatewayLaunchAgentManager {
private static let logger = Logger(subsystem: "com.steipete.clawdis", category: "gateway.launchd")
private static let supportedBindModes: Set<String> = ["loopback", "tailnet", "lan", "auto"]
private static var plistURL: URL {
@@ -16,6 +17,26 @@ enum GatewayLaunchAgentManager {
"\(bundlePath)/Contents/Resources/Relay"
}
private static func gatewayProgramArguments(bundlePath: String, port: Int, bind: String) -> [String] {
#if DEBUG
let projectRoot = CommandResolver.projectRoot()
if let localBin = CommandResolver.projectClawdisExecutable(projectRoot: projectRoot) {
return [localBin, "gateway", "--port", "\(port)", "--bind", bind]
}
if let entry = CommandResolver.gatewayEntrypoint(in: projectRoot),
case let .success(runtime) = CommandResolver.runtimeResolution()
{
return CommandResolver.makeRuntimeCommand(
runtime: runtime,
entrypoint: entry,
subcommand: "gateway",
extraArgs: ["--port", "\(port)", "--bind", bind])
}
#endif
let gatewayBin = self.gatewayExecutablePath(bundlePath: bundlePath)
return [gatewayBin, "gateway-daemon", "--port", "\(port)", "--bind", bind]
}
static func status() async -> Bool {
guard FileManager.default.fileExists(atPath: self.plistURL.path) else { return false }
let result = await self.runLaunchctl(["print", "gui/\(getuid())/\(gatewayLaunchdLabel)"])
@@ -26,12 +47,16 @@ enum GatewayLaunchAgentManager {
if enabled {
let gatewayBin = self.gatewayExecutablePath(bundlePath: bundlePath)
guard FileManager.default.isExecutableFile(atPath: gatewayBin) else {
self.logger.error("launchd enable failed: gateway missing at \(gatewayBin)")
return "Embedded gateway missing in bundle; rebuild via scripts/package-mac-app.sh"
}
self.logger.info("launchd enable requested port=\(port)")
self.writePlist(bundlePath: bundlePath, port: port)
_ = await self.runLaunchctl(["bootout", "gui/\(getuid())/\(gatewayLaunchdLabel)"])
let bootstrap = await self.runLaunchctl(["bootstrap", "gui/\(getuid())", self.plistURL.path])
if bootstrap.status != 0 {
let msg = bootstrap.output.trimmingCharacters(in: .whitespacesAndNewlines)
self.logger.error("launchd bootstrap failed: \(msg)")
return bootstrap.output.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty
? "Failed to bootstrap gateway launchd job"
: bootstrap.output.trimmingCharacters(in: .whitespacesAndNewlines)
@@ -42,6 +67,7 @@ enum GatewayLaunchAgentManager {
return nil
}
self.logger.info("launchd disable requested")
_ = await self.runLaunchctl(["bootout", "gui/\(getuid())/\(gatewayLaunchdLabel)"])
try? FileManager.default.removeItem(at: self.plistURL)
return nil
@@ -52,12 +78,13 @@ enum GatewayLaunchAgentManager {
}
private static func writePlist(bundlePath: String, port: Int) {
let gatewayBin = self.gatewayExecutablePath(bundlePath: bundlePath)
let relayDir = self.relayDir(bundlePath: bundlePath)
let preferredPath = ([relayDir] + CommandResolver.preferredPaths())
.joined(separator: ":")
let bind = self.preferredGatewayBind() ?? "loopback"
let programArguments = self.gatewayProgramArguments(bundlePath: bundlePath, port: port, bind: bind)
let token = self.preferredGatewayToken()
let password = self.preferredGatewayPassword()
var envEntries = """
<key>PATH</key>
<string>\(preferredPath)</string>
@@ -65,11 +92,22 @@ enum GatewayLaunchAgentManager {
<string>sips</string>
"""
if let token {
let escapedToken = self.escapePlistValue(token)
envEntries += """
<key>CLAWDIS_GATEWAY_TOKEN</key>
<string>\(token)</string>
<string>\(escapedToken)</string>
"""
}
if let password {
let escapedPassword = self.escapePlistValue(password)
envEntries += """
<key>CLAWDIS_GATEWAY_PASSWORD</key>
<string>\(escapedPassword)</string>
"""
}
let argsXml = programArguments
.map { "<string>\(self.escapePlistValue($0))</string>" }
.joined(separator: "\n ")
let plist = """
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@@ -79,12 +117,7 @@ enum GatewayLaunchAgentManager {
<string>\(gatewayLaunchdLabel)</string>
<key>ProgramArguments</key>
<array>
<string>\(gatewayBin)</string>
<string>gateway-daemon</string>
<string>--port</string>
<string>\(port)</string>
<string>--bind</string>
<string>\(bind)</string>
\(argsXml)
</array>
<key>WorkingDirectory</key>
<string>\(FileManager.default.homeDirectoryForCurrentUser.path)</string>
@@ -103,10 +136,17 @@ enum GatewayLaunchAgentManager {
</dict>
</plist>
"""
try? plist.write(to: self.plistURL, atomically: true, encoding: .utf8)
do {
try plist.write(to: self.plistURL, atomically: true, encoding: .utf8)
} catch {
self.logger.error("launchd plist write failed: \(error.localizedDescription)")
}
}
private static func preferredGatewayBind() -> String? {
if CommandResolver.connectionModeIsRemote() {
return nil
}
if let env = ProcessInfo.processInfo.environment["CLAWDIS_GATEWAY_BIND"] {
let trimmed = env.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if self.supportedBindModes.contains(trimmed) {
@@ -133,6 +173,33 @@ enum GatewayLaunchAgentManager {
return trimmed.isEmpty ? nil : trimmed
}
private static func preferredGatewayPassword() -> String? {
// First check environment variable
let raw = ProcessInfo.processInfo.environment["CLAWDIS_GATEWAY_PASSWORD"] ?? ""
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
if !trimmed.isEmpty {
return trimmed
}
// Then check config file (gateway.auth.password)
let root = ClawdisConfigFile.loadDict()
if let gateway = root["gateway"] as? [String: Any],
let auth = gateway["auth"] as? [String: Any],
let password = auth["password"] as? String
{
return password.trimmingCharacters(in: .whitespacesAndNewlines)
}
return nil
}
private static func escapePlistValue(_ raw: String) -> String {
raw
.replacingOccurrences(of: "&", with: "&amp;")
.replacingOccurrences(of: "<", with: "&lt;")
.replacingOccurrences(of: ">", with: "&gt;")
.replacingOccurrences(of: "\"", with: "&quot;")
.replacingOccurrences(of: "'", with: "&apos;")
}
private struct LaunchctlResult {
let status: Int32
let output: String
@@ -177,5 +244,9 @@ extension GatewayLaunchAgentManager {
static func _testPreferredGatewayToken() -> String? {
self.preferredGatewayToken()
}
static func _testEscapePlistValue(_ raw: String) -> String {
self.escapePlistValue(raw)
}
}
#endif
@@ -42,6 +42,7 @@ final class GatewayProcessManager {
private var environmentRefreshTask: Task<Void, Never>?
private var lastEnvironmentRefresh: Date?
private var logRefreshTask: Task<Void, Never>?
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "gateway.process")
private let logLimit = 20000 // characters to keep in-memory
private let environmentRefreshMinInterval: TimeInterval = 30
@@ -53,8 +54,10 @@ final class GatewayProcessManager {
self.stop()
self.status = .stopped
self.appendLog("[gateway] remote mode active; skipping local gateway\n")
self.logger.info("gateway process skipped: remote mode active")
return
}
self.logger.debug("gateway active requested active=\(active)")
self.desiredActive = active
self.refreshEnvironmentStatus()
if active {
@@ -86,6 +89,7 @@ final class GatewayProcessManager {
return
}
self.status = .starting
self.logger.debug("gateway start requested")
// First try to latch onto an already-running gateway to avoid spawning a duplicate.
Task { [weak self] in
@@ -98,6 +102,7 @@ final class GatewayProcessManager {
await MainActor.run {
self.status = .failed("Attach-only enabled; no gateway to attach")
self.appendLog("[gateway] attach-only enabled; not spawning local gateway\n")
self.logger.warning("gateway attach-only enabled; not spawning")
}
return
}
@@ -110,6 +115,7 @@ final class GatewayProcessManager {
self.existingGatewayDetails = nil
self.lastFailureReason = nil
self.status = .stopped
self.logger.info("gateway stop requested")
let bundlePath = Bundle.main.bundleURL.path
Task {
_ = await GatewayLaunchAgentManager.set(
@@ -182,6 +188,8 @@ final class GatewayProcessManager {
self.existingGatewayDetails = details
self.status = .attachedExisting(details: details)
self.appendLog("[gateway] using existing instance: \(details)\n")
self.logger.info("gateway using existing instance details=\(details)")
self.refreshControlChannelIfNeeded(reason: "attach existing")
self.refreshLog()
return true
} catch {
@@ -196,6 +204,7 @@ final class GatewayProcessManager {
self.status = .failed(reason)
self.lastFailureReason = reason
self.appendLog("[gateway] existing listener on port \(port) but attach failed: \(reason)\n")
self.logger.warning("gateway attach failed reason=\(reason)")
return true
}
@@ -267,16 +276,19 @@ final class GatewayProcessManager {
await MainActor.run {
self.status = .failed(resolution.status.message)
}
self.logger.error("gateway command resolve failed: \(resolution.status.message)")
return
}
let bundlePath = Bundle.main.bundleURL.path
let port = GatewayEnvironment.gatewayPort()
self.appendLog("[gateway] enabling launchd job (\(gatewayLaunchdLabel)) on port \(port)\n")
self.logger.info("gateway enabling launchd port=\(port)")
let err = await GatewayLaunchAgentManager.set(enabled: true, bundlePath: bundlePath, port: port)
if let err {
self.status = .failed(err)
self.lastFailureReason = err
self.logger.error("gateway launchd enable failed: \(err)")
return
}
@@ -289,6 +301,8 @@ final class GatewayProcessManager {
let instance = await PortGuardian.shared.describe(port: port)
let details = instance.map { "pid \($0.pid)" }
self.status = .running(details: details)
self.logger.info("gateway started details=\(details ?? "ok")")
self.refreshControlChannelIfNeeded(reason: "gateway started")
self.refreshLog()
return
} catch {
@@ -298,6 +312,7 @@ final class GatewayProcessManager {
self.status = .failed("Gateway did not start in time")
self.lastFailureReason = "launchd start timeout"
self.logger.warning("gateway start timed out")
}
private func appendLog(_ chunk: String) {
@@ -307,9 +322,38 @@ final class GatewayProcessManager {
}
}
private func refreshControlChannelIfNeeded(reason: String) {
switch ControlChannel.shared.state {
case .connected, .connecting:
return
case .disconnected, .degraded:
break
}
self.appendLog("[gateway] refreshing control channel (\(reason))\n")
self.logger.debug("gateway control channel refresh reason=\(reason)")
Task { await ControlChannel.shared.configure() }
}
func waitForGatewayReady(timeout: TimeInterval = 6) async -> Bool {
let deadline = Date().addingTimeInterval(timeout)
while Date() < deadline {
if !self.desiredActive { return false }
do {
_ = try await GatewayConnection.shared.requestRaw(method: .health, timeoutMs: 1500)
return true
} catch {
try? await Task.sleep(nanoseconds: 300_000_000)
}
}
self.appendLog("[gateway] readiness wait timed out\n")
self.logger.warning("gateway readiness wait timed out")
return false
}
func clearLog() {
self.log = ""
try? FileManager.default.removeItem(atPath: LogLocator.launchdGatewayLogPath)
self.logger.debug("gateway log cleared")
}
func setProjectRoot(path: String) {
@@ -17,6 +17,7 @@ struct GeneralSettings: View {
@State private var remoteStatus: RemoteStatus = .idle
@State private var showRemoteAdvanced = false
private let isPreview = ProcessInfo.processInfo.isPreview
private var isNixMode: Bool { ProcessInfo.processInfo.isNixMode }
var body: some View {
ScrollView(.vertical) {
@@ -125,7 +126,10 @@ struct GeneralSettings: View {
}
if self.state.connectionMode == .local {
self.gatewayInstallerCard
// In Nix mode, gateway is managed declaratively - no install buttons.
if !self.isNixMode {
self.gatewayInstallerCard
}
TailscaleIntegrationSection(
connectionMode: self.state.connectionMode,
isPaused: self.state.isPaused)
+49 -4
View File
@@ -1,10 +1,27 @@
import Foundation
import Network
import Observation
import OSLog
import SwiftUI
struct HealthSnapshot: Codable, Sendable {
struct Telegram: Codable, Sendable {
struct Probe: Codable, Sendable {
struct Bot: Codable, Sendable {
let id: Int?
let username: String?
}
let ok: Bool
let status: Int?
let error: String?
let elapsedMs: Double?
let bot: Bot?
}
let configured: Bool
let probe: Probe?
}
struct Web: Codable, Sendable {
struct Connect: Codable, Sendable {
let ok: Bool
@@ -30,9 +47,11 @@ struct HealthSnapshot: Codable, Sendable {
let recent: [SessionInfo]
}
let ok: Bool?
let ts: Double
let durationMs: Double
let web: Web
let telegram: Telegram?
let heartbeatSeconds: Int?
let sessions: Sessions
}
@@ -95,6 +114,7 @@ final class HealthStore {
guard !self.isRefreshing else { return }
self.isRefreshing = true
defer { self.isRefreshing = false }
let previousError = self.lastError
do {
let data = try await ControlChannel.shared.health(timeout: 15)
@@ -102,22 +122,41 @@ final class HealthStore {
self.snapshot = decoded
self.lastSuccess = Date()
self.lastError = nil
if previousError != nil {
Self.logger.info("health refresh recovered")
}
} else {
self.lastError = "health output not JSON"
if onDemand { self.snapshot = nil }
if previousError != self.lastError {
Self.logger.warning("health refresh failed: output not JSON")
}
}
} catch {
self.lastError = error.localizedDescription
let desc = error.localizedDescription
self.lastError = desc
if onDemand { self.snapshot = nil }
if previousError != desc {
Self.logger.error("health refresh failed \(desc, privacy: .public)")
}
}
}
private static func isTelegramHealthy(_ snap: HealthSnapshot) -> Bool {
guard let tg = snap.telegram, tg.configured else { return false }
// If probe is missing, treat it as "configured but unknown health" (not a hard fail).
return tg.probe?.ok ?? true
}
var state: HealthState {
if let error = self.lastError, !error.isEmpty {
return .degraded(error)
}
guard let snap = self.snapshot else { return .unknown }
if !snap.web.linked { return .linkingNeeded }
if !snap.web.linked {
// WhatsApp Web linking is optional if Telegram is healthy; don't paint the whole app red.
return Self.isTelegramHealthy(snap) ? .degraded("Not linked") : .linkingNeeded
}
if let connect = snap.web.connect, !connect.ok {
let reason = connect.error ?? "connect failed"
return .degraded(reason)
@@ -129,7 +168,13 @@ final class HealthStore {
if self.isRefreshing { return "Health check running…" }
if let error = self.lastError { return "Health check failed: \(error)" }
guard let snap = self.snapshot else { return "Health check pending" }
if !snap.web.linked { return "Not linked — run clawdis login" }
if !snap.web.linked {
if let tg = snap.telegram, tg.configured {
let tgLabel = (tg.probe?.ok ?? true) ? "Telegram ok" : "Telegram degraded"
return "\(tgLabel) · Not linked — run clawdis login"
}
return "Not linked — run clawdis login"
}
let auth = snap.web.authAgeMs.map { msToAge($0) } ?? "unknown"
if let connect = snap.web.connect, !connect.ok {
let code = connect.status.map(String.init) ?? "?"
@@ -0,0 +1,232 @@
@_exported import Logging
import Foundation
import OSLog
import os
typealias Logger = Logging.Logger
enum AppLogSettings {
static let logLevelKey = appLogLevelKey
static func logLevel() -> Logger.Level {
if let raw = UserDefaults.standard.string(forKey: self.logLevelKey),
let level = Logger.Level(rawValue: raw)
{
return level
}
return .info
}
static func setLogLevel(_ level: Logger.Level) {
UserDefaults.standard.set(level.rawValue, forKey: self.logLevelKey)
}
static func fileLoggingEnabled() -> Bool {
UserDefaults.standard.bool(forKey: debugFileLogEnabledKey)
}
}
enum AppLogLevel: String, CaseIterable, Identifiable {
case trace
case debug
case info
case notice
case warning
case error
case critical
static let `default`: AppLogLevel = .info
var id: String { self.rawValue }
var title: String {
switch self {
case .trace: "Trace"
case .debug: "Debug"
case .info: "Info"
case .notice: "Notice"
case .warning: "Warning"
case .error: "Error"
case .critical: "Critical"
}
}
}
enum ClawdisLogging {
private static let labelSeparator = "::"
private static let didBootstrap: Void = {
LoggingSystem.bootstrap { label in
let (subsystem, category) = Self.parseLabel(label)
let osHandler = ClawdisOSLogHandler(subsystem: subsystem, category: category)
let fileHandler = ClawdisFileLogHandler(label: label)
return MultiplexLogHandler([osHandler, fileHandler])
}
}()
static func bootstrapIfNeeded() {
_ = Self.didBootstrap
}
static func makeLabel(subsystem: String, category: String) -> String {
"\(subsystem)\(Self.labelSeparator)\(category)"
}
static func parseLabel(_ label: String) -> (String, String) {
guard let range = label.range(of: Self.labelSeparator) else {
return ("com.steipete.clawdis", label)
}
let subsystem = String(label[..<range.lowerBound])
let category = String(label[range.upperBound...])
return (subsystem, category)
}
}
extension Logging.Logger {
init(subsystem: String, category: String) {
ClawdisLogging.bootstrapIfNeeded()
let label = ClawdisLogging.makeLabel(subsystem: subsystem, category: category)
self.init(label: label)
}
}
extension Logger.Message.StringInterpolation {
mutating func appendInterpolation<T>(_ value: T, privacy: OSLogPrivacy) {
self.appendInterpolation(String(describing: value))
}
}
struct ClawdisOSLogHandler: LogHandler {
private let osLogger: os.Logger
var metadata: Logger.Metadata = [:]
var logLevel: Logger.Level {
get { AppLogSettings.logLevel() }
set { AppLogSettings.setLogLevel(newValue) }
}
init(subsystem: String, category: String) {
self.osLogger = os.Logger(subsystem: subsystem, category: category)
}
subscript(metadataKey key: String) -> Logger.Metadata.Value? {
get { self.metadata[key] }
set { self.metadata[key] = newValue }
}
// swiftlint:disable:next function_parameter_count
func log(
level: Logger.Level,
message: Logger.Message,
metadata: Logger.Metadata?,
source: String,
file: String,
function: String,
line: UInt)
{
let merged = Self.mergeMetadata(self.metadata, metadata)
let rendered = Self.renderMessage(message, metadata: merged)
self.osLogger.log(level: Self.osLogType(for: level), "\(rendered, privacy: .public)")
}
private static func osLogType(for level: Logger.Level) -> OSLogType {
switch level {
case .trace, .debug:
return .debug
case .info, .notice:
return .info
case .warning:
return .default
case .error:
return .error
case .critical:
return .fault
}
}
private static func mergeMetadata(
_ base: Logger.Metadata,
_ extra: Logger.Metadata?) -> Logger.Metadata
{
guard let extra else { return base }
return base.merging(extra, uniquingKeysWith: { _, new in new })
}
private static func renderMessage(_ message: Logger.Message, metadata: Logger.Metadata) -> String {
guard !metadata.isEmpty else { return message.description }
let meta = metadata
.sorted(by: { $0.key < $1.key })
.map { "\($0.key)=\(stringify($0.value))" }
.joined(separator: " ")
return "\(message.description) [\(meta)]"
}
private static func stringify(_ value: Logger.Metadata.Value) -> String {
switch value {
case let .string(text):
text
case let .stringConvertible(value):
String(describing: value)
case let .array(values):
"[" + values.map { stringify($0) }.joined(separator: ",") + "]"
case let .dictionary(entries):
"{" + entries.map { "\($0.key)=\(stringify($0.value))" }.joined(separator: ",") + "}"
}
}
}
struct ClawdisFileLogHandler: LogHandler {
let label: String
var metadata: Logger.Metadata = [:]
var logLevel: Logger.Level {
get { AppLogSettings.logLevel() }
set { AppLogSettings.setLogLevel(newValue) }
}
subscript(metadataKey key: String) -> Logger.Metadata.Value? {
get { self.metadata[key] }
set { self.metadata[key] = newValue }
}
// swiftlint:disable:next function_parameter_count
func log(
level: Logger.Level,
message: Logger.Message,
metadata: Logger.Metadata?,
source: String,
file: String,
function: String,
line: UInt)
{
guard AppLogSettings.fileLoggingEnabled() else { return }
let (subsystem, category) = ClawdisLogging.parseLabel(self.label)
var fields: [String: String] = [
"subsystem": subsystem,
"category": category,
"level": level.rawValue,
"source": source,
"file": file,
"function": function,
"line": "\(line)",
]
let merged = self.metadata.merging(metadata ?? [:], uniquingKeysWith: { _, new in new })
for (key, value) in merged {
fields["meta.\(key)"] = Self.stringify(value)
}
DiagnosticsFileLog.shared.log(category: category, event: message.description, fields: fields)
}
private static func stringify(_ value: Logger.Metadata.Value) -> String {
switch value {
case let .string(text):
text
case let .stringConvertible(value):
String(describing: value)
case let .array(values):
"[" + values.map { stringify($0) }.joined(separator: ",") + "]"
case let .dictionary(entries):
"{" + entries.map { "\($0.key)=\(stringify($0.value))" }.joined(separator: ",") + "}"
}
}
}
+1 -1
View File
@@ -3,7 +3,6 @@ import Darwin
import Foundation
import MenuBarExtraAccess
import Observation
import OSLog
import Security
import SwiftUI
@@ -30,6 +29,7 @@ struct ClawdisApp: App {
}
init() {
ClawdisLogging.bootstrapIfNeeded()
_state = State(initialValue: AppStateStore.shared)
}
@@ -14,11 +14,14 @@ struct MenuContent: View {
private let heartbeatStore = HeartbeatStore.shared
private let controlChannel = ControlChannel.shared
private let activityStore = WorkActivityStore.shared
@Bindable private var pairingPrompter = NodePairingApprovalPrompter.shared
@Environment(\.openSettings) private var openSettings
@State private var availableMics: [AudioInputDevice] = []
@State private var loadingMics = false
@State private var browserControlEnabled = true
@AppStorage(cameraEnabledKey) private var cameraEnabled: Bool = false
@AppStorage(appLogLevelKey) private var appLogLevelRaw: String = AppLogLevel.default.rawValue
@AppStorage(debugFileLogEnabledKey) private var appFileLoggingEnabled: Bool = false
init(state: AppState, updater: UpdaterProviding?) {
self._state = Bindable(wrappedValue: state)
@@ -32,6 +35,13 @@ struct MenuContent: View {
VStack(alignment: .leading, spacing: 2) {
Text(self.connectionLabel)
self.statusLine(label: self.healthStatus.label, color: self.healthStatus.color)
if self.pairingPrompter.pendingCount > 0 {
let repairCount = self.pairingPrompter.pendingRepairCount
let repairSuffix = repairCount > 0 ? " · \(repairCount) repair" : ""
self.statusLine(
label: "Pairing approval pending (\(self.pairingPrompter.pendingCount))\(repairSuffix)",
color: .orange)
}
}
}
.disabled(self.state.connectionMode == .unconfigured)
@@ -49,7 +59,7 @@ struct MenuContent: View {
get: { self.browserControlEnabled },
set: { enabled in
self.browserControlEnabled = enabled
ClawdisConfigFile.setBrowserControlEnabled(enabled)
Task { await self.saveBrowserControlEnabled(enabled) }
})) {
Label("Browser Control", systemImage: "globe")
}
@@ -67,8 +77,8 @@ struct MenuContent: View {
Toggle(isOn: self.voiceWakeBinding) {
Label("Voice Wake", systemImage: "mic.fill")
}
.disabled(!voiceWakeSupported)
.opacity(voiceWakeSupported ? 1 : 0.5)
.disabled(!voiceWakeSupported)
.opacity(voiceWakeSupported ? 1 : 0.5)
if self.showVoiceWakeMicPicker {
self.voiceWakeMicMenu
}
@@ -102,6 +112,13 @@ struct MenuContent: View {
systemImage: "rectangle.inset.filled.on.rectangle")
}
}
Button {
Task { await self.state.setTalkEnabled(!self.state.talkEnabled) }
} label: {
Label(self.state.talkEnabled ? "Stop Talk Mode" : "Talk Mode", systemImage: "waveform.circle.fill")
}
.disabled(!voiceWakeSupported)
.opacity(voiceWakeSupported ? 1 : 0.5)
Divider()
Button("Settings…") { self.open(tab: .general) }
.keyboardShortcut(",", modifiers: [.command])
@@ -123,8 +140,8 @@ struct MenuContent: View {
.onChange(of: self.state.voicePushToTalkEnabled) { _, enabled in
VoicePushToTalkHotkey.shared.setEnabled(voiceWakeSupported && enabled)
}
.onAppear {
self.browserControlEnabled = ClawdisConfigFile.browserControlEnabled()
.task(id: self.state.connectionMode) {
await self.loadBrowserControlEnabled()
}
}
@@ -139,6 +156,35 @@ struct MenuContent: View {
}
}
private func loadBrowserControlEnabled() async {
let root = await ConfigStore.load()
let browser = root["browser"] as? [String: Any]
let enabled = browser?["enabled"] as? Bool ?? true
await MainActor.run { self.browserControlEnabled = enabled }
}
private func saveBrowserControlEnabled(_ enabled: Bool) async {
let (success, _) = await MenuContent.buildAndSaveBrowserEnabled(enabled)
if !success {
await self.loadBrowserControlEnabled()
}
}
@MainActor
private static func buildAndSaveBrowserEnabled(_ enabled: Bool) async -> (Bool,()) {
var root = await ConfigStore.load()
var browser = root["browser"] as? [String: Any] ?? [:]
browser["enabled"] = enabled
root["browser"] = browser
do {
try await ConfigStore.save(root)
return (true, ())
} catch {
return (false, ())
}
}
@ViewBuilder
private var debugMenu: some View {
if self.state.debugPaneEnabled {
@@ -158,6 +204,16 @@ struct MenuContent: View {
} label: {
Label("Send Test Heartbeat", systemImage: "waveform.path.ecg")
}
if self.state.connectionMode == .remote {
Button {
Task { @MainActor in
let result = await DebugActions.resetGatewayTunnel()
self.presentDebugResult(result, title: "Remote Tunnel")
}
} label: {
Label("Reset Remote Tunnel", systemImage: "arrow.triangle.2.circlepath")
}
}
Button {
Task { _ = await DebugActions.toggleVerboseLoggingMain() }
} label: {
@@ -167,6 +223,22 @@ struct MenuContent: View {
: "Verbose Logging (Main): Off",
systemImage: "text.alignleft")
}
Menu {
Picker("Verbosity", selection: self.$appLogLevelRaw) {
ForEach(AppLogLevel.allCases) { level in
Text(level.title).tag(level.rawValue)
}
}
Toggle(isOn: self.$appFileLoggingEnabled) {
Label(
self.appFileLoggingEnabled
? "File Logging: On"
: "File Logging: Off",
systemImage: "doc.text.magnifyingglass")
}
} label: {
Label("App Logging", systemImage: "doc.text")
}
Button {
DebugActions.openSessionStore()
} label: {
@@ -194,10 +266,12 @@ struct MenuContent: View {
Label("Send Test Notification", systemImage: "bell")
}
Divider()
Button {
DebugActions.restartGateway()
} label: {
Label("Restart Gateway", systemImage: "arrow.clockwise")
if self.state.connectionMode == .local, !AppStateStore.attachExistingGatewayOnly {
Button {
DebugActions.restartGateway()
} label: {
Label("Restart Gateway", systemImage: "arrow.clockwise")
}
}
Button {
DebugActions.restartApp()
@@ -311,6 +385,10 @@ struct MenuContent: View {
Text(label)
.font(.caption)
.foregroundStyle(.secondary)
.multilineTextAlignment(.leading)
.lineLimit(nil)
.fixedSize(horizontal: false, vertical: true)
.layoutPriority(1)
}
.padding(.top, 2)
}
@@ -394,6 +472,21 @@ struct MenuContent: View {
return "System default"
}
@MainActor
private func presentDebugResult(_ result: Result<String, DebugActionError>, title: String) {
let alert = NSAlert()
alert.messageText = title
switch result {
case let .success(message):
alert.informativeText = message
alert.alertStyle = .informational
case let .failure(error):
alert.informativeText = error.localizedDescription
alert.alertStyle = .warning
}
alert.runModal()
}
@MainActor
private func loadMicrophones(force: Bool = false) async {
guard self.showVoiceWakeMicPicker else {
@@ -22,8 +22,7 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
private var cachedErrorText: String?
private var cacheUpdatedAt: Date?
private let refreshIntervalSeconds: TimeInterval = 12
private let nodesStore = InstancesStore.shared
private let gatewayDiscovery = GatewayDiscoveryModel()
private let nodesStore = NodesStore.shared
#if DEBUG
private var testControlChannelConnected: Bool?
#endif
@@ -43,7 +42,6 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
}
self.nodesStore.start()
self.gatewayDiscovery.start()
}
func menuWillOpen(_ menu: NSMenu) {
@@ -108,13 +106,7 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
guard let insertIndex = self.findInsertIndex(in: menu) else { return }
let width = self.initialWidth(for: menu)
guard self.isControlChannelConnected else {
menu.insertItem(self.makeMessageItem(
text: "No connection to gateway",
symbolName: "wifi.slash",
width: width), at: insertIndex)
return
}
guard self.isControlChannelConnected else { return }
guard let snapshot = self.cachedSnapshot else {
let headerItem = NSMenuItem()
@@ -197,41 +189,38 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
menu.insertItem(topSeparator, at: cursor)
cursor += 1
guard self.isControlChannelConnected else {
menu.insertItem(
self.makeMessageItem(text: "No connection to gateway", symbolName: "wifi.slash", width: width),
at: cursor)
if let gatewayEntry = self.gatewayEntry() {
let gatewayItem = self.makeNodeItem(entry: gatewayEntry, width: width)
menu.insertItem(gatewayItem, at: cursor)
cursor += 1
let separator = NSMenuItem.separator()
separator.tag = self.nodesTag
menu.insertItem(separator, at: cursor)
return
}
guard self.isControlChannelConnected else { return }
if let error = self.nodesStore.lastError?.nonEmpty {
menu.insertItem(self.makeMessageItem(text: "Error: \(error)", symbolName: "exclamationmark.triangle",
width: width), at: cursor)
menu.insertItem(
self.makeMessageItem(
text: "Error: \(error)",
symbolName: "exclamationmark.triangle",
width: width),
at: cursor)
cursor += 1
} else if let status = self.nodesStore.statusMessage?.nonEmpty {
menu.insertItem(self.makeMessageItem(text: status, symbolName: "info.circle", width: width), at: cursor)
menu.insertItem(
self.makeMessageItem(text: status, symbolName: "info.circle", width: width),
at: cursor)
cursor += 1
}
if entries.isEmpty {
let title = self.nodesStore.isLoading ? "Loading nodes..." : "No nodes yet"
menu.insertItem(self.makeMessageItem(text: title, symbolName: "circle.dashed", width: width), at: cursor)
let title = self.nodesStore.isLoading ? "Loading devices..." : "No devices yet"
menu.insertItem(
self.makeMessageItem(text: title, symbolName: "circle.dashed", width: width),
at: cursor)
cursor += 1
} else {
for entry in entries.prefix(8) {
let item = NSMenuItem()
item.tag = self.nodesTag
item.target = self
item.action = #selector(self.copyNodeSummary(_:))
item.representedObject = NodeMenuEntryFormatter.summaryText(entry)
item.view = HighlightedMenuItemHostView(
rootView: AnyView(NodeMenuRowView(entry: entry, width: width)),
width: width)
item.submenu = self.buildNodeSubmenu(entry: entry)
let item = self.makeNodeItem(entry: entry, width: width)
menu.insertItem(item, at: cursor)
cursor += 1
}
@@ -239,7 +228,7 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
if entries.count > 8 {
let moreItem = NSMenuItem()
moreItem.tag = self.nodesTag
moreItem.title = "More Nodes..."
moreItem.title = "More Devices..."
moreItem.image = NSImage(systemSymbolName: "ellipsis.circle", accessibilityDescription: nil)
let overflow = Array(entries.dropFirst(8))
moreItem.submenu = self.buildNodesOverflowMenu(entries: overflow, width: width)
@@ -259,13 +248,85 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
return false
}
private func gatewayEntry() -> NodeInfo? {
let mode = AppStateStore.shared.connectionMode
let isConnected = self.isControlChannelConnected
let port = GatewayEnvironment.gatewayPort()
var host: String?
var platform: String?
switch mode {
case .remote:
platform = "remote"
let target = AppStateStore.shared.remoteTarget
if let parsed = CommandResolver.parseSSHTarget(target) {
host = parsed.port == 22 ? parsed.host : "\(parsed.host):\(parsed.port)"
} else {
host = target.nonEmpty
}
case .local:
platform = "local"
host = "127.0.0.1:\(port)"
case .unconfigured:
platform = nil
host = nil
}
return NodeInfo(
nodeId: "gateway",
displayName: "Gateway",
platform: platform,
version: nil,
deviceFamily: nil,
modelIdentifier: nil,
remoteIp: host,
caps: nil,
commands: nil,
permissions: nil,
paired: nil,
connected: isConnected)
}
private func makeNodeItem(entry: NodeInfo, width: CGFloat) -> NSMenuItem {
let item = NSMenuItem()
item.tag = self.nodesTag
item.target = self
item.action = #selector(self.copyNodeSummary(_:))
item.representedObject = NodeMenuEntryFormatter.summaryText(entry)
item.view = HighlightedMenuItemHostView(
rootView: AnyView(NodeMenuRowView(entry: entry, width: width)),
width: width)
item.submenu = self.buildNodeSubmenu(entry: entry, width: width)
return item
}
private func makeSessionPreviewItem(
sessionKey: String,
title: String,
width: CGFloat,
maxLines: Int) -> NSMenuItem
{
let item = NSMenuItem()
item.tag = self.tag
item.isEnabled = false
let view = AnyView(SessionMenuPreviewView(
sessionKey: sessionKey,
width: width,
maxItems: 10,
maxLines: maxLines,
title: title))
item.view = self.makeHostedView(rootView: view, width: width, highlighted: false)
return item
}
private func makeMessageItem(text: String, symbolName: String, width: CGFloat) -> NSMenuItem {
let view = AnyView(
Label(text, systemImage: symbolName)
.font(.caption)
.foregroundStyle(.secondary)
.lineLimit(1)
.truncationMode(.tail)
.multilineTextAlignment(.leading)
.lineLimit(nil)
.fixedSize(horizontal: false, vertical: true)
.padding(.leading, 18)
.padding(.trailing, 12)
.padding(.vertical, 6)
@@ -319,6 +380,19 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
private func buildSubmenu(for row: SessionRow, storePath: String) -> NSMenu {
let menu = NSMenu()
let width = self.submenuWidth()
menu.addItem(self.makeSessionPreviewItem(
sessionKey: row.key,
title: "Recent messages (last 10)",
width: width,
maxLines: 3))
let morePreview = NSMenuItem(title: "More preview…", action: nil, keyEquivalent: "")
morePreview.submenu = self.buildPreviewSubmenu(sessionKey: row.key, width: width)
menu.addItem(morePreview)
menu.addItem(NSMenuItem.separator())
let thinking = NSMenuItem(title: "Thinking", action: nil, keyEquivalent: "")
thinking.submenu = self.buildThinkingMenu(for: row)
@@ -413,7 +487,17 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
return menu
}
private func buildNodesOverflowMenu(entries: [InstanceInfo], width: CGFloat) -> NSMenu {
private func buildPreviewSubmenu(sessionKey: String, width: CGFloat) -> NSMenu {
let menu = NSMenu()
menu.addItem(self.makeSessionPreviewItem(
sessionKey: sessionKey,
title: "Recent messages (expanded)",
width: width,
maxLines: 8))
return menu
}
private func buildNodesOverflowMenu(entries: [NodeInfo], width: CGFloat) -> NSMenu {
let menu = NSMenu()
for entry in entries {
let item = NSMenuItem()
@@ -423,27 +507,27 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
item.view = HighlightedMenuItemHostView(
rootView: AnyView(NodeMenuRowView(entry: entry, width: width)),
width: width)
item.submenu = self.buildNodeSubmenu(entry: entry)
item.submenu = self.buildNodeSubmenu(entry: entry, width: width)
menu.addItem(item)
}
return menu
}
private func buildNodeSubmenu(entry: InstanceInfo) -> NSMenu {
private func buildNodeSubmenu(entry: NodeInfo, width: CGFloat) -> NSMenu {
let menu = NSMenu()
menu.autoenablesItems = false
menu.addItem(self.makeNodeCopyItem(label: "ID", value: entry.id))
menu.addItem(self.makeNodeCopyItem(label: "Node ID", value: entry.nodeId))
if let host = entry.host?.nonEmpty {
menu.addItem(self.makeNodeCopyItem(label: "Host", value: host))
if let name = entry.displayName?.nonEmpty {
menu.addItem(self.makeNodeCopyItem(label: "Name", value: name))
}
if let ip = entry.ip?.nonEmpty {
if let ip = entry.remoteIp?.nonEmpty {
menu.addItem(self.makeNodeCopyItem(label: "IP", value: ip))
}
menu.addItem(self.makeNodeCopyItem(label: "Role", value: NodeMenuEntryFormatter.roleText(entry)))
menu.addItem(self.makeNodeCopyItem(label: "Status", value: NodeMenuEntryFormatter.roleText(entry)))
if let platform = NodeMenuEntryFormatter.platformText(entry) {
menu.addItem(self.makeNodeCopyItem(label: "Platform", value: platform))
@@ -453,19 +537,20 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
menu.addItem(self.makeNodeCopyItem(label: "Version", value: self.formatVersionLabel(version)))
}
menu.addItem(self.makeNodeDetailItem(label: "Last seen", value: entry.ageDescription))
menu.addItem(self.makeNodeDetailItem(label: "Connected", value: entry.isConnected ? "Yes" : "No"))
menu.addItem(self.makeNodeDetailItem(label: "Paired", value: entry.isPaired ? "Yes" : "No"))
if entry.lastInputSeconds != nil {
menu.addItem(self.makeNodeDetailItem(label: "Last input", value: entry.lastInputDescription))
if let caps = entry.caps?.filter({ !$0.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty }),
!caps.isEmpty {
menu.addItem(self.makeNodeCopyItem(label: "Caps", value: caps.joined(separator: ", ")))
}
if let reason = entry.reason?.nonEmpty {
menu.addItem(self.makeNodeDetailItem(label: "Reason", value: reason))
}
if let sshURL = self.sshURL(for: entry) {
menu.addItem(.separator())
menu.addItem(self.makeNodeActionItem(title: "Open SSH", url: sshURL))
if let commands = entry.commands?.filter({ !$0.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty }),
!commands.isEmpty {
menu.addItem(self.makeNodeMultilineItem(
label: "Commands",
value: commands.joined(separator: ", "),
width: width))
}
return menu
@@ -484,12 +569,17 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
return item
}
private func makeNodeActionItem(title: String, url: URL) -> NSMenuItem {
let item = NSMenuItem(title: title, action: #selector(self.openNodeSSH(_:)), keyEquivalent: "")
private func makeNodeMultilineItem(label: String, value: String, width: CGFloat) -> NSMenuItem {
let item = NSMenuItem()
item.target = self
item.representedObject = url
item.action = #selector(self.copyNodeValue(_:))
item.representedObject = value
item.view = HighlightedMenuItemHostView(
rootView: AnyView(NodeMenuMultilineView(label: label, value: value, width: width)),
width: width)
return item
}
private func formatVersionLabel(_ version: String) -> String {
let trimmed = version.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return version }
@@ -615,104 +705,6 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
NSPasteboard.general.setString(value, forType: .string)
}
@objc
private func openNodeSSH(_ sender: NSMenuItem) {
guard let url = sender.representedObject as? URL else { return }
if let appURL = self.preferredTerminalAppURL() {
NSWorkspace.shared.open(
[url],
withApplicationAt: appURL,
configuration: NSWorkspace.OpenConfiguration(),
completionHandler: nil)
} else {
NSWorkspace.shared.open(url)
}
}
private func preferredTerminalAppURL() -> URL? {
if let ghosty = self.ghostyAppURL() { return ghosty }
return NSWorkspace.shared.urlForApplication(withBundleIdentifier: "com.apple.Terminal")
}
private func ghostyAppURL() -> URL? {
let candidates = [
"/Applications/Ghosty.app",
("~/Applications/Ghosty.app" as NSString).expandingTildeInPath,
]
for path in candidates where FileManager.default.fileExists(atPath: path) {
return URL(fileURLWithPath: path)
}
return nil
}
private func sshURL(for entry: InstanceInfo) -> URL? {
guard NodeMenuEntryFormatter.isGateway(entry) else { return nil }
guard let gateway = self.matchingGateway(for: entry) else { return nil }
guard let host = self.sanitizedTailnetHost(gateway.tailnetDns) ?? gateway.lanHost else { return nil }
let user = NSUserName()
return self.buildSSHURL(user: user, host: host, port: gateway.sshPort)
}
private func matchingGateway(for entry: InstanceInfo) -> GatewayDiscoveryModel.DiscoveredGateway? {
let candidates = self.entryHostCandidates(entry)
guard !candidates.isEmpty else { return nil }
return self.gatewayDiscovery.gateways.first { gateway in
let gatewayTokens = self.gatewayHostTokens(gateway)
return candidates.contains { gatewayTokens.contains($0) }
}
}
private func entryHostCandidates(_ entry: InstanceInfo) -> [String] {
let raw: [String?] = [
entry.host,
entry.ip,
NodeMenuEntryFormatter.primaryName(entry),
]
return raw.compactMap(self.normalizedHostToken(_:))
}
private func gatewayHostTokens(_ gateway: GatewayDiscoveryModel.DiscoveredGateway) -> [String] {
let raw: [String?] = [
gateway.displayName,
gateway.lanHost,
gateway.tailnetDns,
]
return raw.compactMap(self.normalizedHostToken(_:))
}
private func normalizedHostToken(_ value: String?) -> String? {
guard let value else { return nil }
let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmed.isEmpty { return nil }
let lower = trimmed.lowercased().trimmingCharacters(in: CharacterSet(charactersIn: "."))
if lower.hasSuffix(".localdomain") {
return lower.replacingOccurrences(of: ".localdomain", with: ".local")
}
return lower
}
private func sanitizedTailnetHost(_ host: String?) -> String? {
guard let host else { return nil }
let trimmed = host.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmed.isEmpty { return nil }
if trimmed.hasSuffix(".internal.") || trimmed.hasSuffix(".internal") {
return nil
}
return trimmed
}
private func buildSSHURL(user: String, host: String, port: Int) -> URL? {
var components = URLComponents()
components.scheme = "ssh"
components.user = user
components.host = host
if port != 22 {
components.port = port
}
return components.url
}
// MARK: - Width + placement
private func findInsertIndex(in menu: NSMenu) -> Int? {
@@ -755,6 +747,16 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
return self.currentMenuWidth(for: menu)
}
private func submenuWidth() -> CGFloat {
if let openWidth = self.menuOpenWidth {
return max(300, openWidth)
}
if let cached = self.lastKnownMenuWidth {
return max(300, cached)
}
return self.fallbackWidth
}
private func menuWindowWidth(for menu: NSMenu) -> CGFloat? {
var menuWindow: NSWindow?
for item in menu.items {
@@ -767,23 +769,14 @@ final class MenuSessionsInjector: NSObject, NSMenuDelegate {
return width
}
private func sortedNodeEntries() -> [InstanceInfo] {
let entries = self.nodesStore.instances.filter { entry in
let mode = entry.mode?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
return mode != "health"
}
private func sortedNodeEntries() -> [NodeInfo] {
let entries = self.nodesStore.nodes.filter { $0.isConnected }
return entries.sorted { lhs, rhs in
let lhsGateway = NodeMenuEntryFormatter.isGateway(lhs)
let rhsGateway = NodeMenuEntryFormatter.isGateway(rhs)
if lhsGateway != rhsGateway { return lhsGateway }
let lhsLocal = NodeMenuEntryFormatter.isLocal(lhs)
let rhsLocal = NodeMenuEntryFormatter.isLocal(rhs)
if lhsLocal != rhsLocal { return lhsLocal }
if lhs.isConnected != rhs.isConnected { return lhs.isConnected }
if lhs.isPaired != rhs.isPaired { return lhs.isPaired }
let lhsName = NodeMenuEntryFormatter.primaryName(lhs).lowercased()
let rhsName = NodeMenuEntryFormatter.primaryName(rhs).lowercased()
if lhsName == rhsName { return lhs.ts > rhs.ts }
if lhsName == rhsName { return lhs.nodeId < rhs.nodeId }
return lhsName < rhsName
}
}
@@ -4,18 +4,23 @@ import JavaScriptCore
enum ModelCatalogLoader {
static let defaultPath: String = FileManager.default.homeDirectoryForCurrentUser
.appendingPathComponent("Projects/pi-mono/packages/ai/src/models.generated.ts").path
private static let logger = Logger(subsystem: "com.steipete.clawdis", category: "models")
static func load(from path: String) async throws -> [ModelChoice] {
let expanded = (path as NSString).expandingTildeInPath
self.logger.debug("model catalog load start file=\(URL(fileURLWithPath: expanded).lastPathComponent)")
let source = try String(contentsOfFile: expanded, encoding: .utf8)
let sanitized = self.sanitize(source: source)
let ctx = JSContext()
ctx?.exceptionHandler = { _, exception in
if let exception { print("JS exception: \(exception)") }
if let exception {
self.logger.warning("model catalog JS exception: \(exception)")
}
}
ctx?.evaluateScript(sanitized)
guard let rawModels = ctx?.objectForKeyedSubscript("MODELS")?.toDictionary() as? [String: Any] else {
self.logger.error("model catalog parse failed: MODELS missing")
throw NSError(
domain: "ModelCatalogLoader",
code: 1,
@@ -33,12 +38,14 @@ enum ModelCatalogLoader {
}
}
return choices.sorted { lhs, rhs in
let sorted = choices.sorted { lhs, rhs in
if lhs.provider == rhs.provider {
return lhs.name.localizedCaseInsensitiveCompare(rhs.name) == .orderedAscending
}
return lhs.provider.localizedCaseInsensitiveCompare(rhs.provider) == .orderedAscending
}
self.logger.debug("model catalog loaded providers=\(rawModels.count) models=\(sorted.count)")
return sorted
}
private static func sanitize(source: String) -> String {
@@ -17,29 +17,35 @@ actor MacNodeBridgePairingClient {
let connection = NWConnection(to: endpoint, using: .tcp)
let queue = DispatchQueue(label: "com.steipete.clawdis.macos.bridge-client")
defer { connection.cancel() }
try await AsyncTimeout.withTimeout(seconds: 8, onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "connect timed out",
])
}) {
try await self.startAndWaitForReady(connection, queue: queue)
}
try await AsyncTimeout.withTimeout(
seconds: 8,
onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "connect timed out",
])
},
operation: {
try await self.startAndWaitForReady(connection, queue: queue)
})
onStatus?("Authenticating…")
try await self.send(hello, over: connection)
let first = try await AsyncTimeout.withTimeout(seconds: 10, onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "hello timed out",
])
}) { () -> ReceivedFrame in
guard let frame = try await self.receiveFrame(over: connection) else {
throw NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "Bridge closed connection during hello",
let first = try await AsyncTimeout.withTimeout(
seconds: 10,
onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "hello timed out",
])
}
return frame
}
},
operation: { () -> ReceivedFrame in
guard let frame = try await self.receiveFrame(over: connection) else {
throw NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "Bridge closed connection during hello",
])
}
return frame
})
switch first.base.type {
case "hello-ok":
@@ -68,28 +74,31 @@ actor MacNodeBridgePairingClient {
over: connection)
onStatus?("Waiting for approval…")
let ok = try await AsyncTimeout.withTimeout(seconds: 60, onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "pairing approval timed out",
])
}) {
while let next = try await self.receiveFrame(over: connection) {
switch next.base.type {
case "pair-ok":
return try self.decoder.decode(BridgePairOk.self, from: next.data)
case "error":
let e = try self.decoder.decode(BridgeErrorFrame.self, from: next.data)
throw NSError(domain: "Bridge", code: 2, userInfo: [
NSLocalizedDescriptionKey: "\(e.code): \(e.message)",
])
default:
continue
let ok = try await AsyncTimeout.withTimeout(
seconds: 60,
onTimeout: {
NSError(domain: "Bridge", code: 0, userInfo: [
NSLocalizedDescriptionKey: "pairing approval timed out",
])
},
operation: {
while let next = try await self.receiveFrame(over: connection) {
switch next.base.type {
case "pair-ok":
return try self.decoder.decode(BridgePairOk.self, from: next.data)
case "error":
let e = try self.decoder.decode(BridgeErrorFrame.self, from: next.data)
throw NSError(domain: "Bridge", code: 2, userInfo: [
NSLocalizedDescriptionKey: "\(e.code): \(e.message)",
])
default:
continue
}
}
}
throw NSError(domain: "Bridge", code: 3, userInfo: [
NSLocalizedDescriptionKey: "Pairing failed: bridge closed connection",
])
}
throw NSError(domain: "Bridge", code: 3, userInfo: [
NSLocalizedDescriptionKey: "Pairing failed: bridge closed connection",
])
})
return ok.token
@@ -48,17 +48,23 @@ actor MacNodeBridgeSession {
try await Self.waitForReady(stateStream, timeoutSeconds: 6)
try await AsyncTimeout.withTimeout(seconds: 6, onTimeout: {
TimeoutError(message: "operation timed out")
}) {
try await self.send(hello)
}
try await AsyncTimeout.withTimeout(
seconds: 6,
onTimeout: {
TimeoutError(message: "operation timed out")
},
operation: {
try await self.send(hello)
})
guard let line = try await AsyncTimeout.withTimeout(seconds: 6, onTimeout: {
TimeoutError(message: "operation timed out")
}, operation: {
try await self.receiveLine()
}),
guard let line = try await AsyncTimeout.withTimeout(
seconds: 6,
onTimeout: {
TimeoutError(message: "operation timed out")
},
operation: {
try await self.receiveLine()
}),
let data = line.data(using: .utf8),
let base = try? self.decoder.decode(BridgeBaseFrame.self, from: data)
else {
@@ -294,28 +300,31 @@ actor MacNodeBridgeSession {
_ stream: AsyncStream<NWConnection.State>,
timeoutSeconds: Double) async throws
{
try await AsyncTimeout.withTimeout(seconds: timeoutSeconds, onTimeout: {
TimeoutError(message: "operation timed out")
}) {
for await state in stream {
switch state {
case .ready:
return
case let .failed(err):
throw err
case .cancelled:
throw NSError(domain: "Bridge", code: 20, userInfo: [
NSLocalizedDescriptionKey: "Connection cancelled",
])
default:
continue
try await AsyncTimeout.withTimeout(
seconds: timeoutSeconds,
onTimeout: {
TimeoutError(message: "operation timed out")
},
operation: {
for await state in stream {
switch state {
case .ready:
return
case let .failed(err):
throw err
case .cancelled:
throw NSError(domain: "Bridge", code: 20, userInfo: [
NSLocalizedDescriptionKey: "Connection cancelled",
])
default:
continue
}
}
}
throw NSError(domain: "Bridge", code: 21, userInfo: [
NSLocalizedDescriptionKey: "Connection closed",
])
}
throw NSError(domain: "Bridge", code: 21, userInfo: [
NSLocalizedDescriptionKey: "Connection closed",
])
})
}
}
@@ -135,6 +135,7 @@ final class MacNodeModeCoordinator {
let capsSet = Set(caps)
if capsSet.contains(ClawdisCapability.camera.rawValue) {
commands.append(ClawdisCameraCommand.list.rawValue)
commands.append(ClawdisCameraCommand.snap.rawValue)
commands.append(ClawdisCameraCommand.clip.rawValue)
}
@@ -228,29 +229,32 @@ final class MacNodeModeCoordinator {
_ stream: AsyncStream<NWConnection.State>,
timeoutSeconds: Double) async throws
{
try await AsyncTimeout.withTimeout(seconds: timeoutSeconds, onTimeout: {
NSError(domain: "Bridge", code: 22, userInfo: [
NSLocalizedDescriptionKey: "operation timed out",
])
}) {
for await state in stream {
switch state {
case .ready:
return
case let .failed(err):
throw err
case .cancelled:
throw NSError(domain: "Bridge", code: 20, userInfo: [
NSLocalizedDescriptionKey: "Connection cancelled",
])
default:
continue
try await AsyncTimeout.withTimeout(
seconds: timeoutSeconds,
onTimeout: {
NSError(domain: "Bridge", code: 22, userInfo: [
NSLocalizedDescriptionKey: "operation timed out",
])
},
operation: {
for await state in stream {
switch state {
case .ready:
return
case let .failed(err):
throw err
case .cancelled:
throw NSError(domain: "Bridge", code: 20, userInfo: [
NSLocalizedDescriptionKey: "Connection cancelled",
])
default:
continue
}
}
}
throw NSError(domain: "Bridge", code: 21, userInfo: [
NSLocalizedDescriptionKey: "Connection closed",
])
}
throw NSError(domain: "Bridge", code: 21, userInfo: [
NSLocalizedDescriptionKey: "Connection closed",
])
})
}
private func resolveBridgeEndpoint(timeoutSeconds: Double) async -> NWEndpoint? {
@@ -103,10 +103,13 @@ actor MacNodeRuntime {
}
let params = (try? Self.decodeParams(ClawdisCameraSnapParams.self, from: req.paramsJSON)) ??
ClawdisCameraSnapParams()
let delayMs = min(10_000, max(0, params.delayMs ?? 2000))
let res = try await self.cameraCapture.snap(
facing: CameraFacing(rawValue: params.facing?.rawValue ?? "") ?? .front,
maxWidth: params.maxWidth,
quality: params.quality)
quality: params.quality,
deviceId: params.deviceId,
delayMs: delayMs)
struct SnapPayload: Encodable {
var format: String
var base64: String
@@ -135,6 +138,7 @@ actor MacNodeRuntime {
facing: CameraFacing(rawValue: params.facing?.rawValue ?? "") ?? .front,
durationMs: params.durationMs,
includeAudio: params.includeAudio ?? true,
deviceId: params.deviceId,
outPath: nil)
defer { try? FileManager.default.removeItem(atPath: res.path) }
let data = try Data(contentsOf: URL(fileURLWithPath: res.path))
@@ -151,6 +155,19 @@ actor MacNodeRuntime {
hasAudio: res.hasAudio))
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload)
case ClawdisCameraCommand.list.rawValue:
guard Self.cameraEnabled() else {
return BridgeInvokeResponse(
id: req.id,
ok: false,
error: ClawdisNodeError(
code: .unavailable,
message: "CAMERA_DISABLED: enable Camera in Settings"))
}
let devices = await self.cameraCapture.listDevices()
let payload = try Self.encodePayload(["devices": devices])
return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload)
case MacNodeScreenCommand.record.rawValue:
let params = (try? Self.decodeParams(MacNodeScreenRecordParams.self, from: req.paramsJSON)) ??
MacNodeScreenRecordParams()
@@ -265,7 +282,7 @@ actor MacNodeRuntime {
guard let raw = await GatewayConnection.shared.canvasHostUrl() else { return nil }
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty, let baseUrl = URL(string: trimmed) else { return nil }
return baseUrl.appendingPathComponent("__clawdis__/a2ui/").absoluteString
return baseUrl.appendingPathComponent("__clawdis__/a2ui/").absoluteString + "?platform=macos"
}
private func isA2UIReady(poll: Bool = false) async -> Bool {
@@ -2,6 +2,7 @@ import AppKit
import ClawdisIPC
import ClawdisProtocol
import Foundation
import Observation
import OSLog
import UserNotifications
@@ -15,6 +16,7 @@ enum NodePairingReconcilePolicy {
}
@MainActor
@Observable
final class NodePairingApprovalPrompter {
static let shared = NodePairingApprovalPrompter()
@@ -26,6 +28,8 @@ final class NodePairingApprovalPrompter {
private var isStopping = false
private var isPresenting = false
private var queue: [PendingRequest] = []
var pendingCount: Int = 0
var pendingRepairCount: Int = 0
private var activeAlert: NSAlert?
private var activeRequestId: String?
private var alertHostWindow: NSWindow?
@@ -104,6 +108,7 @@ final class NodePairingApprovalPrompter {
self.reconcileOnceTask?.cancel()
self.reconcileOnceTask = nil
self.queue.removeAll(keepingCapacity: false)
self.updatePendingCounts()
self.isPresenting = false
self.activeRequestId = nil
self.alertHostWindow?.orderOut(nil)
@@ -292,6 +297,7 @@ final class NodePairingApprovalPrompter {
private func enqueue(_ req: PendingRequest) {
if self.queue.contains(req) { return }
self.queue.append(req)
self.updatePendingCounts()
self.presentNextIfNeeded()
self.updateReconcileLoop()
}
@@ -362,6 +368,7 @@ final class NodePairingApprovalPrompter {
} else {
self.queue.removeAll { $0 == request }
}
self.updatePendingCounts()
self.isPresenting = false
self.presentNextIfNeeded()
self.updateReconcileLoop()
@@ -501,6 +508,8 @@ final class NodePairingApprovalPrompter {
} else {
self.queue.removeAll { $0 == req }
}
self.updatePendingCounts()
self.isPresenting = false
self.presentNextIfNeeded()
self.updateReconcileLoop()
@@ -599,6 +608,12 @@ final class NodePairingApprovalPrompter {
}
}
private func updatePendingCounts() {
// Keep a cheap observable summary for the menu bar status line.
self.pendingCount = self.queue.count
self.pendingRepairCount = self.queue.filter { $0.isRepair == true }.count
}
private func reconcileOnce(timeoutMs: Double) async {
if self.isStopping { return }
if self.reconcileInFlight { return }
@@ -643,6 +658,7 @@ final class NodePairingApprovalPrompter {
return
}
self.queue.removeAll { $0.requestId == resolved.requestId }
self.updatePendingCounts()
Task { @MainActor in
await self.notify(resolution: resolution, request: request, via: "remote")
}
+136 -54
View File
@@ -2,51 +2,69 @@ import AppKit
import SwiftUI
struct NodeMenuEntryFormatter {
static func isGateway(_ entry: InstanceInfo) -> Bool {
entry.mode?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() == "gateway"
static func isGateway(_ entry: NodeInfo) -> Bool {
entry.nodeId == "gateway"
}
static func isLocal(_ entry: InstanceInfo) -> Bool {
entry.mode?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() == "local"
static func isConnected(_ entry: NodeInfo) -> Bool {
entry.isConnected
}
static func primaryName(_ entry: InstanceInfo) -> String {
static func primaryName(_ entry: NodeInfo) -> String {
if self.isGateway(entry) {
let host = entry.host?.nonEmpty
if let host, host.lowercased() != "gateway" { return host }
return "Gateway"
return entry.displayName?.nonEmpty ?? "Gateway"
}
return entry.host?.nonEmpty ?? entry.id
return entry.displayName?.nonEmpty ?? entry.nodeId
}
static func summaryText(_ entry: InstanceInfo) -> String {
entry.text.nonEmpty ?? self.primaryName(entry)
}
static func roleText(_ entry: InstanceInfo) -> String {
if self.isGateway(entry) { return "gateway" }
if let mode = entry.mode?.nonEmpty { return mode }
return "node"
}
static func detailLeft(_ entry: InstanceInfo) -> String {
let role = self.roleText(entry)
if let ip = entry.ip?.nonEmpty { return "\(ip) · \(role)" }
return role
}
static func detailRight(_ entry: InstanceInfo) -> String? {
var parts: [String] = []
if let platform = self.platformText(entry) { parts.append(platform) }
static func summaryText(_ entry: NodeInfo) -> String {
if self.isGateway(entry) {
let role = self.roleText(entry)
let name = self.primaryName(entry)
var parts = ["\(name) · \(role)"]
if let ip = entry.remoteIp?.nonEmpty { parts.append("host \(ip)") }
if let platform = self.platformText(entry) { parts.append(platform) }
return parts.joined(separator: " · ")
}
let name = self.primaryName(entry)
var prefix = "Node: \(name)"
if let ip = entry.remoteIp?.nonEmpty {
prefix += " (\(ip))"
}
var parts = [prefix]
if let platform = self.platformText(entry) {
parts.append("platform \(platform)")
}
if let version = entry.version?.nonEmpty {
let short = self.compactVersion(version)
parts.append("v\(short)")
parts.append("app \(self.compactVersion(version))")
}
if parts.isEmpty { return nil }
parts.append("status \(self.roleText(entry))")
return parts.joined(separator: " · ")
}
static func platformText(_ entry: InstanceInfo) -> String? {
static func roleText(_ entry: NodeInfo) -> String {
if entry.isConnected { return "connected" }
if self.isGateway(entry) { return "disconnected" }
if entry.isPaired { return "paired" }
return "unpaired"
}
static func detailLeft(_ entry: NodeInfo) -> String {
let role = self.roleText(entry)
if let ip = entry.remoteIp?.nonEmpty { return "\(ip) · \(role)" }
return role
}
static func headlineRight(_ entry: NodeInfo) -> String? {
self.platformText(entry)
}
static func detailRightVersion(_ entry: NodeInfo) -> String? {
guard let version = entry.version?.nonEmpty else { return nil }
return self.shortVersionLabel(version)
}
static func platformText(_ entry: NodeInfo) -> String? {
if let raw = entry.platform?.nonEmpty {
return self.prettyPlatform(raw) ?? raw
}
@@ -99,8 +117,22 @@ struct NodeMenuEntryFormatter {
return trimmed
}
static func leadingSymbol(_ entry: InstanceInfo) -> String {
if self.isGateway(entry) { return self.safeSystemSymbol("dot.radiowaves.left.and.right", fallback: "network") }
private static func shortVersionLabel(_ raw: String) -> String {
let compact = self.compactVersion(raw)
if compact.isEmpty { return compact }
if compact.lowercased().hasPrefix("v") { return compact }
if let first = compact.unicodeScalars.first, CharacterSet.decimalDigits.contains(first) {
return "v\(compact)"
}
return compact
}
static func leadingSymbol(_ entry: NodeInfo) -> String {
if self.isGateway(entry) {
return self.safeSystemSymbol(
"antenna.radiowaves.left.and.right",
fallback: "dot.radiowaves.left.and.right")
}
if let family = entry.deviceFamily?.lowercased() {
if family.contains("mac") {
return self.safeSystemSymbol("laptopcomputer", fallback: "laptopcomputer")
@@ -116,9 +148,11 @@ struct NodeMenuEntryFormatter {
return "cpu"
}
static func isAndroid(_ entry: InstanceInfo) -> Bool {
static func isAndroid(_ entry: NodeInfo) -> Bool {
let family = entry.deviceFamily?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
return family == "android"
if family == "android" { return true }
let platform = entry.platform?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
return platform?.contains("android") == true
}
private static func safeSystemSymbol(_ preferred: String, fallback: String) -> String {
@@ -128,7 +162,7 @@ struct NodeMenuEntryFormatter {
}
struct NodeMenuRowView: View {
let entry: InstanceInfo
let entry: NodeInfo
let width: CGFloat
@Environment(\.menuItemHighlighted) private var isHighlighted
@@ -146,11 +180,32 @@ struct NodeMenuRowView: View {
.frame(width: 22, height: 22, alignment: .center)
VStack(alignment: .leading, spacing: 2) {
Text(NodeMenuEntryFormatter.primaryName(self.entry))
.font(.callout.weight(NodeMenuEntryFormatter.isGateway(self.entry) ? .semibold : .regular))
.foregroundStyle(self.primaryColor)
.lineLimit(1)
.truncationMode(.middle)
HStack(alignment: .firstTextBaseline, spacing: 8) {
Text(NodeMenuEntryFormatter.primaryName(self.entry))
.font(.callout.weight(NodeMenuEntryFormatter.isConnected(self.entry) ? .semibold : .regular))
.foregroundStyle(self.primaryColor)
.lineLimit(1)
.truncationMode(.middle)
.layoutPriority(1)
Spacer(minLength: 8)
HStack(alignment: .firstTextBaseline, spacing: 6) {
if let right = NodeMenuEntryFormatter.headlineRight(self.entry) {
Text(right)
.font(.caption.monospacedDigit())
.foregroundStyle(self.secondaryColor)
.lineLimit(1)
.truncationMode(.middle)
.layoutPriority(2)
}
Image(systemName: "chevron.right")
.font(.caption.weight(.semibold))
.foregroundStyle(self.secondaryColor)
.padding(.leading, 2)
}
}
HStack(alignment: .firstTextBaseline, spacing: 8) {
Text(NodeMenuEntryFormatter.detailLeft(self.entry))
@@ -161,21 +216,15 @@ struct NodeMenuRowView: View {
Spacer(minLength: 0)
HStack(alignment: .firstTextBaseline, spacing: 6) {
if let right = NodeMenuEntryFormatter.detailRight(self.entry) {
Text(right)
.font(.caption.monospacedDigit())
.foregroundStyle(self.secondaryColor)
.lineLimit(1)
.truncationMode(.middle)
}
Image(systemName: "chevron.right")
.font(.caption.weight(.semibold))
if let version = NodeMenuEntryFormatter.detailRightVersion(self.entry) {
Text(version)
.font(.caption.monospacedDigit())
.foregroundStyle(self.secondaryColor)
.padding(.leading, 2)
.lineLimit(1)
.truncationMode(.middle)
}
}
.frame(maxWidth: .infinity, alignment: .leading)
}
.frame(maxWidth: .infinity, alignment: .leading)
@@ -215,3 +264,36 @@ struct AndroidMark: View {
}
}
}
struct NodeMenuMultilineView: View {
let label: String
let value: String
let width: CGFloat
@Environment(\.menuItemHighlighted) private var isHighlighted
private var primaryColor: Color {
self.isHighlighted ? Color(nsColor: .selectedMenuItemTextColor) : .primary
}
private var secondaryColor: Color {
self.isHighlighted ? Color(nsColor: .selectedMenuItemTextColor).opacity(0.85) : .secondary
}
var body: some View {
VStack(alignment: .leading, spacing: 4) {
Text("\(self.label):")
.font(.caption.weight(.semibold))
.foregroundStyle(self.secondaryColor)
Text(self.value)
.font(.caption)
.foregroundStyle(self.primaryColor)
.multilineTextAlignment(.leading)
.fixedSize(horizontal: false, vertical: true)
}
.padding(.vertical, 6)
.padding(.leading, 18)
.padding(.trailing, 12)
.frame(width: max(1, self.width), alignment: .leading)
}
}
+100
View File
@@ -0,0 +1,100 @@
import Foundation
import Observation
import OSLog
struct NodeInfo: Identifiable, Codable {
let nodeId: String
let displayName: String?
let platform: String?
let version: String?
let deviceFamily: String?
let modelIdentifier: String?
let remoteIp: String?
let caps: [String]?
let commands: [String]?
let permissions: [String: Bool]?
let paired: Bool?
let connected: Bool?
var id: String { self.nodeId }
var isConnected: Bool { self.connected ?? false }
var isPaired: Bool { self.paired ?? false }
}
private struct NodeListResponse: Codable {
let ts: Double?
let nodes: [NodeInfo]
}
@MainActor
@Observable
final class NodesStore {
static let shared = NodesStore()
var nodes: [NodeInfo] = []
var lastError: String?
var statusMessage: String?
var isLoading = false
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "nodes")
private var task: Task<Void, Never>?
private let interval: TimeInterval = 30
private var startCount = 0
func start() {
self.startCount += 1
guard self.startCount == 1 else { return }
guard self.task == nil else { return }
self.task = Task.detached { [weak self] in
guard let self else { return }
await self.refresh()
while !Task.isCancelled {
try? await Task.sleep(nanoseconds: UInt64(self.interval * 1_000_000_000))
await self.refresh()
}
}
}
func stop() {
guard self.startCount > 0 else { return }
self.startCount -= 1
guard self.startCount == 0 else { return }
self.task?.cancel()
self.task = nil
}
func refresh() async {
if self.isLoading { return }
self.statusMessage = nil
self.isLoading = true
defer { self.isLoading = false }
do {
let data = try await GatewayConnection.shared.requestRaw(method: "node.list", params: nil, timeoutMs: 8000)
let decoded = try JSONDecoder().decode(NodeListResponse.self, from: data)
self.nodes = decoded.nodes
self.lastError = nil
self.statusMessage = nil
} catch {
if Self.isCancelled(error) {
self.logger.debug("node.list cancelled; keeping last nodes")
if self.nodes.isEmpty {
self.statusMessage = "Refreshing devices…"
}
self.lastError = nil
return
}
self.logger.error("node.list failed \(error.localizedDescription, privacy: .public)")
self.nodes = []
self.lastError = error.localizedDescription
self.statusMessage = nil
}
}
private static func isCancelled(_ error: Error) -> Bool {
if error is CancellationError { return true }
if let urlError = error as? URLError, urlError.code == .cancelled { return true }
let nsError = error as NSError
if nsError.domain == NSURLErrorDomain, nsError.code == NSURLErrorCancelled { return true }
return false
}
}
@@ -5,6 +5,8 @@ import UserNotifications
@MainActor
struct NotificationManager {
private let logger = Logger(subsystem: "com.steipete.clawdis", category: "notifications")
private static let hasTimeSensitiveEntitlement: Bool = {
guard let task = SecTaskCreateFromSelf(nil) else { return false }
let key = "com.apple.developer.usernotifications.time-sensitive" as CFString
@@ -17,8 +19,12 @@ struct NotificationManager {
let status = await center.notificationSettings()
if status.authorizationStatus == .notDetermined {
let granted = try? await center.requestAuthorization(options: [.alert, .sound, .badge])
if granted != true { return false }
if granted != true {
self.logger.warning("notification permission denied (request)")
return false
}
} else if status.authorizationStatus != .authorized {
self.logger.warning("notification permission denied status=\(status.authorizationStatus.rawValue)")
return false
}
@@ -37,15 +43,23 @@ struct NotificationManager {
case .active:
content.interruptionLevel = .active
case .timeSensitive:
content.interruptionLevel = Self.hasTimeSensitiveEntitlement ? .timeSensitive : .active
if Self.hasTimeSensitiveEntitlement {
content.interruptionLevel = .timeSensitive
} else {
self.logger.debug(
"time-sensitive notification requested without entitlement; falling back to active")
content.interruptionLevel = .active
}
}
}
let req = UNNotificationRequest(identifier: UUID().uuidString, content: content, trigger: nil)
do {
try await center.add(req)
self.logger.debug("notification queued")
return true
} catch {
self.logger.error("notification send failed: \(error.localizedDescription)")
return false
}
}
Loaded 100 of 431 files, more files were not shown because too many files have changed in this diff. Show more