From c91f76039dc2a024f3fc8800b2f1645e20664e6a Mon Sep 17 00:00:00 2001 From: bob-boat Date: Wed, 6 May 2026 14:36:45 -0400 Subject: [PATCH] Initial: README, spec/agent-watcher.md v1 (signed off by Bob 2026-05-06), .gitignore Co-Authored-By: Claude Opus 4.7 (1M context) --- .gitignore | 7 + README.md | 24 +++ spec/agent-watcher.md | 376 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 407 insertions(+) create mode 100644 .gitignore create mode 100644 README.md create mode 100644 spec/agent-watcher.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..d860037 --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +/bin/ +*.swp +*.swo +.DS_Store +/dist/ +*.test +*.out diff --git a/README.md b/README.md new file mode 100644 index 0000000..09f78b8 --- /dev/null +++ b/README.md @@ -0,0 +1,24 @@ +# agent-watcher + +Push-delivery layer for [`agent-ping`](http://localhost:3300/angus/agent-ping). The "secondary nervous system" for Claude Code agents on this network. + +`agent-ping` queues messages in inbox files; `agent-watcher` notices them (and other external events) and wakes the recipient agent without a human in the loop. + +Two layers: + +- **Collector** — small Go daemon, `systemd --user`, always on, brain-blind. Converts external events (HTTP webhooks, drop-folder file arrivals) into ping inbox writes. Runs whether or not any agent is alive. +- **MCP Watcher** — Claude Code MCP subprocess, declared in each agent's `mcp.json`. Watches the agent's inbox via inotify and surfaces events into the live session via Channels (research preview). Provides reply tools (`ack`, `respond`, `mark_handled`). + +Filesystem is the queue. OpenBrain is not involved. + +## Spec + +[`spec/agent-watcher.md`](spec/agent-watcher.md). Read that for architecture, decisions, scope. + +## Status + +v1 spec signed off by Bob (VPS) 2026-05-06. Implementation pending. + +## Install + +Per CLAUDE.md rule #2, **Angus runs the install commands** — agents do not modify their own configuration. Install script will land alongside `INSTALL.md` once the binaries are built. diff --git a/spec/agent-watcher.md b/spec/agent-watcher.md new file mode 100644 index 0000000..9f4c778 --- /dev/null +++ b/spec/agent-watcher.md @@ -0,0 +1,376 @@ +# Agent Watcher — v1 Spec + +**Author:** Foreman (laptop) +**Date:** 2026-05-06 +**Status:** Draft for Bob review, then Angus sign-off +**Sibling to:** `agent-ping-system.md` (this layer extends it) +**Repo (planned):** Forgejo `agent-watcher`, sibling to `agent-ping` + +--- + +## 1. Problem + +The ping system is async-on-next-prompt: pings land in inbox files but the +recipient agent only sees them when something else triggers a prompt. If +nobody types at the agent, pings sit unread. + +We want push semantics — external events (a ping landing, a webhook firing) +wake the recipient agent without a human in the loop. The watcher is the +"secondary nervous system": its only job is to notice events and notify the +right agent. + +--- + +## 2. Architecture + +Two layers, one durable substrate. + +``` + ┌──────────────────────────┐ +external event ─────► │ Collector (Go daemon) │ +(webhook, file │ systemd --user, always │ + arrival, future │ on, brain-blind │ + sources) └────────────┬─────────────┘ + │ writes + ▼ + ~/Nyx/workspace/pings/.inbox (durable queue) + │ + │ inotify + ▼ + ┌──────────────────────────┐ + │ MCP Watcher │ ──► Channels event + │ (subprocess of Claude │ into agent session + │ Code via mcp.json) │ + └──────────────────────────┘ + │ + │ falls back to + ▼ + UserPromptSubmit hook (existing agent-ping) + on hosts where MCP Watcher is not running +``` + +Key properties: + +- **Filesystem is the queue.** Inbox files are the source of truth. + Both layers read/write the same files the `ping` CLI already uses. + No new persistence dependency. +- **Brain-blind.** The watcher does not capture to OpenBrain. Receiving + agents capture deliberately. (Same poison-loop rule as compaction + summaries.) +- **Two-layer split = uptime independence.** Collector runs whether or + not any agent is alive. Events captured to disk regardless. +- **Channels, not RemoteTrigger.** RemoteTrigger is deferred/unshipped. + Channels (MCP-over-stdio, research preview, Claude Code v2.1.80+) is + the actual primitive. See §6 for fallback strategy. + +--- + +## 3. Layer 1 — Collector + +A small Go daemon, single static binary, runs as `systemd --user` unit on +every host. Cross-compiles for Linux/Android/Pi from one source. + +### 3.1 v1 sources + +Two only: + +1. **HTTP webhook listener** — bind `127.0.0.1:18790` (loopback only + in v1; Caddy + bearer-token reverse proxy is the v2 upgrade path). + Receives POSTs, applies a routing table, writes a JSONL line to the + target inbox file. + +2. **inotify on a designated drop folder** — `~/Nyx/workspace/incoming/` + (NOT the inbox itself; that would loop). External producers drop + `.json` files; the Collector ingests, writes to inbox, deletes the + drop file. + + The drop folder is Syncthing-replicated (it lives under `workspace/`), + which is intentional: a producer on any host can drop into another + host's Collector. Lifecycle: producer drops `event-123.json` → + Syncthing pulls → Collector ingests + deletes → Syncthing replicates + the delete back. One-shot semantics. Do not place permanent files + in `incoming/`; they will disappear. + + **Drop file schema:** + + ```json + { + "recipient": "bob", + "type": "INFO", + "priority": "normal", + "payload": "...", + "sentinel": "/path/optional" + } + ``` + + On schema validation failure: log structured error and move the file + to `~/Nyx/workspace/incoming/.dead-letter/` rather than silently + dropping. External producers can monitor the dead-letter folder for + their own malformed events. + +Cron, sensors, and serial sources are explicitly **deferred to v2**. +Pin v1 small. + +### 3.2 Configuration + +Single per-host file: `~/.config/agent-watcher/collector.yaml`. + +```yaml +agent: foreman # this host's identity + +sources: + webhook: + listen: 127.0.0.1:18790 + routes: + /forgejo/push: + recipient: bob + type: INFO + payload_template: "forgejo push to {{ .repo }} by {{ .actor }}" + /openrouter/billing-alert: + recipient: bob + type: NEEDS-RESPONSE + priority: urgent + payload_template: "billing alert: {{ .message }}" + drop_folder: + path: ~/Nyx/workspace/incoming/ + poll_fallback_seconds: 30 # safety net if inotify misses +``` + +Hot-reload: SIGHUP re-reads config. Invalid config logs and keeps the +old config running. + +### 3.3 Output format + +The Collector writes the same JSONL line the `ping` CLI writes today. +The watcher and existing hook cannot tell the difference — every line +in the inbox looks like a ping regardless of source. + +Adds one optional field: `"source": "webhook:/forgejo/push"` for +debugging. + +### 3.4 Resource budget + +Target: <15MB resident, <1% CPU at idle. Confirm on first deploy. +Logs to journald with structured fields (`source`, `recipient`, +`event_id`, `latency_ms`). + +### 3.5 Health + +`/health` endpoint on the same loopback port returns JSON with +per-source counters (received, emitted, errors) and uptime. Useful +for `journalctl --user -u agent-watcher` correlation. + +--- + +## 4. Layer 2 — MCP Watcher + +A Claude Code MCP server, declared in each agent's `mcp.json`, spawned +as a subprocess at session start. Communicates over stdio. + +### 4.1 Behavior + +On startup: +1. Read inbox file `~/Nyx/workspace/pings/.inbox` and HWM file + `.hwm`. +2. Emit a `notifications/claude/channel` event for any unread pings. +3. Set up inotify on the inbox file. + +On inotify fire (new line appended): +4. Read the new line(s). +5. Emit one `notifications/claude/channel` event per new ping. +6. Advance HWM atomically (write-tmp-rename). +7. Append to acks log. + +### 4.2 Tools exposed + +The MCP watcher exposes reply tools the agent can call: + +- `ack(ping_id, message?)` — write an ack to the channel folder. +- `respond(ping_id, payload)` — emit a response ping back to sender. +- `mark_handled(ping_id)` — mark a ping as processed without replying + (for INFO/ACK-REQUEST that don't need a response). + +**Cross-host write discipline.** Reply tools always write to +local-filesystem paths only (this host's inbox/ack/channel files); +Syncthing replicates outward. The watcher must not write directly to +a remote-host inbox path. Same rule as the `ping` CLI. + +### 4.3 Coexistence with the existing hook (Bob's load-bearing #2) + +**Decision: option (c) — MCP Watcher fully replaces the hook on hosts +where it is running. Hook remains as fallback.** + +Mechanism: +- The MCP watcher writes a sentinel file + `~/Nyx/workspace/pings/..watcher-active` on startup, removes + it on shutdown. +- The existing UserPromptSubmit hook checks for that sentinel; if + present, the hook does nothing (the watcher is delivering). +- If the watcher crashes or is not configured, the hook takes over — + zero-config degradation back to next-prompt delivery. + +**The sentinel must be local-only — Syncthing-excluded.** If it +replicates, Foreman's watcher being up would silence Bob's hook even +when Bob's watcher isn't running. Add to `.stignore` alongside the +existing HWM exclusion: + +``` +pings/.*.hwm +pings/.*.watcher-active +``` + +Same shape and same reasoning as the HWM-not-synced rule. + +This avoids duplicate-delivery and HWM race conditions. One reader at +any time. + +### 4.4 Configuration + +Per-agent `mcp.json` entry: + +```json +{ + "mcpServers": { + "agent-watcher": { + "command": "agent-watcher-mcp", + "args": ["--agent", "foreman"] + } + } +} +``` + +The MCP watcher reads the same inbox/hwm/acks paths as the existing +hook; no extra config. + +--- + +## 5. Cross-host writes & Syncthing + +Same discipline as the existing ping CLI: **each host only writes to +local-filesystem inboxes**; Syncthing replicates outward. The Collector +writes to `~/Nyx/workspace/pings/.inbox` only on the +*sender's* host. Recipient host receives it via Syncthing. + +This means the recipient's MCP watcher sees inotify events from +Syncthing's writes. Confirm Syncthing fires `IN_CLOSE_WRITE` reliably +on rename — it should, but flag for first-deploy testing. If it +doesn't, fall back to a 5-second poll on the inbox file. + +JSONL conflict markers from concurrent writes are an existing risk +documented in `agent-ping-system.md`. Not addressed in v1; flagged for +v2 if hit in practice. + +--- + +## 6. Channels research-preview status (Bob's load-bearing #1) + +Channels is in research preview. The watcher's read/ack contract must +survive API drift. + +### 6.1 Fallback ladder + +If Channels is healthy: MCP watcher delivers events mid-session via +`notifications/claude/channel`. Reply tools work via MCP tool-call. + +If Channels degrades or disappears: MCP watcher stops, sentinel file +removed, the existing UserPromptSubmit hook resumes async-on-next-prompt +delivery. Zero new code needed for fallback — the hook is already +deployed. + +If MCP changes incompatibly: the watcher is a single Go binary, easy +to swap for a different push primitive. The Collector is unaffected. + +### 6.2 Naming the value + +The watcher's value is **during-turn delivery** (event surfaces while +agent is active or wakes an idle agent). It is *not* "delivery at +all" — that already works via the hook. Sets correct expectations. + +--- + +## 7. Scope (Bob's load-bearing #3) + +### v1 (this spec) +- Collector with HTTP webhook + inotify-on-drop-folder sources only. +- MCP watcher with inbox watching and three reply tools. +- systemd --user unit, loopback-only HTTP, single YAML config. +- Hook coexistence via sentinel file (option c). + +### v2 (future, separate spec) +- Cron source on the Collector. +- Sensor sources (service health on VPS, hardware on Pi/phone). +- Caddy + bearer-token webhook auth for cross-host posting. +- TTL on emitted pings (`expires_at`); hook drops or warns on expired. +- Coalesce dupes (e.g. 100 alert webhooks → one summary). +- Cross-host JSONL conflict resolution. + +### Out of scope, ever +- OpenBrain integration. Watcher stays brain-blind. Agents capture. +- Mid-turn preemption. Not possible in current Claude Code; not + attempted. +- Spawning new agents. Use Anthropic Routines for that pattern. + +--- + +## 8. Install & ownership + +Per CLAUDE.md rule 2 ("no agent edits its own code or config"), the +watcher is installed on each agent's host by Angus from the canonical +Forgejo repo. Same pattern as `agent-ping`. + +Install script delivers: +- `agent-watcher` binary → `~/.local/bin/` +- `agent-watcher-mcp` binary → `~/.local/bin/` +- systemd unit → `~/.config/systemd/user/agent-watcher.service` +- Config skeleton → `~/.config/agent-watcher/collector.yaml` +- mcp.json snippet → printed to stdout for manual paste into the + agent's mcp config (Angus or Foreman applies) + +Verify hooks: `agent-watcher --self-check` reports config validity, +inbox path access, MCP binary presence. + +--- + +## 9. Testing + +- **Unit:** routing rules, payload templates, JSONL append correctness, + HWM advancement. +- **Integration (single host):** webhook POST → inbox line → MCP event + → agent sees `[INCOMING FROM …]`. +- **Integration (cross-host):** Foreman ping CLI → Bob inbox via + Syncthing → Bob MCP watcher → Bob sees event mid-session (target + latency: <30s including Syncthing). +- **Failover:** kill MCP watcher mid-session → hook takes over on next + prompt. Restart watcher → hook silent again. + +--- + +## 10. What I need from Angus + Bob to finalize + +1. **Repo creation.** Confirm `agent-watcher` as Forgejo repo name and + that I should match the layout of `agent-ping` (CLI + hook + spec + in repo, install script at root). + +2. **Webhook binding.** Confirm `127.0.0.1:18790` is acceptable as the + v1 default port. (18790 chosen to sit just past the agent-ping + range; happy to change.) + +3. **Drop folder path.** Confirm `~/Nyx/workspace/incoming/` is the + right location, or pick another. It should NOT be inside `pings/`. + +4. **MCP binary name.** Confirm `agent-watcher-mcp` as a separate + binary, vs. `agent-watcher mcp` as a subcommand. I lean separate + binary — clearer in `ps`, simpler mcp.json. + +5. **Health endpoint exposure.** Loopback-only fine, or do you want + it gated behind a token even on loopback? + +6. **Bob: confirm option (c) for hook coexistence** — watcher fully + replaces hook when running, sentinel file gates the hook. If you + prefer (a) shared HWM or (b) live-flag split, say so before code. + +7. **First deploy host.** Foreman/laptop or Bob/VPS first? Foreman + is lower-stakes for testing; Bob is where mobile webhooks will + eventually land. + +Answers fold into the spec; we don't iterate them in chat.