bob-boat c91f76039d Initial: README, spec/agent-watcher.md v1 (signed off by Bob 2026-05-06), .gitignore

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-06 14:36:45 -04:00

13 KiB

Raw Blame History

Agent Watcher — v1 Spec

Author: Foreman (laptop) Date: 2026-05-06 Status: Draft for Bob review, then Angus sign-off Sibling to: agent-ping-system.md (this layer extends it) Repo (planned): Forgejo agent-watcher, sibling to agent-ping

1. Problem

The ping system is async-on-next-prompt: pings land in inbox files but the recipient agent only sees them when something else triggers a prompt. If nobody types at the agent, pings sit unread.

We want push semantics — external events (a ping landing, a webhook firing) wake the recipient agent without a human in the loop. The watcher is the "secondary nervous system": its only job is to notice events and notify the right agent.

2. Architecture

Two layers, one durable substrate.

                            ┌──────────────────────────┐
external event   ─────►     │   Collector (Go daemon)  │
(webhook, file              │   systemd --user, always │
 arrival, future            │   on, brain-blind        │
 sources)                   └────────────┬─────────────┘
                                         │ writes
                                         ▼
                          ~/Nyx/workspace/pings/<r>.inbox     (durable queue)
                                         │
                                         │ inotify
                                         ▼
                            ┌──────────────────────────┐
                            │   MCP Watcher            │ ──► Channels event
                            │   (subprocess of Claude  │     into agent session
                            │   Code via mcp.json)     │
                            └──────────────────────────┘
                                         │
                                         │ falls back to
                                         ▼
                            UserPromptSubmit hook (existing agent-ping)
                            on hosts where MCP Watcher is not running

Key properties:

Filesystem is the queue. Inbox files are the source of truth. Both layers read/write the same files the ping CLI already uses. No new persistence dependency.
Brain-blind. The watcher does not capture to OpenBrain. Receiving agents capture deliberately. (Same poison-loop rule as compaction summaries.)
Two-layer split = uptime independence. Collector runs whether or not any agent is alive. Events captured to disk regardless.
Channels, not RemoteTrigger. RemoteTrigger is deferred/unshipped. Channels (MCP-over-stdio, research preview, Claude Code v2.1.80+) is the actual primitive. See §6 for fallback strategy.

3. Layer 1 — Collector

A small Go daemon, single static binary, runs as systemd --user unit on every host. Cross-compiles for Linux/Android/Pi from one source.

3.1 v1 sources

Two only:

HTTP webhook listener — bind 127.0.0.1:18790 (loopback only in v1; Caddy + bearer-token reverse proxy is the v2 upgrade path). Receives POSTs, applies a routing table, writes a JSONL line to the target inbox file.
inotify on a designated drop folder — ~/Nyx/workspace/incoming/ (NOT the inbox itself; that would loop). External producers drop .json files; the Collector ingests, writes to inbox, deletes the drop file.

The drop folder is Syncthing-replicated (it lives under workspace/), which is intentional: a producer on any host can drop into another host's Collector. Lifecycle: producer drops event-123.json → Syncthing pulls → Collector ingests + deletes → Syncthing replicates the delete back. One-shot semantics. Do not place permanent files in incoming/; they will disappear.

Drop file schema:
```
{
  "recipient": "bob",
  "type": "INFO",
  "priority": "normal",
  "payload": "...",
  "sentinel": "/path/optional"
}
```
On schema validation failure: log structured error and move the file to ~/Nyx/workspace/incoming/.dead-letter/ rather than silently dropping. External producers can monitor the dead-letter folder for their own malformed events.

Cron, sensors, and serial sources are explicitly deferred to v2. Pin v1 small.

3.2 Configuration

Single per-host file: ~/.config/agent-watcher/collector.yaml.

agent: foreman   # this host's identity

sources:
  webhook:
    listen: 127.0.0.1:18790
    routes:
      /forgejo/push:
        recipient: bob
        type: INFO
        payload_template: "forgejo push to {{ .repo }} by {{ .actor }}"
      /openrouter/billing-alert:
        recipient: bob
        type: NEEDS-RESPONSE
        priority: urgent
        payload_template: "billing alert: {{ .message }}"
  drop_folder:
    path: ~/Nyx/workspace/incoming/
    poll_fallback_seconds: 30   # safety net if inotify misses

Hot-reload: SIGHUP re-reads config. Invalid config logs and keeps the old config running.

3.3 Output format

The Collector writes the same JSONL line the ping CLI writes today. The watcher and existing hook cannot tell the difference — every line in the inbox looks like a ping regardless of source.

Adds one optional field: "source": "webhook:/forgejo/push" for debugging.

3.4 Resource budget

Target: <15MB resident, <1% CPU at idle. Confirm on first deploy. Logs to journald with structured fields (source, recipient, event_id, latency_ms).

3.5 Health

/health endpoint on the same loopback port returns JSON with per-source counters (received, emitted, errors) and uptime. Useful for journalctl --user -u agent-watcher correlation.

4. Layer 2 — MCP Watcher

A Claude Code MCP server, declared in each agent's mcp.json, spawned as a subprocess at session start. Communicates over stdio.

4.1 Behavior

On startup:

Read inbox file ~/Nyx/workspace/pings/<self>.inbox and HWM file <self>.hwm.
Emit a notifications/claude/channel event for any unread pings.
Set up inotify on the inbox file.

On inotify fire (new line appended): 4. Read the new line(s). 5. Emit one notifications/claude/channel event per new ping. 6. Advance HWM atomically (write-tmp-rename). 7. Append to acks log.

4.2 Tools exposed

The MCP watcher exposes reply tools the agent can call:

ack(ping_id, message?) — write an ack to the channel folder.
respond(ping_id, payload) — emit a response ping back to sender.
mark_handled(ping_id) — mark a ping as processed without replying (for INFO/ACK-REQUEST that don't need a response).

Cross-host write discipline. Reply tools always write to local-filesystem paths only (this host's inbox/ack/channel files); Syncthing replicates outward. The watcher must not write directly to a remote-host inbox path. Same rule as the ping CLI.

4.3 Coexistence with the existing hook (Bob's load-bearing #2)

Decision: option (c) — MCP Watcher fully replaces the hook on hosts where it is running. Hook remains as fallback.

Mechanism:

The MCP watcher writes a sentinel file ~/Nyx/workspace/pings/.<self>.watcher-active on startup, removes it on shutdown.
The existing UserPromptSubmit hook checks for that sentinel; if present, the hook does nothing (the watcher is delivering).
If the watcher crashes or is not configured, the hook takes over — zero-config degradation back to next-prompt delivery.

The sentinel must be local-only — Syncthing-excluded. If it replicates, Foreman's watcher being up would silence Bob's hook even when Bob's watcher isn't running. Add to .stignore alongside the existing HWM exclusion:

pings/.*.hwm
pings/.*.watcher-active

Same shape and same reasoning as the HWM-not-synced rule.

This avoids duplicate-delivery and HWM race conditions. One reader at any time.

4.4 Configuration

Per-agent mcp.json entry:

{
  "mcpServers": {
    "agent-watcher": {
      "command": "agent-watcher-mcp",
      "args": ["--agent", "foreman"]
    }
  }
}

The MCP watcher reads the same inbox/hwm/acks paths as the existing hook; no extra config.

5. Cross-host writes & Syncthing

Same discipline as the existing ping CLI: each host only writes to local-filesystem inboxes; Syncthing replicates outward. The Collector writes to ~/Nyx/workspace/pings/<recipient>.inbox only on the sender's host. Recipient host receives it via Syncthing.

This means the recipient's MCP watcher sees inotify events from Syncthing's writes. Confirm Syncthing fires IN_CLOSE_WRITE reliably on rename — it should, but flag for first-deploy testing. If it doesn't, fall back to a 5-second poll on the inbox file.

JSONL conflict markers from concurrent writes are an existing risk documented in agent-ping-system.md. Not addressed in v1; flagged for v2 if hit in practice.

6. Channels research-preview status (Bob's load-bearing #1)

Channels is in research preview. The watcher's read/ack contract must survive API drift.

6.1 Fallback ladder

If Channels is healthy: MCP watcher delivers events mid-session via notifications/claude/channel. Reply tools work via MCP tool-call.

If Channels degrades or disappears: MCP watcher stops, sentinel file removed, the existing UserPromptSubmit hook resumes async-on-next-prompt delivery. Zero new code needed for fallback — the hook is already deployed.

If MCP changes incompatibly: the watcher is a single Go binary, easy to swap for a different push primitive. The Collector is unaffected.

6.2 Naming the value

The watcher's value is during-turn delivery (event surfaces while agent is active or wakes an idle agent). It is not "delivery at all" — that already works via the hook. Sets correct expectations.

7. Scope (Bob's load-bearing #3)

v1 (this spec)

Collector with HTTP webhook + inotify-on-drop-folder sources only.
MCP watcher with inbox watching and three reply tools.
systemd --user unit, loopback-only HTTP, single YAML config.
Hook coexistence via sentinel file (option c).

v2 (future, separate spec)

Cron source on the Collector.
Sensor sources (service health on VPS, hardware on Pi/phone).
Caddy + bearer-token webhook auth for cross-host posting.
TTL on emitted pings (expires_at); hook drops or warns on expired.
Coalesce dupes (e.g. 100 alert webhooks → one summary).
Cross-host JSONL conflict resolution.

Out of scope, ever

OpenBrain integration. Watcher stays brain-blind. Agents capture.
Mid-turn preemption. Not possible in current Claude Code; not attempted.
Spawning new agents. Use Anthropic Routines for that pattern.

8. Install & ownership

Per CLAUDE.md rule 2 ("no agent edits its own code or config"), the watcher is installed on each agent's host by Angus from the canonical Forgejo repo. Same pattern as agent-ping.

Install script delivers:

agent-watcher binary → ~/.local/bin/
agent-watcher-mcp binary → ~/.local/bin/
systemd unit → ~/.config/systemd/user/agent-watcher.service
Config skeleton → ~/.config/agent-watcher/collector.yaml
mcp.json snippet → printed to stdout for manual paste into the agent's mcp config (Angus or Foreman applies)

Verify hooks: agent-watcher --self-check reports config validity, inbox path access, MCP binary presence.

9. Testing

Unit: routing rules, payload templates, JSONL append correctness, HWM advancement.
Integration (single host): webhook POST → inbox line → MCP event → agent sees [INCOMING FROM …].
Integration (cross-host): Foreman ping CLI → Bob inbox via Syncthing → Bob MCP watcher → Bob sees event mid-session (target latency: <30s including Syncthing).
Failover: kill MCP watcher mid-session → hook takes over on next prompt. Restart watcher → hook silent again.

10. What I need from Angus + Bob to finalize

Repo creation. Confirm agent-watcher as Forgejo repo name and that I should match the layout of agent-ping (CLI + hook + spec in repo, install script at root).
Webhook binding. Confirm 127.0.0.1:18790 is acceptable as the v1 default port. (18790 chosen to sit just past the agent-ping range; happy to change.)
Drop folder path. Confirm ~/Nyx/workspace/incoming/ is the right location, or pick another. It should NOT be inside pings/.
MCP binary name. Confirm agent-watcher-mcp as a separate binary, vs. agent-watcher mcp as a subcommand. I lean separate binary — clearer in ps, simpler mcp.json.
Health endpoint exposure. Loopback-only fine, or do you want it gated behind a token even on loopback?
Bob: confirm option (c) for hook coexistence — watcher fully replaces hook when running, sentinel file gates the hook. If you prefer (a) shared HWM or (b) live-flag split, say so before code.
First deploy host. Foreman/laptop or Bob/VPS first? Foreman is lower-stakes for testing; Bob is where mobile webhooks will eventually land.

Answers fold into the spec; we don't iterate them in chat.

13 KiB Raw Blame History