Initial: README, spec/agent-watcher.md v1 (signed off by Bob 2026-05-06), .gitignore
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
bob-boat
commit
c91f76039d
3 changed files with 407 additions and 0 deletions
376
spec/agent-watcher.md
Normal file
376
spec/agent-watcher.md
Normal file
|
|
@ -0,0 +1,376 @@
|
|||
# Agent Watcher — v1 Spec
|
||||
|
||||
**Author:** Foreman (laptop)
|
||||
**Date:** 2026-05-06
|
||||
**Status:** Draft for Bob review, then Angus sign-off
|
||||
**Sibling to:** `agent-ping-system.md` (this layer extends it)
|
||||
**Repo (planned):** Forgejo `agent-watcher`, sibling to `agent-ping`
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem
|
||||
|
||||
The ping system is async-on-next-prompt: pings land in inbox files but the
|
||||
recipient agent only sees them when something else triggers a prompt. If
|
||||
nobody types at the agent, pings sit unread.
|
||||
|
||||
We want push semantics — external events (a ping landing, a webhook firing)
|
||||
wake the recipient agent without a human in the loop. The watcher is the
|
||||
"secondary nervous system": its only job is to notice events and notify the
|
||||
right agent.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
Two layers, one durable substrate.
|
||||
|
||||
```
|
||||
┌──────────────────────────┐
|
||||
external event ─────► │ Collector (Go daemon) │
|
||||
(webhook, file │ systemd --user, always │
|
||||
arrival, future │ on, brain-blind │
|
||||
sources) └────────────┬─────────────┘
|
||||
│ writes
|
||||
▼
|
||||
~/Nyx/workspace/pings/<r>.inbox (durable queue)
|
||||
│
|
||||
│ inotify
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ MCP Watcher │ ──► Channels event
|
||||
│ (subprocess of Claude │ into agent session
|
||||
│ Code via mcp.json) │
|
||||
└──────────────────────────┘
|
||||
│
|
||||
│ falls back to
|
||||
▼
|
||||
UserPromptSubmit hook (existing agent-ping)
|
||||
on hosts where MCP Watcher is not running
|
||||
```
|
||||
|
||||
Key properties:
|
||||
|
||||
- **Filesystem is the queue.** Inbox files are the source of truth.
|
||||
Both layers read/write the same files the `ping` CLI already uses.
|
||||
No new persistence dependency.
|
||||
- **Brain-blind.** The watcher does not capture to OpenBrain. Receiving
|
||||
agents capture deliberately. (Same poison-loop rule as compaction
|
||||
summaries.)
|
||||
- **Two-layer split = uptime independence.** Collector runs whether or
|
||||
not any agent is alive. Events captured to disk regardless.
|
||||
- **Channels, not RemoteTrigger.** RemoteTrigger is deferred/unshipped.
|
||||
Channels (MCP-over-stdio, research preview, Claude Code v2.1.80+) is
|
||||
the actual primitive. See §6 for fallback strategy.
|
||||
|
||||
---
|
||||
|
||||
## 3. Layer 1 — Collector
|
||||
|
||||
A small Go daemon, single static binary, runs as `systemd --user` unit on
|
||||
every host. Cross-compiles for Linux/Android/Pi from one source.
|
||||
|
||||
### 3.1 v1 sources
|
||||
|
||||
Two only:
|
||||
|
||||
1. **HTTP webhook listener** — bind `127.0.0.1:18790` (loopback only
|
||||
in v1; Caddy + bearer-token reverse proxy is the v2 upgrade path).
|
||||
Receives POSTs, applies a routing table, writes a JSONL line to the
|
||||
target inbox file.
|
||||
|
||||
2. **inotify on a designated drop folder** — `~/Nyx/workspace/incoming/`
|
||||
(NOT the inbox itself; that would loop). External producers drop
|
||||
`.json` files; the Collector ingests, writes to inbox, deletes the
|
||||
drop file.
|
||||
|
||||
The drop folder is Syncthing-replicated (it lives under `workspace/`),
|
||||
which is intentional: a producer on any host can drop into another
|
||||
host's Collector. Lifecycle: producer drops `event-123.json` →
|
||||
Syncthing pulls → Collector ingests + deletes → Syncthing replicates
|
||||
the delete back. One-shot semantics. Do not place permanent files
|
||||
in `incoming/`; they will disappear.
|
||||
|
||||
**Drop file schema:**
|
||||
|
||||
```json
|
||||
{
|
||||
"recipient": "bob",
|
||||
"type": "INFO",
|
||||
"priority": "normal",
|
||||
"payload": "...",
|
||||
"sentinel": "/path/optional"
|
||||
}
|
||||
```
|
||||
|
||||
On schema validation failure: log structured error and move the file
|
||||
to `~/Nyx/workspace/incoming/.dead-letter/` rather than silently
|
||||
dropping. External producers can monitor the dead-letter folder for
|
||||
their own malformed events.
|
||||
|
||||
Cron, sensors, and serial sources are explicitly **deferred to v2**.
|
||||
Pin v1 small.
|
||||
|
||||
### 3.2 Configuration
|
||||
|
||||
Single per-host file: `~/.config/agent-watcher/collector.yaml`.
|
||||
|
||||
```yaml
|
||||
agent: foreman # this host's identity
|
||||
|
||||
sources:
|
||||
webhook:
|
||||
listen: 127.0.0.1:18790
|
||||
routes:
|
||||
/forgejo/push:
|
||||
recipient: bob
|
||||
type: INFO
|
||||
payload_template: "forgejo push to {{ .repo }} by {{ .actor }}"
|
||||
/openrouter/billing-alert:
|
||||
recipient: bob
|
||||
type: NEEDS-RESPONSE
|
||||
priority: urgent
|
||||
payload_template: "billing alert: {{ .message }}"
|
||||
drop_folder:
|
||||
path: ~/Nyx/workspace/incoming/
|
||||
poll_fallback_seconds: 30 # safety net if inotify misses
|
||||
```
|
||||
|
||||
Hot-reload: SIGHUP re-reads config. Invalid config logs and keeps the
|
||||
old config running.
|
||||
|
||||
### 3.3 Output format
|
||||
|
||||
The Collector writes the same JSONL line the `ping` CLI writes today.
|
||||
The watcher and existing hook cannot tell the difference — every line
|
||||
in the inbox looks like a ping regardless of source.
|
||||
|
||||
Adds one optional field: `"source": "webhook:/forgejo/push"` for
|
||||
debugging.
|
||||
|
||||
### 3.4 Resource budget
|
||||
|
||||
Target: <15MB resident, <1% CPU at idle. Confirm on first deploy.
|
||||
Logs to journald with structured fields (`source`, `recipient`,
|
||||
`event_id`, `latency_ms`).
|
||||
|
||||
### 3.5 Health
|
||||
|
||||
`/health` endpoint on the same loopback port returns JSON with
|
||||
per-source counters (received, emitted, errors) and uptime. Useful
|
||||
for `journalctl --user -u agent-watcher` correlation.
|
||||
|
||||
---
|
||||
|
||||
## 4. Layer 2 — MCP Watcher
|
||||
|
||||
A Claude Code MCP server, declared in each agent's `mcp.json`, spawned
|
||||
as a subprocess at session start. Communicates over stdio.
|
||||
|
||||
### 4.1 Behavior
|
||||
|
||||
On startup:
|
||||
1. Read inbox file `~/Nyx/workspace/pings/<self>.inbox` and HWM file
|
||||
`<self>.hwm`.
|
||||
2. Emit a `notifications/claude/channel` event for any unread pings.
|
||||
3. Set up inotify on the inbox file.
|
||||
|
||||
On inotify fire (new line appended):
|
||||
4. Read the new line(s).
|
||||
5. Emit one `notifications/claude/channel` event per new ping.
|
||||
6. Advance HWM atomically (write-tmp-rename).
|
||||
7. Append to acks log.
|
||||
|
||||
### 4.2 Tools exposed
|
||||
|
||||
The MCP watcher exposes reply tools the agent can call:
|
||||
|
||||
- `ack(ping_id, message?)` — write an ack to the channel folder.
|
||||
- `respond(ping_id, payload)` — emit a response ping back to sender.
|
||||
- `mark_handled(ping_id)` — mark a ping as processed without replying
|
||||
(for INFO/ACK-REQUEST that don't need a response).
|
||||
|
||||
**Cross-host write discipline.** Reply tools always write to
|
||||
local-filesystem paths only (this host's inbox/ack/channel files);
|
||||
Syncthing replicates outward. The watcher must not write directly to
|
||||
a remote-host inbox path. Same rule as the `ping` CLI.
|
||||
|
||||
### 4.3 Coexistence with the existing hook (Bob's load-bearing #2)
|
||||
|
||||
**Decision: option (c) — MCP Watcher fully replaces the hook on hosts
|
||||
where it is running. Hook remains as fallback.**
|
||||
|
||||
Mechanism:
|
||||
- The MCP watcher writes a sentinel file
|
||||
`~/Nyx/workspace/pings/.<self>.watcher-active` on startup, removes
|
||||
it on shutdown.
|
||||
- The existing UserPromptSubmit hook checks for that sentinel; if
|
||||
present, the hook does nothing (the watcher is delivering).
|
||||
- If the watcher crashes or is not configured, the hook takes over —
|
||||
zero-config degradation back to next-prompt delivery.
|
||||
|
||||
**The sentinel must be local-only — Syncthing-excluded.** If it
|
||||
replicates, Foreman's watcher being up would silence Bob's hook even
|
||||
when Bob's watcher isn't running. Add to `.stignore` alongside the
|
||||
existing HWM exclusion:
|
||||
|
||||
```
|
||||
pings/.*.hwm
|
||||
pings/.*.watcher-active
|
||||
```
|
||||
|
||||
Same shape and same reasoning as the HWM-not-synced rule.
|
||||
|
||||
This avoids duplicate-delivery and HWM race conditions. One reader at
|
||||
any time.
|
||||
|
||||
### 4.4 Configuration
|
||||
|
||||
Per-agent `mcp.json` entry:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"agent-watcher": {
|
||||
"command": "agent-watcher-mcp",
|
||||
"args": ["--agent", "foreman"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The MCP watcher reads the same inbox/hwm/acks paths as the existing
|
||||
hook; no extra config.
|
||||
|
||||
---
|
||||
|
||||
## 5. Cross-host writes & Syncthing
|
||||
|
||||
Same discipline as the existing ping CLI: **each host only writes to
|
||||
local-filesystem inboxes**; Syncthing replicates outward. The Collector
|
||||
writes to `~/Nyx/workspace/pings/<recipient>.inbox` only on the
|
||||
*sender's* host. Recipient host receives it via Syncthing.
|
||||
|
||||
This means the recipient's MCP watcher sees inotify events from
|
||||
Syncthing's writes. Confirm Syncthing fires `IN_CLOSE_WRITE` reliably
|
||||
on rename — it should, but flag for first-deploy testing. If it
|
||||
doesn't, fall back to a 5-second poll on the inbox file.
|
||||
|
||||
JSONL conflict markers from concurrent writes are an existing risk
|
||||
documented in `agent-ping-system.md`. Not addressed in v1; flagged for
|
||||
v2 if hit in practice.
|
||||
|
||||
---
|
||||
|
||||
## 6. Channels research-preview status (Bob's load-bearing #1)
|
||||
|
||||
Channels is in research preview. The watcher's read/ack contract must
|
||||
survive API drift.
|
||||
|
||||
### 6.1 Fallback ladder
|
||||
|
||||
If Channels is healthy: MCP watcher delivers events mid-session via
|
||||
`notifications/claude/channel`. Reply tools work via MCP tool-call.
|
||||
|
||||
If Channels degrades or disappears: MCP watcher stops, sentinel file
|
||||
removed, the existing UserPromptSubmit hook resumes async-on-next-prompt
|
||||
delivery. Zero new code needed for fallback — the hook is already
|
||||
deployed.
|
||||
|
||||
If MCP changes incompatibly: the watcher is a single Go binary, easy
|
||||
to swap for a different push primitive. The Collector is unaffected.
|
||||
|
||||
### 6.2 Naming the value
|
||||
|
||||
The watcher's value is **during-turn delivery** (event surfaces while
|
||||
agent is active or wakes an idle agent). It is *not* "delivery at
|
||||
all" — that already works via the hook. Sets correct expectations.
|
||||
|
||||
---
|
||||
|
||||
## 7. Scope (Bob's load-bearing #3)
|
||||
|
||||
### v1 (this spec)
|
||||
- Collector with HTTP webhook + inotify-on-drop-folder sources only.
|
||||
- MCP watcher with inbox watching and three reply tools.
|
||||
- systemd --user unit, loopback-only HTTP, single YAML config.
|
||||
- Hook coexistence via sentinel file (option c).
|
||||
|
||||
### v2 (future, separate spec)
|
||||
- Cron source on the Collector.
|
||||
- Sensor sources (service health on VPS, hardware on Pi/phone).
|
||||
- Caddy + bearer-token webhook auth for cross-host posting.
|
||||
- TTL on emitted pings (`expires_at`); hook drops or warns on expired.
|
||||
- Coalesce dupes (e.g. 100 alert webhooks → one summary).
|
||||
- Cross-host JSONL conflict resolution.
|
||||
|
||||
### Out of scope, ever
|
||||
- OpenBrain integration. Watcher stays brain-blind. Agents capture.
|
||||
- Mid-turn preemption. Not possible in current Claude Code; not
|
||||
attempted.
|
||||
- Spawning new agents. Use Anthropic Routines for that pattern.
|
||||
|
||||
---
|
||||
|
||||
## 8. Install & ownership
|
||||
|
||||
Per CLAUDE.md rule 2 ("no agent edits its own code or config"), the
|
||||
watcher is installed on each agent's host by Angus from the canonical
|
||||
Forgejo repo. Same pattern as `agent-ping`.
|
||||
|
||||
Install script delivers:
|
||||
- `agent-watcher` binary → `~/.local/bin/`
|
||||
- `agent-watcher-mcp` binary → `~/.local/bin/`
|
||||
- systemd unit → `~/.config/systemd/user/agent-watcher.service`
|
||||
- Config skeleton → `~/.config/agent-watcher/collector.yaml`
|
||||
- mcp.json snippet → printed to stdout for manual paste into the
|
||||
agent's mcp config (Angus or Foreman applies)
|
||||
|
||||
Verify hooks: `agent-watcher --self-check` reports config validity,
|
||||
inbox path access, MCP binary presence.
|
||||
|
||||
---
|
||||
|
||||
## 9. Testing
|
||||
|
||||
- **Unit:** routing rules, payload templates, JSONL append correctness,
|
||||
HWM advancement.
|
||||
- **Integration (single host):** webhook POST → inbox line → MCP event
|
||||
→ agent sees `[INCOMING FROM …]`.
|
||||
- **Integration (cross-host):** Foreman ping CLI → Bob inbox via
|
||||
Syncthing → Bob MCP watcher → Bob sees event mid-session (target
|
||||
latency: <30s including Syncthing).
|
||||
- **Failover:** kill MCP watcher mid-session → hook takes over on next
|
||||
prompt. Restart watcher → hook silent again.
|
||||
|
||||
---
|
||||
|
||||
## 10. What I need from Angus + Bob to finalize
|
||||
|
||||
1. **Repo creation.** Confirm `agent-watcher` as Forgejo repo name and
|
||||
that I should match the layout of `agent-ping` (CLI + hook + spec
|
||||
in repo, install script at root).
|
||||
|
||||
2. **Webhook binding.** Confirm `127.0.0.1:18790` is acceptable as the
|
||||
v1 default port. (18790 chosen to sit just past the agent-ping
|
||||
range; happy to change.)
|
||||
|
||||
3. **Drop folder path.** Confirm `~/Nyx/workspace/incoming/` is the
|
||||
right location, or pick another. It should NOT be inside `pings/`.
|
||||
|
||||
4. **MCP binary name.** Confirm `agent-watcher-mcp` as a separate
|
||||
binary, vs. `agent-watcher mcp` as a subcommand. I lean separate
|
||||
binary — clearer in `ps`, simpler mcp.json.
|
||||
|
||||
5. **Health endpoint exposure.** Loopback-only fine, or do you want
|
||||
it gated behind a token even on loopback?
|
||||
|
||||
6. **Bob: confirm option (c) for hook coexistence** — watcher fully
|
||||
replaces hook when running, sentinel file gates the hook. If you
|
||||
prefer (a) shared HWM or (b) live-flag split, say so before code.
|
||||
|
||||
7. **First deploy host.** Foreman/laptop or Bob/VPS first? Foreman
|
||||
is lower-stakes for testing; Bob is where mobile webhooks will
|
||||
eventually land.
|
||||
|
||||
Answers fold into the spec; we don't iterate them in chat.
|
||||
Loading…
Add table
Add a link
Reference in a new issue