# Agent Watcher — v1 Spec

**Author:** Foreman (laptop)
**Date:** 2026-05-06
**Status:** Draft for Bob review, then Angus sign-off
**Sibling to:** `agent-ping-system.md` (this layer extends it)
**Repo (planned):** Forgejo `agent-watcher`, sibling to `agent-ping`

---

## 1. Problem

The ping system is async-on-next-prompt: pings land in inbox files but the
recipient agent only sees them when something else triggers a prompt. If
nobody types at the agent, pings sit unread.

We want push semantics — external events (a ping landing, a webhook firing)
wake the recipient agent without a human in the loop. The watcher is the
"secondary nervous system": its only job is to notice events and notify the
right agent.

---

## 2. Architecture

Two layers, one durable substrate.

```
                            ┌──────────────────────────┐
external event   ─────►     │   Collector (Go daemon)  │
(webhook, file              │   systemd --user, always │
 arrival, future            │   on, brain-blind        │
 sources)                   └────────────┬─────────────┘
                                         │ writes
                                         ▼
                          ~/Nyx/workspace/pings/<r>.inbox     (durable queue)
                                         │
                                         │ inotify
                                         ▼
                            ┌──────────────────────────┐
                            │   MCP Watcher            │ ──► Channels event
                            │   (subprocess of Claude  │     into agent session
                            │   Code via mcp.json)     │
                            └──────────────────────────┘
                                         │
                                         │ falls back to
                                         ▼
                            UserPromptSubmit hook (existing agent-ping)
                            on hosts where MCP Watcher is not running
```

Key properties:

- **Filesystem is the queue.** Inbox files are the source of truth.
  Both layers read/write the same files the `ping` CLI already uses.
  No new persistence dependency.
- **Brain-blind.** The watcher does not capture to OpenBrain. Receiving
  agents capture deliberately. (Same poison-loop rule as compaction
  summaries.)
- **Two-layer split = uptime independence.** Collector runs whether or
  not any agent is alive. Events captured to disk regardless.
- **Channels, not RemoteTrigger.** RemoteTrigger is deferred/unshipped.
  Channels (MCP-over-stdio, research preview, Claude Code v2.1.80+) is
  the actual primitive. See §6 for fallback strategy.

---

## 3. Layer 1 — Collector

A small Go daemon, single static binary, runs as `systemd --user` unit on
every host. Cross-compiles for Linux/Android/Pi from one source.

### 3.1 v1 sources

Two only:

1. **HTTP webhook listener** — bind `127.0.0.1:18790` (loopback only
   in v1; Caddy + bearer-token reverse proxy is the v2 upgrade path).
   Receives POSTs, applies a routing table, writes a JSONL line to the
   target inbox file.

2. **inotify on a designated drop folder** — `~/Nyx/workspace/incoming/`
   (NOT the inbox itself; that would loop). External producers drop
   `.json` files; the Collector ingests, writes to inbox, deletes the
   drop file.

   The drop folder is Syncthing-replicated (it lives under `workspace/`),
   which is intentional: a producer on any host can drop into another
   host's Collector. Lifecycle: producer drops `event-123.json` →
   Syncthing pulls → Collector ingests + deletes → Syncthing replicates
   the delete back. One-shot semantics. Do not place permanent files
   in `incoming/`; they will disappear.

   **Drop file schema:**

   ```json
   {
     "recipient": "bob",
     "type": "INFO",
     "priority": "normal",
     "payload": "...",
     "sentinel": "/path/optional"
   }
   ```

   On schema validation failure: log structured error and move the file
   to `~/Nyx/workspace/incoming/.dead-letter/` rather than silently
   dropping. External producers can monitor the dead-letter folder for
   their own malformed events.

Cron, sensors, and serial sources are explicitly **deferred to v2**.
Pin v1 small.

### 3.2 Configuration

Single per-host file: `~/.config/agent-watcher/collector.yaml`.

```yaml
agent: foreman   # this host's identity

sources:
  webhook:
    listen: 127.0.0.1:18790
    routes:
      /forgejo/push:
        recipient: bob
        type: INFO
        payload_template: "forgejo push to {{ .repo }} by {{ .actor }}"
      /openrouter/billing-alert:
        recipient: bob
        type: NEEDS-RESPONSE
        priority: urgent
        payload_template: "billing alert: {{ .message }}"
  drop_folder:
    path: ~/Nyx/workspace/incoming/
    poll_fallback_seconds: 30   # safety net if inotify misses
```

Hot-reload: SIGHUP re-reads config. Invalid config logs and keeps the
old config running.

### 3.3 Output format

The Collector writes the same JSONL line the `ping` CLI writes today.
The watcher and existing hook cannot tell the difference — every line
in the inbox looks like a ping regardless of source.

Adds one optional field: `"source": "webhook:/forgejo/push"` for
debugging.

### 3.4 Resource budget

Target: <15MB resident, <1% CPU at idle. Confirm on first deploy.
Logs to journald with structured fields (`source`, `recipient`,
`event_id`, `latency_ms`).

### 3.5 Health

`/health` endpoint on the same loopback port returns JSON with
per-source counters (received, emitted, errors) and uptime. Useful
for `journalctl --user -u agent-watcher` correlation.

---

## 4. Layer 2 — MCP Watcher

A Claude Code MCP server, declared in each agent's `mcp.json`, spawned
as a subprocess at session start. Communicates over stdio.

### 4.1 Behavior

On startup:
1. Read inbox file `~/Nyx/workspace/pings/<self>.inbox` and HWM file
   `<self>.hwm`.
2. Emit a `notifications/claude/channel` event for any unread pings.
3. Set up inotify on the inbox file.

On inotify fire (new line appended):
4. Read the new line(s).
5. Emit one `notifications/claude/channel` event per new ping.
6. Advance HWM atomically (write-tmp-rename).
7. Append to acks log.

### 4.2 Tools exposed

The MCP watcher exposes reply tools the agent can call:

- `ack(ping_id, message?)` — write an ack to the channel folder.
- `respond(ping_id, payload)` — emit a response ping back to sender.
- `mark_handled(ping_id)` — mark a ping as processed without replying
  (for INFO/ACK-REQUEST that don't need a response).

**Cross-host write discipline.** Reply tools always write to
local-filesystem paths only (this host's inbox/ack/channel files);
Syncthing replicates outward. The watcher must not write directly to
a remote-host inbox path. Same rule as the `ping` CLI.

### 4.3 Coexistence with the existing hook (Bob's load-bearing #2)

**Decision: option (c) — MCP Watcher fully replaces the hook on hosts
where it is running. Hook remains as fallback.**

Mechanism:
- The MCP watcher writes a sentinel file
  `~/Nyx/workspace/pings/.<self>.watcher-active` on startup, removes
  it on shutdown.
- The existing UserPromptSubmit hook checks for that sentinel; if
  present, the hook does nothing (the watcher is delivering).
- If the watcher crashes or is not configured, the hook takes over —
  zero-config degradation back to next-prompt delivery.

**The sentinel must be local-only — Syncthing-excluded.** If it
replicates, Foreman's watcher being up would silence Bob's hook even
when Bob's watcher isn't running. Add to `.stignore` alongside the
existing HWM exclusion:

```
pings/.*.hwm
pings/.*.watcher-active
```

Same shape and same reasoning as the HWM-not-synced rule.

This avoids duplicate-delivery and HWM race conditions. One reader at
any time.

### 4.4 Configuration

Per-agent `mcp.json` entry:

```json
{
  "mcpServers": {
    "agent-watcher": {
      "command": "agent-watcher-mcp",
      "args": ["--agent", "foreman"]
    }
  }
}
```

The MCP watcher reads the same inbox/hwm/acks paths as the existing
hook; no extra config.

---

## 5. Cross-host writes & Syncthing

Same discipline as the existing ping CLI: **each host only writes to
local-filesystem inboxes**; Syncthing replicates outward. The Collector
writes to `~/Nyx/workspace/pings/<recipient>.inbox` only on the
*sender's* host. Recipient host receives it via Syncthing.

This means the recipient's MCP watcher sees inotify events from
Syncthing's writes. Confirm Syncthing fires `IN_CLOSE_WRITE` reliably
on rename — it should, but flag for first-deploy testing. If it
doesn't, fall back to a 5-second poll on the inbox file.

JSONL conflict markers from concurrent writes are an existing risk
documented in `agent-ping-system.md`. Not addressed in v1; flagged for
v2 if hit in practice.

---

## 6. Channels research-preview status (Bob's load-bearing #1)

Channels is in research preview. The watcher's read/ack contract must
survive API drift.

### 6.1 Fallback ladder

If Channels is healthy: MCP watcher delivers events mid-session via
`notifications/claude/channel`. Reply tools work via MCP tool-call.

If Channels degrades or disappears: MCP watcher stops, sentinel file
removed, the existing UserPromptSubmit hook resumes async-on-next-prompt
delivery. Zero new code needed for fallback — the hook is already
deployed.

If MCP changes incompatibly: the watcher is a single Go binary, easy
to swap for a different push primitive. The Collector is unaffected.

### 6.2 Naming the value

The watcher's value is **during-turn delivery** (event surfaces while
agent is active or wakes an idle agent). It is *not* "delivery at
all" — that already works via the hook. Sets correct expectations.

---

## 7. Scope (Bob's load-bearing #3)

### v1 (this spec)
- Collector with HTTP webhook + inotify-on-drop-folder sources only.
- MCP watcher with inbox watching and three reply tools.
- systemd --user unit, loopback-only HTTP, single YAML config.
- Hook coexistence via sentinel file (option c).

### v2 (future, separate spec)
- Cron source on the Collector.
- Sensor sources (service health on VPS, hardware on Pi/phone).
- Caddy + bearer-token webhook auth for cross-host posting.
- TTL on emitted pings (`expires_at`); hook drops or warns on expired.
- Coalesce dupes (e.g. 100 alert webhooks → one summary).
- Cross-host JSONL conflict resolution.

### Out of scope, ever
- OpenBrain integration. Watcher stays brain-blind. Agents capture.
- Mid-turn preemption. Not possible in current Claude Code; not
  attempted.
- Spawning new agents. Use Anthropic Routines for that pattern.

---

## 8. Install & ownership

Per CLAUDE.md rule 2 ("no agent edits its own code or config"), the
watcher is installed on each agent's host by Angus from the canonical
Forgejo repo. Same pattern as `agent-ping`.

Install script delivers:
- `agent-watcher` binary → `~/.local/bin/`
- `agent-watcher-mcp` binary → `~/.local/bin/`
- systemd unit → `~/.config/systemd/user/agent-watcher.service`
- Config skeleton → `~/.config/agent-watcher/collector.yaml`
- mcp.json snippet → printed to stdout for manual paste into the
  agent's mcp config (Angus or Foreman applies)

Verify hooks: `agent-watcher --self-check` reports config validity,
inbox path access, MCP binary presence.

---

## 9. Testing

- **Unit:** routing rules, payload templates, JSONL append correctness,
  HWM advancement.
- **Integration (single host):** webhook POST → inbox line → MCP event
  → agent sees `[INCOMING FROM …]`.
- **Integration (cross-host):** Foreman ping CLI → Bob inbox via
  Syncthing → Bob MCP watcher → Bob sees event mid-session (target
  latency: <30s including Syncthing).
- **Failover:** kill MCP watcher mid-session → hook takes over on next
  prompt. Restart watcher → hook silent again.

---

## 10. What I need from Angus + Bob to finalize

1. **Repo creation.** Confirm `agent-watcher` as Forgejo repo name and
   that I should match the layout of `agent-ping` (CLI + hook + spec
   in repo, install script at root).

2. **Webhook binding.** Confirm `127.0.0.1:18790` is acceptable as the
   v1 default port. (18790 chosen to sit just past the agent-ping
   range; happy to change.)

3. **Drop folder path.** Confirm `~/Nyx/workspace/incoming/` is the
   right location, or pick another. It should NOT be inside `pings/`.

4. **MCP binary name.** Confirm `agent-watcher-mcp` as a separate
   binary, vs. `agent-watcher mcp` as a subcommand. I lean separate
   binary — clearer in `ps`, simpler mcp.json.

5. **Health endpoint exposure.** Loopback-only fine, or do you want
   it gated behind a token even on loopback?

6. **Bob: confirm option (c) for hook coexistence** — watcher fully
   replaces hook when running, sentinel file gates the hook. If you
   prefer (a) shared HWM or (b) live-flag split, say so before code.

7. **First deploy host.** Foreman/laptop or Bob/VPS first? Foreman
   is lower-stakes for testing; Bob is where mobile webhooks will
   eventually land.

Answers fold into the spec; we don't iterate them in chat.