Collector milestone 6: packaging — install.sh, systemd unit, docs

systemd/agent-watcher.service: --user unit with on-failure restart, ProtectSystem=strict, ProtectHome=read-write, NoNewPrivileges=yes, PrivateTmp=yes. JSON logs to journald. Survives reboot via 'loginctl enable-linger'. examples/collector.yaml: working starter config for both sources with inline comments, per-route examples, and the spec §3.1.2 schema for drop files. install.sh: idempotent installer following the agent-ping pattern. Builds the binary, installs it + the unit, drops the example config if absent, reloads systemd, enables, and (unless --no-start) starts the service. Adds drop-folder lifecycle artifacts (*.tmp, .dead-letter/) to workspace .stignore so they don't replicate during processing. Skips Syncthing-related steps gracefully when ~/Nyx/workspace is not present. INSTALL.md: prerequisites, install, configure, verify (drop-file + webhook end-to-end probes), survive-logout, uninstall, troubleshooting table. README.md: rewritten to reflect actual status — v0 working with 43 tests, packaging ready, Layer 2 in progress on Bob's side. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 16:26:16 -04:00 · 2026-05-06 16:26:16 -04:00 · 4ff8c3f78d
commit 4ff8c3f78d
parent e7d4ea036a
5 changed files with 324 additions and 6 deletions
--- a/INSTALL.md
+++ b/INSTALL.md
@ -0,0 +1,95 @@
+# INSTALL — agent-watcher Collector (Layer 1)
+
+The MCP Watcher (Layer 2) has its own install path; this file covers Layer 1 only.
+
+## Prerequisites
+
+- Linux with systemd
+- Go 1.22+ (`sudo apt install golang-go` on Ubuntu/Mint)
+- An existing agent-ping install on this host (the Collector writes to its inbox dir)
+
+## Install
+
+Per CLAUDE.md rule #2, **Angus runs the install commands** — agents do not modify their own configuration.
+
+```sh
+git clone https://git.botbought.ai/foreman/agent-watcher ~/agent-watcher
+cd ~/agent-watcher
+./install.sh
+```
+
+The script builds, installs to `~/.local/bin/agent-watcher`, drops the systemd unit into `~/.config/systemd/user/`, copies the example config to `~/.config/agent-watcher/collector.yaml` (if absent), reloads systemd, and starts the service.
+
+To install without auto-starting (so you can edit the config first):
+
+```sh
+./install.sh --no-start
+```
+
+## Configure
+
+Edit `~/.config/agent-watcher/collector.yaml`. The example has both sources configured for `agent: foreman` with placeholder routes — adapt to this host. At least one of `webhook:` or `drop_folder:` must be configured.
+
+After editing:
+
+```sh
+systemctl --user restart agent-watcher
+```
+
+## Verify
+
+```sh
+# logs
+journalctl --user -u agent-watcher -f
+
+# health
+curl http://127.0.0.1:18790/health
+
+# end-to-end via drop folder
+echo '{"recipient":"bob","type":"INFO","payload":"hello from drop"}' \
+  > ~/Nyx/workspace/incoming/test.json
+
+# end-to-end via webhook
+curl -X POST http://127.0.0.1:18790/forgejo/push \
+  -H 'Content-Type: application/json' \
+  -d '{"repo":"agent-ping","actor":"angus"}'
+
+# the lines should appear in the recipient's inbox
+tail -2 ~/Nyx/workspace/pings/bob.inbox
+```
+
+## Survive logout
+
+`systemctl --user` units stop when the user logs out. To keep the Collector running across logouts:
+
+```sh
+sudo loginctl enable-linger $USER
+```
+
+## Uninstall
+
+```sh
+systemctl --user disable --now agent-watcher
+rm ~/.local/bin/agent-watcher
+rm ~/.config/systemd/user/agent-watcher.service
+systemctl --user daemon-reload
+# config and example left behind; remove if desired:
+# rm -rf ~/.config/agent-watcher
+```
+
+## Troubleshooting
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| `journalctl` shows `config: ... is required` | YAML field missing | Match the example, save, restart. |
+| `systemctl --user status` shows `address already in use` | port 18790 taken by something else | Edit `webhook.listen` to a free port. |
+| Drop file disappears but no inbox line | check `.dead-letter/` for the file + `.reason` sidecar | Schema validation failed — fix the producer. |
+| Webhook returns 404 | path doesn't match a configured route | Routes must match exactly; check trailing slashes. |
+| Service won't start across reboot | `linger` not enabled | `sudo loginctl enable-linger $USER` |
+| Drop file syncs back from another host before processing | drop folder is in Syncthing scope | This is intended (lets producers on other hosts deliver). Lifecycle artifacts (`.tmp`, `.dead-letter/`) are excluded by `install.sh`. |
+
+## What this does NOT install
+
+- The agent-ping system. Install that first (`~/agent-ping/install.sh <name>`).
+- Layer 2 (MCP Watcher). Different binary, different runtime, different unit — separate install when it lands.
+- Caddy / reverse-proxy webhook auth. v1 is loopback-only; v2 will document the upgrade path.
--- a/README.md
+++ b/README.md
@ -1,13 +1,13 @@
 # agent-watcher

-Push-delivery layer for [`agent-ping`](http://localhost:3300/angus/agent-ping). The "secondary nervous system" for Claude Code agents on this network.
+Push-delivery layer for [`agent-ping`](https://git.botbought.ai/foreman/agent-ping). The "secondary nervous system" for Claude Code agents on this network.

 `agent-ping` queues messages in inbox files; `agent-watcher` notices them (and other external events) and wakes the recipient agent without a human in the loop.

 Two layers:

- **Collector** — small Go daemon, `systemd --user`, always on, brain-blind. Converts external events (HTTP webhooks, drop-folder file arrivals) into ping inbox writes. Runs whether or not any agent is alive.
- **MCP Watcher** — Claude Code MCP subprocess, declared in each agent's `mcp.json`. Watches the agent's inbox via inotify and surfaces events into the live session via Channels (research preview). Provides reply tools (`ack`, `respond`, `mark_handled`).
+- **Collector** (this repo, Go) — small daemon under `systemd --user`. Always on, brain-blind. Converts external events (HTTP webhooks, drop-folder file arrivals) into ping inbox writes. Runs whether or not any agent is alive.
+- **MCP Watcher** (Python, in progress) — Claude Code MCP subprocess declared in each agent's `mcp.json`. Watches the agent's inbox via inotify and surfaces events into the live session via Channels. Provides reply tools (`ack`, `respond`, `mark_handled`).

 Filesystem is the queue. OpenBrain is not involved.

@ -15,10 +15,42 @@ Filesystem is the queue. OpenBrain is not involved.

 [`spec/agent-watcher.md`](spec/agent-watcher.md). Read that for architecture, decisions, scope.

+Channels reference docs (snapshot of Anthropic's official docs, used by Layer 2): [`docs/channels/`](docs/channels/).
+
 ## Status

-v1 spec signed off by Bob (VPS) 2026-05-06. Implementation pending.
+| Layer | Lane | Status |
+|---|---|---|
+| Spec v1 | — | Signed off by Bob 2026-05-06 |
+| Layer 1: Collector | Foreman / Go | **v0 working: 43 tests passing.** End-to-end exercised; binary builds. systemd unit + INSTALL.md ready. |
+| Layer 2: MCP Watcher | Bob / Python | In progress — sandbox CC session being set up on VPS for testing. |

-## Install
+## Install (Layer 1)

-Per CLAUDE.md rule #2, **Angus runs the install commands** — agents do not modify their own configuration. Install script will land alongside `INSTALL.md` once the binaries are built.
+```sh
+git clone https://git.botbought.ai/foreman/agent-watcher ~/agent-watcher
+cd ~/agent-watcher
+./install.sh
+```
+
+Then edit `~/.config/agent-watcher/collector.yaml` and `systemctl --user restart agent-watcher`.
+
+See [`INSTALL.md`](INSTALL.md) for verify steps, troubleshooting, and the `loginctl enable-linger` step required to keep the daemon running across logouts.
+
+Per CLAUDE.md rule #2, **Angus runs the install commands** — agents do not modify their own configuration.
+
+## Quick reference (Layer 1)
+
+```
+Inputs                          Output
+─────────                       ──────
+HTTP POST → port 18790      ┐
+              (routed via   │   <recipient>.inbox  (JSONL, ping-shaped)
+               YAML table)  │   identical format to
+                            ├─→ what `ping <recipient> <payload>` writes;
+File drop in                │   the existing UserPromptSubmit hook and the
+~/Nyx/workspace/incoming/   │   future MCP Watcher consume the stream
+*.json                      ┘   without distinguishing source.
+```
+
+`/health` on the same webhook port returns `{received, emitted, errors, uptime_sec}` for journalctl correlation.
--- a/examples/collector.yaml
+++ b/examples/collector.yaml
@ -0,0 +1,51 @@
+# agent-watcher Collector configuration
+#
+# Lives at: ~/.config/agent-watcher/collector.yaml
+# Override with --config or AGENT_WATCHER_CONFIG.
+#
+# At least one source (webhook OR drop_folder) must be configured.
+
+# This host's identity. Used in logs only; the inbox writer routes by
+# the recipient field on each event, not this.
+agent: foreman
+
+# Optional. Where to write <recipient>.inbox files. Default shown.
+# inbox_dir: ~/Nyx/workspace/pings
+
+sources:
+
+  # HTTP webhook source.
+  # v1 binds loopback only — Caddy + bearer-token reverse-proxy is the
+  # v2 upgrade path for accepting webhooks from external producers.
+  webhook:
+    listen: 127.0.0.1:18790
+    routes:
+      # Path → which inbox to land in, with a Go text/template payload.
+      # Variables come from the request body decoded as JSON.
+      /forgejo/push:
+        recipient: bob
+        type: INFO
+        payload_template: "forgejo push to {{ .repo }} by {{ .actor }}"
+
+      # Empty-body posts work too — fixed-string templates render without
+      # any data.
+      /openrouter/billing-alert:
+        recipient: bob
+        type: NEEDS-RESPONSE
+        priority: urgent
+        payload_template: "billing alert: {{ .message }}"
+
+  # Drop-folder source.
+  # Watches a directory via inotify for *.json files matching the spec
+  # §3.1.2 schema:
+  #   {
+  #     "recipient": "bob",
+  #     "type":      "INFO" | "NEEDS-RESPONSE" | "ACK-REQUEST",
+  #     "priority":  "normal" | "urgent",          # optional
+  #     "payload":   "...",
+  #     "sentinel":  "/path/optional"              # optional
+  #   }
+  # Valid → emit + delete. Invalid → moved to .dead-letter/ with reason.
+  drop_folder:
+    path: ~/Nyx/workspace/incoming/
+    poll_fallback_seconds: 30   # safety net if inotify misses
--- a/install.sh
+++ b/install.sh
@ -0,0 +1,114 @@
+#!/usr/bin/env bash
+# install.sh — set up the agent-watcher Collector on this machine.
+#
+# Usage:
+#   ./install.sh                  # build + install + enable + start
+#   ./install.sh --no-start       # everything but `systemctl start`
+#
+# What it does:
+#   1. Builds the agent-watcher binary (requires Go 1.22+).
+#   2. Installs to ~/.local/bin/agent-watcher.
+#   3. Drops the systemd --user unit into ~/.config/systemd/user/.
+#   4. Drops a starter ~/.config/agent-watcher/collector.yaml if missing
+#      (copies examples/collector.yaml; you edit it before first start).
+#   5. Reloads systemd, enables, and (unless --no-start) starts the unit.
+#   6. Adds drop-folder lifecycle patterns to the workspace .stignore so
+#      drop files do not Syncthing-replicate during local processing.
+#
+# Per CLAUDE.md rule #2, this is intended to be run by a human. The
+# Collector itself, like agent-ping, never invokes this script.
+#
+# Layer 2 (MCP Watcher) has its own install path — different language,
+# different runtime, different unit. See its README when it lands.
+
+set -euo pipefail
+
+NO_START=0
+for arg in "$@"; do
+    case "$arg" in
+        --no-start) NO_START=1 ;;
+        *) echo "unknown arg: $arg" >&2; exit 2 ;;
+    esac
+done
+
+REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
+BIN_DIR="$HOME/.local/bin"
+CONF_DIR="$HOME/.config/agent-watcher"
+UNIT_DIR="$HOME/.config/systemd/user"
+WORKSPACE="$HOME/Nyx/workspace"
+STIGNORE="$WORKSPACE/.stignore"
+
+echo "agent-watcher install"
+echo "repo: $REPO_DIR"
+echo
+
+# 1. Check Go
+echo "[1/6] checking Go toolchain"
+if ! command -v go >/dev/null 2>&1; then
+    echo "  ERROR: 'go' not found. Install Go 1.22+ first." >&2
+    echo "  e.g. sudo apt install golang-go" >&2
+    exit 1
+fi
+GO_VERSION=$(go version | awk '{print $3}')
+echo "  $GO_VERSION"
+
+# 2. Build
+echo "[2/6] building agent-watcher binary"
+mkdir -p "$BIN_DIR"
+( cd "$REPO_DIR" && go build -o "$BIN_DIR/agent-watcher" ./cmd/agent-watcher )
+echo "  installed: $BIN_DIR/agent-watcher"
+
+# 3. systemd unit
+echo "[3/6] installing systemd --user unit"
+mkdir -p "$UNIT_DIR"
+install -m 0644 "$REPO_DIR/systemd/agent-watcher.service" "$UNIT_DIR/agent-watcher.service"
+echo "  installed: $UNIT_DIR/agent-watcher.service"
+
+# 4. Config skeleton
+echo "[4/6] config skeleton"
+mkdir -p "$CONF_DIR"
+if [ -f "$CONF_DIR/collector.yaml" ]; then
+    echo "  $CONF_DIR/collector.yaml exists — leaving as-is"
+else
+    install -m 0644 "$REPO_DIR/examples/collector.yaml" "$CONF_DIR/collector.yaml"
+    echo "  installed example: $CONF_DIR/collector.yaml — EDIT BEFORE START"
+fi
+
+# 5. .stignore — drop folder is replicated, but lifecycle artifacts shouldn't be
+echo "[5/6] .stignore (drop-folder lifecycle artifacts are local-only)"
+if [ -d "$WORKSPACE" ]; then
+    touch "$STIGNORE"
+    for pattern in "incoming/.dead-letter/" "incoming/*.json.tmp"; do
+        if ! grep -qxF "$pattern" "$STIGNORE"; then
+            echo "$pattern" >> "$STIGNORE"
+            echo "  added: $pattern"
+        fi
+    done
+else
+    echo "  $WORKSPACE not present — skipping (Syncthing not set up here)"
+fi
+
+# 6. systemd reload + enable + (optionally) start
+echo "[6/6] systemd reload + enable"
+systemctl --user daemon-reload
+systemctl --user enable agent-watcher.service >/dev/null
+echo "  enabled at user login (loginctl enable-linger required to run when logged out)"
+
+if [ "$NO_START" -eq 0 ]; then
+    echo "  starting…"
+    systemctl --user restart agent-watcher.service
+    sleep 1
+    systemctl --user status agent-watcher.service --no-pager -n 5 || true
+else
+    echo "  --no-start: not starting; run 'systemctl --user start agent-watcher' when ready"
+fi
+
+echo
+echo "installation complete."
+echo
+echo "next steps:"
+echo "  1. EDIT $CONF_DIR/collector.yaml to suit this host."
+echo "  2. systemctl --user restart agent-watcher (after edits)."
+echo "  3. journalctl --user -u agent-watcher -f  (watch logs)."
+echo "  4. curl http://127.0.0.1:18790/health      (sanity check)."
+echo "  5. To survive logout: loginctl enable-linger \$USER"
--- a/systemd/agent-watcher.service
+++ b/systemd/agent-watcher.service
@ -0,0 +1,26 @@
+[Unit]
+Description=agent-watcher Collector — converts external events to ping inbox writes
+Documentation=https://git.botbought.ai/foreman/agent-watcher
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStart=%h/.local/bin/agent-watcher --json-log
+Restart=on-failure
+RestartSec=5
+# small daemon; no need for elevated limits
+LimitNOFILE=4096
+# read-only by intent; the daemon writes only to the inbox dir which is
+# inside $HOME and unaffected by ProtectSystem.
+ProtectSystem=strict
+ProtectHome=read-write
+PrivateTmp=yes
+NoNewPrivileges=yes
+# stdout/stderr go to journald automatically; --json-log makes them parseable
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=agent-watcher
+
+[Install]
+WantedBy=default.target