Collector milestone 2: source interface + drop-folder source
source.Source is the contract every Collector input implements: Name + Run(ctx, emit). Sources don't own state — they convert external events into emit calls. Dispatcher routes. internal/source/dropfolder: watches ~/Nyx/workspace/incoming/ for *.json drop files. fsnotify-driven with periodic poll fallback (default 30s safety net for missed events). Each file: 1. Parsed against the spec §3.1.2 schema with DisallowUnknownFields. 2. Valid → emitted, then file deleted. 3. Invalid (missing fields, bad type/priority, unknown fields, garbage) → moved to .dead-letter/ with a sidecar .reason file for forensics. 4. Emit failure → file retained in place for retry (transient errors shouldn't be permanent dead-letters). Also: initial-scan on Run() drains files that landed before the watcher attached, catching up after a Collector restart. 14 tests in the package — schema validation table for all error cases, initial-scan, live inotify drop, post-emit delete, dead-letter + sidecar, emit-failure retention. Plus the 7 inbox tests still passing. Pinned fsnotify v1.7.0 (Go 1.22 compatible; 1.10.x demanded toolchain 1.23 which isn't in apt yet). go.mod stays at 1.22 to match VPS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
50e8ece83d
commit
f9d81471c4
5 changed files with 576 additions and 1 deletions
35
internal/source/source.go
Normal file
35
internal/source/source.go
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
// Package source defines the contract every Collector source plugin implements.
|
||||
//
|
||||
// A Source is a long-running goroutine that converts external events into
|
||||
// inbox writes. It receives an Emit callback at start; each external event it
|
||||
// observes results in one Emit call. The Source returns when its context is
|
||||
// canceled, or earlier on a fatal error.
|
||||
//
|
||||
// Sources do not own the inbox writer or any other shared state. They emit;
|
||||
// the dispatcher routes. This keeps each source small and testable in
|
||||
// isolation.
|
||||
package source
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"git.botbought.ai/foreman/agent-watcher/internal/inbox"
|
||||
)
|
||||
|
||||
// Emit is the callback a Source uses to push one observed event into the
|
||||
// dispatcher. The dispatcher writes it to recipient's inbox file. Returning
|
||||
// an error means the inbox write failed; sources may log and continue, or
|
||||
// drop, or shut down — that's a per-source policy decision.
|
||||
type Emit func(recipient string, ev *inbox.Event) error
|
||||
|
||||
// Source is a Collector input. Implementations are constructed with their
|
||||
// own configuration and started via Run.
|
||||
type Source interface {
|
||||
// Name is a short identifier for logs and the "source" field on emitted
|
||||
// events ("webhook", "drop-folder", etc.).
|
||||
Name() string
|
||||
|
||||
// Run blocks until ctx is canceled or a fatal error occurs. Each
|
||||
// observed external event becomes one Emit call.
|
||||
Run(ctx context.Context, emit Emit) error
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue