Sutra Feed Guard — governed change control for external data feeds

External feed drift is a change-control and evidence problem, not merely a parsing problem. Sutra Feed Guard observes a CSV or JSON delivery, compares it with an explicitly accepted baseline, and produces a deterministic decision envelope before any production destination is touched.

Problem

External feeds rarely fail cleanly. A partner renames amount to total_amount, drops a required field, strips leading zeroes from an identifier, switches rupees to paise, or uploads a partial file that still parses. Traditional ingestion pipelines discover the change after transformation has begun — often after malformed semantics have crossed into a production table.

The category error is treating every change as an integration task. Before mapping or remediation, the system needs to answer a governance question: is this observation unchanged, an accepted variant, a safe additive change, something a human must review, or a breaking delivery that must be blocked?

Constraints

The build is deliberately local-first and observational. It must work with customer artifacts without sending them to a remote model. It must preserve decimal and identifier semantics instead of relying on implicit coercion. The same inputs and policy must produce the same disposition. AI assistance may propose a field mapping or unit-change hypothesis, but it may not widen the accepted baseline or override the deterministic result. Most importantly, Feed Guard does not transform or publish customer data: it stops at evidence and disposition.

Decisions

1. Observe before transforming. The build rejects the easier default of repairing a feed inline. It parses into a canonical local observation and classifies the difference before a downstream write is possible. That makes the blast radius explicit and keeps remediation separate from detection.
2. Compare canonical meaning, not raw bytes. Row order, field order, quoting, and JSON whitespace should not manufacture incidents. Canonical SHA-256 digests identify equivalent observations while preserving the schema and values that matter. This turns data contracts into an API layer rather than a brittle file-format check.
3. Keep the deterministic core authoritative. Policy rules classify eight concrete scenarios and return integration-friendly exit codes: pass, review, block, or invalid. Optional local AI can suggest aliases and unit traps, but every suggestion is logged and non-binding. It is the same boundary as a hard deterministic guardrail beneath a softer probabilistic layer.
4. Emit evidence, not just an alert. Each check produces a JSON envelope, a human-readable HTML report, exact affected records where parsing permits, content and policy digests, and an append-only journal entry. The incident packet can be replayed from the original artifacts instead of reconstructed from dashboard fragments.

Outcome

The working Python CLI accepts a baseline, loads CSV and JSON observations, canonicalizes them, applies a policy-driven decision tree, and emits stable dispositions for unchanged feeds, safe additions, known aliases, removed required fields, duplicate keys, identifier corruption, unit changes, and malformed deliveries. The CLI's exit codes make those decisions usable from cron, Airflow, Dagster, or an internal orchestrator without turning the classifier itself into a scheduler.

The reference implementation also includes local Ollama assistance for non-binding field-mapping and unit-trap suggestions, an immutable record of model interactions, HTML and JSON evidence reports, acquisition adapters for SFTP, S3, HTTPS, Azure Blob, and GCS, a self-guided demo, and a 38-test suite. The source is available at github.com/Dhristhi/sutra-feed-guard.

What you can run

git clone https://github.com/Dhristhi/sutra-feed-guard
cd sutra-feed-guard
uv sync --extra dev
uv run pytest -q
uv run feedguard accept \
  fixtures/sutra-feed-guard-fixtures-v1/historical/orders_2026-07-03.csv \
  fixtures/sutra-feed-guard-fixtures-v1/contracts/accepted_policy.yaml \
  -o .feedguard
for s in scripts/scenario_*.py; do uv run python "$s"; done

The honest caveat: this is a validated reference build, not a hosted control plane or evidence from production traffic. Its deterministic core and acquisition seams work; multi-user approvals, cryptographic signatures, scheduled operations, and customer-specific policy calibration still belong to a real deployment. The claim is narrower and more useful: external feed changes can be made reviewable and reproducible before they become production incidents.