Data contracts are the API layer of the data platform

Most pipeline breakage has a single root cause: a producer changed a schema and nobody negotiated. A column renamed upstream, a type quietly widened, a field's meaning drifted while its name stayed put. In software engineering we solved this decades ago — you don't change a public API without a version bump, a deprecation window, and a contract test. In data, the equivalent discipline exists and has a name: the data contract. It's not a governance document. It's the API layer of the data platform.

The analogy is precise, not decorative. A real contract — the Open Data Contract Standard structures this well — carries the same sections an API spec does: schema (structure plus semantics), quality guarantees, SLAs, terms of use. Semantic versioning applies directly: additive changes are minor, breaking changes are major, and a major version demands a migration path. The migration patterns port over too — expand/contract parallel change, dual writes during a deprecation window, versioned views so consumers migrate at their own pace. Even contract testing has its analogue: tools like Data Contract CLI verify conformance the way Pact verifies a consumer-driven contract, and the nastiest failure class is identical — the silent breaking change, where the schema validates but the meaning shifted underneath it.

Enforcement at write, not detection at read

What separates a contract from a hopeful wiki page is enforcement at the write boundary. Delta Lake's schema-on-write already rejects structurally invalid appends; pipeline expectations extend that to semantics, with graduated actions — warn and retain, drop the offending row, or fail the update outright — and quarantine tables for what gets rejected. The point isn't rigidity; schema evolution is a feature, and Delta supports it well through mergeSchema and automatic MERGE evolution. The point is that evolution becomes a negotiated act with a version number instead of a surprise discovered by the consumer at 2 a.m. When enforcement is in place, contract violations become events to route and handle, not emergencies to triage — and the catalog, increasingly Unity Catalog in practice, becomes the contract registry with lineage that answers "who breaks if I ship this?"

This matters more now than it did five years ago, for one reason: the consumer is changing species. A human analyst notices when revenue looks off by 40x because units changed. A feature pipeline feeding a model does not — it degrades silently, and the model drifts with it. An agent querying data unsupervised has no institutional memory to smell that something's wrong. You cannot let autonomous systems consume data interfaces that have no semantics, no versioning, and no guarantees, for the same reason you wouldn't point production code at an undocumented endpoint that changes without notice. Contracts are the prerequisite for taking the human out of the read path — the same load-bearing role they play for AI-ready data generally, and for the semantic layer an LLM actually queries.

The honest caveat: contracts have a cost, and applying them everywhere is how the idea dies. A contract is negotiation overhead — producer commitment, versioning ceremony, enforcement infrastructure. Exploratory tables, raw Bronze ingestion, one-analyst scratch datasets don't warrant it; contract-everything mandates produce shelfware and resentment. Contracts belong at product boundaries: the interfaces where a producer team hands data to consumers it doesn't control. That's exactly where API discipline applies in software, and the judgment call is the same one.

Data teams spent a decade envying software engineering's tooling. The lesson worth stealing isn't the tooling — it's the refusal to change an interface without telling the people who depend on it.