The semantic layer is what your LLM actually queries
Text-to-SQL looks solved until you point it at a real schema. On Spider, the academic benchmark, the best systems clear 86% execution accuracy; the numbers I trust for real enterprise schemas show accuracy collapsing from the mid-nineties to around 39%. That gap is not a syntax problem — modern models write fluent SQL. It's a semantics problem, and the error analysis says so plainly: schema linking, picking the right tables and columns, accounts for 30–40% of all failures. The model isn't failing to write the query. It's failing to know what "revenue" means in your company.
Look at what a raw schema actually gives an LLM: table names abbreviated by a DBA in 2014, foreign keys that may or may not be declared, three columns that could each plausibly be "the" revenue figure, and no record of the tribal knowledge that active customers exclude internal test accounts. The model does what any newcomer does — it guesses. Confidently. The failure taxonomy for NL2DB reads like a new analyst's first month: wrong join path, wrong grain, WHERE-clause literals that don't match how values are stored, a metric computed three plausible ways of which one is right.
Move the meaning, not the prompt
The prompt-engineering response is to stuff the missing semantics into context: column descriptions, foreign-key hints, few-shot examples. It helps at the margins — FK hints alone are worth a few points of execution accuracy — but it's the wrong layer. You're maintaining your company's business definitions inside prompt templates, per application, with no governance and no reuse. The platform answer is the semantic layer: metrics defined once as governed contracts, entities and join paths declared explicitly, time-intelligence handled by the layer rather than reinvented per query. Define once, bind at runtime — the same principle that kills metric drift across BI dashboards kills it across LLM-generated queries. The measured effect is large: grounding natural-language querying in a semantic layer lifts accuracy from roughly 40% to 83% in the studies I've seen — a bigger jump than any model upgrade delivers.
The mechanism is worth stating precisely. Against a raw schema, the LLM must infer entities, joins, and metric definitions per query — every question re-derives the business model, and every re-derivation is a fresh chance to be wrong. Against a semantic layer, the model compiles a question into references to named, governed objects: a metric that already knows its formula and filters, entities whose join paths are declared, dimensions that are legal to group by. The search space shrinks from "any syntactically valid SQL" to "any valid combination of defined objects" — hallucination gives way to selection, which is the game LLMs are actually good at. The ecosystem is converging on this shape: MetricFlow and Cube as headless layers, warehouse-native semantic views, MCP emerging as the standard access path from agent to semantic layer, and the ladder extending upward from metrics through semantic graphs toward ontologies as agents need richer context.
The concession: a semantic layer only answers questions it has definitions for. The long tail of genuinely exploratory queries — new questions on unmodeled data — still needs raw-schema SQL, human review, and humility; and for a five-table schema with clean names, direct text-to-SQL may honestly be good enough. But that's the point in reverse. The semantic layer is modeling work, and it's the same modeling work your BI stack already needed — governed by the same discipline as data contracts one layer down. The LLM didn't create the requirement. It just removed the human who was quietly compensating for its absence.