Founders

How to Review an Agent's Schema Guess Before It Writes SQL

Antonio Evans7 min read

A person hands a robot a few index cards showing simplified table outlines with join-key symbols, and the robot holds up a rough boxes-and-arrows diagram with one arrow marked by a red question mark.
Key points

Key points

  1. 1Most wrong SQL from an agent comes from a wrong reading of the schema, not from bad SQL syntax.
  2. 2Give the model a trimmed schema card: only the tables and columns it needs, each with a plain description and the join keys marked.
  3. 3Ask the model to restate the tables, columns, types, relationships and filters it inferred, then compare that restatement to the real schema and the question.
  4. 4When the restatement is wrong, fix the schema card or the data descriptions, not the prompt, and only then allow query generation.

Why the schema read fails first

An agent writing SQL does two jobs in one pass: it reads your schema and it writes a statement. When the output is wrong, most reviewers attack the second job, rewriting the prompt, adding examples, or asking for a retry. The evidence points the other way. A survey of LLM-based text-to-SQL systems names schema understanding as a core difficulty: real schemas are large and vary by domain, and mapping a plain question onto the right tables and columns is where systems break the text-to-SQL review. Research on schema linking goes further, reporting that schema information plays a significant or even dominant role in text-to-SQL quality, and that feeding a whole database schema raises token cost and increases hallucination and error rates X-SQL on schema linking.

The practical consequence is that a query can be perfectly valid SQL and still answer the wrong question. A practitioner guide to text-to-SQL accuracy separates failures into structural errors (syntax, wrong join type, missing GROUP BY) and semantic errors (valid SQL that answers the wrong question, such as filtering on order_date instead of ship_date). It calls semantic errors the silent killers, because the database engine never complains text-to-SQL accuracy practices. A wrong schema read produces exactly this class of error: the agent picks order_adjustments when the question said refunds, or ship_region when the business means customer_region, and the query runs clean and returns a confident wrong number.

There is a second reason to start one step earlier. Once an agent has committed to a table choice inside a generated query, that choice is buried in a join clause and looks deliberate. Reviewers tend to check whether the SQL is well formed and whether the number seems plausible, not whether the agent understood the column. Pulling the schema read out into a separate, reviewable step makes the assumption visible while it is still cheap to correct.

Build the trimmed schema card

A schema card is a compact, written description of the part of the database the agent needs for one question. It is not the full DDL dump. The goal is to remove every table and column that is irrelevant, and to explain every column that remains in plain language.

Annotation is not decoration. The same accuracy guide shows augmented schema passed to a model with inline comments such as a foreign key marker and a note that a status column holds the lifecycle values pending, fulfilled, refunded and cancelled. Those comments steer the model toward the right table and the right filter values text-to-SQL accuracy practices.

A usable card for a retention question looks like this:

-- Table: users (one row per registered account)
--   user_id      int primary key, unique per account
--   signup_date  date, when the account was created (use for cohorts)
--   country      text, billing country at signup

-- Table: user_events (one row per action taken in the product)
--   event_id     int primary key
--   user_id      int, foreign key -> users.user_id
--   event_date   date, when the action happened
--   event_type   text, one of: login | purchase | view
-- join key: user_events.user_id = users.user_id  (many events per user)

Four rules make the card do its job:

  • Trim to the tables the question plausibly needs. Adding two extra tables "just in case" invites the agent to join them.
  • Describe every column in business terms. "Signup date" is clear; a column named created_at on four different tables is not.
  • Mark join keys on both sides and state the cardinality (one row per user, many events per user). The cohort walkthroughs depend on joining activity back to the signup month, so a wrong key silently changes every retained user count a SQL cohort walkthrough.
  • Name the allowed values for any status or type column, so filters use real values instead of invented ones.
Card elementKeep it whenDrop it whenWhy it matters
TableThe question could reference it in one join hopIt only touches the answer through a second joinFewer tables means fewer wrong picks
ColumnIt is selected, filtered, grouped or joinedIt is never referencedNoise competes with signal
DescriptionIt disambiguates a nameThe name is already unambiguousPrevents semantic errors
Join keyIt links two kept tablesOne side is not on the cardStops invented joins
Allowed valuesA column holds a fixed setThe column is free textStops invented filters

Ask the agent to restate the schema before writing SQL

The core technique is a restatement step. Hand the agent the schema card and the question, and ask for no SQL yet. Ask for its understanding in a fixed shape: the tables it will use, the columns, what each column means in its own words, how the tables join, and the filters it plans to apply. The model does not need a schema to produce a sentence like this, and because it never sees the full DDL, it cannot quietly fall back on guessing.

A prompt that works in practice:

You are given a schema card and a question. Do NOT write SQL.

Return exactly these sections:

1. Tables I will use, and why each is needed.
2. Columns I will use, with a one-line meaning for each.
3. How the tables join (column = column, and cardinality).
4. Filters I plan to apply, with the exact column and value.
5. Anything in the question that is ambiguous and how I will resolve it.

Question: <the user question>
Schema card: <the trimmed card>

The ambiguity section matters. Natural language is less precise than SQL, and a model faced with an unclear term tends to answer anyway rather than ask, which produces hallucinated intent Google Cloud on text-to-SQL. A question like "best selling" has no single meaning until someone decides whether it means most orders or most revenue, and the restatement is where that decision gets surfaced. If the model states its reading and you agree, the eventual SQL is grounded. If it states a reading you did not intend, you have caught the failure before a single row was scanned.

The review checklist

Now compare the restatement to two references: the actual schema and the question. Work through five checks in order.

  • Tables. Does every table the agent named exist, and is every table it needs present? A missing user_events table means retention cannot be computed at all, however good the query looks.
  • Columns. Do the named columns exist on the named tables? This is where an agent invents a plausible column such as last_login that no one ever created.
  • Types and meaning. Does the agent's stated meaning match the documented one? A date stored as a timestamp with a time zone will behave differently in date truncation than a plain date. Getting this wrong shifts events into the wrong month and changes every retention percentage.
  • Relationships. Is each join a real foreign key or a documented link, with the right cardinality? A many-to-many link treated as one-to-many duplicates rows and inflates counts.
  • Filters. Are the filter values drawn from the allowed set, and do they match the question's intent? "Refunded orders" should filter on the refund type, not on order status, if those differ.

A worked example makes the value clear. Suppose the question is "what share of users who signed up in January were still active three months later?" and there are 1,000 users in the January cohort. The agent restates the join as user_events.user_id = users.user_id and plans to filter on users.signup_date. That reading is correct, and the eventual query can bucket users by signup month exactly as the standard cohort pattern does a retention SQL guide. Now suppose the agent instead restates the join as user_events.event_date = users.signup_date. That runs, returns numbers, and answers a different question entirely: it only counts users active on the day they signed up. Reviewing the restatement catches this in seconds. Reviewing the finished SQL, with a join condition buried in a WITH clause, often does not.

Fix the card, not the prompt

The instinct when the restatement is wrong is to rewrite the prompt. Resist it. If the agent misread the schema, the fault is in the schema card or the data, and a prompt tweak will paper over it for this question and fail on the next one.

Use this decision rule:

What the restatement got wrongWhere to fix it
Picked a table you did not wantRemove that table from the card, or add a sentence saying which table to prefer and why
Invented a columnThe real column is missing or badly named; add the real column with a description
Stated the wrong meaningRewrite the column description in plain business terms
Used the wrong join keyMark the correct key explicitly and state the cardinality
Applied an invented filter valueList the allowed values on the card

Two supporting habits pay off. First, validate the underlying values, not just the column names. Data validation is the practice of checking that data is complete, unique and consistent with expectations before it is used, and running those checks on the columns in your card tells you whether the agent will see clean values or nulls and duplicates simple data validation in SQL. Second, when the agent eventually does generate SQL, run it against a read-only account with a statement timeout, so a misread join cannot lock a production table while you debug creating a read-only Postgres user. A misread schema that turns into a full scan is a performance incident on top of a wrong answer.

If you want the schema card and the restatement step wired into a working pipeline rather than hand-rolled, a specialist can scope it with you: describe the pipeline you need.

Where this fits with the rest of the stack

The schema review does not replace query-level checks, it precedes them. Once the restatement matches, the agent's SQL can be tested the usual ways. Use EXPLAIN ANALYZE to see whether the plan does what the query's author intended, and remember that it actually executes the statement, so wrap writes in a transaction and roll back PostgreSQL EXPLAIN. Confirm that the row count the agent assumed matches reality rather than trusting a hand-written estimate counting rows after a SELECT. None of these steps will tell you that the agent joined the wrong table on purpose. Only the schema restatement does that.

Teams that run many agent questions over the same warehouse get compounding value from the card. Each wrong restatement produces one card fix, and the card improves for every future question against those tables. That is the opposite of prompt patching, where each fix is local to one question and invisible to the next.

FAQ

FAQ

Why not just read the generated SQL and check it?

Because the wrong assumption is hidden inside a valid statement. A join on the wrong column, or a filter on a column whose meaning the agent guessed, produces SQL that runs without error and returns plausible numbers. The restatement surfaces that assumption as a plain sentence you can argue with, before it is buried in a WITH clause.

How big should the schema card be?

Small enough that a person could read it in under a minute. Keep only the tables the question could plausibly need, and describe every column that remains. Research on schema linking finds that passing whole database schemas raises token cost and increases hallucination, so trimming is part of the accuracy work, not a convenience X-SQL on schema linking.

What if the agent's restatement is vague rather than wrong?

Treat vagueness as a failure. "Joins users to events" is not a reviewable claim; "user_events.user_id equals users.user_id, many events per user" is. Tighten the prompt's required output shape so each section must name a column, and fix the card if the agent cannot be specific because the card itself is vague.

Does this work for questions the card does not cover?

That is the signal to expand the card deliberately, not to let the agent guess. When a new question needs a table you left out, add that table and its descriptions as a reviewed change. The card then acts as a record of which parts of the schema your agents are allowed to interpret, which is easier to audit than a growing prompt.

How do I know the agent understood a column's meaning, not just its name?

Ask for a one-line meaning in its own words and compare it to the documented description. If a status column is described as a lifecycle stage and the agent calls it a payment flag, the restatement has caught a semantic error. When in doubt about a column's real content, validate the values directly with a completeness and uniqueness check simple data validation in SQL.

Sources

Sources

  1. Review of LLM-based Text-to-SQL Systems
  2. X-SQL: Expert Schema Linking and Understanding of Text- ...
  3. Text2SQL Accuracy - RAG, Fine-Tuning, Schema Linking
  4. Quick Guide: Calculate Cohort Retention Analysis with SQL
  5. Techniques for improving text-to-SQL
  6. Retention in SQL: How to Calculate User and Cohort Retention?
  7. Simple Data Validation in SQL
  8. Creating a Read-Only Postgres User
  9. PostgreSQL: Documentation: 18: EXPLAIN
  10. Need a row count after SELECT statement
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles