Schema-First Prompting: Design the Contract First

Before writing prompt text, decide what kind of contract you are enforcing. The two structures look similar because both pass JSON schemas to the API, but they do different jobs.

The decision rule: if the model has all the information it needs and only has to reshape it, use structured outputs with a response format. If the model has to pick an action or collect parameters across turns, use function calling, which is structured output with consequence Machine Learning Mastery. Vellum's guidance matches: reach for function calling when external API requests, tool selection or interactive parameter collection are involved, and use the response format when the assistant's reply is the last step in the pipeline Vellum. One practical wrinkle: provider support differs, and Gemini has historically exposed structured output through function calling rather than a standalone response format Vellum. That matters if you route the same contract across several vendors.

Start with the object your downstream code will actually consume. Design for the parser, not for the model. A workable extraction schema for a support ticket looks like this:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["billing", "shipping", "technical", "account"],
      "description": "The single best-fitting support category."
    },
    "anger_level": {
      "type": "integer",
      "minimum": 0,
      "maximum": 5,
      "description": "0 for calm, 5 for explicit demand for a manager."
    },
    "should_escalate": { "type": "boolean" },
    "summary": {
      "type": "string",
      "description": "One sentence, under 25 words, in the customer's language."
    }
  },
  "required": ["category", "anger_level", "should_escalate", "summary"],
  "additionalProperties": false
}

Four rules make a schema prompt-ready. Keep it simple, because complex and restrictive schemas raise the chance of an invalid response nuric Blog. Put natural language in the description field of every property, since descriptions travel with the schema and are read as instructions Agenta. Pick enum values close to how the model already talks, then map them to your internal taxonomy after parsing, rather than forcing esoteric labels nuric Blog. And break genuinely nested contracts into smaller schemas called in sequence, so the model focuses on one object at a time nuric Blog.

If you write models in Python, generate the schema from a Pydantic model instead of hand-writing JSON. BaseModel.model_json_schema() returns a JSON-schema-compliant dict, and the docs show how defaults, descriptions and constraints land in the output Pydantic docs. Field docstrings and Field(description=...) become schema descriptions automatically, which is exactly the text you want the model to read.

One portability check: strict modes accept only a subset of JSON Schema. Optional fields, unions and numeric constraints like minimum/maximum can be rejected by some strict implementations and have to be rewritten or moved into descriptions LangChain forum. Test your schema against the provider before you build prompt text on top of it.

Now write the prompt as a translation of the schema, not as a fresh description of the task. The model's job is to fill fields, so the instructions should read field by field.

A worked derivation from the ticket schema above:

Notice what drops out. You no longer need to write "respond with JSON only, nothing before nothing after" and hope. Provider documentation makes the point directly: with structured outputs there is no need for strongly worded formatting prompts, because the schema does that work OpenAI structured outputs guide. The prompt's remaining job is semantic: define what good values look like, not what the container looks like. That split is the core insight of the workflow. Formatting is the schema's problem; judgment is the prompt's problem.

For a function-calling contract the derivation shifts slightly. The tool name and parameter descriptions carry the meaning, and the required list tells the model which arguments are non-negotiable, as in the classic weather tool where location is required and unit is an enum Prompt Engineering Guide. Write tool descriptions the way you would write the first sentence of a docstring, then write the prompt to say when the tool should not be used.

Examples are cheapest to generate when the schema already exists, because you can copy permitted values into a filled example instead of inventing them. The guidance is blunt: provide examples of expected output where you can, since they improve the chance of a valid response nuric Blog. Do that, but generate both directions.

Positive example, derived from the schema. For the input "This is the third time my order has not arrived and I want a manager," the expected object is {"category": "shipping", "anger_level": 5, "should_escalate": true, "summary": "Customer reports a third missing delivery and demands a manager."}. Every value is legal under the schema: the enum is satisfied, the integer is inside 0 to 5, no extra keys appear.

Negative examples, also derived from the schema. Each one breaks exactly one constraint, which makes failures readable in the eval report:

• {"category": "delivery", ...} violates the enum.
• {"anger_level": "very angry", ...} violates the integer type.
• {"anger_level": 7, ...} violates the maximum.
• An object missing should_escalate violates required.
• An object with an extra customer_name key violates additionalProperties: false.

The structural inversion is the point. In prompt-first development, test cases come from whatever the model happened to produce, so coverage is accidental. In schema-first development, each constraint generates a test by construction, and a schema change that adds a field or tightens a bound immediately implies new tests.

Turn each field and constraint into an assertion. The suite then checks the contract rather than a vibe.

schema = load_schema("ticket.json")

def check_structure(obj):
    errors = []
    for field in schema["required"]:
        if field not in obj:
            errors.append(f"missing required field: {field}")
    for field, spec in schema["properties"].items():
        value = obj.get(field)
        if value is None:
            continue
        if "enum" in spec and value not in spec["enum"]:
            errors.append(f"{field} not in enum: {value}")
        if spec["type"] == "integer" and not isinstance(value, int):
            errors.append(f"{field} not an integer: {value!r}")
        if "maximum" in spec and isinstance(value, int) and value > spec["maximum"]:
            errors.append(f"{field} above maximum: {value}")
    extra = set(obj) - set(schema["properties"])
    if schema.get("additionalProperties") is False and extra:
        errors.append(f"unexpected fields: {sorted(extra)}")
    return errors

Run it over a fixed set of inputs that covers each field: one case where the enum value is obvious, one where two categories are close, one empty or hostile input, one case that should trigger escalation and one that should not. Record the structural errors, the retry rate and the validation failures per case.

A worked calculation shows why this matters. Suppose a pipeline processes 10,000 documents a day and the prompt-first version fails structure validation on 3 percent of calls, needing one retry each. That is 300 retries a day, plus the downstream failures when a malformed object slips through a loose parser. If schema-first prompting with strict output cuts structural failures to near zero at generation time, the 300 retries vanish from the queue and the remaining failures are semantic ones, which is where your human review time should go. Use your own measured failure rate in place of 3 percent; the decision rule is that any schema constraint with no corresponding test is a retry waiting to happen.

Then validate what the schema cannot see. Shape is not truth: a schema-valid customer ID, amount or date can still violate a domain rule TrueFoundry. So keep a second layer for business rules, such as "a billing ticket cannot have anger_level 5 and should_escalate false," and assert those in the same eval run.

Pass the schema to the API as the response format and parse the result into the typed object, then validate again on your side. The OpenAI Python helper accepts a Pydantic class and returns a parsed object, so the contract you designed is the object your code receives OpenAI structured outputs guide. For tools, register the function with its parameter schema and let the framework convert your Python signature into that schema automatically OpenAI Agents SDK.

Three runtime rules keep the loop honest. First, fix parsing errors by editing the schema and regenerating the prompt, never by patching the output with regex. Second, log the structural errors from your checker with the input that produced them, so the eval suite grows from real traffic. Third, re-run the whole eval suite whenever the schema changes, because a new field changes both the prompt and the tests. If the same contract has to run across several providers, a gateway that accepts one model interface and keeps the response format consistent removes a class of portability work; Quanta Inference exposes text, image and embedding models through one OpenAI-compatible API at https://quantaintelligence.ai/inference, which keeps the schema and the eval suite stable while models underneath change.

Finally, measure two numbers per release: schema-pass rate and retry rate. If the first rises, your contract wording is working. If the second stays flat while the first rises, the failures have moved from structure to semantics, and the fix belongs in descriptions and examples, not in stricter output constraints.