The job of the wrapper is small on purpose: run the validator, catch its failure, read the first failing path and the expected type from the validator's own error structure, build the object, log it once, and return it. It never throws on a validation miss, so callers can branch on a returned value.
Validators already carry most of what you need. The Python jsonschema library's ValidationError exposes path, validator (the name of the failed keyword), validator_value (what the schema asked for), instance (the value being validated) and json_path, plus a context for nested errors (jsonschema error handling docs). JavaScript validators such as Ajv return an array of { instancePath, keyword, message, params } entries, and express-validator's result object exposes path and a formatter so you can reshape errors into your own type (express-validator validation results). The wrapper's job is to translate whichever shape you use into one canonical object.
The decision rule for "first failure" matters. Some validators return every error; some return only the first. If you have a list, prefer the one with the shortest path, or the first in document order depending on which you find more actionable, and record the choice so it does not drift. Do not loop over all errors to build a list: a list of failures per call multiplies your log volume and makes counting by path ambiguous (one request contributes many rows). One row per failed validation keeps the math clean.
Here is a compact Python wrapper. It assumes a jsonschema validator and a JSON-schema-with-$defs design similar to the order schema shown in the API testing JSON schema guide, but the pattern transfers to any validator that exposes a path.
from dataclasses import dataclass, asdict
import json, logging
from jsonschema import Draft202012Validator
from jsonschema.exceptions import ValidationError
log = logging.getLogger("validation")
@dataclass
class ValidationMiss:
path: str
expected: str
received: str
reason: str
prompt_version: str
schema_version: str
def _reason(err: ValidationError) -> str:
if err.validator == "required":
return "schema_missing_required"
if err.validator == "type":
return "schema_type_mismatch"
if err.validator in ("format", "pattern", "minLength"):
return "schema_format_violation"
if err.validator in ("enum", "minimum", "maximum", "minItems"):
return "schema_rejected_value"
return "schema_other"
def _short(value) -> str:
text = json.dumps(value, ensure_ascii=False, default=str)
return text if len(text) <= 120 else text[:117] + "..."
def validate_or_miss(payload, schema, *, prompt_version, schema_version):
validator = Draft202012Validator(schema)
err = next(validator.iter_errors(payload), None)
if err is None:
return None
miss = ValidationMiss(
path=err.json_path, # e.g. "$.items[1].quantity"
expected=str(err.validator_value),
received=_short(err.instance),
reason=_reason(err),
prompt_version=prompt_version,
schema_version=schema_version,
)
log.warning("validation_miss %s", json.dumps(asdict(miss)))
return miss
Two details are load-bearing. First, iter_errors gives you the first error without building the whole list, which matches the "first failure" rule and avoids the cost of validating everything when you are going to stop at one. Second, the log line is the only place the miss is written. Every caller gets the object back but does not log again. That is what keeps each failure to exactly one row.
If your runtime is JavaScript, the same shape works with Ajv:
const miss = (err, promptVersion, schemaVersion) => ({
path: err.instancePath || "$",
expected: err.keyword,
received: short(err.data),
reason: reasonFor(err.keyword),
promptVersion,
schemaVersion,
});
The choice of expected deserves a note. Some teams want the JSON type ("integer"), others want the failed keyword ("type"). They answer different questions. The type tells you what to coerce or what to tell the model; the keyword tells you how the schema is failing overall. For aggregation, the keyword is usually the better grouping key; for repair, the type is. I would record both: expected as the type, and fold the keyword into reason.
Antonio Evans
