How to Write a Model Decision Log Schema

A useful decision log is a small structured record per material decision, not a transcript and not an archive of hidden reasoning. Start with a decision boundary. Log when the agent crosses from interpretation into a consequential branch or side effect: selecting a tool, classifying a request into a risk tier, moving money, denying an action, escalating to a human the decision log guide from Singularity.

Here is a schema sketch that holds up when the model changes. It is deliberately additive: every block can gain fields without breaking readers.

decision_id            string, unique
occurred_at            RFC 3339 timestamp
tenant_id              string
subject               {type, subject_id}
decision_kind          enum: credit | fraud | route | allow | hold | block
policy                {policy_id, policy_version, verdict, reason_codes[]}
release               {prompt_template_id, prompt_version, model_id,
                       model_provider, sampling{...}, tool_manifest_id}
inputs[]               {input_id, value, source_system, as_of, hash}
outcome               {state: attempted|verified|failed|unknown, result_ref}
redaction_version     string

Three choices carry the weight. First, inputs[] stores immutable input IDs alongside the value the model actually saw, so a later model can be replayed against identical evidence. Second, release captures the prompt template version, model identifier, provider and sampling parameters together, because prompt behavior depends on more than the prompt text alone: the model, parameters, tool configuration and rationale all influence what comes back the prompt versioning guide. Third, policy.reason_codes[] holds versioned codes, never free text.

That last point matters for explainability. With AI and machine learning underwriting, meeting the specific-reasons requirement takes feature-level explainability methods such as SHAP or LIME, a governed process for mapping those outputs to Regulation B reason code language, and validation that the reasons are accurate and tied to a specific model version the Trussed lending guide. A versioned code library is the only way that mapping survives a retrain.

The single most common failure is logging feature names without values, or values fetched at read time. A credit decision made in March cannot be explained in June if the bureau score, debt-to-income ratio and account age have all moved since. The record must freeze each value the model consumed, stamped with its source system and as-of time, and identified by an input ID that never changes.

Prefer references over copies for bulky evidence. In the support agent example, requestRef, targetRef and resultRef point to access-controlled data rather than duplicating it, so a transcript or invoice stays in its existing system under its existing retention rules the decision log guide from Singularity. For small scalar features, inline the value and a hash, so tampering is detectable.

Worked example, credit decision: an applicant is declined. The log's inputs[] holds bureau_score = 612 from crdbureau as of 2026-03-14T09:02:11Z, dti = 0.47 from income_service as of the same window, and months_since_oldest_trade = 9. The release block records prompt_version = 7, model_provider = A, temperature = 0, top_p = 1.0. Six months later, a new model is proposed. You replay the same inputs, get a new verdict, and compare reason codes. That comparison is the audit.

Worked example, fraud alert: a payment is held. inputs[] includes velocity_24h = 11, geo_distance_km = 3400, device_age_days = 2. The policy verdict is hold with reason code fraud.velocity.v2. The outcome block starts as attempted and is later appended as verified with a result_ref pointing at the payment event. Writing the planned record before the side effect gives you a durable idempotency key and distinguishes attempted, verified and unknown instead of silently turning uncertainty into success the decision log guide from Singularity.

The payoff is context. Monitoring systems that answer an alert in isolation cannot tell a routine payroll deposit from a laundering pattern; context about a customer's normal behavior is what separates a false positive from a true one the Flagright false-positive guide. A frozen input set is that context, captured at decision time and never rewritten.

Reason codes are the part of the schema most likely to be quietly broken. A team renames insufficient_income to income_below_threshold, reuses the old code for a new factor, and suddenly two years of decisions mean different things depending on when they were logged. The fix is a rule, enforced in code review: a reason code is permanent. Any change to meaning creates a new versioned code.

Use a dotted, versioned namespace such as credit.income_insufficient.v3. The numeric suffix is the semantics version, not a date. Retiring a code means marking it deprecated in the registry while it stays valid for old records. Regulators take the same view: Regulation B's Appendix C sample reason checklists are illustrative rather than exhaustive, and lenders using AI models should treat them as a starting vocabulary and expand it to reflect the features the model actually uses the Trussed lending guide. A governed, version-controlled mapping process is what keeps disclosed reasons accurate as the model or its inputs change.

A reason-code registry is worth treating as its own artifact. It holds the code, its plain-language description, the owner, the model versions that emit it, and its status. That registry becomes the translator between a raw attribution and applicant-facing language. The mapping must be checked to confirm it reflects principal reasons rather than plausible-sounding but inaccurate explanations, and the CFPB circular notes creditors must be able to validate that disclosed codes correspond to what actually happened the Trussed explainability guide.

Prompt changes are the silent regression in most agent systems. A small wording edit made to handle an edge case can break other scenarios without anyone noticing, and when quality drops, rolling back usually means guessing what the previous prompt looked like the prompt versioning guide. Logging the prompt template version next to every decision turns that guess into a lookup.

Store the template ID and version, not a full copy of the rendered prompt, unless the rendered prompt contains no personal data. Templates and variables need to be versioned together, because changes to template structure or variable handling affect behavior as much as changes to visible text the prompt versioning guide. The rendered prompt belongs in a trace with a stable ID that the decision log links, protected by normal access controls.

Sampling parameters belong in the decision record because they change the output. A decision made at temperature zero and one made with higher temperature are not the same evidence. Record temperature, top-p, max tokens and any seed, all together with the model identifier and provider, so a later replay can reproduce the original conditions or, deliberately, not reproduce them.

Version immutability is the underlying rule: once a prompt version exists, it should never be edited, because later references in logs and evaluations depend on it the prompt versioning guide. Apply the same rule to templates, tool manifests and policy bundles.

Schemas evolve. The goal is that old records stay valid and new records get richer. That means every schema change must be readable by code that has never seen the new field, and every reader must tolerate records from before the field existed.

Practical rules:

• Add optional fields, never repurpose existing ones. A new field defaults to absent. Old records simply lack it, and reports say "not captured" rather than inventing a value.
• Namespace new blocks. If you add a retrieval block for RAG context, version it as retrieval_v1 so future versions coexist.
• Backfill by replay, not by guess. To populate a new field on historical records, replay the frozen inputs against the original model, prompt version and parameters, and write the results as clearly labeled derived data with a derived_from marker. Never overwrite the original verdict.
• Keep readers decoupled from writers. Readers should consume the versioned fields they need and treat unknown fields as absent.
• Migrate policy and reason codes via the registry. A new model that emits fraud.velocity.v3 must not retire v2; both stay valid.

Validation checks belong in a small script that runs on every record before it is written and in a batch job over historical records. Run the checks on every insert, or you will discover the gap during an audit.

def validate(record):
    errors = []
    if not record.get("decision_id"):
        errors.append("missing decision_id")
    if not record.get("release", {}).get("prompt_version"):
        errors.append("missing prompt_version")
    if not record.get("release", {}).get("model_id"):
        errors.append("missing model_id")
    if not record.get("inputs"):
        errors.append("no inputs recorded")
    for i, inp in enumerate(record.get("inputs", [])):
        if not inp.get("input_id") or inp.get("as_of") is None:
            errors.append(f"input {i} lacks id or as_of")
        if inp.get("value") is None and inp.get("hash") is None:
            errors.append(f"input {i} has neither value nor hash")
    if record.get("policy", {}).get("verdict") in ("block", "hold"):
        for code in record["policy"].get("reason_codes", []):
            if not is_registered(code):
                errors.append(f"unregistered reason code: {code}")
    if record.get("outcome", {}).get("state") not in (
        "attempted", "verified", "failed", "unknown"
    ):
        errors.append("bad outcome state")
    return errors

The unknown state is not a nicety. A timeout or worker crash is itself important evidence, and distinguishing attempted, verified and unknown keeps uncertainty visible instead of folding it into success the decision log guide from Singularity.

Yes, if you measure the right thing. The cost of a weak schema shows up as investigator time and as regulatory risk. Anti-money laundering systems built on static threshold rules routinely produce very high false-positive rates; one practitioner paper reports industry false-positive rates above ninety percent and proposes a hybrid architecture showing thirty to forty percent reductions in false positives with twenty to thirty percent improvements in detection precision the AI transaction monitoring paper. Another analysis cites research that in traditional AML systems ninety percent of alerts are false positives the Lucinity overview.

Take a worked example. Suppose you process 10,000 fraud alerts a month at a ninety percent false-positive rate, so 9,000 are cleared after review. If an average review costs 10 minutes, that is 90,000 minutes, or 1,500 hours, of investigator time per month. Now suppose a replay harness lets you test a candidate model against six months of frozen inputs before deployment and shows it would have flagged 40 percent fewer of those legitimate cases. The saving is 3,600 reviews a month, or 36,000 minutes, roughly 600 hours. The exact numbers depend on your base rate; the structure of the calculation is what you should copy.

The decision rule is simple. If a proposed change saves less investigator time than the schema work costs, skip the change; if it saves more, require that the replay evidence ships with the change request. Without frozen inputs and versioned reason codes, you cannot compute the first term at all, which is why the schema comes first.

This is also where explainability investment pays. Modern detection techniques and explainable models are what make reduced false positives defensible to regulators, since excessive alerts from outdated rule-based approaches signal weak systems the Facctum explanation. A log that captures evidence and reasons is the substrate for that defensibility.

Agent systems make the schema problem harder, because one workflow may call several models through different providers. The log should treat model routing as part of the release metadata: record which provider and model handled each step, and keep the tool manifest version so the set of available tools is part of the evidence. If your agents call text, image and embedding models, keeping a single record per decision rather than per call prevents the log from fragmenting into thousands of rows nobody can read.

A useful pattern is one decision record with child spans. The parent holds the verdict, reason codes and frozen inputs. Child spans hold the individual model calls, their parameters and their trace IDs. Reviewers read the parent; engineers drill into the children. Teams that need to route many model types through one interface can hold that contract constant; Quanta Inference exposes text, image, video, audio and embedding models behind one OpenAI-compatible API key, which keeps the provider field meaningful as a metadata value rather than a scattering of SDKs.