How to Choose an AI Model for Your CRM Notes

Do not evaluate on public transcripts or synthetic calls. Your calls have your vocabulary: SAFEs versus priced rounds, extension rounds, pro rata, bridge notes. A SAFE is a simple agreement for future equity that became the default early stage instrument partly because it avoids agreeing a pre-money valuation according to Wyrick's SAFE deep dive. If your model does not know that, it will mislabel a SAFE as a priced round and your pipeline will show a wrong stage.

Steps that work in practice:

1. Export the last 150 to 200 call transcripts, prioritizing calls where your team already wrote a note by hand.
2. Have one person label the required fields for each call, from the transcript, not from the existing note. Where the call is genuinely ambiguous, mark the field as null and record why.
3. Freeze a dev split of 50 calls and a held-out test split of 100 to 150. Tune prompts on dev only.
4. Write down the acceptance bar in numbers before you run anything, for example "stage and round size must be 98 percent correct, next step 95 percent, sentiment 90 percent."

Label quality sets your ceiling. If two people disagree on 10 percent of stage labels, no model can score above roughly 90 percent against that ground truth. Run a 20 call double-label check first and fix the schema where humans disagree.

Also decide what counts as a note. If your team wants a 120 word summary plus five structured fields, benchmark that exact output. Changing the output shape between models makes the comparison meaningless.

A worked example makes the cost math concrete. Assume a typical transcript is 12,000 input tokens and the note plus JSON is 600 output tokens. That is 12 million input tokens and 0.6 million output tokens per 1,000 notes. At published rates of $2.00 per million input and $10.00 per million output, a mid-tier model costs about $24 for input and $6 for output, so roughly $30 per 1,000 notes using CostGoat's listed rates. A budget tier at $0.10 input and $0.50 output would come to about $1.50, and a frontier tier at $10 input and $50 output about $150. Same transcripts, a hundredfold spread.

That is why list price alone cannot pick a winner. Input and output prices vary widely by tier, and output rates are typically several times input rates as Amnic's cost comparison shows. More important, reasoning tokens bill as output. A model that "thinks" before answering can turn an expected bill into several times that figure according to the developer cost breakdown cited in Amnic's analysis. Turn reasoning off for extraction if the vendor allows it, or measure it.

Latency is the number teams forget. Test it end to end: upload, transcribe if needed, model call, schema validation, CRM write. A batch API is cheaper but unsuitable if your analysts expect pipeline updates within the same call. Measure at your real concurrency, because ten parallel calls behave differently from one. Use the median, and check the 95th percentile too so one slow call does not block a partner meeting.

Accuracy is the number that actually constrains cost. Score per field, not per note. A model can score 90 percent per note while getting round size wrong on the largest checks, which are the ones your partners read. Weight fields by how much damage a wrong value does.

Run every candidate on the held-out set, twice, and average the two runs to smooth variance. Use temperature zero or the lowest setting available. Keep the prompt identical across models, with only the schema and output mechanics adjusted to what each provider enforces. Log raw outputs so you can audit failures later.

A decision rule that survives contact with real data:

• Drop any model below your stage and round size accuracy bar. Those fields gate everything else.
• Among survivors, drop any model above your latency ceiling for same day updates.
• Rank the rest by cost per 1,000 notes and pick the cheapest whose accuracy bar you can defend with a human review sample of 10 to 20 percent of notes.

Then look at the errors that remain. If most failures are sentiment (a soft "let's keep in touch" read as a pass), a cheaper model plus a tightened enum may close the gap. If failures are round size, no prompt tweak saves you; step up a tier. Be careful with the accuracy bar itself: benchmarks are not your job. Published quality scores are useful for shortlisting but should never replace your own labeled set, which is why CostGoat separates quality scores from value scores in its comparison.

A small JSON contract in your application keeps the comparison honest and makes swapping models cheap:

{
  "type": "object",
  "properties": {
    "investor_name": {"type": "string"},
    "round_size_usd": {"type": ["number", "null"]},
    "stage": {"type": "string", "enum": ["pre-seed", "seed", "series-a", "series-b", "other"]},
    "next_step": {"type": "string"},
    "due_date": {"type": ["string", "null"]},
    "sentiment": {"type": "string", "enum": ["hot", "warm", "neutral", "cold"]}
  },
  "required": ["investor_name", "stage", "next_step", "sentiment"]
}

Validate the response against the schema before any CRM write. Reject and retry once, then route to a human queue. That single rule counted for one of the three metrics; do not let retries go uncounted.

The model you pick is a hypothesis. Instrument the pipeline so you can test it every month without another bake-off. Log token counts, retries, schema rejections and human corrections per note. Corrections are your cheapest ongoing accuracy signal: if analysts override the stage on 4 percent of notes, that is a real, current number.

Watch three cost traps that rate cards hide. First, tokenizers differ by vendor, so the same transcript can split into different token counts, with documentation noting some tokenizers consume noticeably more tokens for the same text as Amnic reports from provider docs. Second, caching changes the economics: repeated instructions and schema can be cached at a fraction of input rates, which matters when every call shares the same long system prompt CostGoat lists batch and cache tiers side by side. Third, prompt bloat: every field you add to the schema, every example you paste, inflates input on every call. Audit the prompt monthly.

You may find the best result is not one model. A cheap tier for routine calls, a stronger tier when the transcript mentions a term sheet or a number outside your expected range. Report both cost and corrections to the person who owns the CRM, weekly, in one table. Fundraising volume is lumpy: one team reached out to more than 100 VC firms, met 80, sent documents to 16 and got 7 offers per the Fidelity session recap. At that volume, a $30 per 1,000 notes model costs single digit dollars a month. The decision is almost entirely about accuracy and latency, not budget.

When you are ready to run the bake-off without wiring a separate SDK per vendor, one API key across text models simplifies the comparison and keeps the prompt and schema identical through Quanta Inference.

Treat the table as a hypothesis to test, not a ranking. Your labeled set decides.