Scoring AI Projects: A Weighted Matrix for Small Businesses

Five criteria are enough. More than that and the scoring session turns into a committee meeting; fewer and you lose something important. The five below appear, in various forms, across the published frameworks, and each maps to a specific failure mode.

• Value. The expected effect on cost, revenue, or a metric the business already tracks. The Umbrex AI use-case prioritization framework recommends quantifying with real KPIs rather than adjectives, and considering upside and downside ranges. For a small business, "saves 4 hours of bookkeeping per week" beats "improves efficiency."
• Feasibility. Whether you can build it with the tools, people, and time you have. Umbrex folds data availability, model complexity, integration effort and change load on frontline teams into this. For a 15-person firm, change load is often the binding constraint, not technology.
• Data readiness. Whether the data exists, is accessible at the cadence you need, and is clean enough. This deserves its own criterion rather than hiding inside feasibility, because it is the most common reason a technically easy project fails. Unit8's AI project selection guide is blunt about this: in contrast to other types of innovation, data availability is a core requirement for AI projects.
• Risk. Safety, privacy, cybersecurity, algorithmic bias, and regulatory exposure. Umbrex notes that higher-risk use cases can still be attractive but need stronger guardrails.
• Strategic fit. Whether the project advances the priority you have already named. The fountaincity framework makes the case that you should rank your strategic objectives before ranking projects, so that this criterion has something concrete to score against.

Note that speed to value and reusability, which appear in larger enterprise frameworks, are deliberately excluded here. For a small business, speed is largely captured by feasibility and data readiness, and reusability across dozens of use cases is not yet a meaningful benefit when you have four candidates.

Assign each criterion a weight as a percentage. The five weights must sum to exactly 100. The published guidance converges loosely: Umbrex gives example ranges of value 35 to 45 percent, feasibility 25 to 35 percent, time to value 10 to 15 percent, reuse 10 to 15 percent, and risk 5 to 10 percent. That is an enterprise portfolio and includes two criteria we are not using, so translate rather than copy.

A workable starting point for a small business making a first or second AI bet:

Two adjustments are worth making deliberately. If you operate in a regulated sector (health, finance, legal), raise risk to 20 or 25 and reduce value to compensate, so the total stays at 100. If you are in a genuine cash crunch and need measurable savings within a quarter, raise value and lower strategic fit. What you should not do is adjust the weights after seeing the scores. Write them down, date the page, and score every candidate against the same set.

The Toptal prioritization case study is useful here for context on why this discipline matters: it cites Gartner research that a large majority of CIOs were increasing AI investment while a majority of CFOs reported that AI initiatives had underperformed expectations. Weights agreed in advance are how you avoid being on the wrong side of that split.

Score every project 1 to 5 on every criterion. The anchoring is what makes the scores comparable between people and between meetings, so write the anchors down and use the same ones every time.

Two rules keep the scoring honest. First, a score must come with a one-line justification in the cell next to it. If you cannot write the sentence, the score is a guess. Second, score risk in the direction of safety: 5 means low risk, so that every criterion points the same way and a higher total is always better. Mixing directions is the most common way these matrices produce nonsense.

Andrew Ng's guidance that a valuable AI system may be possible with perhaps as few as 100 to 1,000 data records, per the HBR article, is worth keeping in view when scoring data readiness. A score of 1 or 2 should mean the data genuinely does not exist or cannot be reached, not that you have less than a million rows. Savvi's litmus test makes the same point more simply: the workflow needs to be data rich, operationally relevant, and recurring.

The weighted score is the sum of each criterion score multiplied by its weight, divided by 100:

weighted_score = (value*Wv + feasibility*Wf + data*Wd + risk*Wr + fit*Ws) / 100

With the default weights (30, 25, 20, 15, 10), a project that scores 5 on value, 3 on feasibility, 2 on data readiness, 4 on risk and 3 on strategic fit produces:

(5*30 + 3*25 + 2*20 + 4*15 + 3*10) / 100
= (150 + 75 + 40 + 60 + 30) / 100
= 355 / 100
= 3.55

That lands in the pilot band. Now compare two real candidates from a services firm with twelve staff.

Project A: automated proposal drafting. Reps spend roughly six hours per proposal and write about eight a month. Value 4 (48 hours a month recovered, but only partly). Feasibility 4 (existing document tools plus an API). Data readiness 4 (past proposals are already in a shared drive, reasonably consistent). Risk 4 (internal documents only, no client data leaves your systems). Strategic fit 3 (supports growth but not the named top priority).

(4*30 + 4*25 + 4*20 + 4*15 + 3*10) / 100 = (120 + 100 + 80 + 60 + 30) / 100 = 3.90

Project B: predictive reorder for inventory. Value 5 (stockouts are the single biggest source of lost revenue). Feasibility 2 (needs a forecasting pipeline nobody on staff can build). Data readiness 2 (sales history lives in two systems, one of which exports inconsistently). Risk 4 (no personal data involved). Strategic fit 5 (this is the stated top priority).

(5*30 + 2*25 + 2*20 + 4*15 + 5*10) / 100 = (150 + 50 + 40 + 60 + 50) / 100 = 3.50

The higher-value, higher-strategy project scores lower than the unglamorous proposal tool, and the reason is visible in the arithmetic: 70 points of weight sit on feasibility and data readiness combined. That visibility is the point. Asana's guide to decision matrices notes that weighting is what you use for complex decisions where some factors matter more than others, and that the structured process is what turns an argument into a comparison.

Set the thresholds before you score anything, and treat them as a gate rather than a suggestion.

The bounded pilot band is where most first projects should land, and that is fine. Andrew Ng's six-to-twelve-month guidance and Unit8's "dream big, but start small" principle both describe the same thing: use a small proof of concept to earn organizational buy-in for the next project. A pilot with a stop date is not a weaker commitment than full funding; it is the correct commitment for anything you have not done before.

Ties happen, and they usually happen right at a boundary, which is the worst place for an unstructured argument. The rule: on equal weighted scores, fund the project with the higher data-readiness score. If data readiness also ties, fund the higher-risk score (lower risk). Only if both tie should you fall back to business judgment.

The reasoning is not arbitrary. Data readiness is the criterion most likely to have been scored optimistically in the meeting and most likely to be discovered as a problem after money has been spent, and Unit8's guide is explicit that data availability is a core requirement rather than a nice-to-have. A tie broken by data readiness is effectively a tie broken in favor of the project you are most likely to actually finish. Save the worked logic somewhere your team can see it, because in six months the argument for why you picked the proposal tool over the inventory model needs to still make sense.

A scoring matrix is easy to game and easy to abandon. Three habits keep it useful.

First, keep the candidate list short and written down. The Agility at Scale guide to use case prioritization separates discovery from prioritization and warns that running them as one step tends to fund whichever project surfaced most loudly. Score only what is on the written list.

Second, revisit scores after a pilot, not during it. If the pilot reveals that data readiness was a 1 rather than a 3, record the correction. Over three or four projects your anchor descriptions will improve, and so will your scores. Do not retroactively change the weights to make a past decision look better.

Third, use the matrix output to define the pilot's success metric. BizTech's step-by-step guide for small business AI puts business objectives first and notes that starting with targeted use cases keeps spend on measurable outcomes. If value was scored 4 because you expect to recover 48 hours a month, then 48 hours a month is the number the pilot reports against.

If the scoring session itself is where your team gets stuck, a short structured conversation about goals and constraints often unsticks it faster than another round of internal debate. That is what a free 30-minute AI discovery call is for: your situation, and a tailored recommendation with no obligation.