A Review Checklist for LLM Features Before Launch

Run the review as two ordered gates, not one meeting.

Gate 1 is an acceptance gate, and it should happen weeks before launch. For every risk that applies to your build, record one row: the question, whether the exposure exists, whether the mitigation is implemented, the residual risk, the evidence link, and the accepting owner. Any residual high risk needs a written decision by a named person with authority to accept it. This gate answers whether you are ready to produce evidence at all, and it routinely reveals that a tool needs removing or a credential needs narrowing. Those are cheap fixes and should happen before any testing.

Gate 2 is the launch gate, and it happens days before release. Here you check that every row from Gate 1 has evidence that a reviewer can open, ideally by clicking a link in this article's checklist section rather than trusting the description in a status field. If a row fails evidence, the decision is not "launch with a caveat." The decision is to defer that capability, or ship the feature with the risky action disabled, until the evidence exists.

The reason to insist on artifacts is that a status checkbox hides the difference between a mitigation that exists and one that works. OWASP has published a cybersecurity and governance checklist aimed at executives, security, privacy, compliance, and engineering audiences the OWASP checklist. Treat it as a prompt for the questions, not as the proof. The proof is the log.

The table below turns each OWASP LLM risk into one question to ask, one artifact to collect, and a pass condition. Where a risk does not apply to your architecture, record "not applicable" with one sentence of reasoning. "Not applicable" without a reason is a finding.

A worked example of scope-first ordering. Suppose the feature is an email assistant that summarizes a shared inbox and drafts replies, with a send action. Ask first: does it need the send tool for the reviewed task? Per OWASP, granting an extension that can read, modify and delete when only reading is needed, or leaving a trial plugin available because nothing removed it, is excessive functionality LLM06:2025 Excessive Agency. Removing send costs minutes and shrinks several risks at once. If send stays, the same source says an application should not perform high-impact deletions without confirmation from the user, so require an explicit approval step per send. Now your evidence artifacts are much smaller: a test log for indirect injection through email bodies, and a per-user credential that only sees that user's mailbox, not a shared privileged account. That second point comes straight from the OWASP example of an extension reading a user's documents through a privileged account that can see everyone's files.

Here is the row format you can paste into the review pack:

Risk: LLM06 Excessive Agency
Question: Is the send tool scoped to the user and gated by approval?
Answer: Partially. Send available; approval step present in UI; credential is shared service account.
Residual: High (shared credential can read all mailboxes)
Evidence: link to credential scope export (pending), approval UI recording (attached)
Owner: platform-security lead
Decision: FAIL until per-user credentials; send disabled in v1 if not resolved

That kind of row is the difference between a checklist and a review process: it produces a decision, a name, and a next action.

Prompt injection is the risk most likely to be "handled" with prose. OWASP's own entry is unusually direct: given the stochastic nature of models, it is unclear whether fool-proof prevention methods exist, so the goal is to mitigate impact LLM01:2025 Prompt Injection. Reviewers know this. What they want from you is evidence about impact, not confidence about defense.

Your evidence should cover both flavors. Direct injection is a user typing instructions that override behavior. Indirect injection is content the model reads from websites, files, or tool results carrying hidden instructions. OWASP notes that injected content does not need to be human-readable, only parsed by the model, and that multimodal inputs like images can carry instructions next to benign text LLM01:2025 Prompt Injection. Practical artifacts:

• A dated test log with at least payloads for instruction override, role change, encodings (base64, homoglyphs, right-to-left text), and instructions hidden in documents or images.
• Traces showing which tools the model called for each payload, not just final answers.
• Results from reviewing whether the model treated external content as data or as instructions, which is the segregation control OWASP recommends LLM01:2025 Prompt Injection.

For the defense-in-depth side, the same entry suggests constraining the system prompt, validating output formats with deterministic code, filtering input and output, applying least privilege to credentials the model can trigger, and requiring human approval for high-risk actions. Industry comparisons often describe injection as attackers manipulating prompts to hijack logic or approve an unintended action such as a transfer Oligo's LLM security overview, while vendor write-ups split it into direct jailbreaking and indirect injection through external content Cloudflare's OWASP overview. The review question is not which taxonomy you use. It is whether an injected command produces a damaging effect or a logged, blocked attempt.

Sensitive information disclosure covers PII, financial and health records, credentials, legal documents, and confidential business data, and it can surface not only through answers but through logs, traces, and retrieval LLM02:2025 Sensitive Information Disclosure. The review question to ask is blunt: can a user get data they are not authorized to see, by asking, by retrieving, or by reading our logs. The artifacts are the authorization tests and a sample of redacted traces. Note that OWASP warns restrictions in the system prompt "may not always be honored and could be bypassed via prompt injection," so authorization has to live in code, not in a polite sentence to the model LLM02.

Vector and embedding weaknesses are the retrieval version of the same question. RAG systems with shared vector stores can leak across tenants if partitioning is loose; embeddings containing sensitive content can be inverted to recover source text; and poisoned documents with hidden instructions can steer answers LLM08:2025 Vector and Embedding Weaknesses. A good evidence artifact is a negative test: a query from tenant A that should never return tenant B's chunk, executed across several phrasings, with the retrieval result recorded. A summary of the same risk notes that hidden text such as white-on-white instructions in an ingested document can survive extraction and manipulate responses A10 Networks on RAG security. Ingest validation and hidden-text detection belong in the artifact list, not in a note to self.

Excessive agency deserves its own review meeting, because it is where the damage lives. OWASP's root causes are excessive functionality, excessive permissions, and excessive autonomy LLM06:2025 Excessive Agency. The most useful review insight here is a decision rule many teams miss: count how many of the three agentic properties your feature combines. Can it ingest untrusted content? Can it reach sensitive data or systems? Can it change state or communicate externally? If all three are present, require per-action human approval before launch; if untrusted input is paired with either of the other two, require a written risk assessment Anaconda's review guide. Cutting one property is a cheaper fix than months of monitoring.

Improper output handling is the plain engineering one. Model output that flows into a browser, a shell, a query, or an admin tool without validation and encoding is untrusted input like any other. Reviewers will ask for the code review link and the test that proves it. One source lists exploitation outcomes including cross-site scripting, server-side request forgery and remote code execution, and recommends treating the model like any user and validating its output before it drives other functions Cloudflare's OWASP overview.

System prompt leakage now appears as hidden context exposure, and the review question has shifted. OWASP lists extracting or inferring hidden instructions as a risk the risk index, and practitioner summaries of the newer edition propose keeping sensitive data out of hidden context, using deterministic guardrails, and enforcing authorization outside the model Superwise.ai's comparison. For evidence, provide the system prompt in the review pack with secrets removed and a test showing that disclosure reveals nothing that can be reused.

Unbounded consumption is a wallet and availability risk, and it has a specimen attack worth testing: the denial of wallet, where a high volume of operations exploits cost-per-use pricing LLM10:2025 Unbounded Consumption. It also covers model extraction, functional replication, and side-channel analysis of exposed logits. The artifacts are small: a load test that hits limits and shows the failure mode, plus a screenshot or export of the rate limits and spending caps enforced at the gateway and at the vendor account. Run a cap drill: set a low cap in staging, confirm the system degrades safely, and record the alert. For comparison, independent practitioner checklists frequently put rate limiting and anomaly detection on the model-DoS line Lasso Security's checklist, and prompt-injection TLDRs list supply chain, poisoning, and unbounded consumption together and note that rate limits and logit exposure are controls you can verify in configuration Promptfoo's TLDR.

Supply chain evidence for an LLM feature is not just package scanning. Include model and adapter provenance, pinned versions, hashes, serving platform, and plugin sources, with an owner per component.

Misinformation needs a policy answer more than a technical one. The review question: which outputs lead to actions, and what verifies them before the action fires? For a customer-facing answer, the bar may be retrieval grounding and a visible citation. For an action like issuing a refund, the bar is human confirmation or a deterministic rule outside the model. Record the answer, because that is what a reviewer is accepting.

Keep the pack small enough to read and specific enough to audit. The artifact list below is a practical target, and it scales down for a small feature.

Two process habits make this sustainable. First, version the pack alongside the feature, and re-run only the rows affected by a change, so you keep the full pack fresh without rerunning everything. Second, add one launch-blocking condition to your deployment checklist: if the pack's Gate 2 record for every applicable risk was not signed in the last change window, deployment fails. That converts the review from etiquette into control.

If your team is preparing a review and wants a second pair of eyes on the risk table before security arrives, a free discovery call is a sensible next step: https://quantaintelligence.ai/calendar.