How to Write a Security Questionnaire for an AI Agent Vendor

Behavioral rules are legal and HR documents, so their job differs from a vendor questionnaire, but they teach one useful discipline: name the specific risky action instead of the technology. A good employee AI policy bans uploading confidential data, making legal or financial decisions, committing the company contractually and replacing professional judgment, rather than banning "AI" (CRI). Do the same in the questionnaire. "Do you use AI?" produces a marketing answer. "List every tool the agent can call and the scope of each credential" produces an engineer's answer.

Three logistics make the difference between a questionnaire and a rubber stamp:

1. Send it after the use case is fixed. A shortlisted vendor plus a defined workflow is the right moment. It is a procurement artifact, not a general research exercise (Wavect).
2. Require named answers per section. A single reply from security@ means nobody owned the question. Ask for the engineer accountable for each domain.
3. Re-classify existing vendors. A tool approved as a project tracker can ship agentic features later. Re-classification is one of the most commonly skipped steps in AI risk programs (Reco).

Build a scoring table before you send anything, so the answers land on a scale instead of a feeling. This one is a starting point you can adapt:

Anyone who has built a workplace AI policy knows the failure mode of a generic template: the specifics of the business are missing, so nobody follows it (Littler). The same applies here. Weight the table toward whichever action would hurt most in your environment, whether that is a payment, a deploy or a customer record.

This section decides the blast radius. Ask for the full tool inventory first, then work outward.

Questions to ask

• List every tool, API, MCP server or internal endpoint the agent can call, with the exact scope or permission for each.
• Does the agent act under its own service identity, the requesting user's identity, or a delegated token? For each, what does the downstream system see?
• How is a permission granted to one user for one task, and how is it revoked? Can a scope be time-limited to the duration of a task?
• Can a user grant the agent read access without write access to the same system?
• How do you prevent standing privilege, meaning a credential left attached after a pilot ends?
• Which humans or groups can create new agents or new tool connections without a security review?

Good answers name a per-tool scope, show a delegated identity so the downstream log carries the human requester, and describe revoke as a tested operation with a time. Deal-breakers are a single svc- account shared across all customers, a scope list the vendor "can't enumerate," and write access granted by default because read-only "caused errors in testing."

The audit reason to be strict: agents multiply machine identities far faster than quarterly access reviews can track, and a role grant tells an auditor the agent could read a table, not whether this read was appropriate for this user at this moment (PlainID). If the vendor cannot reconstruct an authorization decision, you own an audit finding later.

Autonomy without isolation is the core agent risk. This section asks where code runs and what it can reach.

Questions to ask

• Where does agent execution happen: a container per task, a shared host, or the customer's own infrastructure?
• What filesystem, network and credential mounts are present inside the sandbox?
• Which outbound destinations are allowed by default, and who approves a new one?
• Is there a domain allowlist for browsing, and how are newly registered or untrusted domains handled?
• Does the agent's environment hold long-lived secrets, or are credentials minted per task and short-lived?
• Can the agent's tool output be treated as untrusted input, or does a fetched page or document get to issue instructions?

Good answers describe a per-task container with an explicit egress allowlist, short-lived credentials and the assumption that tool output is hostile. Deal-breakers are an agent on a shared host with open internet access and a standing API key, and any answer that treats a fetched web page as trusted instruction.

Ask for a diagram. A vendor who knows their own architecture can produce one in an hour; a vendor who cannot is describing something they did not build. And do not accept "MCP" as a security answer: it is a connection standard, and each server, permission, token and command still needs review (Kingy AI).

For a model, a log captures a decision. For an agent, it must capture a decision, an action and the chain of steps between them: prompts, retrieved context, tool inputs, the decision trace, the systems touched and whether each was permitted (Collibra). If the vendor logs the agent the way they log a model, they miss the part that matters.

Questions to ask

• For one completed task, show a redacted sample log. It must include actor, requesting user, timestamp, model or agent version, tool called, input arguments, result and whether a policy allowed it.
• Are logs immutable or tamper-evident, and where are they stored?
• Can we stream logs to our own SIEM, in what format, and how quickly?
• How long are logs retained, and can we set the retention?
• Do you log blocked actions and guardrail triggers, not just completed ones?
• Can we query "every record this agent read for customer X last quarter" and get an answer in minutes?

Good answers produce the sample log as an attachment with the questionnaire, share one identifier across prompt, tool-call and API logs, and answer the last question with a screenshot. Deal-breakers are logs without a human requester, logs the vendor can edit, and a retention setting that only the model provider controls.

Put the sample log in the contract as an exhibit. Answers are claims made on one day about a product that changes weekly, so the strongest programs verify claims against live telemetry from their own environment (Reco). A pinned log schema is far easier to verify than a promise.

Agent memory is the newest thing in this questionnaire and the easiest to get wrong, because it accumulates on purpose. Every stored preference, summary and embedding is a copy of your data with a lifetime.

Questions to ask

• What does the agent store between tasks: conversation history, learned preferences, retrieved documents, embeddings, cached tool results?
• Where is memory scoped: per user, per team, per tenant, global?
• How long is each memory type retained, and can we set it to a specific period or zero?
• How does a user delete their memory, and how long until backups and derived copies are gone?
• Is our data used for model training or product improvement? Get a written no-training term covering prompts, outputs, metadata and logs.
• Is tenant data logically or physically isolated at the model and vector-store layer?
• Can we exclude specific data classes, such as PII, PHI or source code, from AI processing entirely?
• Who are the subprocessors, including model providers, and will you notify us before adding one?

Good answers name a retention period per memory type, describe a purge that covers backups and derived copies, and provide a signed no-training term. Deal-breakers are indefinite retention, shared vector stores across customers, and a training clause that depends on a setting a customer might toggle.

The comparison that helps here is between what a standard SaaS review checks and what an agent review must add:

Field-level scope matters more than a data category. Ask for a field-by-field list of every element the agent will touch, including inputs, outputs, metadata, logs and support copies, rather than accepting "we need the whole chart" (Censinet).

An agent that is halfway through a multi-step task when something goes wrong poses a problem a chatbot never did. The answer you want is a mid-run stop, not a key rotation.

Questions to ask

• Can you disable one running agent mid-task, and what happens to the actions already taken? Is there a compensating action?
• What is the measured time from "stop the agent" to "no further tool calls," and when was that last tested?
• Can we trigger the stop ourselves, through a console or API, without contacting support?
• How do you detect a compromised credential or a hostile tool response during a run, and what rate or anomaly thresholds trigger a hold?
• Who is the named security contact, with a backup, and what is the contractual breach notification window?
• What is the rollback path for actions like a sent email, a merged pull request or a refund?

Good answers include a written procedure, a tested stop time, a customer-accessible control and a rollback for each consequential action. Deal-breakers are "contact support and we will rotate the key," no customer-side stop, and no defined window for breach notice.

Evidence standard for the whole review: a sample audit log, a sandbox architecture diagram, a written kill-switch procedure with a tested time, a retention schedule, a subprocessor list and a named security contact. If the vendor will not put those into the contract or its exhibits, treat the deal as unresolved, because paper beats promises (Censinet).

A questionnaire that ends in a PDF gets ignored. Convert it into one of four outcomes: approve, approve with conditions, pilot with constraints, or decline. The subsections above should already tell you which.

A simple worked example shows how to keep it honest. Suppose a vendor scores well on identity and logging but answers the kill-switch question with "under an hour, via support ticket," and the agent can issue refunds up to a set cap. The conditional approval is: pilot only with read access and a manual approval step for every write action, a customer-configurable spend cap, and a contract clause requiring a tested customer-side stop within a stated time by a fixed date. Re-review at renewal with the same table. That is more useful than a score of 7 out of 10.

Three habits keep the program from decaying:

• Keep the questionnaire short enough to finish. A tightly scoped set of questions per domain, each with a defined red-flag response, beats a 200-item document nobody completes (Wavect).
• Tie it to your internal policy. If your own rules require approval before any tool is integrated with internal software, the questionnaire should ask the same thing of the vendor (Fisher Phillips).
• Match controls to first-party standards. If you enforce MFA and least privilege internally, require the equivalent for vendor and agent access into your environment, or you have created a vendor-only loophole (CyberProof).

If you are building or procuring agent platforms rather than only buying them, the same checklist doubles as a build specification, and a team that can review agent architecture and write the evidence pack is worth having in the room early. Quanta Intelligence runs AI experts who produce deliverables like this against your own context, and its inference API puts text, image, audio and embedding models behind one OpenAI-compatible key, which simplifies the subprocessor list in Section 4.

A minimal log record to ask for

Ask vendors to return one redacted record per task in a schema like this. If they cannot fill it, they are not logging at agent granularity.

{
  "task_id": "tsk_01HX",
  "actor_agent": "support-agent:v14",
  "acting_for": "user:[email protected]",
  "started_at": "2026-03-02T10:14:05Z",
  "steps": [
    {
      "step": 1,
      "tool": "crm.read",
      "scope": "crm:read:account",
      "input": {"account_id": "acc_778"},
      "policy_decision": "allow",
      "policy_id": "pol_crm_read_v3",
      "result_summary": "12 fields returned"
    },
    {
      "step": 2,
      "tool": "billing.refund",
      "scope": "billing:write:refund",
      "input": {"amount": 40, "currency": "USD"},
      "policy_decision": "deny",
      "reason": "approval_required_over_25"
    }
  ],
  "ended_at": "2026-03-02T10:14:22Z",
  "outcome": "halted_for_approval"
}

The field that separates a real agent log from a model log is policy_decision with a policy identifier. Without it, you can see what happened and not why it was allowed.