A Vendor Questionnaire for AI Tools: What to Ask Before You Sign

Ask: "What data was the model trained on, who checked it, and does any customer data, prompt, output, metadata or log get used for training or model improvement? If yes, can we contractually exclude ours?"

The provenance half of the question matters because undated training data is a compliance exposure you cannot see. A vendor that trained on scraped web data may have pulled in personal information without consent, and a vendor that trained on other clients' data has quietly enrolled you in a sharing arrangement you never negotiated, as Atlas Systems' questionnaire guide puts it.

The exclusion half is the one to put in the contract. Censinet's six question checklist treats training use as a stop point: ban use of PHI, prompts, outputs, metadata and logs for model improvement, in writing, not in a help center article. The Wavect 45 question questionnaire puts training and retention first in its question order for the same reason.

If a foundation model provider sits in the request path, you have a second vendor in the room whose pricing and roadmap you never negotiated, which is why the Agathon pre signature guide tells buyers to ask explicitly where inference runs.

Ask: "How long do you retain prompts, outputs, embeddings, logs, backups and support copies, can we set retention to zero, and what exactly happens when we request deletion?"

Retention is the question vendors answer most loosely. A defensible answer separates each data type and gives a number of days for each. Embeddings and logs are the two that get forgotten, and both are recoverable text in practice. The Reco 40 question questionnaire flags indefinite retention, or retention set only by the model provider, as a red flag for exactly this reason.

Deletion needs the same specificity. Ask for deletion dates for prompts, logs, backups and third party copies, which is the wording Censinet recommends. Then pin down the backup tail: if backups roll off on a schedule, a deletion request may not clear a copy for weeks. Put the number in the contract and require written confirmation after deletion completes.

A worked example makes the cost visible. Suppose a team of 40 runs about 60 prompts a day each on a tool that retains prompts for 90 days. That is roughly 216,000 prompts in the retention window at any moment. If the vendor's answer to "can you set retention to zero" is no, you are accepting a 216,000 item archive you do not control, every day, forever. That is the number to put in front of the approver.

Ask: "Give us the current named list of subprocessors, including model, cloud, speech and support vendors, and tell us which countries process our data. How much notice do we get before the list changes?"

This is fourth party risk, and it is why AI vendors need stricter vetting than ordinary SaaS. Censinet's analysis notes that an AI vendor may depend on a model provider, cloud platform, speech to text service or vector database your team never chose, and cites HHS guidance that a cloud service provider can be a business associate even when it cannot view encrypted data or hold the decryption key. That is a legal point, not a technical one, and it survives encryption.

A second layer: the vendor you approved may have shipped new AI features since. Reco's guide warns that reclassification is one of the most commonly skipped steps, and that a tool approved as a project tracker can quietly become an AI vendor. Residency deserves a hard line if you are subject to data protection rules. The Wavect questionnaire devotes a full section to model and infrastructure location, and treats EU residency claims as evidence to be produced rather than accepted.

On attestations, ask which are current, what their scope and period are, and whether you can see the report under NDA. The trap is accepting the logo instead of the report. A SOC 2 Type II report covers a defined system description over a defined period, so check whether the AI features you are buying sit inside that boundary. Censinet asks for a current report as a baseline and pairs it with encryption, MFA, audit logs, penetration test records and incident response steps.

The gap shows up in breach data. Atlas Systems cites IBM's data breach report finding that 13% of organizations reported breaches involving AI models or applications, and that 97% of those lacked proper AI access controls.

Audit has two meanings, and you should separate them. The first is tenant level: who ran what prompt, against which data, and what came back, exportable to your own SIEM. The Wavect questionnaire treats audit logs as their own section, which is right, because a log you cannot export is a log you cannot investigate with. Access rights follow the same logic: ask what OAuth scopes the product requests into core systems, since read write access to email, CRM, code or files extends the blast radius well past the vendor's own product, per Reco. The second meaning is a right to audit the vendor. Large buyers get it and smaller ones usually get a questionnaire. If you cannot get a right to audit, get a commitment to notify you of material control changes and to answer an annual reassessment inside a fixed number of days, in the contract.

Ask two separate questions here, because vendors often merge them.

Incident notice: "Within what window will you tell us about a breach affecting our data, what will the notice contain, and who acts first?" Censinet recommends breach terms with deadlines in the 24 to 72 hour range, plus stated remediation and audit support obligations. Note that the clock starts when the vendor knows, so ask them to define "knows".

Model change notice: "How much notice before the model, provider or default settings change, and can we pin a version?" This is the question almost nobody asks and almost everybody needs. Reco's table lists products that ship weekly and swap models as a distinct risk class, and notes that questionnaire answers are claims made on one day about a product that changes. Agathon makes the testing point directly: run your evaluation against a pinned model version rather than whatever the default endpoint serves that day, or you have measured a moving target.

A decision rule for the change clause: if the tool touches Tier 3 data or drives a customer facing workflow, require notice before any model change and the right to re test before it takes effect. If the tool drafts internal emails, notice after the fact is fine.

Ask: "Who owns the outputs we generate, can you use them for any purpose, and do you indemnify us if a third party claims the output infringes?"

Get the answer in the contract rather than the marketing page. Note that ownership of an output is not the same as the right to use it, and neither settles the copyright question for AI generated material. The Taylor and Francis AI policy is a useful model for how a careful organization handles this: it requires authors to disclose tool name and use, keeps accountability human, and notes that generative tools lack standard attribution practices and may reuse input or output for training, which can affect rights.

For employment and HR uses, add one more line. Littler's analysis notes that vendors of AI powered HR tools are common and that a policy improves the relationship by setting clear expectations with them. If the tool touches hiring or performance, the Fisher Phillips sample policy is blunt about the red line: no AI tool for employment decisions about applicants or employees, and no confidential or personal information uploaded at all. A vendor answer that shrugs at this is a stop.

Then ask exit: "Can we export our prompts, outputs, configuration and any fine tuned artifacts in a usable format, at what cost, and what happens to our data after termination?" Export format matters more than export existence. JSON or CSV with embeddings excluded is very different from a PDF. Ask for the schema up front, run one export during the pilot, and re import it into a test account to prove it works before signature. Then set the post termination clock: deletion of production data, then logs, then backups, each with a date and a confirmation. The Wavect questionnaire separates data deletion from exit and portability for precisely this reason, and both should be answered before you commit.

Score each question as evidenced, claimed or missing. Evidenced means a document, report or clause you can point to. Claimed means an email. Missing means silence. Then apply a simple rule that keeps procurement honest:

That last row is the one leaders miss. A vendor can honestly pass a standard assessment and still present a materially different risk from the one you documented, because the assessed product and the product you are buying are not the same object. If your shortlist includes several model providers or an internal multi model stack, the review also has to cover the routing layer, which is where a single API such as Quanta Inference changes the subprocessor question: one set of terms, one bill, one place to ask about retention and model changes instead of five.

When the policy is written but the review is stuck, a free 30 minute discovery call is a reasonable place to pressure test your tier definitions against a real shortlist.