# Inside My AI Vendor Questionnaire: 10 Design Decisions and the Alternatives I Rejected

> Ten design decisions behind my AI vendor questionnaire: triage tiers, deal-breakers, written evidence, model providers, AI security, agents, and reviews that expire.

Canonical: https://nosistech.com/ai-vendor-questionnaire-design-decisions/
Published: 2026-11-18
Last updated: 2026-11-18
Author: Carlos, NosisTech LLC

My last post introduced an AI vendor questionnaire for small and mid-size companies. That post was for business owners. This one is for the people who run vendor reviews: IT leads, security and compliance people, and consultants who advise clients.

Before I designed it, I studied public vendor questionnaires, AI security checklists, and risk frameworks. They included the NIST AI Risk Management Framework, its Generative AI Profile, and the OWASP lists. Weak AI vendor reviews failed in the same ways:

- They sent the same long questionnaire to every vendor, so small vendors gave up and big vendors pasted marketing text.
- They reused a general security questionnaire, with one line about AI at the end.
- They asked "Do you train on our data?" and stopped there.
- They checked the vendor, but not the AI model company behind it.
- They added up points, so a strong answer on logins hid a weak answer on data use.
- They happened once, at purchase, and never again.

I designed the template around these failures. Below are the ten decisions that matter most, the alternatives that I rejected, and the frameworks behind each one. I quote the template directly, so you can judge the wording yourself.

---

## Decision 1: Three tiers, chosen by a short triage

**The alternative I rejected:** one complete questionnaire for every vendor.

A writing tool that sees only public text does not need 42 questions. If it gets them, the review takes weeks, and people learn to skip reviews. So the template starts with six triage questions. The answers set the tier:

- **Light:** 6 questions.
- **Standard:** 27 questions.
- **Deep:** all 42 questions.

The Excel file applies the rule. If any of T2 to T5 is "Yes", the tier is Deep. If not, T1 or T6 makes it Standard. Otherwise, it is Light. Each question carries its tier, so the "Send?" column marks the questions for the vendor.

The [NIST Generative AI Profile](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf) suggests the same idea. [GV-6.1-005](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=25) asks for a use-case based supplier risk assessment. NIST AI RMF [MAP 4.1](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf#page=32) asks you to map the risks of third-party software and data.

## Decision 2: The tier follows the use, not the price or the brand

**The alternatives I rejected:** tiers by contract value, and tiers by vendor reputation.

Many procurement teams set review depth by spend. For AI, that fails. A no-cost browser extension can read every page that your staff open. A famous vendor can still run a risky use.

So the triage asks only about your use: the data, the audience, decisions about people, actions, and how much you depend on the tool.

> **From the template, section 5:**
>
> - T4. Will the output help to decide about people, for example hiring, pay, credit, health care, housing, insurance, or education?
>
> - T5. Can the tool take actions or connect to our systems, for example send messages, change records, or read our files?

One sentence protects the triage from optimism: "If you are not sure about a triage answer, answer 'Yes'."

The review also covers AI features inside software that you already use. Section 2 starts a review "before we approve a new AI tool or a new AI feature". [GV-6.1-009](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=25) makes the same point: vendor due diligence must address solutions that rely on embedded generative AI.

## Decision 3: Deal-breakers, not a weighted score

**The alternative I rejected:** a points score, for example 100 points with a pass mark of 70.

A points score looks objective. But it lets strengths cancel weaknesses. A vendor with perfect single sign-on and a strong certificate can still pass while it trains on your data.

So the template has no total score. It marks eight questions as deal-breakers:

> **From the template, section 6:**
>
> Some answers stop the approval as planned. In the Excel file, these questions show "Yes" in the Deal-breaker column. If the answer to one of them is "Not acceptable", we do not approve the use as planned:

The eight cover training (B1), sub-processors (B6), deletion (B7), and incident notice (G2). They also cover the data processing agreement (H1), regulated data (H2), approval of actions (F3), and switch-off (F4). Each one protects something that no other answer can make up for.

A failed deal-breaker still leaves a path. The approver can choose a smaller use, for example without Restricted data, and run the triage again.

This follows NIST AI RMF [GOVERN 6.1](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf#page=29): policies and procedures that address AI risks from third parties.

## Decision 4: Four statuses, judged against the use

**The alternative I rejected:** a yes/no checklist, where "Yes" means "pass".

A yes/no checklist rewards the vendor that writes "Yes" to everything. It also ignores context. Thirty days of retention is fine for public marketing text and too long for patient notes.

> **From the template, section 7.2:**
>
> 1. Judge each answer against our use, not in general. A long retention period can be acceptable for public data and not acceptable for customer data.
>
> 2. A missing answer is "Follow-up", never "Acceptable".

The four statuses are Acceptable, Follow-up, Not acceptable, and Not applicable. "Not applicable" needs a written reason. Without that rule, "N/A" becomes the easiest way to skip a hard question.

## Decision 5: A deal-breaker needs a written source

**The alternative I rejected:** accepting the questionnaire answers as the evidence.

A questionnaire answer is a statement by a salesperson or a support engineer. It rarely binds the vendor. So every question has an "Evidence" field, and B2 asks the vendor to name the document and the section for its training answer.

> **From the template, section 7.2:**
>
> 3. For each deal-breaker, the answer must point to a contract, the vendor's terms, or an independent report. A sales email or a marketing page is not enough.
>
> 4. If the vendor's answer and its contract disagree, the contract counts.

This follows [GV-6.1-006](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=25), which asks for contract clauses that let you evaluate third-party generative AI. It also follows [GV-6.2-007](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=26), which asks you to review vendor contracts for terms that allow secondary data use.

## Decision 6: Training, retention, and feedback are three separate questions

**The alternative I rejected:** one question about "data use".

"We do not train on your data" is the answer that every vendor wants to give. It is often true. It is also incomplete. So the template splits the topic:

- **B1** asks about training, by the vendor and by its model providers.
- **B3** asks whether feedback features, for example "rate this answer" buttons, change how they use your data.
- **B4** asks how long they keep inputs and outputs, including logs that they keep to detect abuse.

> **From the template, section 7.2:**
>
> 5. "We do not train on your data" does not mean "we do not keep your data". Read the answers to B1 and B4 together.

B3 is the question that most questionnaires miss. A feedback button can send a conversation down a different path from the normal data flow. Section B as a whole addresses Sensitive Information Disclosure, LLM02 in the [OWASP Top 10 for LLM Applications 2026](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/).

## Decision 7: Follow the data to the model provider

**The alternative I rejected:** questions about the vendor only.

Many AI products run on a model from another company. Your data then reaches a second company that you never signed with. So the questions follow the data:

- **A2** asks for each model provider by name.
- **B1** and **B4** ask about the model providers too, not only the vendor.
- **B6**, a deal-breaker, asks for the sub-processors, including AI model providers, and their countries.
- **D3** asks where the models come from, and how the vendor checks them.

> **From the template, section 7.2:**
>
> 6. If the vendor uses another company's AI model, ask the same data questions about that company.

[GV-6.1-007](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=25) asks you to inventory all third parties with access to your content. [MG-3.1-005](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=47) asks you to review system cards and model cards for third-party models, which A4 requests. Section D covers Supply Chain, LLM04 on the OWASP list.

## Decision 8: AI security questions, not a general security form with an AI line

**The alternative I rejected:** our existing security questionnaire, plus one question about AI.

A general security questionnaire covers encryption, backups, and access control. Those still matter, and C1 and C6 cover the basics. But AI adds attacks that a general form never asks about:

> **From the vendor questionnaire, section C:**
>
> **C2** (Standard). How do you keep our data separate from other customers' data, including in AI search indexes and caches?
>
> **C3** (Standard). How do you protect the AI features against prompt injection, including hidden instructions in documents, emails, or web pages?

C3 names indirect prompt injection on purpose. An AI tool that reads emails or web pages can receive instructions from a stranger. That is LLM01 on the OWASP list. C2 and B8 cover AI search indexes and embeddings, which LLM09, Vector and Embedding Weaknesses, addresses.

C5 asks whether the testing goes "beyond simple jailbreak attempts". A test that only tries to make the model say something rude does not prove much about data leaks or agent misuse.

## Decision 9: Agents get two deal-breakers

**The alternative I rejected:** treating agents like any other AI feature.

An AI that writes a draft is one risk. An AI that sends the message, changes the record, or approves the refund is another. Section F asks what the AI can do, which systems it reaches, and who sees the logs. Two of its questions are deal-breakers:

> **From the template, section 6:**
>
> 7. **F3. Approval of actions.** The AI can take actions that are difficult to undo, and we cannot require a person's approval first.
>
> 8. **F4. Switch-off.** We cannot switch off the AI feature or an agent quickly.

F2 also asks about connectors, plugins, and MCP servers, and whether you can limit their access. Together, these address Excessive Agency, LLM03 on the OWASP list. F4 is also a contingency plan. NIST AI RMF [GOVERN 6.2](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf#page=29) asks for contingency processes for failures in high-risk third-party AI.

Section G connects the vendor's incidents to yours. G2, a deal-breaker, asks how fast the vendor tells you, and whether that time is in the contract. That supports [MANAGE 4.3](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf#page=38), on communicating incidents, and [GV-6.2-003](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf#page=26), on incident response plans for third-party generative AI.

## Decision 10: A review has an expiry date

**The alternative I rejected:** one review at purchase.

AI products change faster than contracts. A vendor can change its model, its terms, or its owner in a month. So D1 asks how much notice the vendor gives before a model change. D2 asks whether you can keep the earlier model or switch the feature off.

The guide also sets a review interval for each tier:

| Tier | Review again at least every |
| --- | --- |
| Deep | 12 months |
| Standard | 24 months, or at renewal if sooner |
| Light | At contract renewal |

And it lists events that force an early review:

> **From the template, section 9:**
>
> - the vendor changes the model that a feature uses
>
> - the vendor changes its data terms, its sub-processors, or its owner
>
> - the vendor has a breach or a serious AI incident
>
> - we start to use the tool with a higher data tier, or for a new use
>
> - the tool gets new actions or new connections to our systems

This follows [MANAGE 3.1](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf#page=37), which asks you to monitor third-party AI risks regularly. The decision then goes into the AI tool register from Template 2, which has its own review dates.

---

## What changes in the three examples

The three industry versions use fictional companies. Each one shows how the same questionnaire bends to a different business.

- **Riverbend Family Health**, clinics in Texas and California. The patient data agreement must cover the AI feature and every company in the data path. Extra questions cover recording consent and how long the vendor keeps visit audio.
- **Lumorra Data**, a B2B AI software company. Here the company is the buyer of model APIs. The review focuses on retention options, regional processing, and notice before a model version goes away.
- **Keystone Advisory**, a 12-person consulting firm. Client data comes first. The review focuses on "no training" in the contract, admin controls, and AI notetakers.

## About the references in this post

Each version ends with a short note on security risks. It names the items on the 2026 OWASP list that the questions cover. The healthcare version also lists the US laws behind its extra questions.

Two notes on the references in this post:

1. **I checked every NIST reference against the original NIST documents.** Each link opens the correct page.
2. **A reference is not a compliance claim.** NIST and OWASP help to explain why a question exists. Using the questionnaire does not prove that you meet a law.

## How to adapt it

- **Start with the industry version** that is closest to you, even if your industry is different. The questions transfer.
- **Keep the eight deal-breakers** for the first year. Add one if your laws or contracts require it. Remove one only with a written reason.
- **Run the triage with the requester, not alone.** The person who wants the tool knows what data it will see.
- **Review your riskiest vendor first.** One complete Deep review teaches more than five rushed Light ones.

## Get the templates

All four versions are on GitHub under CC BY 4.0. Each version has an Excel questionnaire, a review guide, and a vendor questionnaire, in Markdown and Word: [github.com/nosistech/governance-templates](https://github.com/nosistech/governance-templates).

Next in this series is Template 4: the AI incident-response playbook. It covers what to do when an AI tool goes wrong.

Have a question or feedback about these templates? [Contact NosisTech](https://nosistech.com/connect/).

### Keep reading: AI governance templates

Previous article: [What Does Your AI Vendor Do With Your Data? Ask Before You Sign](https://nosistech.com/ai-vendor-questionnaire-template/)

Template 2: [Inside My AI Tool Register: 10 Design Decisions and the Alternatives I Rejected](https://nosistech.com/ai-tool-register-design-decisions/)

Topic hub: [AI Governance](https://nosistech.com/ai-governance/)

All articles: [NosisTech Writing](https://nosistech.com/blog/)

*This post is general information, not legal advice. The example companies are fictional. Any resemblance to a real company is a coincidence. The legal references are current as of November 2026, and laws about AI change often. Ask your legal counsel before you rely on any template.*
