AI Vendor Risk Assessment: Questions Your CISO Should Be Asking

Every mid-market company is adopting AI tools right now. Marketing bought an AI content generator. Sales is using an AI-powered CRM plugin. Engineering deployed a code assistant. HR is piloting an AI resume screener. And in most cases, nobody asked a single security question before swiping the corporate card.

This is not hypothetical. This is what we see in nearly every cybersecurity assessment we perform. Companies with 50 to 2000 employees are integrating AI-powered SaaS products at a pace that far outstrips their vendor risk management processes — creating an unexamined attack surface that most security teams do not even know exists.

An AI vendor risk assessment is not optional anymore. It is a foundational part of any serious AI governance program. And the questions you ask during that assessment matter far more than the checkbox compliance exercises most companies default to.

This guide gives you a practical, category-by-category framework for evaluating AI vendors. These are the questions your CISO should be asking — or the questions you should be asking if you do not have a CISO yet.

Why Traditional Vendor Assessments Fall Short for AI

Standard vendor risk questionnaires were built for conventional SaaS applications. They cover data encryption, uptime SLAs, SOC 2 compliance, and access controls. Those things still matter, but AI vendors introduce risks that traditional assessments completely miss.

A traditional SaaS tool processes your data according to fixed logic. An AI tool may learn from your data, retain it in model weights, use it to improve services for other customers, or produce outputs that are unpredictable and potentially harmful. The attack surface is fundamentally different.

Your security architecture needs to account for these new vectors. That means your vendor assessment process needs to evolve too.

If your company has not yet built a formal framework for evaluating AI tools, you are not alone. Our guide on building an AI risk management program without a dedicated team covers how to get started with limited resources.

Category 1: Data Handling and Privacy

Data is the single most important area of concern with any AI vendor. How they collect, store, process, and retain your data will determine the bulk of your risk exposure. Start here.

Questions to ask:

• What data do you collect from our usage, and what is each data element used for? Do not accept vague answers like "to improve our services." Demand a specific inventory.
• Is our data used to train or fine-tune your AI models? This is the critical question. If yes, your proprietary data could influence outputs for other customers.
• Can we opt out of model training entirely, and is that opt-out contractually binding? A settings toggle is not sufficient. You need it in the agreement.
• Where is our data stored geographically, and does any processing occur outside our jurisdiction? This matters for companies subject to GDPR, state privacy laws, or industry regulations.
• What is your data retention policy, and can we request full deletion including from model training datasets? Deletion from a trained model is technically complex. Understand what "deletion" actually means.
• Do you use sub-processors for AI inference or training, and if so, who are they? Your data may pass through third parties you have never evaluated.
• How do you handle data segregation between customers in multi-tenant AI environments? Model contamination between tenants is a real and under-discussed risk.

Companies subject to emerging AI regulations should also review our breakdown of EU AI Act compliance for US companies, as data handling requirements are tightening globally.

Category 2: Model Transparency and Explainability

You cannot assess risk for something you do not understand. Model transparency is not just a nice-to-have — it is a prerequisite for meaningful risk management.

Questions to ask:

• What type of AI/ML model powers your product, and can you provide documentation on its architecture? You do not need to understand every parameter, but you need to know if you are dealing with a large language model, a decision tree, or something else entirely.
• What data was your model trained on, and how do you ensure training data quality and bias mitigation? If the vendor cannot answer this, they either do not know or will not tell you. Neither is acceptable.
• How do you test for and mitigate bias, hallucination, and harmful outputs? Ask for specifics. "We have guardrails" is not an answer.
• Can you explain how the model reaches its outputs or decisions? For AI involved in decisions affecting people — hiring, lending, risk scoring — explainability is a legal and ethical requirement.
• How frequently is the model updated or retrained, and how are customers notified? A model update can fundamentally change behavior. You need advance notice.
• Do you publish model cards or similar documentation? Model cards are becoming a standard transparency practice. Their absence is a yellow flag.

Category 3: Security Controls

AI systems have unique security requirements beyond standard application security. Adversarial attacks, prompt injection, data poisoning, and model theft are real threats that your vendor should be actively defending against.

Questions to ask:

• What protections do you have against prompt injection and adversarial input attacks? If the vendor does not understand this question, that tells you everything you need to know.
• How do you protect your model weights and intellectual property from extraction? A compromised model could expose patterns learned from your data.
• What input validation and output filtering do you implement? Understand the guardrails between your users and the model.
• Do you conduct regular adversarial testing or red teaming of your AI systems? Ask for evidence, not assertions.
• How do you monitor for data poisoning in ongoing training or fine-tuning? If the model continues to learn, it remains vulnerable to poisoned inputs.
• What is your vulnerability disclosure and patching process for AI-specific vulnerabilities? AI vulnerabilities differ from traditional software bugs. The process should reflect that.
• Do you maintain an AI Bill of Materials (AI-BOM) that inventories model components, training data sources, and dependencies? This is the AI equivalent of an SBOM, and it is becoming a best practice.

These questions should integrate into your broader security architecture review. AI vendors do not operate in isolation — they connect to your systems, access your data, and interact with your users.

Category 4: Compliance and Regulatory Alignment

The regulatory landscape for AI is shifting fast. Your vendors need to be ahead of it.

Questions to ask:

• What AI-specific regulations or frameworks do you comply with? Look for references to NIST AI RMF, ISO 42001, the EU AI Act, or state-level AI legislation.
• Do you hold SOC 2 Type II certification, and does the audit scope include your AI/ML systems? Many SOC 2 reports explicitly exclude AI model operations. Read the scope carefully.
• How do you classify your AI system under the EU AI Act risk categories? Even if your company is US-based, your vendor may process EU resident data. Their classification tells you about their risk posture.
• Can you provide a Data Processing Agreement (DPA) that specifically addresses AI model training and inference? Standard DPAs often do not cover AI-specific data uses.
• How do you handle regulatory changes, and what is your timeline for compliance with new requirements? A vendor that cannot articulate a regulatory strategy will create compliance exposure for you.

A solid AI governance framework makes it far easier to evaluate whether a vendor's compliance posture meets your requirements. If you have not built that framework yet, start with our AI governance checklist for mid-market companies.

Category 5: Incident Response and Liability

When something goes wrong with an AI system — and it will — you need to know who is responsible and what happens next.

Questions to ask:

• What is your incident response plan for AI-specific failures, including model hallucination, data leakage, or adversarial compromise? A generic incident response plan is insufficient.
• How quickly will you notify us of a security incident involving our data or AI model compromise? Get a specific SLA, not a vague "promptly."
• Who bears liability for decisions or outputs generated by your AI that cause harm? This is where contracts get uncomfortable — and where they matter most.
• Do you carry cyber insurance that covers AI-specific incidents? If their insurance does not cover AI failures, that risk falls to you.
• Can you provide post-incident forensic data including model behavior logs, input/output records, and root cause analysis? Without this, you cannot conduct your own investigation or satisfy regulatory reporting requirements.
• What is your process for rolling back a model update that causes issues? Model rollback is not as simple as reverting a code deployment.

Red Flags and Deal-Breakers

Not every vendor risk can be mitigated. Some are disqualifying. Here are the red flags that should stop an AI vendor engagement immediately.

Immediate deal-breakers:

• The vendor cannot or will not explain how your data is used in model training. Opacity about data usage is the biggest red flag. Walk away.
• No opt-out from model training, or the opt-out is not contractually enforceable. A settings toggle the vendor can change unilaterally is meaningless.
• No incident response plan addressing AI-specific scenarios. If they have not planned for model failure, they are not ready for enterprise use.
• The vendor dismisses questions about bias, explainability, or adversarial attacks. A vendor that does not take these seriously does not understand the technology they sell.
• No independent security audit or certification that includes AI systems. Self-attestation is not assurance.

Serious yellow flags that require deeper investigation:

• The vendor is a startup with less than 18 months of operating history and no SOC 2 or equivalent.
• Terms of service grant broad rights to use customer data for "service improvement" without specific limitations.
• They cannot identify their sub-processors or downstream model providers.
• They have no published security or responsible AI documentation.
• Their pricing model incentivizes sending more data to the platform than necessary.

These red flags apply regardless of how impressive the product demo looks. A tool that creates unmanaged security exposure is not a solution — it is a liability.

Building a Repeatable Assessment Process

Asking the right questions is step one. Building a repeatable process is what separates companies that manage AI risk from companies that just worry about it.

Step 1: Create a standardized AI vendor questionnaire. Take the questions from this guide and adapt them to your organization's risk tolerance and regulatory requirements. This becomes your baseline.

Step 2: Tier your vendors by risk level. Not every AI tool carries the same risk. An AI grammar checker that processes text locally is different from a platform that ingests your customer database. Define tiers — high, medium, low — and calibrate your assessment depth accordingly.

Step 3: Assign ownership. Someone needs to own this process. In mid-market companies, this typically falls to a virtual CISO or a security-minded IT leader. Without clear ownership, assessments do not happen consistently.

Step 4: Centralize your vendor risk data. Tracking assessments in spreadsheets breaks down quickly as your AI tool portfolio grows. A platform like Radius360 lets you centralize vendor assessments, track risk posture over time, and maintain a living inventory of AI vendor relationships.

Step 5: Reassess on a defined cadence. AI products change faster than traditional software. Models get updated and data handling policies shift. Annual reviews are the minimum. Quarterly reviews for high-risk vendors are better.

If your organization is also dealing with employees adopting AI tools without going through any approval process, our piece on shadow AI security risks covers how to get visibility into unapproved AI usage.

Who Should Lead AI Vendor Assessments

In large enterprises, vendor risk management is handled by a dedicated GRC team with support from procurement, legal, and security. Mid-market companies rarely have that luxury.

The most effective approach we see is assigning AI vendor assessment responsibility to whoever owns your broader AI governance program — whether that is an internal security leader, your IT director, or a virtual CISO service. The key is that the person leading assessments understands both the technical risks of AI and the business context of how the tool will be used.

Cross-functional input matters. Legal should review data processing terms. The business unit requesting the tool should articulate the use case and data requirements. But one person needs to own the final risk decision and maintain the assessment record.

For companies that want to track vendor risk posture centrally and maintain audit-ready records, Radius360 provides a single pane of glass for managing vendor relationships alongside your broader security program.

Does My Company Really Need a Formal AI Vendor Risk Assessment?

Yes. If your company uses any AI-powered tool that touches company data, customer data, or business decisions, you need a formal assessment process. The question is not whether you can afford to do vendor assessments — it is whether you can afford the consequences of not doing them. A single AI vendor mishandling your customer data can result in regulatory fines, breach notification costs, litigation, and reputational damage that far exceeds the cost of a proper assessment.

What If an AI Vendor Refuses to Answer Our Assessment Questions?

A vendor that refuses to answer reasonable security and data handling questions is telling you something important about how they operate. Transparency is a baseline expectation, not a special request. If a vendor will not disclose how they handle your data or whether your data trains their models, treat that as a disqualifying factor. There are enough AI vendors in the market that you do not have to accept opacity. If the tool is truly irreplaceable, escalate to their security team or executive leadership and document the refusal for your risk register.

How Often Should We Reassess AI Vendors?

At minimum, conduct a full reassessment annually. For high-risk AI vendors — those that process sensitive data, make decisions affecting people, or are deeply integrated into critical workflows — reassess quarterly. You should also trigger a reassessment whenever the vendor announces a major model update, changes their terms of service, or experiences a security incident. Continuous monitoring is ideal. Track vendor security posture changes and regulatory actions between formal assessments. Building this cadence into your overall cybersecurity program ensures it actually happens.

Can We Use Our Existing Vendor Risk Questionnaire for AI Vendors?

Your existing questionnaire is a starting point, but it is not sufficient on its own. Traditional questionnaires cover important fundamentals like encryption, access controls, and business continuity — but they miss AI-specific risks entirely: model training on customer data, adversarial attack resilience, bias, hallucination, and explainability. The most effective approach is to create an AI-specific addendum that supplements your existing questionnaire, maintaining consistency while ensuring AI-specific risks are properly evaluated.

What Is the Biggest AI Vendor Risk That Mid-Market Companies Overlook?

Data leakage through model training is the risk we see overlooked most frequently. Many AI vendors use customer data to train and improve their models by default. This means your proprietary business data, customer information, and internal communications can influence the model's outputs for other customers — including your competitors. Most mid-market companies do not realize this is happening because they never asked, and the default terms of service permit it. The fix is straightforward: ask every AI vendor explicitly whether your data is used for model training, demand a contractual opt-out, and verify it is actually implemented. This single question will do more to reduce your AI vendor risk than any other step you take.