Question 01
Which features of the product we license use AI or machine learning, and when was each introduced?
Establishes scope and, critically, timing. Compare the dates against your last assessment: anything introduced afterwards has never been assessed, however clean the file looks.
A good answer Names specific features with dates, and distinguishes AI that is on by default from AI a user must enable.
A non-answer “AI is embedded throughout the platform” — which sounds forthcoming and tells you nothing you can assess.
Question 02
Is any of our data — inputs, outputs, logs or metadata — used to train, fine-tune or evaluate any model, yours or a third party’s?
The most important question, and the one most often answered misleadingly. “We do not use customer data for training” is frequently true and beside the point: data may still be retained, reviewed by staff, used to evaluate output quality, or sent to a model provider at inference. Name all four activities so a narrow denial cannot cover the field.
A good answer Addresses each activity separately, and says what happens at inference — not only at training.
A non-answer Repeating the word “training” back at you without defining it.
Question 03
Which third parties or subprocessors provide, host or have access to the AI, and in which countries is our data processed?
Most vendors have not built the model; they are calling somebody else’s. That provider is a fourth party to you, often not named in the agreement you signed, and frequently in a different jurisdiction. This is where your supply chain got one layer deeper without anyone telling you.
A good answer Names the provider and the processing locations, and points to where the subprocessor list is maintained.
A non-answer A refusal on confidentiality grounds. Record it: you are being asked to accept an unnamed processor.
Question 04
How long are our inputs and outputs retained by the AI component, and who can see them — including your staff and any human reviewers?
Retention and human review are where confidential material actually leaks. A thirty-day retention window for abuse monitoring is ordinary and usually acceptable; what matters is that it exists, that you know about it, and that your users have not been pasting privileged or regulated content into a field that stores it.
A good answer Gives a retention period, names the purpose, and is explicit about human access.
A non-answer “Data is retained per our privacy policy.” Read the policy, then ask again citing it.
Question 05
What notice will we receive before AI functionality, model providers or data handling change, and is that commitment contractual?
This is the only answer that protects you going forward, and the one vendors resist most, because it constrains a roadmap. Everything else describes today. Without a notice commitment you will be repeating this exercise indefinitely and finding out after the fact each time. Push for it at renewal even if the rest of the answers are good.
A good answer Cites a clause, or offers to add one. Thirty days’ written notice of material change is a reasonable ask.
A non-answer A reference to the product changelog. A changelog is not notice; it is a place notice might appear.
Question 06
Can the AI features be disabled for our tenant, and does disabling them change the service, the support terms or the price?
The opt-out test, and the fastest way to learn what you are really dealing with. If AI can be switched off cleanly, you have a decision to make and time to make it. If it cannot, the AI is not a feature you adopted — it is a change to a product you already depend on, which is a materially different conversation and belongs in front of whoever owns the risk.
A good answer Confirms a tenant-level control and states the consequences plainly.
A non-answer “It is core to the platform experience.” That is a no, and it should be recorded as one.