What to Ask Your Legal-AI Vendor: A Buyer's Checklist for Indian Firms
The 22 questions every managing partner should put in writing before signing a legal-AI contract. Covers data residency under DPDP, training rights, indemnity, evaluation methodology and exit terms, with the answers a serious vendor should be willing to commit to in the MSA.
Three MSAs, four questions, no answers
A managing partner at a Mumbai full-service firm told us this in April: “I have signed three legal-AI MSAs in eighteen months. I sat down last weekend with all three on my desk and could not, on rereading them, answer four basic questions about any of them.” The four questions were where the data sits, who trains on it, what happens on exit, and what the vendor pays if it goes wrong. He is not unusual. He is, on our sample, the median.
This piece is the 22 questions you should be able to answer about every legal-AI contract you sign. Not the boilerplate ones. The ones a serious vendor will answer straight, and a thin one will fog up.
A disclosure before we start. Firmtalk publishes this piece and is itself a legal-AI vendor. The 22 questions below are the ones we would expect a serious managing partner to ask us. We have written them at the level that hurts us most, not least. If you think we have ducked one of them when you ask us, write in and we will publish the correction with attribution.
Why 22 and not 5
The five-question checklists doing the rounds in legal procurement decks were useful in 2023. They are not useful any more. Vendors have learned the standard questions and have well-rehearsed answers ready. “Do you train on our data?” gets a confident no, with no contractual hook behind it. “Is the data in India?” gets a yes that turns out to mean the front-end is, while embeddings sit in Virginia.
The 22 questions below are the ones where vendors still differ, and where the differences matter. They are clustered into six categories: data residency and DPDP, training rights and model behaviour, indemnity and IP, evaluation methodology, service levels and reliability, and commercial and exit. The point of each one is to make the vendor say something specific, on paper, that you can hold them to later.
Two cross-currents run through the list. The first is DPDP 2025, which has made data residency a legal exposure rather than a preference. The second is what we found in our review of the best legal AI tools for Indian law firms: most Indian firms are buying in the wrong order, and most legal-AI contracts are drafted to suit the vendor’s economics, not the firm’s. The procurement conversation is where you fix both.
A serious vendor will give you a specific, dated, on-paper answer. A thin one will give you a confident yes and ask to move on.On Vendor Selection
A. Data residency and DPDP exposure
The four questions in this category are the ones that, if you get them wrong, become a regulator’s problem rather than a procurement problem. Read them alongside our DPDP 2025 compliance playbook; the obligations there are not the vendor’s problem, they are yours, and the contract is where they get pushed back down the chain.
Question 1. Where is our data processed and stored, country and region, for each of: prompts, outputs, vector embeddings, and logs?
A serious answer is a four-line table with named regions. AWS Mumbai, ap-south-1, for prompts and outputs. Azure Pune for embeddings. CloudWatch in Mumbai for logs. Specific. Auditable. The contract names the region and the vendor commits not to move it without notice. A serious vendor will also tell you which sub-component briefly leaves India in transit, if any, and why.
Red flag: “Our data is in India.” That is not an answer. Ask which data, in which service, in which region, behind what control plane. If the vendor cannot decompose the question, they are reading a marketing line, not their architecture.
Question 2. Are you a data fiduciary or a data processor under DPDP 2025 for this engagement, and will you sign a Data Processing Addendum with the obligations spelled out?
A serious answer is unambiguous: the vendor is a processor, the firm is the fiduciary, and there is a DPA template ready to sign. The template enumerates breach-notification windows, sub-processor obligations, audit rights, and the specific DPDP sections it maps to. They have done this before and will not flinch at redlines.
Red flag: “DPDP does not really apply to us because we are a US company.” It does. If the vendor cannot tell you, on the call, whether they are a processor or a fiduciary on the matter you are buying, they have not done the work, and you will be doing it for them.
Question 3. Who are your sub-processors, by named entity and jurisdiction, and what is your change-of-sub-processor notice period?
A serious answer is a named list with jurisdictions. The base model provider, the vector store, the observability vendor, the email service. Notice of any change is 30 days minimum, in writing, with a right to terminate without penalty if you object to the new sub-processor. The list lives on a URL the vendor maintains.
Red flag: “We don’t share that for competitive reasons.” You are about to send them privileged client material. The opacity is the answer.
Question 4. If we exit, in what format and within how many days will all our data be returned and deleted, with attestation?
A serious answer is on paper: 30 days for export in a documented format, raw text for prompts and outputs, JSON for matter metadata, parquet or similar for vectorised content, plus an SQL dump or CSV export of structured records. Deletion follows within a further 30 days across primary and backup systems, with a signed attestation. The clause survives termination.
Red flag: “We’ll work with you on that at the time.” The time to negotiate exit is now, when the vendor still wants you to sign. After the relationship sours, leverage is gone.
B. Training rights and model behaviour
This category is where the vendor’s long-run economics most directly conflict with the firm’s confidentiality obligations. The contractual position has to be tight, because the technical default at most providers is permissive.
Question 5. Do you train, fine-tune, or otherwise use our prompts, documents, or outputs to improve any model, in any form, on any tier? If yes on a lower tier, what is the contractual opt-out?
A serious answer is a flat no on the enterprise tier, with the no extended to all sub-processors in the DPA, and a yes-but-with-opt-out on lower tiers with the opt-out mechanism documented. The contract specifies that training includes fine-tuning, reinforcement learning, retrieval augmentation built into the vendor’s base product, and any human-review pipeline for model evaluation.
Red flag: “We do not use your data to train models” said quickly. The question is layered, and a fast answer usually means an unconsidered one. Push for the sub-processor commitment in writing.
Question 6. Which base models power the product today, and what is your notice period if you switch base model providers?
A serious answer names the models, by version, and commits to 60 to 90 days’ notice before a switch, with right of termination if the new model fails an agreed quality threshold. The vendor maintains a public model card or equivalent that you can point to in disputes.
Red flag: “We abstract that away from the customer.” That is a vendor’s preference, not a position you should accept. The base model affects quality, latency, cost, and Indian-context accuracy. You should be told when it changes.
Question 7. What is your output retention default, and how do we shorten it?
A serious answer states the default in days, names where the data is held, and gives you a tenant-level control to shorten or zero out retention. Thirty days is a defensible default; ninety days is not, unless there is a specific operational reason and an explicit opt-out.
Red flag: “Indefinite, but only for service improvement.” Service improvement is training by another name in 80% of cases.
Question 8. What logging is enabled by default that contains client data, and how do we turn it off?
A serious answer distinguishes between operational logs that need to exist for the service to work, debug logs that can be turned down to a sampled fraction, and content logs that should be off by default in legal tenants. The vendor gives you tenant-level controls for the last two.
Red flag: vague language about “telemetry” that does not separate metadata from content. Privileged drafts in a third-party log store are a problem whether the vendor calls it telemetry or not.
C. Indemnity and IP
This is the category where MSAs are weakest at Indian firms, in our reading of 23 legal-AI contracts signed across our 2026 benchmark cohort. The standard template shifts almost all model-output risk back to the customer. That is the vendor’s opening position, not the answer.
Question 9. Will you indemnify against IP infringement of model outputs, and what is the cap, the carve-outs, and the trigger, in writing?
A serious answer offers a real IP indemnity, capped at a multiple of the annual fee, triggered by a third-party claim against the customer arising from use of the tool within agreed terms. Carve-outs are limited to gross misuse and fine-tunes the customer built themselves on prohibited content. Two of the eight vendors we evaluated in the AI tools piece offer this without a fight; three offer it with a redline; three resist entirely.
Red flag: “Outputs are the customer’s responsibility.” That is the model card. It is not a contract position.
Question 10. Will you indemnify against confidentiality breach if a sub-processor exposes our data?
A serious answer treats sub-processor breach as the vendor’s breach for contractual purposes. The cap may be higher than the IP cap, the trigger is a confirmed exposure, and the obligations include cost of notification to affected parties and regulatory engagement.
Red flag: a back-to-back clause that says the vendor will assign whatever rights it has against the sub-processor. That is the vendor passing through its own limited recourse and calling it an indemnity. It is not.
Question 11. What is your liability cap on hallucination, and are there carve-outs for material misstatement?
A serious answer acknowledges that hallucinations are a known failure mode and offers a defined liability for cases where the vendor’s published quality claims were materially false. The cap may be modest, but the principle that vendors carry some risk for what their marketing promises is the right one.
Red flag: “We make no warranties about output quality.” If the deck promises a 92% citation accuracy rate, the contract should not disavow the deck. Either the number is real or it is not.
Question 12. Who owns the output, including fine-tuned model weights derived from our content?
A serious answer assigns output to the customer outright. Fine-tuned weights derived from customer content are jointly owned or assigned to the customer, with a licence back to the vendor for the limited purpose of delivering the service to that customer alone, never to be used in any other tenant or shared model.
Red flag: “Fine-tunes become part of the platform.” That is the firm’s knowledge being commoditised across the vendor’s book of business, including, sometimes, your competitors.
D. Evaluation methodology and benchmarks
The decks claim numbers. The questions in this category are whether you can stand behind those numbers when a partner asks where they came from. In our sample, six of eight vendors fold under a careful version of question 13.
Question 13. What is your published evaluation methodology for the claims you make in the deck, and will you let us run our own evaluation on our own matters before signing?
A serious answer is a methodology document with sample size, scorers, rubric, and date. The vendor will run the same evaluation on a sample of your matters in a two-week sandbox, no charge, before contract execution. The numbers may move somewhat on your data; the vendor will tell you why.
Red flag: the deck quotes accuracy figures with no link to a methodology and no willingness to repeat them on your sample. The numbers are either real and repeatable or they are marketing decoration. Find out which before you sign.
Question 14. What is your hallucination rate on Indian legal citations, measured how, on which sample?
A serious answer gives a percentage, a measurement protocol, a sample size and composition, and the date of the last measurement. The percentage is honest. The vendor will tell you the failure modes: wrong year on a correctly-named case, correct case cited for the wrong proposition, fabricated case entirely. They will also tell you which Indian sources they have indexed and which they have not.
Red flag: a single number with no protocol. We have seen vendors quote 96% accuracy on Indian citations with no published methodology. When we asked, in writing, for the test set, the conversation ended. Those are the vendors whose citation discipline our two-partner review marked lowest.
Question 15. Will you provide a 60-day side-by-side pilot against an incumbent of our choosing, with success criteria agreed up front?
A serious answer is yes, with a written pilot plan: matters covered, evaluators named, rubric agreed in advance, decision date fixed. The vendor will price the pilot at zero or near zero because they expect to win on the merits.
Red flag: pilot pricing close to the full annual licence, or insistence on a decision before pilot data is in. Either suggests the vendor does not expect to win on the merits and wants you committed before you find out.
E. Service levels, support and reliability
Legal-AI vendors are mostly young companies. Some have grown faster than their operations teams. The questions in this category surface where the cracks are.
Question 16. What is your uptime SLA, what is the credit regime, and what was your actual uptime in the last 12 months?
A serious answer is a 99.5% or 99.9% SLA, a credit regime that escalates with repeated breach, and a 12-month uptime history shared on request. The credit is meaningful, not 5% of the monthly fee for a 12-hour outage.
Red flag: an SLA without an uptime history, or an SLA with credits so trivial that the vendor is not actually exposed to its own number.
Question 17. What is the P1 response time, and is the person on the other end of the line Indian-counsel-aware or first-line generic?
A serious answer is a defined P1 SLA, a named escalation path, and a support team that knows the difference between a writ petition and a winding-up petition. For India-based firms, this often means a dedicated India support pod or at least an India-time-zone first-line.
Red flag: a US-only support footprint with email-only contact and a 24-hour first-response target. By the time you get a useful response, the partner has moved on without the tool.
Question 18. What is your change-management policy: do you push model upgrades into production without our approval?
A serious answer is a documented change-management process, with material model upgrades behind a tenant-level opt-in or at least a 30-day preview window. The vendor will tell you what counts as material and what is treated as a routine patch.
Red flag: “We continuously improve the product.” That is fine for a consumer product. For a tool generating client deliverables, silent model swaps are a quality and audit problem.
F. Commercial and exit
The four commercial questions are where the deal economics actually live. In our review of the 23 Indian legal-AI contracts in the benchmark cohort, the firms that did best on three-year economics were not the ones that negotiated the steepest year-one discount. They were the ones who negotiated the renewal and exit clauses hardest.
Question 19. What is the pricing basis: per seat, per matter, per token, or all-you-can-eat? And what happens if our usage doubles?
A serious answer maps the pricing basis to a usage forecast and tells you, in writing, what happens at 1.5x, 2x and 3x your contracted volume. Overage rates are defined, capped, and reset annually. The vendor will offer you a usage review at 90 days to right-size, without penalty.
Red flag: per-token pricing with no overage cap. That is a metered tap that runs while the firm sleeps, and we have seen INR 18 lakh monthly bills surface from what was meant to be a INR 60 lakh annual deal.
Question 20. What is your contractual exit clause, including data export format and migration assistance?
A serious answer is in the contract, not in a side letter or in good-faith language. Export is in named formats. Migration assistance is a defined number of hours of vendor-side engineering time, free, plus a transition period during which your data remains accessible. The clause survives termination by either side.
Red flag: language about “reasonable cooperation” with no specifics. Reasonable cooperation, once the relationship has broken down, is whatever the vendor decides it is.
Question 21. Will you contractually commit to no rate increase above CPI for the initial term?
A serious answer is yes, with CPI defined and a hard cap if CPI itself runs wild. For multi-year deals, the cap matters more than the headline discount; the deals we have seen go wrong almost always go wrong at the price-step at renewal.
Red flag: “Pricing is reviewed annually at our discretion.” That language gives the vendor a renegotiation right while denying you the same.
Question 22. Do you offer a renewal break clause at 12 months, no questions asked, if we are not seeing benefit?
A serious answer is yes, or yes-with-modest-condition, on a multi-year deal. Vendors confident in their product offer it without flinching. The condition, if there is one, is usually a quarterly business review process to give the vendor a chance to fix problems before the break is exercised.
Red flag: insistence on a full three-year term with no exit. That is the vendor’s revenue desk speaking, not the product team. The product team knows that customers who cannot leave do not advocate for the product internally; only customers who could leave but chose not to, do.
The two answers, side by side
Over the 22 questions, a pattern emerges. Serious vendors give one shape of answer. Thin vendors give another. The difference is rarely in the headline word, the yes or the no. It is in the specificity that follows.
- Named region, named service, named sub-processor.
- DPA template ready, marked up against DPDP 2025 sections.
- Indemnity in the body of the contract, with a real cap.
- Evaluation methodology document, sample size disclosed.
- Exit clause with named formats and committed engineering hours.
- Pricing schedule with overage caps and CPI ceiling.
- “Our data is in India,” with no decomposition.
- “DPDP does not really apply to us.”
- “Outputs are the customer’s responsibility.”
- Single accuracy number, no published protocol.
- “We’ll work with you on exit at the time.”
- Per-token pricing, no overage cap, three-year lock.
The eight red flags worth a screenshot
Below is the short list we now send managing partners ahead of a vendor meeting. If three of the eight show up in one conversation, that vendor is not ready for an Indian law firm yet, however slick the demo.
| # | Red flag heard on the call | What it actually means |
|---|---|---|
| 1 | “Our data is in India.” | The front-end is. Embeddings and logs may not be. |
| 2 | “DPDP does not really apply to us.” | The vendor has not done the legal work. You will. |
| 3 | “We don’t share our sub-processor list.” | You are sending privileged material into an opaque chain. |
| 4 | “We do not use your data to train,” said quickly. | The sub-processor commitment is missing. |
| 5 | “Outputs are the customer’s responsibility.” | No IP indemnity. The risk lives with you. |
| 6 | Single accuracy figure with no methodology. | Marketing number, not a measured one. |
| 7 | “We’ll work with you on exit at the time.” | No leverage when you most need it. |
| 8 | Per-token pricing, no overage cap, three-year term. | The vendor is hedged. You are not. |
What to ask in the first 20 minutes
If you only have a short meeting, before the legal-AI procurement process opens formally, five questions will tell you whether this vendor is worth a full RFP. The point is not to extract perfect answers; it is to see how the vendor handles specificity under light pressure.
Where do our embeddings sit, by named region? If the answer is not a region code, the vendor is not engineering-led.
Will you sign a DPDP Data Processing Addendum and a no-training clause that covers your sub-processors? If the answer is “our standard MSA covers it,” the answer is no.
What is your hallucination rate on Indian citations, and can we replicate the test on our matters? The replicability question is the one that separates real numbers from marketing.
What happens at month 12 if we are not seeing benefit? The vendor’s answer tells you how confident they are in their own product.
If we exit at the end of the term, how do we get our data back? The fluency of the answer correlates almost perfectly with the maturity of the vendor’s operations team.
What changes once you ask these questions
The candid bit. Three things happen, on the evidence of the dozen Indian firms we have coached through this checklist in the last year.
Some vendors fold. They cannot commit to the specifics in writing, and they would rather walk than be held to them. Roughly a quarter of the vendors we have seen the checklist run against do this. It is the cheapest, fastest part of the procurement process; the firm has just saved itself an MSA it would have regretted at month 11.
Some vendors upgrade their offer. They had been hoping the standard MSA would go through, because that is how procurement usually goes. Asked to put DPDP, training rights, indemnity and exit on paper, they do. Pricing sometimes moves with the rewrites, sometimes does not. Half of the vendors we have seen behave this way.
Some vendors walk. They are not interested in customers who ask 22 questions. That is a signal too. Vendors who walk from this conversation are vendors who would have walked from a renewal conversation at year three, when leverage is reversed and your data is inside their system. Better to know now.
The triage is the point. The 22 questions are not a hostility test. They are a maturity test. Mature vendors enjoy them. Immature ones reveal themselves. Either way, you have shortened the list and earned the right to push hard on the survivors.
What to do this quarter
Three actions, in order. The first two cost nothing.
One. Run the 22-question checklist against every legal-AI contract you have already signed. Score each contract red, amber, green per question. The exercise will take a senior associate two days per contract. Most firms we have done this for find that no signed contract is green across all 22; the median is 9 reds. That is the gap to close at renewal.
Two. Build a one-page procurement standard from this list and circulate it to your COO and head of risk. The standard says these 22 questions are mandatory in any legal-AI RFP, and these eight red flags trigger an automatic escalation to the managing partner. Once the standard exists in writing, the firm has institutional memory between procurement cycles.
Three. For your next legal-AI MSA, agree the answers with the vendor in a working session before drafting. Two hours, joint document, vendor counsel and your COO in the room. The drafting that follows is twice as fast and the contract is three times as defensible. We have run this for four firms now; in every case the vendor admitted afterwards that it was the most productive procurement meeting they had had with a customer that year.
A standing offer
Most legal-AI procurement at Indian firms is run by the COO and a senior associate, on the back of a vendor-drafted MSA, with the partnership rubber-stamping at the end. The conversations we are recommending are not technically hard. They are uncomfortable, and they take time. They also pay back, in our experience, twice in the first year: once in commercial terms negotiated, and once in vendors avoided.
The firms that take this seriously are the same firms that are pulling away on every other operational metric in our 2026 Indian Law Firm Benchmark. Procurement discipline is not glamorous. It is, in 2026, one of the highest-leverage two-day projects a managing partner can sponsor.
A note on the data. The 23 legal-AI MSAs referenced in this piece were shared, in anonymised form, by Indian firms in the same survey set as the 2026 Indian Law Firm Benchmark. Vendor behaviour observations are drawn from our own evaluation calls with eight named legal-AI vendors during the benchmark period, plus structured conversations with twelve managing partners and four COOs at firms ranging from 35 to 220 lawyers. All firm and vendor specifics referenced anecdotally are anonymised.
Firmtalk and this view. We are a legal-AI vendor and we sell into the same procurement processes we are describing here. Of the 22 questions above, we have set ourselves the discipline of having a written, specific answer to all of them in our own MSA. We do not get every one right; the indemnity language and the published evaluation methodology are where we have invested most heavily over the last six months, because they are where we were weakest a year ago. When a customer scores us on this checklist, we want to see where we still are not serious.
