How to check a QA vendor's references properly

Updated 2026-09-24

To check vendor references properly, choose who you call rather than accepting the vendor's list, ask for at least one former client, and speak to the person who managed the vendor day to day, not only the executive sponsor. Ask about specific events: the people who started and who is still there, the last release, the worst week, extra charges, and how hard it would be to leave. Record the answers against the same questions for every vendor so they can be scored.

What to check

  1. Ask for a list of clients with similar work, then choose which ones to call yourself.
  2. Ask for at least one former client, and ask why the engagement ended.
  3. Speak to the person who managed the testers day to day, not only the sponsor who signed.
  4. Use the same core questions for every reference, and write the answers down during the call.
  5. Ask each reference whether the vendor asked them to take the call and how often that happens.
  6. Compare what references say with the proposal, and follow up on any difference with the vendor.

Ask the vendor

  • Can you give us a list of current clients with work similar to ours in size, domain and model, so that we can choose which ones to call?
  • Can you give us a former client whose engagement ended in the last two years, and tell us why it ended?
  • Can we speak to the person on the client side who worked with your testers day to day?
  • Which of the people proposed for our account worked on these reference accounts?

Why most reference calls tell you little

Vendors choose their references, brief them, and ask the same few happy clients again and again. A reference call that asks "are you satisfied?" gets a yes. The call becomes useful only when you choose who to speak to and ask about what actually happened.

Past performance is a standard part of source selection in public procurement. US federal rules, for example, treat past performance as an evaluation factor and let evaluators use information from sources other than those the offeror names. You can apply the same idea: the vendor's list is a starting point, not the whole picture.

Choose who you call

Ask for a long list. Ask each vendor for clients with work similar to yours in size, domain and engagement model, with contact names, service and dates. Then choose. If the vendor offers only two or three names, ask for more.

Ask for a former client. A client that has left is the only one who can tell you how the vendor handles the end of a contract. Ask the vendor why the engagement ended, then ask the client the same question.

Speak to the tester-facing person. The executive sponsor who signed the contract knows the price and the relationship. The engineering manager or QA lead who worked with the testers every day knows whether the work was good, whether people changed, and how much management it needed.

Check the fit. A glowing reference from a very different client, such as a large enterprise with a long waterfall cycle when you ship weekly, tells you less than a lukewarm one from a client like you.

Questions that get real answers

The reference check tool generates a call script from the framework's reference questions. The ones that tend to surface the most:

Topic Question
Staffing Are the people who started on your account still on it? If some left, how was the change handled?
Bait and switch Were the people you met during the sale the people who then did the work?
Ramp-up How long did it take a new tester on their side to become useful on your product?
Real work Think of the last release they tested. What did they find, and did anything important get through?
Problems Tell me about the worst week you have had with them. What went wrong and what did they do?
Cost Has the cost ended up close to what was agreed? Where did extra charges come from?
Automation Who owns and can run the automated tests today? Could your team keep them going if the vendor left?
Exit If you ended the contract, how hard would it be to move the work?
Honesty check Did the vendor ask you to take this call, and how often do they ask?

Ask about events, not opinions. "How was their communication?" invites a polite answer. "When they found a serious problem late in the day, how and how quickly did you hear about it?" invites a story.

Listen for what is not said

Some signals in a reference call are in the gaps:

  • hesitation on the question about staff continuity;
  • "we manage them quite closely" (the vendor needs direction);
  • "the first few months were rough" with no explanation of what changed;
  • a reference who cannot name anyone on the vendor's team;
  • answers that repeat the vendor's marketing phrases.

Follow up gently: "What would you do differently if you were starting again?" often gets the most honest answer of the call.

Record, score and compare

Write answers down during the call. Score each vendor's references on the same criteria in the scorecard, such as relevant past performance and staff continuity. Then compare with the proposal. Where a reference contradicts the proposal, for example on team stability or automation ownership, put it back to the vendor and record its answer.

References complement other checks rather than replacing them. The due diligence checklist covers the legal entity, insurance and financial signals, and how to verify ISO 27001 and SOC 2 covers security claims. For the staffing questions in more depth, see how QA vendors staff projects.

Tools for this

Questions buyers ask

The vendor says its clients are under NDA and cannot be named. What now?
Some clients genuinely do not allow it. Ask the vendor to request permission from a few, and ask for a call rather than a written statement. If no client at all will speak to you, treat past performance as unverified when you score it.
How many references should I call?
Enough to hear a pattern. For a shortlist, a few calls per vendor, including one former client, usually shows whether the picture from the proposal holds. Keep the calls short and focused.
Can I use contacts I find myself rather than the vendor's list?
You can, and it is often more informative. Be open about who you are and why you are asking, respect confidentiality, and do not share what one reference said with another.

Sources

  1. FAR Part 15: Contracting by Negotiation (source selection), US Federal Acquisition Regulation
  2. The Sourcing and Consultancy Playbooks, UK Cabinet Office
  3. ISO 37500:2014 Guidance on outsourcing, ISO