Reference call script: [Vendor]
| Item | Detail |
|---|---|
| Reference | [Reference company, contact name and role] |
| Date of call | |
| Called by | |
| Engagement described |
Opening: thank them, say the call takes about 30 minutes, and that you will not share their answers with the vendor. Ask them to describe their engagement before you ask anything else.
Context
1. What did you hire them to do, and how long have they been working with you?
Listen for: A specific scope and start date that match what the vendor told you. If the reference describes a much smaller or different engagement, weigh the rest of the call accordingly.
Notes:
2. How similar is your product and setup to ours: the technology, release frequency and regulation you work under?
Listen for: Concrete overlap with your situation. A reference from a very different product tells you about the vendor's manners, not about how they would handle your risks.
Notes:
Staffing continuity
3. Are the people who started on your account still on it? If some left, how was the change handled?
Listen for: Names and rough dates. Notice given, overlap between leaver and replacement, and whether the reference had any say. Vague answers such as "they always find someone" suggest rotation the reference has stopped tracking.
Notes:
4. Were the people you met during the sale the people who then did the work?
Listen for: Whether senior people from the pitch stayed involved after signing. A plain "no, we never saw them again" is a useful answer.
Notes:
5. How long did it take a new tester on their side to become useful on your product?
Listen for: A number of weeks and what helped or slowed it. Good vendors bring documentation and pairing; weak ones rely on the client's staff to train each new person.
Notes:
Delivery
6. Think of the last release they tested. What did they find, and did anything important get through to production?
Listen for: A recent, specific story with both catches and misses. Only praise, or no example at all, usually means the reference is not close to the work.
Notes:
7. How do they decide what to test first when time is short?
Listen for: A description of risk-based choices the reference has seen in practice, including what was left out and who agreed to that. "They test everything" is not an answer.
Notes:
8. Did they keep to the service levels in your contract, and how do you know?
Listen for: Whether the reference has its own measures or relies on the vendor's reports. Ask whether any service credit was ever claimed and how that went.
Notes:
Automation handover
9. Who owns and can run the automated tests today? Could your team keep them going if the vendor left tomorrow?
Listen for: Tests stored in the client's repository and run in the client's pipeline, with in-house engineers who have changed them. Hesitation here suggests dependence on the vendor.
Notes:
10. How much of their time goes on fixing unstable or broken tests, and do they report it?
Listen for: Awareness of flaky tests and a routine for dealing with them. If the reference does not know, the vendor is probably not reporting it.
Notes:
Communication
11. When they find a serious problem late in the day, how and how quickly do you hear about it?
Listen for: A named channel and a real example with timing. Watch for problems that surfaced only in a weekly report.
Notes:
12. What do their regular reports tell you, and which parts do you actually use?
Listen for: Reports that help decisions about releases and risk, not only counts of test cases and defects. A reference who never reads them may not have a view on quality at all.
Notes:
Commercials
13. Has the cost ended up close to what was agreed at the start? Where did extra charges come from?
Listen for: Specific sources of extra cost such as change requests, rate rises or extra headcount, and whether they were agreed in advance.
Notes:
14. When you asked for something outside the original scope, how did they respond?
Listen for: Whether small requests were absorbed sensibly or each one became a priced change. Either can be fair; the question is whether it matched what was agreed.
Notes:
Problems and recovery
15. Tell me about the worst week you have had with them. What went wrong and what did they do?
Listen for: An honest account of a real problem and a clear recovery, including who on the vendor side took responsibility. A reference who can think of nothing may be too distant from the work, or chosen for politeness.
Notes:
16. Have you ever had to escalate above your day-to-day contact? What happened?
Listen for: Whether an escalation route exists, who was on it and how fast things changed. Repeated escalations for the same issue are a signal worth noting.
Notes:
Security and data
17. What access to your systems and data do their people have, and have they ever had a security or data incident on your account?
Listen for: Named accounts, limited access and an incident either denied with confidence or described with how it was reported. Uncertainty about who has access is itself informative.
Notes:
Exit
18. If you ended the contract, how hard would it be to move the work in-house or to someone else?
Listen for: Whether documentation, test assets and knowledge sit with the client. If the reference has not thought about it, ask what they would lose.
Notes:
19. Have you seen them hand work over at the end of a phase or contract? How complete was it?
Listen for: A real handover with documents, code, walk-throughs and time allowed. Answers based only on what the vendor promised are weaker evidence.
Notes:
Overall
20. Would you hire them again for the same work? What would you change in the contract if you did?
Listen for: The changes they name are often the most useful part of the call: they point to clauses you should add now.
Notes:
21. Did the vendor ask you to take this call, and how often do they ask?
Listen for: Frequent requests suggest a small set of references used for every buyer. Ask the vendor for a client that has left them, or one of similar size to you.
Notes:
After the call
- Write down specifics the reference gave: names, dates, numbers.
- Note questions they avoided or answered in general terms.
- Compare with what the vendor's proposal claims about this engagement.
- Score the vendor's past performance criterion in the scorecard.