Methodology

Who, by name and role, will work on our account in the first three months? Please share their CVs and confirm these are the people who will start, not examples.

Why: The people named in a proposal are often not the people who arrive. Asking for named CVs up front lets you hold the vendor to the team you evaluated.

Strong answer: Named individuals with CVs, their current allocation, a start date for each, and a willingness to list them as key personnel in the contract.

Weak answer: "Sample profiles", "a team like this", or "we will assign the right people once the contract is signed".

What is the seniority mix of the proposed team, and how many years of testing experience does each person have on products like ours?

Why: A proposal priced for senior testers and staffed with juniors is a common gap between sale and delivery. The seniority mix also tells you who will design tests and who will run them.

Strong answer: A table of each role with seniority level, years in testing, relevant domain experience, and who reviews the work of less experienced staff.

Weak answer: "A balanced team of experts" with no breakdown, or seniority titles that do not match the experience in the CVs.

If a named team member leaves or is moved to another client, how much notice do you give us, who approves the replacement, and how is knowledge handed over?

Why: Staff turnover is where outsourced testing loses context. Notice, buyer approval and an overlap period are the controls that keep continuity.

Strong answer: A stated notice period, the buyer's right to interview and approve replacements, a paid or unpaid overlap for handover, and a written handover note.

Weak answer: "We have low attrition" or "we will replace them quickly" without notice, approval or overlap commitments.

What proportion of the proposed team are your own employees, and will any work be done by contractors or subcontractors? If so, which roles and where are they based?

Why: Subcontracted staff change who holds your data and who you can hold accountable. You need to know before contract, not after.

Strong answer: A clear split of employees and contractors per role, named subcontractors if any, their locations, and a commitment to seek consent before adding new ones.

Weak answer: "We work with a wider network of specialists" or reluctance to say whether the testers are on the vendor's payroll.

Which testing qualifications do the proposed testers hold, and how can we verify them?

Why: Certifications do not prove skill, but they show a shared vocabulary and can be checked. A claim that cannot be verified should carry no weight.

Strong answer: Named qualifications per person, such as ISTQB levels, with certificate numbers or a way to check them against the issuing board's register.

Weak answer: "Our team is fully certified" with no names, levels or verification route.

Who is the single accountable lead on your side, how much of their time is allocated to us, and what decisions can they make without escalating?

Why: A lead spread across many accounts cannot make timely decisions. Knowing their allocation and authority tells you how fast problems get resolved.

Strong answer: A named lead with a stated percentage of time on your account, a list of decisions they own, and the escalation route for the rest.

Weak answer: "Your account manager will coordinate everything" with no allocation or decision rights stated.

How will your testers work inside our team day to day: which of our meetings do they attend, whose backlog do they work from, and who on our side assigns their work?

Why: Testing works well when testers sit in the delivery team rather than receive builds over a wall. The answer shows whether the vendor expects to be embedded or separate.

Strong answer: A concrete working model: which ceremonies they join, the shared tools and backlog they use, and a named buyer contact who prioritises their work.

Weak answer: "We adapt to any process" with no description of how, or a model where testing happens only at the end of each release.

How many working hours overlap between your team and ours, and who is available during our core hours when a release is blocked?

Why: Time zone gaps turn a one-hour question into a one-day delay. The overlap and on-call arrangements decide how quickly blocking defects are handled.

Strong answer: Stated overlap hours per location, named people covering your core hours, and how urgent issues are reached outside the overlap.

Weak answer: "We are flexible with time zones" or "we follow the sun" without hours, names or a contact route.

How quickly can you add or reduce testers, what notice do you need in each direction, and does the rate change when the team size changes?

Why: Testing demand rises and falls with releases. Scaling terms decide whether you pay for idle capacity or wait weeks for extra hands.

Strong answer: Written notice periods for scaling up and down, the lead time to onboard a new tester, and a rate card that states whether volume changes the price.

Weak answer: "We can scale instantly" with no notice periods, or scaling terms that only become clear after signing.

Can we interview the proposed testers before we sign, and can we run a short paid pilot with the same people?

Why: An interview and a pilot show how the actual team thinks and communicates, which a proposal cannot. Public buyers use pilots to test delivery before committing.

Strong answer: Agreement to interviews with the named people and a scoped pilot with defined deliverables, duration and exit criteria, staffed by the same team.

Weak answer: Interviews only with a sales engineer or solution architect, or a pilot staffed by people who will not continue on the account.

Please share an anonymised test strategy or test plan you have written for another client. What sections does it contain?

Why: A real document shows how the vendor plans testing, far better than a description of their method. ISO/IEC/IEEE 29119-3 sets out what test plans and strategies are expected to cover.

Strong answer: A redacted document covering scope, risks, test levels and types, entry and exit criteria, environments, data, roles and schedule, broadly in line with ISO/IEC/IEEE 29119-3.

Weak answer: A marketing slide describing "our proprietary methodology" or a template with no sign it has been used on a real project.

How do you decide what to test first and what not to test? Walk us through how you would assess risk on our product.

Why: No team can test everything. A risk-based approach shows the vendor will spend effort where failure would hurt you most.

Strong answer: A described method for rating risk by likelihood and impact, how it is agreed with the buyer, and how it drives test depth and order.

Weak answer: "We aim for full coverage" or "we test everything thoroughly" with no way of choosing between tests.

What are your entry and exit criteria for a test cycle, and who has the authority to say a release is not ready?

Why: Without agreed exit criteria, "testing is done" means whatever the schedule allows. You need to know who can halt a release and on what grounds.

Strong answer: Written, measurable entry and exit criteria agreed per release, and a clear statement that the vendor reports readiness while the buyer decides.

Weak answer: "We test until the deadline" or an answer that avoids saying who can block a release.

What does a defect report from your team contain, and can you show us a few real, anonymised examples?

Why: Poor defect reports waste developer time on reproduction. Real examples show the quality you will receive every day.

Strong answer: Examples with clear steps to reproduce, expected and actual results, environment and build, severity and priority, and evidence such as logs or screenshots.

Weak answer: A template only, or examples that say "does not work" without steps, environment or evidence.

How do you fit testing into an agile, iterative delivery cycle? At what point in a sprint or release does your team start testing a change?

Why: Testing that starts only after development ends turns every sprint into a queue. Government digital guidance expects testing to run throughout delivery, not at the end.

Strong answer: Testers involved in refinement, tests written alongside development, regression run on every build, and a description of what happens when a story fails testing late.

Weak answer: "We have a dedicated testing phase after development" presented as the default for an iterative team.

How do you plan and record exploratory testing so that we can see what was covered and what was found?

Why: Exploratory testing finds problems scripted tests miss, but unrecorded sessions cannot be reviewed or paid for fairly. Records make the work visible.

Strong answer: Session charters, time boxes, notes of areas covered and findings, and a summary per session or per release that the buyer can read.

Weak answer: "Our testers explore freely" with no charters or records, or exploratory hours that appear only as a line on the invoice.

How do you cover non-functional quality such as performance, accessibility and security? Which of these do you do yourselves, and which would you leave to us or a specialist?

Why: Functional testing alone leaves large quality characteristics untested. You need to know which gaps remain so you can plan for them.

Strong answer: A clear list of what is in and out of scope, the standards used (for example ISO/IEC 25010 characteristics, WCAG 2.2 for accessibility), and the skills of the people who would do it.

Weak answer: "We cover all types of testing" with no scope limits or named standards.

How do you get and manage test data? Will your team ever need access to production data, and if so, how is it masked or protected?

Why: Test data is where personal data most often leaks into environments with weaker controls. The answer shows whether the vendor has thought about this before you ask.

Strong answer: A described approach using synthetic or masked data by default, named controls when real data is unavoidable, and agreement that the buyer approves any use of production data.

Weak answer: "Just give us a copy of the production database" or no distinction between test data and live data.

What test environments do you need from us, and how do you make sure results in those environments reflect how the product behaves in production?

Why: Tests run in environments unlike production give false confidence. The vendor's environment needs also affect your cost and timeline.

Strong answer: A list of environments, access and data required, how differences from production are identified and tracked, and who maintains each environment.

Weak answer: "We will use whatever you have" with no view on how environment differences affect test results.

In your first 30 days on our account, what will you produce and how will we know onboarding has worked?

Why: The start of an outsourcing relationship sets its pattern. A defined transition plan with outputs gives you an early checkpoint to act on.

Strong answer: A dated onboarding plan with named outputs such as a test strategy draft, a risk map and a first test run, plus the criteria for calling transition complete.

Weak answer: "We will ramp up over the first few months" with no deliverables or checkpoints.

Where will the test automation code live? Will it be in a repository that we own and control from the first commit?

Why: If the test code sits in the vendor's repository, leaving them means losing it or paying to get it back. Owning the repository from day one removes that lever.

Strong answer: Code committed to the buyer's repository under the buyer's organisation account from the start, with the vendor given access rather than the other way round.

Weak answer: "We host it in our secure repository and share access" or "we hand over the code at the end of the contract".

Who owns the intellectual property in the test code, test cases, data and documentation you produce for us, and does any of it depend on your own libraries or accelerators?

Why: Vendor libraries embedded in your tests can leave you unable to run or change them without a licence. Ownership needs to be written into the contract.

Strong answer: Clear transfer or assignment of all deliverables to the buyer, a list of any pre-existing vendor components, and a perpetual licence to use those if they remain.

Weak answer: "Our proprietary framework speeds things up" with no statement of who owns it or what happens to it when the contract ends.

Which automation framework would you use for our product, and why that one? What would make you choose differently?

Why: The framework should fit your stack and your team's skills, not the vendor's habit. A reasoned choice also shows the vendor understands trade-offs.

Strong answer: A choice tied to your languages, platforms and team, with trade-offs named, for example why Playwright, Cypress, Selenium or Appium fits or does not, and a preference for open standards and open-source tools you can maintain yourself.

Weak answer: The same framework for every client regardless of stack, or a closed tool that only the vendor can run.

How do you define, detect and deal with flaky tests? What do you do with a test that passes and fails on the same code?

Why: Flaky tests teach a team to ignore red builds. How a vendor handles them shows whether their automation will stay trustworthy over time.

Strong answer: A definition of flakiness, how it is measured, quarantine rules, an owner and deadline for each fix, and reporting of flaky rates to the buyer.

Weak answer: "We just rerun failed tests" or "our tests are never flaky".

How will the automated tests run in our CI pipeline, and how long should a full run take before it slows our developers down?

Why: Automation that does not run on every change finds problems late. Run time matters because slow suites get skipped.

Strong answer: Tests triggered on each change in the buyer's pipeline, a target run time, a plan for splitting fast and slow suites, and results visible to developers.

Weak answer: Tests run manually by the vendor on their own machines, or nightly only, with results sent by email.

How do you keep test code maintainable? Do you have coding standards, code review and a way of structuring tests that our developers could follow?

Why: Test code is code. If it is hard to read or change, it becomes a cost you carry long after the vendor has gone.

Strong answer: Written coding standards, peer review on every change, a described structure such as page objects or similar patterns, and linting in the pipeline.

Weak answer: "Our engineers know what they are doing" or recorded scripts with no review process.

How do you decide which tests to automate and which to keep manual, and how will you report automation coverage to us?

Why: Automating everything is expensive and automating the wrong things adds maintenance without value. You need a rationale and a way to see progress.

Strong answer: Criteria for automation such as repetition, risk and stability, a view on the test pyramid, and coverage reported against agreed risks or features rather than test counts alone.

Weak answer: "We will automate every test" or coverage reported only as a growing number of test cases.

If we ended the contract tomorrow, could our own engineers run, read and extend the test suite without you? What documentation would they have?

Why: This tests whether you are buying a capability or renting a dependency. Public digital guidance warns against arrangements that lock the buyer in.

Strong answer: A README and setup guide in the repository, documented architecture, no hidden dependencies on vendor systems, and an offer to demonstrate a clean setup by your team.

Weak answer: "Our team would handle the transition" or documentation that exists only in the vendor's internal wiki.

What access will your engineers need to our code, pipelines and cloud accounts, and how will that access be granted, reviewed and removed?

Why: Automation work often needs broad access. Least-privilege access with regular review limits what can go wrong through the supplier.

Strong answer: A list of access needed per role, named individual accounts rather than shared ones, periodic access review, and removal on the day someone leaves the account.

Weak answer: Requests for admin access "to save time" or shared team logins.

If you are taking over automation from a previous supplier or our own team, how will you assess the existing suite before deciding to keep, fix or rewrite it?

Why: A new vendor may prefer to rewrite everything, which is billable and not always needed. An assessment first protects what you already paid for.

Strong answer: A time-boxed assessment with a written report covering stability, coverage, maintainability and flaky tests, and a costed recommendation for each part.

Weak answer: "We will rebuild it using our framework" before seeing the existing code.

Which tools will you use for test management, defect tracking and reporting, and will they be our accounts or yours?

Why: Test cases and defect history held in the vendor's accounts are hard to recover when the contract ends. Using the buyer's tools keeps the record with you.

Strong answer: Work done in the buyer's existing tools and accounts, or in new accounts owned by the buyer, with the vendor added as users.

Weak answer: "We use our own internal platform" with exports offered only on request or at extra cost.

Are any of the tools you propose licensed by you and charged to us, directly or inside your rate? What happens to those licences if we part ways?

Why: Tool licences bundled into rates hide cost and can tie you to the vendor. You need to see them separately to compare proposals fairly.

Strong answer: A list of every tool with who holds the licence, its cost, whether it is in the rate, and what the buyer must buy to keep using the work after exit.

Weak answer: "Tools are included" with no list, or licences that lapse on exit leaving the test assets unusable.

Can all test cases, results and defect history be exported in an open format such as CSV, JSON or plain text at any time, not only at exit?

Why: Open formats and regular exports keep your testing history from being trapped in one tool. Public technology guidance asks buyers to favour open standards for this reason.

Strong answer: A named export route in an open format, available on demand, and an offer to run a sample export during the pilot.

Weak answer: "We can provide a report" or exports only in a format readable by one commercial tool.

How will your tools connect to our existing pipeline, issue tracker and chat channels? Is there anything we would need to install or change?

Why: Tools that do not fit your pipeline create manual handoffs and duplicated work. Knowing the integration work up front avoids surprise cost.

Strong answer: A description of each integration, what the buyer needs to set up, and an example from another engagement of results flowing into the issue tracker automatically.

Weak answer: "Our tools integrate with everything" with no specifics about your setup.

Which devices, browsers and operating systems will you test on, and are they real devices, emulators or a cloud device service? Who pays for that service?

Why: Device coverage drives both confidence and cost. The buyer needs to know what is covered and whose account the device service sits in.

Strong answer: A coverage matrix agreed from your usage data, the mix of real and virtual devices, the device service account owner, and the cost per month shown separately.

Weak answer: "We test on all major devices" with no list or a matrix that ignores your user data.

Which tools and methods do you use for accessibility testing, and how do you combine automated checks with manual checks against WCAG 2.2?

Why: Automated accessibility scanners catch only part of the issues WCAG covers. Manual checks with assistive technology are needed for the rest.

Strong answer: Named WCAG 2.2 conformance level, open or standard checkers for automated scans, manual testing with screen readers and keyboard only, and results mapped to WCAG success criteria.

Weak answer: "We run an accessibility scanner" presented as full accessibility testing.

If security testing is in scope, which methodology and tools do you follow, and what do you not cover that a specialist penetration tester would?

Why: Security testing claims vary widely. Tying the work to a public methodology lets you see what has and has not been checked.

Strong answer: Reference to a public methodology such as the OWASP Web Security Testing Guide, the checks in and out of scope, and a clear line where specialist testing is needed.

Weak answer: "We include security testing" meaning an automated scan with no methodology or scope.

Do you use any AI or code-generation tools on client work? If so, which data from our product is sent to them, and can we opt out?

Why: Generation tools can send your code, test data or screenshots to third parties. This is a data-processing question as much as a tooling one.

Strong answer: A clear yes or no, the data involved, where it is processed, contractual controls over the tool provider, and a buyer opt-out.

Weak answer: "We use the latest AI to speed up testing" with no answer on what data leaves your environment.

How is test progress shown to us during a cycle? Can we see a live dashboard in our own tools, and which measures does it show?

Why: Visibility during a cycle lets you act before release day. Measures tied to delivery outcomes are more useful than counts of tests run.

Strong answer: A live view in buyer-accessible tools showing progress against plan, open defects by severity, pass rates on the main branch and pipeline run times.

Weak answer: A weekly slide deck, or dashboards that report only the number of test cases executed.

Where are your tools and any test artefacts hosted, in which countries, and does any of our data leave our environment when your tools run?

Why: Hosting location affects data protection obligations and transfer rules. You need to know before test data or screenshots start moving.

Strong answer: A list of tools with hosting region, whether they store buyer data, and the transfer mechanism used for any data leaving your jurisdiction.

Weak answer: "Everything is in the cloud and secure" without locations or data flows.

Can you share your most recent SOC 2 Type II report under NDA, and tell us its audit period, the Trust Services Criteria in scope, and which offices and systems it covers?

Why: A Type II report tests whether controls worked over a period, not just whether they existed on one day. The scope decides whether the team that would test your product is covered at all.

Strong answer: Offers the report under NDA, names the period end date, the criteria in scope, the locations and systems covered, and confirms the delivery team for your work sits inside that scope. Points you to any exceptions and the management response.

Weak answer: Offers only a logo, a Type I report, a report from a parent or hosting provider, or says the report is being prepared without a date.

What does your ISO/IEC 27001 certificate cover: which legal entity, which sites and which services? Which certification body issued it, which accreditation body accredits that body, and when does it expire?

Why: A certificate only protects you if it covers the entity and sites doing your work, and it only means something if an accredited body issued it. You can check this yourself in a public register.

Strong answer: Gives the certificate number, legal entity, scope statement, sites, issuing body, accreditation body and expiry date, and expects you to verify it in the IAF register.

Weak answer: Says the company is certified without naming the entity or scope, cites a certificate held by a group company elsewhere, or names a certification body that is not accredited.

Where will our test data, test results, screenshots and logs be stored, in which countries, and on whose systems? Who on your side can reach them?

Why: Test artefacts often contain personal data or product secrets. Location and access decide which laws apply and whether an international transfer mechanism is needed.

Strong answer: Names the storage systems, the countries or regions, and the roles with access, and offers to keep artefacts in your own environments where you prefer. Explains how access is granted, reviewed and removed.

Weak answer: Says data stays secure without saying where it is, or assumes testers can copy artefacts to personal devices or shared drives.

Will your testers ever need production data? If so, how do you mask, subset or synthesise it, and who approves any exception?

Why: Copying live personal data into test environments is a common way data ends up where it should not. A vendor should have a default that avoids it.

Strong answer: Defaults to synthetic or masked data, describes the masking approach and who runs it, and requires written approval from you for any use of live data with a documented legal basis and deletion date.

Weak answer: Asks for a production copy as a matter of course, or treats masking as your problem without offering a process.

If you process personal data for us, will you sign a data processing agreement that covers every point in GDPR Article 28(3), including acting only on our documented instructions, confidentiality, security, assistance, deletion or return, and audits?

Why: Article 28 sets the minimum content of a controller to processor contract. A vendor that resists these terms is telling you how it will behave when something goes wrong.

Strong answer: Has a standard processing agreement mapped to each Article 28(3) point, accepts your template or reviews it quickly, and names who handles data protection requests on their side.

Weak answer: Says a confidentiality clause in the main contract is enough, or refuses audit and assistance obligations.

Which subprocessors, subcontractors or freelancers would touch our systems or data? How do you tell us before adding or changing one, and can we object?

Why: Under GDPR a processor needs the controller's authorisation to engage another processor, and the same obligations must flow down. Undisclosed subcontracting also weakens your control over who is testing your product.

Strong answer: Provides a current subprocessor list with locations and roles, commits to advance written notice of changes with a right to object, and confirms the same data protection terms flow down to each one.

Weak answer: Cannot list subprocessors, says staff augmentation partners are not subprocessors, or offers notice only after a change.

If our data will be accessed from outside the UK or EEA, which transfer mechanism will you use, and have you carried out a transfer risk assessment for those countries?

Why: Remote testers accessing personal data from another country is a restricted transfer. The mechanism and supporting assessment need to exist before work starts.

Strong answer: Names the mechanism, such as the current EU Standard Contractual Clauses with the right module, identifies the countries involved, and can share or summarise its transfer assessment.

Weak answer: Says remote access is not a transfer, or has not considered which countries the team works from.

How do you control access to our environments: named accounts, multi-factor authentication, least privilege, managed devices, and removal when someone leaves the team? How quickly is access removed?

Why: Testers usually hold wide access to pre-release systems. Offboarding gaps are a common source of exposure when teams rotate.

Strong answer: Describes named accounts only, enforced multi-factor authentication, company-managed and encrypted devices, a stated time to revoke access after a role change, and periodic access reviews it shares with you.

Weak answer: Uses shared test accounts, allows personal laptops, or relies on the buyer to notice when someone leaves.

Will you complete our security questionnaire, or a standard one such as the CSA CAIQ, and back each answer with evidence we can sample?

Why: A standard questionnaire lets you compare vendors on the same controls. Evidence behind the answers separates policy on paper from practice.

Strong answer: Returns a completed questionnaire within an agreed time, attaches or offers evidence for sampled controls, and names a security contact who can answer follow-ups.

Weak answer: Returns yes to every control with no evidence, or sends marketing material instead of answers.

How will you notify us of a security incident or personal data breach affecting our data, within what time, and what will the first notice contain?

Why: Processors must tell the controller about a personal data breach without undue delay so the controller can meet its own deadlines. Sector rules such as HIPAA and DORA add their own requirements.

Strong answer: Commits to a specific notification time in hours, names the contact route, lists what the first notice will include, and describes how they support your investigation. Accepts sector terms such as a business associate agreement where relevant.

Weak answer: Promises to notify promptly without a time, or only once the vendor has finished its own investigation.

How do you manage supply chain risk in your own tools and dependencies, such as test frameworks, plugins, browser extensions and cloud services used on our work?

Why: Test tooling runs inside your pipelines and environments. A compromised dependency on the vendor side can become your incident.

Strong answer: Keeps an inventory of tools and dependencies used on client work, has an approval process for new ones, applies updates on a schedule, and can explain how this maps to a recognised supply chain practice.

Weak answer: Lets each tester install whatever tools they like, or has never been asked about this.

Can you show us a real, anonymised weekly status report from a current engagement, with nothing added for the sales process?

Why: The report you will receive each week is the main window into the work. A real sample shows whether it helps decisions or only shows activity.

Strong answer: Shares a redacted report that covers progress against plan, coverage, open defects by severity, risks, blockers and decisions needed, in a format consistent with a test status report.

Weak answer: Sends a template or a designed mock-up, or says every client gets something bespoke.

How exactly do you define and count your defect metrics, such as severity, priority, defect leakage, reopen rate and fix verification time? Where are those definitions written down?

Why: The same metric name can mean different things between vendors. Without written definitions you cannot compare vendors or track change over time.

Strong answer: Provides written definitions with formulas, explains who sets severity and how disputes are resolved, and agrees to align the definitions with yours in the contract.

Weak answer: Reports counts of test cases executed or bugs found with no definitions, or changes definitions from month to month.

Which measurable outcomes would you agree to be held to, how are they measured, and what happens when one is missed?

Why: Performance-based contracting ties the work to measurable standards and a way of checking them. It moves the conversation from hours spent to results delivered.

Strong answer: Proposes a short list of measurable service levels or outcomes, states the measurement method and data source, and links misses to a remedy such as a service credit or corrective plan.

Weak answer: Offers only effort measures, such as hours logged or tests executed, or declines any measurable commitment.

Who is our day-to-day contact, who is their escalation point, and how fast do they respond to a blocking issue during our working hours?

Why: When a release is blocked, you need a named person and a known response time, not a shared inbox.

Strong answer: Names the roles in an escalation path with response times for each level, and states the hours of cover that overlap with your team.

Weak answer: Gives a generic support address, or says the account manager will handle everything.

How many working hours will your team overlap with ours each day, and which meetings and ceremonies will your testers attend live?

Why: Testing depends on fast questions and answers with developers and product owners. Too little overlap turns small questions into day-long delays.

Strong answer: States the overlap in hours by location, lists the meetings testers attend, and explains how questions are handled outside the overlap.

Weak answer: Promises flexibility without specifics, or plans for all communication to go through one coordinator.

What does your test completion report at the end of a release or phase contain, and can you show one?

Why: A completion report records what was tested, what was not, and the residual risk you are accepting. It is also evidence for audits and for any later dispute.

Strong answer: Shares an anonymised completion report that states scope tested, scope not tested and why, deviations from plan, open defects and residual risk, in line with test documentation standards.

Weak answer: Says a final call covers it, or the report only lists totals of passed and failed tests.

How do you report flaky or unreliable automated tests, and how do you keep them from hiding real failures in our reports?

Why: Flaky tests make pass rates look better or worse than reality and train teams to ignore failures.

Strong answer: Tracks flaky tests separately with a stated threshold, quarantines and fixes them with an owner and a date, and shows the flaky count in each report.

Weak answer: Reruns failures until they pass and reports the final result as green.

Will we have direct, read access to your test management data, defect tracker and automation results at all times, rather than only through reports?

Why: Direct access lets you verify reports and keeps the record in your hands if the relationship ends.

Strong answer: Works in your tools or gives you full read access to theirs, with exports available on request, and treats the underlying data as yours.

Weak answer: Shares only summary reports or screenshots, or says access to their tools needs an extra licence fee.

How often will we review the service together, who attends from your side, and what data will those reviews use?

Why: Regular reviews with the right people are how problems surface before they become disputes. Governance should be agreed up front.

Strong answer: Proposes a governance rhythm, such as weekly operational and quarterly service reviews, with named attendees, a standard agenda and data drawn from agreed metrics.

Weak answer: Suggests meeting when needed, or only offers a sales contact for reviews.

How will you tell us about risks you see in our product or process, such as untestable requirements or unstable environments, and how quickly?

Why: A testing vendor sees problems early. The value is lost if those observations wait for a monthly report or are softened to keep the client happy.

Strong answer: Has a risk log you can see, raises blocking risks the same day with a proposed action, and can give an example of a risk it raised with a client that changed a plan.

Weak answer: Says it focuses on execution and leaves risk to the client, or reports risks only in retrospectives.

Is your price based on time and materials, fixed price or outcomes, and exactly which activities are inside the price and which are billed separately?

Why: The pricing model decides who carries the risk of the work taking longer. Hidden extras, such as environment setup or tool licences, change the real cost.

Strong answer: States the model, lists what is included and excluded, and gives a priced example for a typical month or release using your scope.

Weak answer: Quotes a day rate only, or says extras will be agreed as they come up.

If we pay for time, how are hours recorded, approved and evidenced on invoices, and can we see timesheets by person and activity?

Why: Time based billing only works if you can see what the time bought. Unclear records make disputes hard to resolve.

Strong answer: Uses timesheets by named person and task, has you approve them before invoicing, and ties invoice lines to those approved records.

Weak answer: Invoices a monthly block with no breakdown, or says timesheets are internal.

Can you lock your rates for the initial term, and what is the mechanism and cap for any increase after that?

Why: Rate increases at renewal can erode the business case, especially once switching is costly.

Strong answer: Fixes rates for the initial term in writing, links later increases to a named index with a cap, and gives advance notice before any change.

Weak answer: Reserves the right to review rates annually with no cap, or ties increases to its own cost changes.

How do you handle change requests: who can raise them, how are they estimated and approved, and what happens to work that starts before approval?

Why: Scope changes are normal in testing. A clear change process stops small additions from turning into unexpected invoices.

Strong answer: Describes a written change process with an impact estimate, a named approver on your side, and a rule that no unapproved work is billed.

Weak answer: Says changes are handled informally, or bills extra work after the fact.

Can you show how your price is built up, such as roles, seniority mix, rates, management overhead and tooling, so we can compare it with our own estimate?

Why: A should-cost view helps you tell whether a low bid is realistic or relies on junior staffing and later change requests.

Strong answer: Breaks the price down by role, seniority, allocation and overhead, and explains assumptions you can test against your own estimate.

Weak answer: Gives a single total and declines to break it down, or the breakdown does not match the proposed team.

What service credits or other remedies apply when an agreed service level is missed, and how are they calculated and applied?

Why: Service levels without remedies are targets, not commitments. The remedy should be simple enough to apply without a dispute.

Strong answer: Proposes credits tied to specific service levels with a clear formula, applied automatically on the next invoice, alongside a corrective action plan for repeated misses.

Weak answer: Offers no remedies, or only after a formal claim process that is hard to use.

What are your payment terms, and would you accept tying part of the payment to acceptance of deliverables or milestones?

Why: Paying in advance or on time alone removes leverage when quality slips. Linking some payment to acceptance keeps incentives aligned.

Strong answer: Accepts invoicing in arrears on standard terms, and agrees that milestone or deliverable payments follow written acceptance against agreed criteria.

Weak answer: Requires large upfront payments, or refuses any link between payment and acceptance.

What minimum commitment, notice period and early termination fees apply, and how can we scale the team down or up?

Why: Minimum terms and exit fees decide how easily you can respond if the work changes or the service disappoints.

Strong answer: States any minimum term plainly, offers reasonable notice to scale down, and ties any early termination charge to documented unrecovered costs.

Weak answer: Requires a long minimum term with fees for the full remaining value, or keeps these terms out of the proposal.

What insurance do you hold, such as professional indemnity, cyber and general liability, at what limits, and will you share certificates?

Why: Insurance shows whether the vendor could meet its liabilities if something goes seriously wrong. Limits should match the risk of the work.

Strong answer: Lists each policy type with its limit and expiry, shares certificates, and commits to keep cover in place for the contract term.

Weak answer: Says it is fully insured without giving types or limits, or cover lapses before the contract would end.

What limitation of liability do you propose, and are data protection breaches, confidentiality breaches and intellectual property claims treated separately?

Why: A low general cap can leave you carrying the cost of a data breach or an ownership dispute caused by the vendor.

Strong answer: Proposes a cap related to contract value and risk, with separate or higher caps for data protection, confidentiality and intellectual property indemnities, and is willing to discuss them.

Weak answer: Caps all liability at a few months of fees with no exceptions, or excludes data loss entirely.

What exactly will you hand over when the contract ends: test cases, automation code, test data, environments configuration, defect history, documentation and credentials?

Why: If the handover list is not agreed at the start, you may find at exit that key assets live only in the vendor's tools or heads.

Strong answer: Provides a written inventory of deliverables handed over on exit in usable formats, confirms you own them, and agrees to include the list in the contract.

Weak answer: Says everything will be handed over without listing it, or treats frameworks and test assets as its own property.

Will you agree a written exit plan within the first months of the contract, keep it updated, and test it?

Why: Financial services regulators require exit plans for important outsourcing, and the same discipline helps any buyer avoid a disorderly change of vendor.

Strong answer: Commits to a documented exit plan by a set date, with roles, timeline, handover steps and costs, reviewed at least yearly.

Weak answer: Says an exit plan can be written when needed, or charges separately to produce one.

How long a knowledge transfer period will you support at exit, what does it include, and how is it priced?

Why: The period when one vendor hands over to another is where coverage gaps and lost knowledge usually appear.

Strong answer: States a transfer period in weeks, lists sessions, shadowing and documentation walkthroughs, names who will take part, and sets the price or rate in advance.

Weak answer: Offers a single handover call, or leaves the scope and price of transfer to be agreed at the time.

What notice period applies to termination for convenience and for cause, and will service levels continue to apply during the notice period?

Why: Service can drift once a vendor knows it is leaving. Keeping obligations in force during notice protects you.

Strong answer: States both notice periods, confirms service levels and staffing commitments stay in force until the end, and supports a parallel run with the incoming team.

Weak answer: Has no termination for convenience, or says staff will be moved to other clients once notice is given.

When we take over or move the automation suite, can it run in our own pipeline without your licences, private libraries or internal tools?

Why: An automation suite that depends on the vendor's proprietary components is hard to move and becomes a reason you cannot leave.

Strong answer: Confirms the suite runs on open or buyer-licensed components, keeps it in your repository from day one, and shows a build running in your pipeline.

Weak answer: Relies on a proprietary framework or accelerator you can only use while the contract runs.

Who will own the intellectual property in test cases, scripts and documentation created for us, and will the contract assign it to us in writing?

Why: Work made by a contractor is not automatically owned by the client in every jurisdiction. Ownership needs clear written terms.

Strong answer: Assigns ownership of all deliverables created for you in the contract, lists any pre-existing components with a perpetual licence for you to keep using them, and has employee and subcontractor agreements that support the assignment.

Weak answer: Grants only a licence while the contract runs, or says ownership will be discussed at the end.

How will you delete or return our data at the end of the contract, and will you certify that it is done?

Why: Processors must delete or return personal data at the end of the service unless law requires them to keep it. You need evidence, not a promise.

Strong answer: Offers return in a stated format followed by deletion within a stated time, including backups and personal devices, and a written certificate of deletion.

Weak answer: Says data will be deleted in due course, or cannot say how backups are handled.

How do you keep the service going if key people leave, fall ill or are moved to another client? What notice do we get and how do you replace them?

Why: Continuity on a testing team depends on people who know your product. Unplanned rotation throws that away.

Strong answer: Commits to named key roles, advance notice before planned changes, a replacement with similar skills within a stated time, an overlap period at no cost, and documentation kept current so no one person holds the knowledge.

Weak answer: Reserves the right to rotate staff freely, or replaces people without notice or overlap.

What is your business continuity plan for the delivery team, such as loss of an office, power, connectivity or a regional disruption, and when was it last tested?

Why: A distributed test team can be affected by local events you will not hear about until work stops.

Strong answer: Shares a summary of the plan relevant to your delivery location, states recovery targets, describes the last test and what it changed.

Weak answer: Says staff can work from home without a plan, or has never tested it.

Will you give us, and our regulators where relevant, the right to audit and access information during the contract and through the exit period?

Why: Regulated buyers must keep access and audit rights over outsourced work, and those rights matter most when a relationship is ending.

Strong answer: Accepts audit and information rights for you and your regulators, including during exit, with reasonable notice and without extra fees for regulator requests.

Weak answer: Limits audits to its own summary reports, or ends all access on the termination date.

No named team High severity

The proposal describes roles and headcount but names no individuals and attaches no CVs, so the buyer cannot judge who will actually do the work.

Ask next: Which named people will work on our account in the first three months, and can we see their CVs and speak to the lead before we sign?

Pooled resources across clients Medium severity

Testers are described as a shared pool or bench that serves several clients at once, with no commitment on how much of each person's time is ours.

Ask next: What share of each named tester's week is committed to us, and what happens to that time when another client has a release?

Automation percentage without scope Medium severity

The proposal promises a percentage of automated coverage without saying what the percentage is measured against: which features, which test levels, which environments.

Ask next: Percentage of what, exactly? Please list the features, test levels and environments the figure applies to, and how you will measure it each month.

Proprietary framework or tool High severity

The automation is built on the vendor's own framework, accelerator or platform that the buyer cannot run, maintain or licence without the vendor.

Ask next: If we end the contract, can our own engineers run and change every test without a licence from you? Please show us the dependencies and licence terms.

Test code kept in the vendor's repository High severity

Test scripts, data and pipelines live in repositories the vendor controls, with access or transfer to the buyer only on request or at the end.

Ask next: Will all test code, data and pipeline configuration be committed to a repository we own from day one, with our team holding admin rights?

Vendor keeps the rights to test assets High severity

The proposal or draft terms say the vendor retains ownership of test cases, scripts or frameworks created for the buyer and grants only a licence to use them.

Ask next: Which of the assets you create for us will we own outright, and which do you class as your pre-existing material? Please list the second group item by item.

Hours billed with no outcome measures Medium severity

Pricing is purely hours or headcount, and the proposal contains no measurable outcomes, acceptance criteria or way to tell good delivery from poor.

Ask next: Which three measurable outcomes would you accept being reviewed against each month, and what happens to the invoice if we miss them?

Vague SLA without remedies High severity

Service levels are described with words such as timely or reasonable, or they are measurable but carry no service credit, remedy or escalation when missed.

Ask next: For each service level, what is the number, how is it measured, who reports it, and what credit or remedy applies when it is missed?

Certification claimed without scope or number High severity

The proposal mentions ISO 27001 or a similar certificate by logo only, with no certificate number, certification body, scope statement or expiry date.

Ask next: Please send the certificate number, the accredited certification body, the scope statement and the expiry date, so we can check it ourselves.

SOC 2 Type I offered as Type II High severity

The vendor refers to a SOC 2 report without saying which type, or offers a Type I report (design at a point in time) where the buyer needs a Type II (operation over a period).

Ask next: Is the report Type I or Type II, what period and which trust services criteria does it cover, and which systems and locations are in scope?

Subcontracting not disclosed High severity

The proposal does not say whether any work goes to subcontractors or freelancers, or reserves the right to subcontract without the buyer's prior approval.

Ask next: Will any part of this work be done by a subcontractor, freelancer or affiliate? If so, who, where, and will you seek our written approval before adding one?

Request to test with production data High severity

The vendor asks for copies of production data, including personal data, for test environments, without proposing masking, synthetic data or a lawful basis.

Ask next: Why do you need production data, and what is your plan to use masked or synthetic data instead? If personal data is still needed, how will you process and delete it?

No data processing terms High severity

The testers will handle personal data, but the proposal has no data processing agreement or equivalent terms on instructions, security, subprocessors and deletion.

Ask next: Please send your data processing agreement, including your subprocessor list and how you delete our data at the end of the contract.

Data transfer abroad without a mechanism Medium severity

Testers sit in another jurisdiction and will access personal data, but the proposal does not name the transfer mechanism or the countries involved.

Ask next: From which countries will people access our data, and which transfer mechanism, such as standard contractual clauses, will cover it?

Broad access to production systems Medium severity

The vendor asks for administrator or shared accounts on production or build systems, with no mention of least privilege, named accounts or access review.

Ask next: Which systems do your people need access to, at what level, through named accounts, and how often will that access be reviewed and removed?

No exit or knowledge transfer section High severity

The proposal covers onboarding in detail but says nothing about how the work is handed back to the buyer or to a new supplier at the end.

Ask next: Please describe your exit plan: what you hand over, in what format, over how many weeks, at what cost, and who from your side stays until it is done.

Auto-renewal with rate escalation Medium severity

The term renews automatically unless cancelled in a narrow window, and rates rise each year by a fixed percentage or at the vendor's discretion.

Ask next: Can renewal require our written agreement, and can any rate change be capped and tied to a published index with notice we can act on?

Change requests billed without limit Medium severity

Scope is defined narrowly and anything outside it becomes a separately priced change request, with no agreed rates or process for assessing them.

Ask next: Which kinds of work would become change requests, how will they be priced, and can we agree the rates and approval steps in the contract?

Pilot refused, or free pilot with strings Medium severity

The vendor will not agree a short paid pilot with exit, or offers a free proof of concept that is tied to a longer commitment or uses their own tooling.

Ask next: Can we run a paid pilot of four to eight weeks with agreed success criteria, where we own everything produced and can stop without further commitment?

Defect counts as the main success measure Medium severity

Success is described mainly as the number of defects or test cases logged, which rewards volume rather than lower risk in releases.

Ask next: Besides counts, how will you report defects that escaped to production, test cycle time, and whether the right risks were covered?

Complete coverage promised High severity

The proposal promises 100% test coverage or bug-free releases. Complete testing is not achievable; testing should be scoped by risk.

Ask next: Coverage of what, and how will you decide which risks are tested first and which are accepted? Please show the risk-based approach in writing.

Full automation promised in weeks Medium severity

The proposal promises a fully automated regression suite within a few weeks, with no allowance for test design, environment work or ongoing maintenance.

Ask next: What will be automated by the end of month one, month two and month three, and how much effort per month do you budget for keeping those tests stable?

No test strategy or plan Medium severity

The proposal jumps from headcount to price without a test strategy or plan: no scope, test levels, environments, entry and exit criteria or risks.

Ask next: Please send an outline test strategy for our product, covering scope, test levels, environments, entry and exit criteria and the main risks.

Offshore handoff with no overlap hours Medium severity

The team works in a distant time zone and the proposal relies on overnight handoffs, with no stated overlap hours or named contact during the buyer's working day.

Ask next: How many hours a day will your team overlap with ours, who is our contact during our working day, and how are urgent issues handled outside the overlap?

Key people not named in the contract High severity

The lead and other key people appear in the pitch but the contract lets the vendor replace them at its discretion, without notice or buyer approval.

Ask next: Will you name the key people in the contract, commit to minimum notice before any change, and give us a say in who replaces them?

Liability capped at a month of fees Medium severity

The vendor's total liability is capped at one month of fees or a similarly small amount, including for data breaches and confidentiality failures.

Ask next: Can the liability cap reflect the realistic loss from this contract, with separate or uncapped treatment for data protection and confidentiality breaches?

No audit or information rights Medium severity

The proposal or terms exclude buyer audits, or limit information to what the vendor chooses to share, which regulated buyers usually cannot accept.

Ask next: Will the contract give us, our auditors and our regulator access to the information and premises needed to check how this work is done?

Vendor signs off its own readiness High severity

The vendor proposes to certify release or go-live readiness itself, and the buyer is not given the evidence or criteria to challenge that judgement.

Ask next: What evidence will you give us before each release so that we, not you, can decide whether it is ready? Please show an example report.

Flaky tests handled by reruns Low severity

The proposal treats unstable tests as normal and handles them by automatic reruns, without tracking flakiness or fixing its causes.

Ask next: How will you measure test flakiness, report it to us, and decide when a flaky test is fixed, quarantined or removed?

Automation outside the delivery pipeline Medium severity

Automated tests run on the vendor's machines or on a schedule they control, rather than in the buyer's build and deployment pipeline.

Ask next: Will the tests run in our own pipeline on every change, with results visible to our engineers without asking you?

Security testing without a method Medium severity

Security testing is listed as included, but the proposal names no method, scope, tester qualifications or reporting format.

Ask next: Which method will you follow for security testing, what exactly is in scope, and can we see an anonymised sample report?

Accessibility testing without a standard Low severity

Accessibility testing is offered without naming the standard and level it tests against, or relies only on automated scanners.

Ask next: Which standard and conformance level will you test against, and which checks will be manual rather than automated?

Certificates used as proof of seniority Low severity

Seniority is argued mainly by the number of certified testers. Entry-level certification shows knowledge of terms, not years of practice on products like the buyer's.

Ask next: For the named people on our account, how many years have they tested products like ours, and what did they do on their last two projects?

Contracting entity unclear Medium severity

The proposal comes from a brand, and the legal entity that will sign, its registration number and its country are missing or differ from the one on the invoice.

Ask next: Which legal entity will sign the contract, what is its registration number and country, and is it the same entity that employs the people doing the work?

Ownership of test code and test assets

Test scripts, automation frameworks, test data sets and test documentation are paid for by the buyer and are needed to keep testing after the contract ends. In the US, software written by an outside company is usually not a work made for hire, so ownership moves to the buyer only through a written assignment.

Repository and tool accounts held by the buyer

If the test code lives in the vendor's repository, the buyer depends on the vendor's cooperation to get it back. Holding the repository, pipelines and test management accounts in the buyer's name keeps access and return of the work simple.

Knowledge transfer on exit

The buyer needs a way to move testing to another supplier or in-house without losing what the outgoing team knows. Regulators ask firms to plan exit so that the service can be transferred without disruption.

Exit plan and exit assistance period

A written exit plan agreed early is easier to negotiate than one agreed during a dispute. An exit assistance period keeps the service running while the buyer moves it elsewhere.

Staff replacement terms

Rotating testers take product knowledge with them, and the buyer ends up paying for the new person to learn. Notice and an overlap period at the vendor's cost keep that cost with the party that caused the change.

Key personnel

The people presented during selection are often part of why the vendor was chosen. Naming them in the contract stops a switch after signature without the buyer's agreement.

Service levels with measurable targets

Service levels only help if both sides can measure them the same way. Each one needs a definition, a target, a measurement method and a reporting cycle.

Service credits and remedies

Without a stated consequence, a missed service level becomes a discussion rather than a remedy. Credits and a clear path to termination for repeated failure give service levels weight.

Data processing agreement

When a testing vendor handles personal data for the buyer, the GDPR requires a written contract with specific content. Testers can see personal data in test environments, logs and screenshots even when that is not the plan.

Subprocessor approval and data location

The buyer is responsible for knowing who handles its data and where. Prior approval of subprocessors and a limit on transfers keep that under the buyer's control.

No production data without written approval

Copying live customer data into test environments puts personal or health data where controls are usually weaker. The default should be synthetic or masked data, with any exception approved in writing.

Security obligations and incident notification

Test teams often hold credentials to staging and sometimes production systems. The buyer needs stated minimum controls and early notice of any incident so it can meet its own reporting duties.

Audit and information rights

The buyer, and in regulated sectors its regulator, may need to check how the service is delivered. A written right to information and audit avoids having to negotiate access when it is needed.

Restriction on subcontracting

A buyer who assesses one vendor can end up with work done by another company it never checked. Subcontracting should need the buyer's consent, and the main vendor should stay responsible.

Confidentiality

Testers see unreleased features, defects and security weaknesses before anyone else. Confidentiality needs to cover that information and survive the end of the contract.

Rate lock and price review

A low first-year rate followed by unplanned increases changes the value of the whole contract. Fixing rates for a period and linking later changes to a stated index keeps costs predictable.

Change control

Scope in testing work changes often. A written change procedure keeps price, timing and service levels tied to what was actually agreed.

Termination for convenience and for cause

The buyer should be able to leave a relationship that is not working without having to prove breach, and should be able to leave quickly when there is a serious breach. Both routes should trigger the exit terms.

Acceptance of deliverables

Paying for a test plan or an automation suite that does not run leaves the buyer with little leverage. Acceptance criteria agreed in advance let the buyer check the work before paying for it.

Limitation of liability with carve-outs

A general liability cap is normal, but some losses should sit outside it, because a small cap would leave the buyer carrying most of the harm. Common carve-outs cover data protection, confidentiality, IP infringement and deliberate misconduct.

Insurance

Liability terms only help if the vendor can pay. Required insurance, with evidence on request, supports the liability position the buyer agreed to.

Legal entity and jurisdiction: The company that will sign the contract exists, is active, and is the same entity described in the proposal.

How to verify: Ask for the full legal name, registration number and registered address of the contracting entity. Look it up in the national register (Companies House for the UK, SAM.gov entity records for US federal registrants) or in OpenCorporates for other countries. Confirm the status is active and the name and number match the proposal and draft contract exactly.

Legal entity and jurisdiction: Where the delivery team sits and which entity employs them, if it differs from the contracting entity.

How to verify: Ask the vendor to list every entity involved in delivery, with country and the role each plays. Look each one up in OpenCorporates or the relevant national register and note the parent and group relationships. Confirm the contract makes the contracting entity responsible for all of them.

Legal entity and jurisdiction: The EU VAT number given on the proposal or invoice is valid and belongs to the contracting entity.

How to verify: Enter the member state and VAT number in the European Commission VIES validation service. Check that the returned name and address match the contracting entity.

Legal entity and jurisdiction: Neither the vendor, its parent, nor its named directors appear on sanctions lists.

How to verify: Search the entity names and director names in the OFAC Sanctions List Search, and in any sanctions list your own jurisdiction requires. Record exact and close matches and resolve any close match before proceeding.

Legal entity and jurisdiction: The vendor is not debarred or excluded from public contracts.

How to verify: Search the World Bank listing of ineligible firms and individuals, and the exclusion records on SAM.gov. Where the buyer is a public body, also check the exclusion grounds that apply under its own procurement rules.

Insurance: The vendor holds professional indemnity, cyber and other insurance at the levels the contract will require.

How to verify: Ask for a certificate of insurance for each policy, issued by the insurer or broker. Check the insured entity is the contracting entity, the cover limits meet the contract, and the policy dates cover the start of the work.

Insurance: The cyber and professional indemnity policies cover the kind of work the vendor will do, including access to your systems and data.

How to verify: Ask the vendor or its broker to confirm in writing whether the policies cover services provided to clients, data breaches caused by the vendor's staff, and work done by subcontractors. Note any exclusions that apply to your use.

References: References come from clients with work similar in size, domain and engagement model to yours.

How to verify: Ask for at least [3] references with contact names, the service delivered and dates. Choose which ones to call, ask for one former client if possible, and speak to the person who managed the vendor day to day rather than only a sponsor.

References: Any adverse public information about the vendor, such as litigation, regulatory action or public statements from former clients.

How to verify: Search public court and regulator records in the vendor's home jurisdiction and news sources for the entity name and main directors. Ask the vendor to disclose any material litigation or regulatory action in the last [5] years and compare with what you found.

Security posture and certifications: Any ISO/IEC 27001 certificate the vendor cites is valid, current and issued by an accredited certification body.

How to verify: Ask for a copy of the certificate. Search the certificate or company in IAF CertSearch and confirm the certification body is accredited, the certificate is current and the entity named matches the contracting entity.

Security posture and certifications: The ISO/IEC 27001 scope covers the sites, teams and services that will deliver your work.

How to verify: Read the scope statement on the certificate and ask for the statement of applicability. Check that the delivery locations and the testing service are inside the scope, not only a head office or a different business unit.

Security posture and certifications: Any SOC 2 report is a Type II report that covers a recent period and the services you are buying.

How to verify: Request the SOC 2 Type II report under NDA. Read the period covered, the system description and scope, the trust services criteria included, the auditor's opinion and the listed exceptions. If the period ended more than a few months ago, ask for a bridge letter covering the gap.

Security posture and certifications: Exceptions and complementary user entity controls in the SOC 2 report are understood and addressed.

How to verify: List each exception the auditor reported and ask the vendor how it was fixed. List the complementary user entity controls the report expects you to operate and confirm who on your side will do each one.

Security posture and certifications: Where no certificate or report exists, the vendor's security controls are assessed directly and in proportion to the access they will have.

How to verify: Send a security questionnaire covering access control, device security, staff onboarding and offboarding, incident response and secure handling of credentials. Ask for supporting documents for the answers that matter most to your risk.

Security posture and certifications: The vendor has a working incident response process and can notify you in the time the contract will require.

How to verify: Ask for the incident response policy and the named contact for security incidents. Ask for an anonymised example of how a past incident was handled and how quickly clients were told.

Subcontracting: Whether any part of the work will be done by subcontractors or freelancers, and who they are.

How to verify: Ask the vendor to state in writing whether any team member will be a subcontractor, freelancer or employee of another company. For each subcontractor, get the legal name, country and work to be done, and run the entity and sanctions checks above.

Subcontracting: The vendor passes its security, confidentiality and IP obligations down to subcontractors.

How to verify: Ask for the template agreement the vendor uses with subcontractors, or the relevant clauses. Check it covers confidentiality, IP assignment, security requirements and audit rights at least equal to your contract.

Staff continuity: The people proposed are employed by the vendor and available for the planned start date.

How to verify: Ask for CVs of the proposed team, their employing entity and confirmation of availability. Interview the lead and at least one team member. Confirm the proposal does not list people who are not yet hired.

Staff continuity: Staff turnover and how the vendor handles departures from client teams.

How to verify: Ask for the vendor's staff turnover figure for delivery staff over the last [12] months and how it is calculated. Ask how handovers are run when someone leaves a client team, and confirm the answer with references.

Staff continuity: Staff background checks match the level of access they will have.

How to verify: Ask what pre-employment checks the vendor runs (identity, right to work, employment history, criminal record where lawful) and whether these apply to subcontractors. Compare with what your own policy requires for people with the same access.

IP and code ownership: The vendor owns the work of its staff and subcontractors, so it can assign test code and assets to you.

How to verify: Ask the vendor to confirm in writing that employment and subcontractor agreements assign IP created during the work to the vendor. For US work, note that commissioned software is not usually a work made for hire, so a written assignment is needed at each step.

IP and code ownership: Which pre-existing tools, frameworks or libraries the vendor plans to use, and on what terms.

How to verify: Ask for a list of any proprietary tools or accelerators the vendor will bring, and any open-source components with their licences. Confirm you can keep using the test suite after exit without a licence from the vendor.

Data handling: Whether the vendor will act as a processor of personal data and, if so, can meet GDPR Article 28 requirements.

How to verify: Map what personal data testers could see, including in test environments, logs and screenshots. If any, send your data processing terms and ask the vendor to confirm it can meet them, including instructions, confidentiality, security, subprocessors, assistance, deletion and audit.

Data handling: Where data will be accessed from, and the transfer mechanism for any access from outside your jurisdiction.

How to verify: Ask for the countries from which staff and subcontractors will access your systems and data. For access from outside the EEA or UK, confirm the transfer mechanism, such as the Standard Contractual Clauses, and ask for the vendor's transfer risk assessment.

Data handling: How test data is created and how the vendor avoids using production data.

How to verify: Ask the vendor to describe how it creates or obtains test data, how it masks any real data, and how it deletes data at the end of an engagement. Ask for an example of a deletion confirmation given to another client, with details removed.

Financial stability signals: The vendor files its accounts on time and the filed figures show no obvious distress.

How to verify: Download the latest filed accounts from Companies House or the relevant national register. Check filing dates, any overdue filings, net assets and whether auditors raised a going concern issue. For private companies with no public filings, ask for the latest accounts under NDA.

Financial stability signals: Your contract value is not a large share of the vendor's revenue, and the vendor does not depend on one or two clients.

How to verify: Ask the vendor for its annual revenue band and the share of revenue from its largest clients. Compare your expected annual spend with its revenue. A high share on either side means the relationship matters more if either party leaves.

Financial stability signals: Charges, insolvency notices and recent changes in ownership or directors.

How to verify: Check the company filing history in Companies House or OpenCorporates for charges registered against the company, insolvency notices, and recent changes of directors or owners. Ask the vendor to explain any recent change.

Financial stability signals: What happens to your service and your test assets if the vendor fails or is sold.

How to verify: Ask the vendor how it would keep services running and hand over your assets in an insolvency or change of control. Check this against the exit plan, the repository clause and your ability to terminate on change of control.

Context: What did you hire them to do, and how long have they been working with you?

Listen for: A specific scope and start date that match what the vendor told you. If the reference describes a much smaller or different engagement, weigh the rest of the call accordingly.

Context: How similar is your product and setup to ours: the technology, release frequency and regulation you work under?

Listen for: Concrete overlap with your situation. A reference from a very different product tells you about the vendor's manners, not about how they would handle your risks.

Staffing continuity: Are the people who started on your account still on it? If some left, how was the change handled?

Listen for: Names and rough dates. Notice given, overlap between leaver and replacement, and whether the reference had any say. Vague answers such as "they always find someone" suggest rotation the reference has stopped tracking.

Staffing continuity: Were the people you met during the sale the people who then did the work?

Listen for: Whether senior people from the pitch stayed involved after signing. A plain "no, we never saw them again" is a useful answer.

Staffing continuity: How long did it take a new tester on their side to become useful on your product?

Listen for: A number of weeks and what helped or slowed it. Good vendors bring documentation and pairing; weak ones rely on the client's staff to train each new person.

Delivery: Think of the last release they tested. What did they find, and did anything important get through to production?

Listen for: A recent, specific story with both catches and misses. Only praise, or no example at all, usually means the reference is not close to the work.

Delivery: How do they decide what to test first when time is short?

Listen for: A description of risk-based choices the reference has seen in practice, including what was left out and who agreed to that. "They test everything" is not an answer.

Delivery: Did they keep to the service levels in your contract, and how do you know?

Listen for: Whether the reference has its own measures or relies on the vendor's reports. Ask whether any service credit was ever claimed and how that went.

Automation handover: Who owns and can run the automated tests today? Could your team keep them going if the vendor left tomorrow?

Listen for: Tests stored in the client's repository and run in the client's pipeline, with in-house engineers who have changed them. Hesitation here suggests dependence on the vendor.

Automation handover: How much of their time goes on fixing unstable or broken tests, and do they report it?

Listen for: Awareness of flaky tests and a routine for dealing with them. If the reference does not know, the vendor is probably not reporting it.

Communication: When they find a serious problem late in the day, how and how quickly do you hear about it?

Listen for: A named channel and a real example with timing. Watch for problems that surfaced only in a weekly report.

Communication: What do their regular reports tell you, and which parts do you actually use?

Listen for: Reports that help decisions about releases and risk, not only counts of test cases and defects. A reference who never reads them may not have a view on quality at all.

Commercials: Has the cost ended up close to what was agreed at the start? Where did extra charges come from?

Listen for: Specific sources of extra cost such as change requests, rate rises or extra headcount, and whether they were agreed in advance.

Commercials: When you asked for something outside the original scope, how did they respond?

Listen for: Whether small requests were absorbed sensibly or each one became a priced change. Either can be fair; the question is whether it matched what was agreed.

Problems and recovery: Tell me about the worst week you have had with them. What went wrong and what did they do?

Listen for: An honest account of a real problem and a clear recovery, including who on the vendor side took responsibility. A reference who can think of nothing may be too distant from the work, or chosen for politeness.

Problems and recovery: Have you ever had to escalate above your day-to-day contact? What happened?

Listen for: Whether an escalation route exists, who was on it and how fast things changed. Repeated escalations for the same issue are a signal worth noting.

Security and data: What access to your systems and data do their people have, and have they ever had a security or data incident on your account?

Listen for: Named accounts, limited access and an incident either denied with confidence or described with how it was reported. Uncertainty about who has access is itself informative.

Exit: If you ended the contract, how hard would it be to move the work in-house or to someone else?

Listen for: Whether documentation, test assets and knowledge sit with the client. If the reference has not thought about it, ask what they would lose.

Exit: Have you seen them hand work over at the end of a phase or contract? How complete was it?

Listen for: A real handover with documents, code, walk-throughs and time allowed. Answers based only on what the vendor promised are weaker evidence.

Overall: Would you hire them again for the same work? What would you change in the contract if you did?

Listen for: The changes they name are often the most useful part of the call: they point to clauses you should add now.

Overall: Did the vendor ask you to take this call, and how often do they ask?

Listen for: Frequent requests suggest a small set of references used for every buyer. Ask the vendor for a client that has left them, or one of similar size to you.