Situation

Enterprise procurement of a QA vendor: risk and governance

You are buying testing services through a formal enterprise process, with procurement, legal, security and vendor risk management all involved. Delivery quality still matters, but the contract will also be judged on how well it controls risk: who else touches your systems, what happens when service levels are missed, and whether you can audit the supplier.

What changes in criteria and weights

Subcontracting transparency, measurable service levels, insurance and liability, and audit and information rights rise to 5; security posture, data handling, the exit plan and the contracting entity and jurisdiction rise to 4. Capacity and speed to start drops to 2, because an enterprise timeline is set by the review process and a vendor's ability to pass it matters more than how fast it can start.

Criteria whose weight differs from the default
CriterionDefault weightWeight here
Subcontracting transparency35
Measurable service levels35
Insurance and liability35
Audit and information rights35
Security posture and certifications34
Test data and personal data handling34
Exit plan and assistance34
Contracting entity and jurisdiction34
Capacity and speed to start32

Weights run from 0 (not scored) to 5. Every other criterion keeps its default weight of 3. The scoring template loads these weights when you choose this situation.

Which tools, in which order

  1. RFP generator: Publish the evaluation criteria and weights in the RFP so every vendor knows how it will be scored and the process can be defended afterwards.
  2. Vendor questionnaire: Filter by this situation and use it alongside your vendor risk questionnaire so delivery questions and risk questions reach vendors together.
  3. Due-diligence checklist: Run entity, sanctions, debarment, insurance, certification and financial checks on each shortlisted vendor with the verification steps given.
  4. Proposal red-flag checker: Catch undisclosed subcontracting, missing audit rights, low liability caps and an unclear contracting entity before legal review.
  5. Scoring template: Score with the enterprise weights and record the reason for each score, so the decision can be explained to stakeholders and auditors.
  6. Reference-call script: Ask enterprise references how the vendor handled audits, security reviews and missed service levels.
  7. Contract clauses: Bring the clause library to legal review: subcontracting control, service credits, audit rights, insurance, liability and exit.
  8. Cost model: Include management overhead, security review effort and exit costs in the total, not only the vendor's rate card.

Typical mistakes

  • Starting the security and vendor risk review after the vendor is chosen, so it delays the start by months.
  • Scoring proposals on criteria that were not stated in the RFP, which weakens the defence of the decision.
  • Accepting service levels without remedies, so a missed target has no consequence.
  • Not asking for the full list of subcontractors and where they work.
  • Agreeing a liability cap so low it bears no relation to the harm a data incident could cause.
  • Treating a testing vendor with access to production-like environments as low risk without assessing that access.

Bring the reviewers in early

Security, legal, privacy and vendor risk teams each have their own queue and questionnaire. Share the draft RFP with them before release and ask what they need from vendors. Put their requirements in the RFP so vendors answer them once, alongside the delivery questions, and the reviews can run in parallel with evaluation.

Tier the risk

Describe what the vendor will access: environments, test data, source code, production systems. That determines how deep the review needs to be. Designing the engagement to use synthetic data and isolated environments can lower the tier and shorten the review.

Control the supply chain

Ask each vendor to list every subcontractor and freelancer who might work on your account, where they are based, and what they would access. Make any change subject to your approval in the contract, and require that security, confidentiality and IP obligations flow down.

Service levels with consequences

Define measurable service levels for the things that matter to delivery, such as turnaround per build, response to blocking issues, and staff continuity. Attach service credits or other remedies and a route to termination for repeated failure. Review them at a regular governance meeting with named owners on both sides.

Document the decision

Keep the scoring sheets, the reasons behind each score, reference call notes and due-diligence results together. A well-documented decision is easier to defend and to revisit at renewal.

Questions buyers ask

Can we use a standard vendor risk questionnaire for a testing vendor?
Yes, as the security and risk layer. Standard questionnaires such as the CSA CAIQ cover controls well but say little about testing delivery, staffing or ownership of test code. Pair them with the delivery questions from the questionnaire tool.
How do we set the depth of due diligence?
Base it on what the vendor will access and how critical the service is. A vendor testing with synthetic data in an isolated environment needs less scrutiny than one with access to production systems or personal data.
What audit rights are reasonable?
The right to receive information and reports, to audit on reasonable notice, and to have a regulator audit where your sector requires it. Financial entities in the EU have specific requirements under DORA and the EBA outsourcing guidelines.
How do we keep the process fair to vendors?
Give every vendor the same information, deadlines and evaluation criteria, answer questions in writing to all bidders, and record how each score was reached. Public bodies must also follow their procurement law.

Sources

  1. Interagency Guidance on Third-Party Relationships: Risk Management (2023), US Federal Reserve, FDIC, OCC
  2. ISO/IEC 27036-1:2021 Cybersecurity, Supplier relationships, Part 1: Overview and concepts, ISO/IEC
  3. NIST SP 800-161r1 Cybersecurity Supply Chain Risk Management Practices, NIST
  4. Consensus Assessments Initiative Questionnaire (CAIQ) v4, Cloud Security Alliance
  5. Regulation (EU) 2022/2554 (DORA), including Article 30 key contractual provisions, European Union
  6. Guidelines on outsourcing arrangements (EBA/GL/2019/02), being replaced by the EBA guidelines on third-party risk management, European Banking Authority
  7. Model Services Contract, UK Cabinet Office
  8. FAR Part 15: Contracting by Negotiation (source selection), US Federal Acquisition Regulation
  9. FAR Subpart 37.6: Performance-Based Acquisition, US Federal Acquisition Regulation
  10. OFAC Sanctions List Search, US Treasury
  11. SAM.gov entity and exclusion records, US General Services Administration