How to evaluate RFP responses from QA vendors

Updated 2026-09-24

Evaluate RFP responses in a fixed order: apply pass/fail gates first, have each evaluator score independently against the criteria and weights you published, meet to reconcile scores with evidence, then normalise prices to one basis and compare cost with quality. Score only against what the RFP asked, and send written clarification questions when a vendor answered a different question.

What to check

  1. Fix the criteria, weights and scoring scale before the first response is opened, and keep them as published in the RFP.
  2. Screen every response against pass/fail gates before anyone scores quality.
  3. Have at least two evaluators score each response alone, writing the evidence for each score.
  4. Normalise prices to the same scope, team size and period before comparing them.
  5. Record every clarification question and answer, and send the same kind of question to every vendor where it applies.
  6. Keep a short written rationale for the final decision that someone outside the panel could follow.

Ask the vendor

  • Your response proposes a different scope from the one in the RFP. Please price the scope as written, and list your proposed changes separately.
  • Please restate your price as a monthly cost for the team and period in section X, including all tooling and management fees.
  • Please confirm the named people in your response are the people who will start on this engagement.
  • Which assumptions in your response, if wrong, would change your price, and by how much?

Decide how you will judge before you read

One evaluation problem to avoid is changing the rules after reading the responses. A persuasive proposal makes its strengths feel like the criteria that matter. Fix the method first.

US federal source selection rules in FAR Part 15 require agencies to evaluate proposals solely on the factors stated in the solicitation. You are not bound by FAR, but the discipline is useful: publish criteria and weights in the RFP, then score against them and nothing else. The scorecard lets you set weights by situation and export them into the RFP, and weighting a vendor scorecard explains how to choose them.

Agree a scoring scale with definitions, for example:

Score Meaning
0 Not addressed
1 Addressed, with no evidence
2 Addressed, with general evidence
3 Addressed with specific evidence for a product like ours
4 As 3, plus named people, samples or verifiable references

Step 1: pass/fail gates

Before scoring quality, screen each response against mandatory requirements. Typical gates for QA services:

  • Response complete and in the required format.
  • Willing to sign a data processing agreement if personal data is involved.
  • Test code and assets owned by you, in your repository.
  • No undisclosed subcontracting.
  • Legal entity named and verifiable.

A vendor that fails a gate is out, however good the rest looks. Record the reason. If a gate removes everyone, the gate may be wrong; fix it for all vendors.

Step 2: independent scoring

Each evaluator scores alone, in writing, with a reference to the page or answer behind each score. This matters more than the number of evaluators. When people score together, the first voice sets the score.

Split sections by expertise. Engineers score approach and automation; a QA lead scores process and reporting; procurement or finance scores commercials and terms. Everyone can score the same core criteria.

Step 3: handle off-brief responses

Some vendors answer a different question: they propose a different scope, a different model, or a price for a smaller team. Do not score the alternative against your criteria as if it were a response.

Send a written clarification asking them to respond to the scope as written, and to list proposed changes separately. Treat alternatives as information for later, not as a reason to change the RFP for one vendor. If an alternative is genuinely better, change the requirement for every bidder.

Keep clarifications narrow. A clarification lets a vendor explain what it wrote; it should not become a chance to rewrite the proposal.

Step 4: reconcile

Meet once all scores are in. Look at criteria where evaluators differ by more than one point. Each person explains the evidence; the panel agrees a consensus score and writes down why. Scores can go down as well as up.

Step 5: normalise prices

Prices arrive in different shapes: hourly rates, monthly team fees, fixed prices per phase, with or without tooling, travel and management. Restate each one on the same basis:

  • Same scope and deliverables.
  • Same team size and seniority mix, or the vendor's proposed team priced over the same period.
  • Tooling, device services and licences included.
  • Management, onboarding and exit costs included.
  • Same period, such as 12 and 24 months.

The cost model does this with the vendors' quotes and your in-house comparison. Rate cards and what they hide lists items that often sit outside the headline rate.

Step 6: compare quality and cost

Decide in advance how you combine them. Two common methods:

  • Weighted total. Price is one weighted criterion, scored against the lowest normalised price.
  • Quality first. Rank by quality score, then ask whether any difference in price is worth the difference in quality.

Whichever you use, a low price should not rescue a weak proposal, and a high score should be backed by evidence you can check.

Step 7: verify and decide

Before you sign, verify what scored well: call references, check certificates in the due-diligence checklist, and consider a paid pilot with the people named in the proposal. Then write a one-page rationale. It helps you explain the choice internally and gives unsuccessful vendors honest feedback.

The RFP generator produces a response template, which makes every step above faster because answers arrive in the same structure. The selection path shows how evaluation fits into the full timeline.

Tools for this

Questions buyers ask

Should price be a scored criterion or a separate comparison?
Either works if you decide before responses arrive. One option is to score quality first and then compare quality against normalised price, so a low price does not hide a weak proposal. Write down the method in the RFP.
How many evaluators do I need?
At least two, ideally from different functions, such as engineering and QA, with procurement or finance for the commercial part. Independent scores followed by a reconciliation meeting reduce the effect of one person's first impression.
What if every response fails a gate?
Check whether the gate was realistic. If it was, reissue the RFP with clearer wording. If it was not, change it for every vendor and tell all of them.

Sources

  1. FAR Part 15: Contracting by Negotiation (source selection), US Federal Acquisition Regulation
  2. The Sourcing and Consultancy Playbooks, UK Cabinet Office
  3. ISO 37500:2014 Guidance on outsourcing, ISO