Failures
NHS National Programme for IT: lessons for buyers of assurance
2026-09-24
The National Programme for IT in the NHS was meant to give every patient in England an electronic care record. The National Audit Office (NAO) reported on it several times. Its May 2011 update on detailed care records systems is the clearest account of what went wrong between a large public buyer, the Department of Health, and the suppliers it contracted. Much of it is about how the buyer measured what it was getting, which is the core problem in buying any kind of assurance. This post draws out the points that apply to a buyer of testing services and refers to the Department's contractors as the suppliers.
The contracts and the gap in delivery
According to the NAO, in 2003-04 the Department awarded five 10-year contracts, totalling some £5 billion, to four suppliers for local care records systems. By 2011, one supplier had exited the Programme and another's contract had been terminated, leaving two local service providers across three Programme areas.
The original aim was for every patient to have an electronic care record by 2010. The NAO reported that the contracted systems were not all expected until 2015-16, and that in the North, Midlands and East just four of 97 systems had been delivered to acute hospital trusts in seven years. To meet the revised deadline there, more than two systems a month would need to be delivered for the next five years. In London, care records systems for 1,243 GP practices and the London Ambulance Service had been removed from the Programme and the number of acute trust systems had roughly halved, without a significant reduction in the total contract value.
Where systems were in place, the NAO found they were mainly providing administrative benefits rather than the expected clinical ones, such as prescribing and administering drugs in hospitals.
How "delivered" was measured
The most useful section for a QA buyer is about measurement. When asked what functionality had been delivered, the Department reported that 91 per cent of the acute system's functionality had been proven to work in London and the South, and 64 per cent had been developed in the North, Midlands and East.
The NAO said its initial view was that this risked presenting an overly positive position. The measure did not show how much functionality was available and in use across acute trusts. It measured the technical readiness of individual modules, and counted development as complete once a module had been delivered in one care setting, such as a ward, in one NHS organisation. It was not weighted by the complexity or potential benefit of each module, and it was measured against a minimum specification agreed with clinicians in 2008 rather than the original contracts. The NAO also noted that the latest release in one Programme area had not yet been signed off as meeting the requirements of any of the receiving trusts.
The Department's own position, recorded in the report, was that no supplier was paid for a system until the trust had verified that it was deployed successfully. Both statements can be true at once: payment was tied to deployment, while the overall measure of progress still overstated what users could do.
Value for money
On that basis the NAO concluded that the £2.7 billion spent on care records systems did not represent value for money, and that it had no grounds for confidence that the remaining planned £4.3 billion would be different. The Department estimated that making systems work together under its revised approach would cost at least £220 million.
In June 2013, reviewing the Department's final benefits statement, the NAO reported whole-life estimated costs of £9.8 billion against estimated benefits of £10.7 billion, but with very considerable uncertainty: around two-thirds of the benefits (£6.6 billion) were forecast to arise after March 2012, and for three programmes 98 per cent of the estimated benefits were still to be realised.
Lessons for a buyer of assurance
Define "done" as in use, in your settings. A test result from one site, one environment or one user group shows readiness there, not everywhere. Write acceptance criteria that name the settings and the users that must be covered.
Weight what you measure. A percentage of passed test cases or delivered modules treats a trivial check and a critical workflow the same. Ask the vendor to report coverage weighted by business risk.
Measure against the contract, not a moving baseline. If scope is reduced or reset, record the new baseline, and keep reporting against what was contracted as well.
Keep scope disputes visible. The NAO timeline records a dispute over whether requirements were remedial fixes or new, chargeable work. Agree in advance how defects, change requests and new requirements are classified and who decides.
Watch concentration. The NAO's timeline records the Committee of Public Accounts questioning the remaining suppliers' capacity to deliver and raising concerns about the strength of the Department's negotiating position. A buyer with one testing vendor should plan exit and knowledge transfer from the start.
What this means for your selection
Set acceptance criteria and reporting measures before you sign, using the contract builder and the statement of work produced by the RFP tool. Ask how vendors weight coverage and report progress in the questionnaire, and check real outcomes with past clients using the reference check tool. The guides on KPIs for a QA vendor, defect leakage as a vendor metric and exit planning cover the measures and the exit route.
Questions buyers ask
- Why is the NHS National Programme for IT considered a failure?
- In May 2011 the National Audit Office concluded that the £2.7 billion spent on care records systems did not represent value for money: far fewer systems were being delivered than planned, with little reduction in cost, and delivered systems were mainly providing administrative rather than clinical benefits.
- What is the lesson for acceptance testing?
- The NAO found that the Department's measure of functionality counted a module as complete once it worked in one care setting in one organisation, which did not show what was delivered and in use across the NHS. Acceptance measures should reflect use in the buyer's real settings.
Sources
- The National Programme for IT in the NHS: an update on the delivery of detailed care records systems, UK National Audit Office
- The National Programme for IT in the NHS: an update on the delivery of detailed care records systems (HC 888, 18 May 2011), report page and summary, UK National Audit Office
- Review of the final benefits statement for programmes previously managed under the National Programme for IT in the NHS (6 June 2013), UK National Audit Office