Test automation vendor lock-in: how to avoid it

Updated 2026-09-24

Test automation lock-in happens when the suite only runs with the vendor's framework, libraries, runner or accounts. Avoid it by requiring an open-source or widely used framework, code in your own repository from the first commit, tests running in your own CI pipeline, test data and results exportable in open formats, IP assigned to you, and a proof before exit that your engineers can run the suite without the vendor.

What to check

  1. Ask for the full list of dependencies the suite will need, and flag any that are vendor-owned, licensed or closed source.
  2. Require the repository to be under your organisation's account from the first commit, with the vendor as a contributor.
  3. Require tests to run in your CI pipeline on every change, not on the vendor's machines or schedule.
  4. Check that test cases, results and defect history can be exported in open formats at any time.
  5. Read the IP clause: the contract should assign test code and assets to you, not license them.
  6. Once a quarter, have one of your engineers clone the repository and run the suite from the README alone.

Ask the vendor

  • Does any part of the proposed automation depend on your own framework, accelerator, library or platform? If so, what happens to it when the contract ends?
  • Will the automation code live in a repository we own and control from the first commit?
  • Can our engineers run the full suite in our own pipeline without your licences, private packages or internal tools?
  • Can all test cases, results and defect history be exported in CSV, JSON or plain text at any time, not only at exit?
  • Who owns the intellectual property in the test code, and will the contract assign it to us in writing?

Where lock-in comes from

You pay for automation so that tests keep running after the people who wrote them move on. Lock-in means they do not. It rarely comes from one decision. It builds up through several choices made early, often inside the proposal.

The main sources:

Source What it looks like What to require
Proprietary framework Tests written on the vendor's own framework or accelerator Open-source or widely used framework, or source code delivered with a perpetual licence
Private libraries Tests import vendor packages from a private registry All dependencies in your repository or from public registries
Vendor repository Code lives in the vendor's account, shared on request Repository under your organisation from day one
Hosted runner Tests run on the vendor's machines or schedule Tests run in your CI pipeline
Tool accounts Test management, device service or reporting in the vendor's accounts Accounts in your name, or export on demand
Closed formats Test cases stored in a format only one tool reads Export to CSV, JSON or plain text
Knowledge Only the vendor's team knows how the suite is built Documentation, code review, your engineers in the loop
Contract Vendor keeps IP, licenses the code to you IP assigned to you

Framework choice

Ask each vendor which framework it proposes and why. Open-source frameworks such as Selenium, Playwright, Cypress and Appium, or the test tools that come with your language and platform, have public documentation and a hiring market. Your engineers can maintain them.

A vendor's own framework or accelerator can save time early. The question is what happens later. The UK Technology Code of Practice asks public bodies to use open source and open standards and to consider contractual limitations in their purchasing strategy. The same reasoning applies to your test suite: open code and open formats keep your options open.

If a vendor insists on its own framework, ask for the source code, a perpetual free licence to use and modify it, and documentation good enough for your engineers. If it will not agree, treat it as a red flag.

Where the code lives and runs

Two rules prevent most lock-in:

  1. Your repository from the first commit. The vendor's engineers work in a repository under your organisation's account. You control access. Nothing needs to be transferred at the end because nothing was ever elsewhere.
  2. Your pipeline on every change. Tests run in your CI, triggered by your builds. Your developers see results where they already work. If tests only run on the vendor's machines or a schedule it controls, you cannot run them without it.

Also ask where test data, environment configuration and secrets live, and make sure they sit with you too.

Ownership in the contract

Who writes the code and who owns it are separate questions. Under US copyright law, a work created by an independent contractor is a "work made for hire" only in specific categories and with a written agreement; US Copyright Office Circular 30 explains the conditions. Other countries have their own rules. The safe course is an explicit written assignment of IP in test code, test cases, data and documentation to you, plus a licence to anything pre-existing the vendor embeds.

The contract clauses tool includes clauses for ownership of test code and for repositories and tool accounts held by the buyer. Test automation code ownership covers the wording in more detail.

Maintainability is part of lock-in

A suite in your repository that nobody on your side can read is still locked in. Ask for:

  • Coding standards and a structure your developers would recognise.
  • Code review, with your engineers reviewing at least some changes.
  • A README that explains how to install, run and extend the suite.
  • Flaky test tracking, so the suite you inherit is one you can trust.

Test the exit before you need it

Do not wait for the end of the contract to find out. Periodically, have one of your engineers clone the repository on a clean machine, follow the README, and run the suite in your pipeline without help. If it fails, fix the gap while the vendor is still engaged. Put this check into the exit plan.

Questions to put in the RFP

The questionnaire includes the automation and exit questions above, with what a strong and a weak answer sound like. Add them to your RFP so every vendor states its position on frameworks, repositories and ownership before you score anything.

The selection path shows where to raise these points: in the RFP, again in scoring, and finally in the contract.

Tools for this

Questions buyers ask

Is a vendor accelerator always a problem?
Not always. It is acceptable if the source code is delivered to you with a perpetual, free licence to use and modify it, and your engineers can maintain it. It is a problem if you need the vendor, or a paid licence, to keep running your tests.
Does using an open-source framework remove lock-in?
It removes framework lock-in. You can still be locked in through repositories, runners, device service accounts, undocumented structure or knowledge held by a few vendor engineers. Check each of these.
What about low-code or record-and-playback tools?
They can speed up early work, but tests are often stored in the tool's own format. Ask how tests export and whether they run outside the tool before you let a vendor build on one.

Sources

  1. The Technology Code of Practice, UK Central Digital and Data Office
  2. Circular 30: Works Made for Hire, US Copyright Office
  3. The Digital, Data and Technology Playbook, UK Cabinet Office