The engineering manager has a demo on her laptop. Claude Code wrote forty Playwright tests overnight from the app's routes, and thirty-six of them pass. The question on the table is simple: do we keep going like this, or do we pay for a testing platform?
Six months later, the question looks different. Who fixes the failures? Who notices when the agent "fixes" a test by deleting the assertion? Who owns the CI runners and the flaky-test channel?
Build your own end-to-end tests with Playwright and a coding agent if you have engineers who will own the suite, a web-only product and a code-first culture. Buy a platform if non-coders need to write tests, you need mobile, desktop or API in the same suite, or maintenance and flake triage are already eating your sprint.
Key takeaways
- A coding agent makes authoring cheap. It doesn't make ownership, maintenance or flake triage cheap.
- Count every cost line: authoring, review, maintenance, flake triage, CI infrastructure, reporting and the people who own it.
- DIY Playwright wins with a strong SDET team, stable web UI and tests that live in the repository.
- A platform wins when testers outnumber automation engineers, or when the journeys cross web, mobile, desktop and APIs.
What are end-to-end testing tools?
End-to-end testing tools drive a real application the way a user would, through the UI and the services behind it, and check that a complete journey works: sign up, buy, refund, report. They sit at the top of the testing pyramid, above unit and integration tests.
Developer-first frameworks such as Playwright, Cypress and Selenium give you a library and a runner; you write and host everything else. Platforms such as mabl, testRigor, Katalon and Robonito bundle authoring, execution, reporting and maintenance, usually with low-code or no-code authoring.
If E2E is new to your team, start with 5 end-to-end testing pitfalls to avoid and our map of the levels of software testing.
What changed with coding agents?
Until recently, the main cost of a DIY E2E suite was writing it. Now Claude Code, Cursor and GitHub Copilot can read your app, write Playwright tests and fix the obvious failures. Playwright itself now ships Test Agents: a planner that explores the app and writes a test plan, a generator that turns the plan into test files, and a healer that reruns failing tests and patches them, or skips a test if it decides the feature is broken. You set them up for VS Code, Claude Code, Codex or opencode with one init-agents command. Our Playwright MCP guide covers the browser-control side.
So authoring is no longer the bottleneck. What hasn't changed is everything after the first green run:
- Someone has to review what the agent wrote. A generated test can pass while checking nothing useful.
- Someone has to own failures. A healer that patches tests can also patch away a real bug, unless a person reviews each change.
- Someone has to run the infrastructure and read the results: runners, browsers, secrets, environments, and what blocks a release.
The build-vs-buy question in 2026 is really a question about ownership, not authoring.
What does it really cost to build E2E tests yourself?
A DIY suite has no licence fee, which makes it look free. List every line before you compare it with a platform quote.
| Cost line | What drives it | How to estimate it |
|---|---|---|
| Authoring | New journeys per sprint; agent output still needs prompting and fixing | Engineer hours per new test, including review |
| Code review | Every generated test and every agent "heal" should be reviewed | Reviewer hours per pull request that touches tests |
| Maintenance | UI changes, renamed labels, new flows, framework upgrades | Hours per week spent updating tests that broke for non-bug reasons |
| Flake triage | Timing, test data, environments, third parties | Hours per week spent rerunning and investigating red builds |
| CI infrastructure | Runners, browsers, parallel workers, artifact storage, or a cloud grid | Monthly compute or grid bill, plus who maintains it |
| Reporting | Dashboards, history, flaky-test tracking, release sign-off | Build time for in-house reporting, or a paid reporting service |
| Test data and environments | Seeded accounts, resets, stubs, feature flags | Engineer hours to build and keep it working |
| Ownership and turnover | One SDET who understands the framework leaves | Ramp-up time for whoever inherits it |
Two lines are easy to underestimate. Maintenance grows with suite size and the pace of UI change, however the tests were written. Flake triage is pure overhead: Playwright's retries docs label a test "flaky" when it fails first and passes on retry, which tells you it happened, not why.
Infrastructure has real prices too. Cypress's open-source app is free under the MIT licence, but Cypress Cloud features such as flake detection start on the Team plan at $67 a month. BrowserStack Automate's desktop browser plan starts at $99 a month billed annually. Self-hosting runners costs compute and someone's time instead.
We worked through these lines for Selenium and Appium in what in-house automation really costs.
What does it cost to buy a platform?
A platform moves most of those lines into a subscription, but not all of them.
- Fees. Per seat, per run, per minute or custom. Check what's metered.
- Still yours: what to test, test data, environments and reviewing results.
- Migration and lock-in. Running two suites for a while, and whether you can export tests if you leave.
- Fit gaps. A platform that misses one of your UIs leaves you with a second tool anyway.
The honest comparison is the DIY total (engineer hours times loaded cost, plus infrastructure) against the platform total (fees plus the hours that remain yours).
When does DIY Playwright clearly win?
Building your own is the better choice when most of these are true:
- You have a strong SDET or developer team that wants to own tests and treats them as production code.
- Code-first culture. Tests live in the repository, go through code review and ship with the feature.
- Web only. Playwright supports Chromium, WebKit and Firefox, with mobile emulation of Chrome for Android and Mobile Safari, not native mobile apps.
- Stable UI. The interface doesn't change shape every sprint, so maintenance stays modest.
- You already run CI well. Playwright's CI guide generates a GitHub Actions workflow that uploads an HTML report with traces, and your team is comfortable building from there. In that setup, Playwright plus a coding agent is a very good choice. Use the agent for drafts, keep a human reviewer on every test change, and track flake rate as a first-class metric.
When does buying a platform win?
A platform is the better choice when most of these are true:
- Non-coders need to write tests. QA analysts, product owners or business users own the journeys, and engineers are scarce.
- More than web. Journeys cross a mobile app, a desktop client or APIs, and you want one suite and one report.
- Maintenance is already the problem. The team spends more time fixing tests than adding them.
- Flake triage is eating sprints. Red builds get rerun until green.
- Evidence matters. Release managers, auditors or customers need video, logs and readable reports without opening a terminal.
Build or buy? A decision checklist
Score each row. If most answers land in one column, that's your direction.
| Question | Points to build (Playwright + agent) | Points to buy (platform) |
|---|---|---|
| Who will write and own tests? | Developers and SDETs | QA analysts, product, business users |
| Which platforms do journeys cover? | Web only | Web plus mobile, desktop or API |
| How often does the UI change? | Rarely, in planned releases | Every sprint |
| Where should tests live? | In the repo, code-reviewed | In a shared tool, readable by non-engineers |
| Who maintains CI runners and reports? | An existing platform or DevOps team | Nobody has time |
| How do you handle flaky tests today? | Tracked and fixed by owners | Rerun until green |
| What evidence do approvers need? | CI logs and traces are enough | Videos, reports and sign-off records |
| What does the budget favour? | Engineer time over licences | Licences over engineer time |
Many teams end up hybrid: developer-owned Playwright tests in the repository, and a platform for cross-platform regression owned by QA.
How do you add E2E tests to CI for AI-generated code?
AI-generated code ships faster and with less human reading, so E2E tests become the safety net.
- Write tests from requirements, not from the code. A test generated from the code it checks tends to confirm its bugs. Start from the acceptance criteria.
- Run a smoke suite on every pull request. Five to fifteen critical journeys, fast enough that developers wait for it.
- Run the full regression on merge or nightly, in parallel.
- Block merges on critical failures, and quarantine known-flaky tests visibly instead of retrying them silently.
- Keep evidence. Traces, videos and screenshots for every failure, linked from the pull request.
- Review test changes like code changes, especially agent fixes.
Our playbook on how to test AI-generated code goes deeper, and CI/CD test automation covers pipeline patterns.
Which end-to-end testing tools are available in 2026?
Facts below come from each vendor's own site, as of November 2026.
Developer-first frameworks
- Playwright. Open source, for Chromium, WebKit and Firefox, in Node.js, Python, Java and .NET, with built-in Test Agents for planning, generating and healing tests. See Robonito vs Playwright.
- Cypress. The Cypress app is open source under MIT. Cypress Cloud has a free Starter plan (500 test results a month), Team at $67 a month and Business at $267 a month, with Enterprise custom. See Robonito vs Cypress.
- Selenium. An open-source umbrella project with WebDriver, Grid for parallel runs and IDE for record and playback, with bindings for Java, Python, C#, Ruby, JavaScript and Kotlin. See Robonito vs Selenium.
Platforms
- mabl. Covers web, mobile, AI apps and APIs; you can describe a flow or paste a Jira ticket to generate tests, and it integrates with GitHub, GitLab and Jenkins. Pricing on request.
- testRigor. Plain-English tests covering web, native and hybrid mobile, Windows desktop (paid plans), API, email and SMS. Free public plan, pay-as-you-go and custom Enterprise per its sign-up page.
- Katalon. Web, mobile, API and desktop with record, low-code keywords or Groovy and Java. Professional $1,800 per seat per year. See Robonito vs Katalon.
- Robonito. AI agents create, run and maintain tests for web on every plan, and mobile, desktop and API on Pro and Enterprise.
| Tool | Category | Who writes tests | Platforms | Pricing (as published) |
|---|---|---|---|---|
| Playwright | Developer-first | Developers, SDETs, coding agents | Web | Open source |
| Cypress | Developer-first | Developers | Web | App free (MIT); Cloud from free to $267/mo, Enterprise custom |
| Selenium | Developer-first | Developers, SDETs | Web | Open source |
| mabl | Platform, low-code and agentic | QA | Web, mobile, API | Pricing on request |
| testRigor | Platform, no-code | QA, manual testers | Web, mobile, desktop, API | Free public plan; pay-as-you-go; Enterprise custom |
| Katalon | Platform, low-code plus scripting | QA, SDETs | Web, mobile, API, desktop | Professional $1,800/seat/year |
| Robonito | Platform, no-code AI agents | QA, product, developers | Web; mobile, desktop, API on Pro | Free; Starter $20/mo; Pro $199/mo; Enterprise custom |
End-to-end testing with Robonito
Robonito is an agentic QA platform built for the "buy" side of this decision. End-to-end testing in Robonito works like this:
- Plain-language authoring. In AI Studio, describe the journey, paste a URL or reference a Jira issue, and the agent works through the app step by step. It can also generate tests from requirement documents, Jira stories and epics, which keeps tests tied to requirements rather than to generated code.
- Other ways in. Record a flow in the desktop app, build no-code tests from reusable steps, or use low-code extensions for advanced cases.
- Self-healing with an audit trail. Steps adapt when labels, layouts or locators change, and every healed step is logged for review.
- Execution and evidence. Parallel runs in isolated cloud workers, with video, network traces and an AI root cause for every failure, plus pass rates, trends and risk analysis across runs.
- One suite across platforms. Web on every plan; mobile (Android and iOS), desktop and API testing on Pro and Enterprise.
- CI and tools. Jenkins, GitHub Actions, GitLab CI and Azure DevOps, plus Jira and Slack, on Starter and above.
- An exit path. Robonito can generate executable code for test cases.
Pricing. Free covers web with 100 test cases and unlimited local runs. Starter is $20 a month and adds CI/CD, Jira and Slack. Pro is $199 a month and adds mobile, desktop and API. Enterprise is custom. Local runs on the desktop app or your own CI runners are free on every plan. See pricing.
Where it doesn't fit. If you have a strong SDET team, a web-only product and a culture where every test lives in the repository and goes through code review, Playwright with a coding agent may serve you better and cost less. Robonito also isn't the right tool for unit or component tests; keep those in your framework. And the Free plan doesn't include CI integration, so a pipeline trial needs Starter or above.
with one journey your team keeps fixing, or on your web app.
Frequently asked questions
Should we build our own E2E automation with Playwright and a coding agent, or buy?
Build if engineers will own the suite, the product is web only, the UI is fairly stable and tests should live in the repository. Buy if non-coders write tests, journeys cross mobile, desktop or APIs, or maintenance and flaky-test triage already take more time than new tests. Many teams use both, split by who owns which layer.
What are the top end-to-end testing tools?
Developer-first frameworks include Playwright, Cypress and Selenium for web. Platforms include mabl, testRigor, Katalon and Robonito, which add low-code or no-code authoring, hosted execution and reporting. The right choice depends on who writes tests, which platforms you cover and how much infrastructure you want to run.
What's the difference between developer-first and no-code E2E platforms?
Developer-first tools give engineers a library and runner; tests are code in the repository, and you host execution and reporting. No-code platforms let testers build tests in plain language, recorders or reusable steps, and include execution, reporting and maintenance features. The first maximises control; the second widens who can write and maintain tests.
Can a coding agent maintain our Playwright tests?
Partly. Agents, including Playwright's own healer, can rerun failing tests, inspect the UI and patch selectors or steps. But a patch that makes a test pass can also hide a real bug. Treat agent changes to tests as code changes: review them, and track why each test was healed, not just that it went green.
How do I add E2E tests to CI for AI-generated code?
Write tests from requirements, not from the generated code. Run a fast smoke suite on every pull request, the full regression on merge or nightly, and block merges on critical failures. Keep traces or videos for each failure, quarantine flaky tests visibly, and review any test change an agent makes before it merges.
Are open-source E2E tools really free?
The licence is free; running them isn't. You pay in engineer time for authoring, review, maintenance and flake triage, plus CI compute or a cloud browser grid and reporting. Add those up before comparing them with a platform subscription, which covers some of those lines but not all.
Decide with your own numbers, not a demo of forty green tests. to run one of your real journeys end to end, or .



