Key takeaway
- Agentic software testing explained: how planner, generator and healer agents work, what Playwright, Selenium and Appium offer, and the controls to adopt.
Agentic software testing uses AI agents that plan what to test, generate test code, run it, repair failing tests and report results, with people approving the decisions that matter. It differs from scripted automation because the agent works towards a quality goal instead of replaying fixed steps. For a CTO or head of engineering, the question is not whether agents can write a test, but which parts of the test lifecycle can be handed over safely and how that work is governed.
This guide covers what the official Playwright, Selenium and Appium documentation say, how an agentic test loop works, where it fits and where it does not, and how to adopt it in stages. It complements our guides to agentic process automation testing and testing data quality with agentic AI, which cover process-level and data-level testing.
What is agentic software testing?
In scripted automation, an engineer writes each step and each assertion, and the suite fails when the application changes. In agentic testing, an agent is given a goal such as "cover guest checkout", explores the application, proposes a plan, writes the tests and fixes them when they break. The ability to act on the application and on the test code is what makes the system agentic. It builds on the same ideas as agentic workflows and the wider agentic AI stack, applied to quality assurance.
| Aspect | Scripted automation | Agentic testing |
|---|---|---|
| Who decides what to test | Engineer, upfront | Agent proposes a plan, engineer approves |
| Test creation | Written by hand or recorded | Generated from a plan and live exploration of the application |
| When the UI changes | Test fails until someone edits it | Agent replays the failing steps, proposes a patch and reruns |
| Human role | Author and maintainer | Reviewer of plans, patches and results |
| Main risk | Maintenance effort | Unreviewed changes, wrong assertions, over-broad agent access |
What do the official testing tools provide today?
The major open-source tools do not all ship agents. The table separates what each project documents on its own site from what a team still has to build around it.
| Tool (source) | What its documentation says | Fit for agentic testing |
|---|---|---|
| Playwright Test Agents | Three built-in agents: a planner that explores the app and writes a Markdown test plan, a generator that turns the plan into Playwright tests, and a healer that runs failing tests, inspects the UI, suggests a patch and reruns. Started with npx playwright init-agents, with VS Code, Claude Code, Codex and OpenCode listed as supported tools. A seed test supplies setup. | Closest to a ready plan, generate, heal loop for web applications |
| Playwright MCP | A Model Context Protocol server that lets an LLM drive a browser through the page accessibility tree, not pixels. It can navigate, click, fill forms, inspect network requests and run custom Playwright code, which the docs call RCE-equivalent and advise enabling only for trusted clients. Needs Node.js 20 or newer. | Gives your own agent browser control; needs strict access limits |
| Playwright locators | Recommends role locators first, notes that CSS and XPath selectors can break when the DOM changes, and says locators are strict and re-resolve the element on each use. | Stable locators reduce what an agent has to repair |
| Selenium | An umbrella project: WebDriver for browser automation, IDE for recording, Grid for running tests across machines and platforms. | Execution layer your agents can call; agents are not part of the project |
| Appium | Open-source, cross-platform automation for native, hybrid, mobile web and desktop apps, built on the W3C WebDriver protocol with drivers per platform and official clients for Java, Python, Ruby and .NET. | Mobile and desktop execution layer; agents are not part of the project |
Read this as a split between the agent (reasoning about what to test and how to repair it) and the execution layer (the framework that drives the browser or device). Selenium and Appium are the second kind in their own documentation; Playwright documents both.
How does an agentic test loop work?
Whatever the tooling, the loop has five stages. Each stage has an owner and a control.
| Stage | What the agent does | Control to keep |
|---|---|---|
| 1. Plan | Explores the application and drafts a test plan from a request, a seed test and, where available, a requirements document | A person approves the plan before tests are generated |
| 2. Generate | Writes test code from the plan and checks selectors and assertions against the running application | Code review, as for any pull request |
| 3. Run | Executes in CI against a test environment | Agent has no access to production data or credentials |
| 4. Heal | Replays failing steps, inspects the current UI, proposes a patch and reruns; skips the test if it judges the functionality itself to be broken | Patches go through review; skipped tests are triaged by a person |
| 5. Report | Summarises coverage, failures and skipped tests | Results feed your quality metrics and release gates |
The Playwright documentation describes the planner, generator and healer in this order and notes that the generator output may contain initial errors that the healer can fix. That is a reminder that review is part of the design, not an optional extra. The same release gates feed delivery metrics such as those in DevOps metrics.
Where does agentic testing fit, and where does it not?
| Situation | Fit | Reason |
|---|---|---|
| Regression suites that break often after UI changes | Good | Healing targets exactly this maintenance cost |
| Exploratory coverage of a new web flow | Good, with review | Planner and generator can draft coverage quickly |
| Business-critical assertions such as pricing or entitlement rules | Human-owned | The agent can draft; a person must define what correct means |
| Applications with no test environment or with production data in test | Not yet | Agent access to real data is a risk; fix the environment first |
| Safety, financial or regulated release decisions | Support only | Agent output is evidence for a decision, not the decision |
| Test data and data pipelines | Different tooling | See our guide to testing data quality with agentic AI |
What are the governance, security and cost considerations?
- Scope agent access. Playwright MCP can run custom code in the browser context and its own documentation warns that this is RCE-equivalent. Give agents test environments, test accounts and the least access they need.
- Review what the agent changes. Healing edits test code. Treat each patch like a code change, with a pull request, an owner and a record of why it was made.
- Guard against healing real bugs. A healer that "fixes" a test to match broken behaviour hides a defect. The Playwright healer is documented to skip a test when it believes the functionality is broken; keep a person in the triage path for those cases.
- Keep an audit trail. Record the plan, the prompts or goals, the patches and the run results for each agent run, so a release decision can be traced. Our guides to agentic AI risk management and agentic AI governance describe the controls.
- Track cost by outcome. Count model usage, CI minutes and reviewer time against test maintenance hours saved and defects caught earlier. Measure these in your own environment before scaling; do not rely on generic savings figures.
How should you adopt agentic testing in stages?
- Pick one application with a stable test environment and a regression suite that costs real maintenance time.
- Stabilise locators and seed tests first, following the framework's own locator guidance.
- Run the planner and generator in a branch and have a tester review every plan and every generated test.
- Add healing for a bounded set of tests, with patches arriving as pull requests.
- Define quality metrics up front: flaky-test rate, time to repair a failing test, escaped defects.
- Expand to further applications only when those metrics hold, and keep human approval for release gates.
How does XenonStack support agentic testing?
Agentic testing works in an enterprise when the agents are governed and observable and act on trusted context about the application under test.
- ElixirClaw: an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control, which fits running testing agents with approval steps.
- ElixirData: keeps governed context, versioned policy and decision traces in one decision layer above AI agents, so requirements, defect history and release data can inform what agents test.
- AgentQA: according to its site, it continuously tests AI agents' outputs and multi-step workflows. AgentEvaluation is, according to its site, for testing AI agents on real tasks before and after launch.
Teams that want quality checks built into every stage of delivery can also read about QAOps.
To map one testing decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.
Frequently Asked Questions (FAQs)
What is agentic software testing?
It is the use of AI agents that plan tests, generate test code, run it, repair failing tests and report results, with people approving plans, patches and release decisions.
How is agentic testing different from test automation?
Scripted automation replays steps an engineer wrote and fails when the application changes. An agentic approach works from a goal: it explores the application, drafts a plan, writes tests and proposes patches when tests break.
Which open-source tools include testing agents?
According to its documentation, Playwright includes three Test Agents: a planner, a generator and a healer. Playwright MCP lets an LLM drive a browser. The Selenium and Appium documentation describe automation frameworks, not agents.
What are the main risks of agentic testing?
Over-broad agent access, unreviewed test patches, and healing that hides real defects. Playwright MCP documentation warns that running custom code is RCE-equivalent, so use it only with trusted clients and test environments.
Will agentic AI replace QA engineers?
No. Agents take on repetitive work such as test maintenance, while engineers define what correct behaviour is, review plans and patches, and decide on releases.
How do you start with agentic testing?
Start with one application and one bounded task, such as regression test maintenance. Stabilise locators, keep patches in pull requests, define metrics up front and expand only when they hold.
Next Steps in Agentic Testing
Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE