Skip to main content
XenonStack

Artificial Intelligence

XenonStack Blog Agentic Software Testing: Agents, Tools and Controls

and | Published | Updated

Agentic Software Testing: Agents, Tools and Controls
12:44

Key takeaway

  • Agentic software testing explained: how planner, generator and healer agents work, what Playwright, Selenium and Appium offer, and the controls to adopt.

Agentic software testing uses AI agents that plan what to test, generate test code, run it, repair failing tests and report results, with people approving the decisions that matter. It differs from scripted automation because the agent works towards a quality goal instead of replaying fixed steps. For a CTO or head of engineering, the question is not whether agents can write a test, but which parts of the test lifecycle can be handed over safely and how that work is governed.

This guide covers what the official Playwright, Selenium and Appium documentation say, how an agentic test loop works, where it fits and where it does not, and how to adopt it in stages. It complements our guides to agentic process automation testing and testing data quality with agentic AI, which cover process-level and data-level testing.

What is agentic software testing?

In scripted automation, an engineer writes each step and each assertion, and the suite fails when the application changes. In agentic testing, an agent is given a goal such as "cover guest checkout", explores the application, proposes a plan, writes the tests and fixes them when they break. The ability to act on the application and on the test code is what makes the system agentic. It builds on the same ideas as agentic workflows and the wider agentic AI stack, applied to quality assurance.

AspectScripted automationAgentic testing
Who decides what to testEngineer, upfrontAgent proposes a plan, engineer approves
Test creationWritten by hand or recordedGenerated from a plan and live exploration of the application
When the UI changesTest fails until someone edits itAgent replays the failing steps, proposes a patch and reruns
Human roleAuthor and maintainerReviewer of plans, patches and results
Main riskMaintenance effortUnreviewed changes, wrong assertions, over-broad agent access

What do the official testing tools provide today?

The major open-source tools do not all ship agents. The table separates what each project documents on its own site from what a team still has to build around it.

Tool (source)What its documentation saysFit for agentic testing
Playwright Test AgentsThree built-in agents: a planner that explores the app and writes a Markdown test plan, a generator that turns the plan into Playwright tests, and a healer that runs failing tests, inspects the UI, suggests a patch and reruns. Started with npx playwright init-agents, with VS Code, Claude Code, Codex and OpenCode listed as supported tools. A seed test supplies setup.Closest to a ready plan, generate, heal loop for web applications
Playwright MCPA Model Context Protocol server that lets an LLM drive a browser through the page accessibility tree, not pixels. It can navigate, click, fill forms, inspect network requests and run custom Playwright code, which the docs call RCE-equivalent and advise enabling only for trusted clients. Needs Node.js 20 or newer.Gives your own agent browser control; needs strict access limits
Playwright locatorsRecommends role locators first, notes that CSS and XPath selectors can break when the DOM changes, and says locators are strict and re-resolve the element on each use.Stable locators reduce what an agent has to repair
SeleniumAn umbrella project: WebDriver for browser automation, IDE for recording, Grid for running tests across machines and platforms.Execution layer your agents can call; agents are not part of the project
AppiumOpen-source, cross-platform automation for native, hybrid, mobile web and desktop apps, built on the W3C WebDriver protocol with drivers per platform and official clients for Java, Python, Ruby and .NET.Mobile and desktop execution layer; agents are not part of the project

Read this as a split between the agent (reasoning about what to test and how to repair it) and the execution layer (the framework that drives the browser or device). Selenium and Appium are the second kind in their own documentation; Playwright documents both.

How does an agentic test loop work?

Whatever the tooling, the loop has five stages. Each stage has an owner and a control.

Two-column diagram: agent tasks plan, generate, run and heal on the left, mapped to controls approve the plan, review the code, limit access and review patches on the right.
StageWhat the agent doesControl to keep
1. PlanExplores the application and drafts a test plan from a request, a seed test and, where available, a requirements documentA person approves the plan before tests are generated
2. GenerateWrites test code from the plan and checks selectors and assertions against the running applicationCode review, as for any pull request
3. RunExecutes in CI against a test environmentAgent has no access to production data or credentials
4. HealReplays failing steps, inspects the current UI, proposes a patch and reruns; skips the test if it judges the functionality itself to be brokenPatches go through review; skipped tests are triaged by a person
5. ReportSummarises coverage, failures and skipped testsResults feed your quality metrics and release gates

The Playwright documentation describes the planner, generator and healer in this order and notes that the generator output may contain initial errors that the healer can fix. That is a reminder that review is part of the design, not an optional extra. The same release gates feed delivery metrics such as those in DevOps metrics.

Where does agentic testing fit, and where does it not?

SituationFitReason
Regression suites that break often after UI changesGoodHealing targets exactly this maintenance cost
Exploratory coverage of a new web flowGood, with reviewPlanner and generator can draft coverage quickly
Business-critical assertions such as pricing or entitlement rulesHuman-ownedThe agent can draft; a person must define what correct means
Applications with no test environment or with production data in testNot yetAgent access to real data is a risk; fix the environment first
Safety, financial or regulated release decisionsSupport onlyAgent output is evidence for a decision, not the decision
Test data and data pipelinesDifferent toolingSee our guide to testing data quality with agentic AI

What are the governance, security and cost considerations?

  • Scope agent access. Playwright MCP can run custom code in the browser context and its own documentation warns that this is RCE-equivalent. Give agents test environments, test accounts and the least access they need.
  • Review what the agent changes. Healing edits test code. Treat each patch like a code change, with a pull request, an owner and a record of why it was made.
  • Guard against healing real bugs. A healer that "fixes" a test to match broken behaviour hides a defect. The Playwright healer is documented to skip a test when it believes the functionality is broken; keep a person in the triage path for those cases.
  • Keep an audit trail. Record the plan, the prompts or goals, the patches and the run results for each agent run, so a release decision can be traced. Our guides to agentic AI risk management and agentic AI governance describe the controls.
  • Track cost by outcome. Count model usage, CI minutes and reviewer time against test maintenance hours saved and defects caught earlier. Measure these in your own environment before scaling; do not rely on generic savings figures.

How should you adopt agentic testing in stages?

  1. Pick one application with a stable test environment and a regression suite that costs real maintenance time.
  2. Stabilise locators and seed tests first, following the framework's own locator guidance.
  3. Run the planner and generator in a branch and have a tester review every plan and every generated test.
  4. Add healing for a bounded set of tests, with patches arriving as pull requests.
  5. Define quality metrics up front: flaky-test rate, time to repair a failing test, escaped defects.
  6. Expand to further applications only when those metrics hold, and keep human approval for release gates.

How does XenonStack support agentic testing?

Agentic testing works in an enterprise when the agents are governed and observable and act on trusted context about the application under test.

  • ElixirClaw: an agentic OS platform to build, orchestrate and govern AI agents and automation workflows while keeping humans in control, which fits running testing agents with approval steps.
  • ElixirData: keeps governed context, versioned policy and decision traces in one decision layer above AI agents, so requirements, defect history and release data can inform what agents test.
  • AgentQA: according to its site, it continuously tests AI agents' outputs and multi-step workflows. AgentEvaluation is, according to its site, for testing AI agents on real tasks before and after launch.

Teams that want quality checks built into every stage of delivery can also read about QAOps.

To map one testing decision your agents should make, and the context and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently Asked Questions (FAQs)

What is agentic software testing?

It is the use of AI agents that plan tests, generate test code, run it, repair failing tests and report results, with people approving plans, patches and release decisions.

How is agentic testing different from test automation?

Scripted automation replays steps an engineer wrote and fails when the application changes. An agentic approach works from a goal: it explores the application, drafts a plan, writes tests and proposes patches when tests break.

Which open-source tools include testing agents?

According to its documentation, Playwright includes three Test Agents: a planner, a generator and a healer. Playwright MCP lets an LLM drive a browser. The Selenium and Appium documentation describe automation frameworks, not agents.

What are the main risks of agentic testing?

Over-broad agent access, unreviewed test patches, and healing that hides real defects. Playwright MCP documentation warns that running custom code is RCE-equivalent, so use it only with trusted clients and test environments.

Will agentic AI replace QA engineers?

No. Agents take on repetitive work such as test maintenance, while engineers define what correct behaviour is, review plans and patches, and decide on releases.

How do you start with agentic testing?

Start with one application and one bounded task, such as regression test maintenance. Stabilise locators, keep patches in pull requests, define metrics up front and expand only when they hold.

Next Steps in Agentic Testing

Talk to our experts about implementing advanced AI-driven software testing solutions. Discover how industries and departments leverage intelligent workflows and decision intelligence to deliver high-quality software efficiently. Utilize AI to automate and optimize testing processes, enhance accuracy, and accelerate development cycles, ensuring robust and reliable software delivery.

Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.

Table of Contents

navdeep-singh-gill

Navdeep Singh Gill

Global CEO and Founder of XenonStack

LinkedIn profile

Navdeep Singh Gill is serving as Chief Executive Officer and Product Architect at XenonStack. He holds expertise in building SaaS Platform for Decentralised Big Data management and Governance, AI Marketplace for Operationalising and Scaling. His incredible experience in AI Technologies and Big Data Engineering thrills him to write about different use cases and its approach to solutions.

Get the latest articles in your inbox

Subscribe Now

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue