How we shipped the world's first GenAI-native testing agent.

For a decade, QA was stuck in a paradox.

To understand why KaneAI mattered, you have to understand the gap it filled. Releases got faster. CI/CD became standard. Cloud device grids and parallel execution made running tests cheap. But the part where someone actually writes a test stayed manual and slow.

A QA engineer wanting to automate even a moderately complex flow had to read the user story, manually navigate the app, inspect DOM elements for stable locators, write the test as code in Selenium or Playwright, implement Page Object Models, add explicit waits and assertions, debug locally, push to source control, and then — maintain it forever. Every UI change in the product could break it. Renaming a single button class could break 200 tests overnight.

A regression suite of 200 tests was 4–6 person-weeks of authoring, before maintenance even started. The test suite became a liability — a thing engineers feared changing because the downstream impact was unpredictable. This was the world KaneAI was built into.

Simple to state. Hard to do.

The brief was clean and ambitious: let users describe what they want to test in natural language, and let AI do the rest.

No XPaths. No Selenium boilerplate. No Page Object Models. No framework lock-in. Just describe the behavior; the AI handles implementation, identifying elements intelligently, and executing on the LambdaTest grid.

If the goal was just "build a tool that can do that," the AI capability was reachable. The real challenge was building a tool QA teams would actually trust to do that, at scale, on the test suites that protect their production environments.

The AI was reachable. The trust wasn't.

This was my first true 0→1 product. From day one, two things became clear. We were solving two problems simultaneously, and the second was harder than the first.

Think about what you're asking a QA engineer to do. You're asking them to hand over the work that protects their product from breaking, to an AI. If that AI gets a test wrong, a real bug ships to production. If it's over-confident, users find out the hard way. If it's opaque about what it did or why, the engineer has no way to verify.

QA teams are paid to be skeptical. That's literally the job. The default mental model was: "I'd rather write 100 lines of Selenium myself than trust an AI to do it for me, because at least I know what my code does."

Five principles. One product.

These didn't come from a wall on day one. They emerged from constant iteration with our internal QA team and pilot customers — and by the time we shipped MVP, they'd earned their place by being tested against hundreds of small decisions.

The fifth principle was the hardest to live up to. KaneAI had to work for an SDET who could read the underlying code, and a PM who'd never written a test in their life. One product, multiple skill levels, without dumbing down for either. That tension shows up in almost every screen of the product.

Two entry points, one authoring loop.

Most AI products hide their work behind a spinner. We did the opposite — every action the agent takes is visible on screen. The user is in the loop, not waiting outside it.

KaneAI has two ways in. Test Planner is for users who want to plan coverage before writing anything — feed it a PRD, a Jira ticket, screenshots, a feature description, even audio or video, and it produces a set of test cases that cover the feature. The Web Agent is for users who want to skip planning and start authoring directly. Either entry point lands the user in the same authoring environment. Forcing every user through Test Planner would have made KaneAI feel heavy for simple cases. Skipping it entirely would have left the most ambitious AI work on the table.

Test Planner — the strategic layer

Test Planner does the most ambitious AI work in the product.

Translating a single scenario into steps is, frankly, not that hard for a modern model. What Test Planner does is harder: it does test-case decomposition. From a feature description, it reasons about what could go wrong, what scenarios matter, where the coverage gaps are. It's the work a senior QA lead does when planning a release, compressed into something the AI can draft in seconds.

The verification question changes at this layer. Instead of "did the AI translate my scenario correctly?", the user asks "did the AI think of the cases I would have, plus ones I might have missed?" That's a more useful question to be answering, because the AI is helping with work the user couldn't do as well or as fast themselves.

The Web Agent — authoring with the page in view

The Web Agent is where the actual test gets built. The screen is a deliberate split-pane: the user's natural-language instructions and the AI-generated steps on the left, the live target application running in a real browser on the right.

As the user describes what to do — "click the search bar, type 'wireless headphones', filter by 4+ stars" — they're looking at the actual page they're describing. The element is right there. The state of the page is right there. Authoring isn't blind.

This was the single biggest trust-design decision in the product. Most AI authoring tools ask the user to describe a workflow against an abstract idea of an interface. We built it the other way: the page is in front of the user the whole time. The AI's interpretation of each step lands in real time, against the real DOM, and any mismatch is immediately visible.

In user testing, this single layout change did more for trust than any explanation copy we wrote. Users described the product as "transparent" and "honest" without us using either word in the UI. The layout was doing the talking.

Once a test is authored, the user kicks it off to run on HyperExecute — LambdaTest's existing cross-browser cloud. Authoring stays interactive and fast; execution stays parallel and infrastructure-grade. Two different jobs, two different surfaces.

Modules — making it a real authoring environment

Reusable test components: login flows, common navigation paths, frequently-used assertions. Save them once, reuse them across tests.

This solved a major pre-AI problem (duplicated test code) and shifted KaneAI from a one-shot AI to a real authoring environment. Tests stopped being throwaway artifacts and started being a system the team could build on.

Parameters and Variables — tests as systems, not one-offs

Dynamic data inputs for flexible, reusable tests. Run the same test with different data sets without rewriting it.

Adoption hit 70% within a few months of shipping — a strong signal that users had crossed the line from "trying KaneAI for one-off tests" to "building durable test suites with it."

API Testing — same loop, different surface

Backend testing alongside UI testing. Validate API responses, test integrations, ensure services communicate correctly.

Designed to feel like a natural extension of the same authoring loop, not a separate product. The user describes API behavior in natural language, the same way they describe UI behavior. The product stayed coherent even as the surface area grew.

From AI session to a structured test case

Generating a test isn't the same as owning a test. KaneAI included a deliberate handoff step: every authored test could be promoted into a structured test case with a title, description, priority, status, automation flag, tags, and linked variables. Assignable to a teammate, versioned, and queryable from the test manager.

This was the seam between "AI agent that ran something for you" and "test asset your team relies on." Without it, KaneAI would have been a clever demo. With it, the output of an AI session became something a QA team could actually live with — review, schedule, hand off, and trust six months later.

Dev tools have their own design language.

KaneAI's users weren't end consumers. They were technical practitioners — QA engineers, SDETs, developers, DevOps folks. People who already had strong opinions about their tools and very low tolerance for friction. Designing for that audience is its own discipline. It shaped a thousand smaller decisions.

• Respect the existing stack. Tests had to be exportable as real Selenium / Playwright / Cypress code so engineers could review, version, and ship them through their existing CI/CD pipelines.
• Trust the user's mental model. Files, branches, commits, modules, environments, runs. KaneAI's IA mirrored what dev users already think in — closer to an IDE than a marketing page.
• Density over decoration. Information density, keyboard shortcuts, fast navigation. Optimize for screens that show a lot of useful information without feeling cluttered.
• Errors and failures are core flows, not edge cases. A failing test is the most important moment in the product. Logs, screenshots, suggested fixes — designed with the same care as success states.
• Don't pretend the AI is perfect. Honest UI when the AI is uncertain or making assumptions. Dev users see through false confidence immediately.

Tight loops. Real users. Eat our own dog food.

A blank slate is liberating and terrifying in equal parts. Here's how we navigated it.

Working with the co-founder

I partnered closely with our co-founder on product vision, workflow definition, and system architecture. This wasn't a "design hands a spec to engineering" relationship. We worked side by side — debating product decisions, sequencing features, weighing tradeoffs in real time. The co-founder brought deep technical and strategic context. I brought the user, the workflow, and the experience design. Engineering brought the constraints and possibilities of the AI itself.

Internal QA team as continuous design partners

This was the secret weapon. LambdaTest's own QA engineers were exactly our target user. They had real pain points, real opinions, and zero patience for products that didn't earn their trust. Every iteration of KaneAI went through them first. They broke things. They asked for things. They told us when something felt wrong even if they couldn't articulate why.

That dogfooding loop forced us to confront product gaps from the inside. If our own QA team couldn't trust KaneAI to test KaneAI, no external customer would either. The phrase "would you actually use this on your production tests?" became a regular sanity check.

The product didn't ship in one shot. We followed a tight, iterative loop: build, test internally, ship to a small set of pilot customers, gather feedback, refine. The MVP launch on August 22, 2024 wasn't an end state — it was an inflection point. We continued building heavily through April 2025.

Discipline shipped what features couldn't.

The hardest decisions on this project weren't about what to build. They were about what to leave out — and to be specific, what to deliberately defer to ship after MVP, on a foundation users had already learned to trust.

The MVP had one job: prove that the end-to-end loop worked. If we shipped Modules without first earning trust on that core loop, we'd be asking users to invest in a system they hadn't yet decided to believe in.

All three deferred features shipped in the months after MVP, once we had real users, real feedback, and a stable foundation. By the time we shipped them, we knew exactly what shape they needed to take — because we'd watched users live without them. Post-MVP work wasn't fixes for things we'd shipped wrong. It was planned scope, deferred on purpose, shipped on schedule.

The spectrum had a ceiling — and we knew it.

The pitch was "anyone can author a test," and within authoring, that was true. PMs authored in NL, SDETs validated in code, both worked on the same underlying test. But the spectrum had a ceiling beyond authoring, and it's worth being honest about that.

Authoring opened up to non-technical users. The parts of the workflow that came after — execution configuration, debugging failures, analyzing logs, integrating with CI/CD — stayed technical. So during the beta, non-technical users could contribute to the test suite by authoring tests, but the power users who took those tests end-to-end were still SDETs and engineers.

What shipped. What changed.

KaneAI launched MVP on August 22, 2024. The signal in the months that followed was strong, and it came from multiple directions at once.

The fundraise was specifically positioned around AI expansion, with KaneAI as the centerpiece. The product didn't just sell — it became the wedge that made LambdaTest's existing infrastructure (HyperExecute, the cross-browser cloud) accessible to a 10× larger non-technical user base. The AI was the new front door to an established business.

The numbers above are the leadership-deck version. The version I care about more is qualitative: in our follow-up interviews, multiple auditors described the new product as "the first testing tool that doesn't feel like it was made by people who've never written tests." That sentence is on a Post-it on my desk.

What I'd revisit if I were starting over.

Honestly, the MVP shipped well. I'm not sitting on a list of things I'd reverse. The cuts held up, the principles held up, the post-MVP shipping plan held up. The thing I'd revisit isn't a UI decision — it's an architectural one.

KaneAI shipped as a web product. That made sense at the time. It integrated cleanly with LambdaTest's cloud grid, no download barrier for customers, faster iteration on our side, easier support story. All real reasons.

But execution speed and resource handling have always been a soft spot for web-based test automation. The browser is running inside another browser. Network round-trips happen for every action. There's no access to local resources. For users on slower networks or running heavier test suites, the cost shows up.

If I were starting over with what I know now, I'd push hard on whether KaneAI should ship as a desktop app (at least as an option) — using local browser instances, local resources, and local execution. That would have meant materially faster test runs and a tighter debugging experience.

The tradeoffs are real. A desktop app is a download barrier, a different distribution and update model, more engineering investment up front. So I'm not saying we got it wrong. For MVP, web was the right call to validate the thesis. But the right time to revisit the architectural call is post-MVP, once the thesis is validated. That's the call I'd want to test seriously now.

Seven lessons that carried over.