Getting AI to click on pages is easy; proving it clicked the right thing is hard

e2e, the open-source AI testing framework, recently reached 13.8k stars on GitHub, and the team just raised $8 million in funding. Founding team developer evangelist Kacper Kapuściak wrote an engineering practices piece on writing E2E tests for AI Agents. The article isn't short, and its core tension is clear: once an Agent can click through pages for you, test credibility depends even more on assertions and sources of truth.

1. Test user intent, not frontend implementation

Users just want to generate last month's activity report. They don't care which component library the date picker uses or how many DOM layers are nested inside the button. Kacper recommends giving the Agent a business goal directly—in code, it's just agent.act, with the goal being 'generate last month's activity report.' The Agent figures out whether it should select the month, fill in a date, or click a preset. If the frontend switches component libraries or changes button copy, the test case doesn't need to be rewritten. Tests bound to test IDs go red as soon as there's a redesign, and maintenance cost can end up higher than writing the tests.

2. Wait for business state, don't use fixed sleeps

Fixed sleeps are a chronic disease in CI. A five-second wait can make things pass locally, but it also adds five seconds to every successful run. Report generation time varies with data volume and server load. Kacper's approach is to use agent.waitFor with a timeout cap to wait for a strong business state such as 'last month's activity report is now viewable.' The disappearance of a loading indicator is a weak signal; it only means some UI element is gone. One commenter added: dynamic waits must be tied to an observable source of business state; otherwise, 'wait for the report to open' and a fixed sleep are just two different ways of guessing.

3. Use backend APIs as the source of truth, then have an independent Judge reconcile

The execution Agent saying 'click complete' doesn't count as acceptance. The report could have empty data, or it could be showing data from another workspace. Even less should we let an LLM blindly guess what the correct screen is. Kacper's process is to first query the backend API for the real task list, use that as the single source of truth, then call an independent AI Judge to compare the rendered screen details against the API's underlying records item by item. A commenter added a key point in reply: for results like CSV, you can add another deterministic check—column names, row counts, and amounts can be reconciled directly in code—while leaving layout and interaction semantics to the AI Judge. The fewer layers of 'a model judging a model,' the better.

4. Don't let tests depend on each other

A report test shouldn't have to wait for the 'complete task' test to run first. When run alone, retried, or run in parallel, the data may disappear. Kacper recommends calling the API through a reusable Fixture before the test starts, independently creating a dedicated workspace and test data, and cleaning up/restoring in afterEach or afterAll after the run. Each test should set up its own state so retries and concurrency are idempotent. Shared code can stay, but don't package order dependencies as reuse.

5. Split into atomic tests and package failure traces

'Generate report, download CSV, send email' are three actions; split them into clearly named independent tests. On failure, e2e packages a failure trace covering prerequisite steps, application logs, the Agent's reasoning context, and screenshots. Humans can locate the issue quickly, and the trace can be fed directly to a Coding Agent to fix the bug. It doesn't guarantee perfect reproduction, nor does it guarantee pointing directly to the line in the application code that broke, but it's far better than a single 'something went wrong.'

One commenter poured a bit of cold water in the replies: software is ultimately still for people to use, and fully automated AI E2E can at best do a rough job of clearing mines; in some scenarios the results are still mediocre. This isn't a flattering thing to hear, but it's worth putting on the acceptance checklist. An Agent can lower the cost of operations, but it can't define correctness for you. Start by picking one test case the team reruns repeatedly, and revise it according to these five points. Then deliberately break a flow and see whether the test catches it.

Link: 5 Tips for Writing Good Agentic Tests with e2e