Accessibility tree testing
what is accessibility tree testing?
The accessibility tree is the structured representation a browser exposes to screen readers: the role, name, value and state of every element on a page. An agent that drives a browser through this tree picks a named control rather than a screen coordinate, which is why it does not click the wrong thing when a layout shifts or a page renders at a different size.
The alternative is vision: screenshot the page, ask a model where to click, click there. It works, it is impressive to watch, and it fails in ways that are hard to debug, because a coordinate is not a thing, it is a location where a thing happened to be.
Reading the tree also makes a run cheaper and more legible. The model receives text describing controls rather than an image, and the evidence a failure produces names an element rather than a pixel, which is the difference between a report you can act on and a screenshot you have to interpret.
It has limits worth knowing. Canvas-rendered interfaces, custom widgets with no roles and iframes across origins all expose less than a well-built page does, and an agent reading the tree sees exactly what a screen-reader user would see, which is sometimes a lesson in itself.
in smolanalytics
The agent works from the accessibility tree rather than from screenshots, so it selects an element by role and name and never a pixel coordinate. A useful side effect is that a page an agent struggles to use is often a page a screen reader struggles with too, which the free readability check on this site will tell you about without an account.
How it works shows where this fits in the loop, and the docs have the exact behaviour.