Skip to content
Lauren DormanDesign Engineer

Notes

The second reader

Put a person in front of a venue's calendar and they'll know within a second which shows are all ages, what a ticket actually costs and where to buy one. Point a browser agent at the same page and it might see something else: a tree of unlabeled containers, two prices with nothing to say which one applies, and a "Buy" button that's really a div with a click handler. The real version is usually quieter: a gallery where every photo's alt text is the venue's name. A person sees the stage, the bar and the room. Anything reading the markup gets the same name, once per photo.

The page has a second reader now, and it reads differently. Some agents look at screenshots. Some parse the DOM. Some read the accessibility tree, the same structure screen readers use. Playwright's MCP server, a common way to give a model a browser, hands it a snapshot of roles and names instead of a picture.

I'm not arguing that agents are replacing people. Most visitors to these sites are still people. The narrower point is that a page now has to make sense to both, and what makes it make sense to a machine is mostly structure, not appearance.

Version A

Upcoming at The Hollis
Filter
All ages
Saturday
Kid Meridian
Late show
Sat, Nov 721+
$22
Buy
Pale Hours
with Static Bloom
Sat, Nov 14
$25 $34
Buy
Ferro & Lyle
Seated show
Fri, Nov 20
$26
Buy
The Overnights
Early show
Sat, Nov 21
$30
Buy

Version B

Upcoming at The Hollis

Filter
Same look, different markup. Hover any element to find its node in each page's accessibility tree.

The limits of visual meaning

A lot of front-end code communicates meaning through layout and styling alone. That works for sighted people using a mouse. For anything else, the meaning just isn't there.

PatternWhat a person seesWhat a machine gets
A clickable divA buttonText, with no role and no keyboard access
Six "Learn more" linksContext from the card around each oneSix identical links
Styled spans as section titlesClear sectionsOne long run of text
A custom date picker with no accessible nameA calendarAn unnamed group of buttons labeled with numbers
An old price struck through with CSSA discountTwo prices, with no way to tell which applies

Visually, none of these look broken. A person resolves the ambiguity with context. An agent might resolve it with a guess.


Accessibility as infrastructure

Most of what helps here isn't new. Semantic HTML, accessible names, real form controls and exposed states (aria-pressed, aria-expanded, disabled) were built so assistive technology could interpret a page. Agents that read the accessibility tree benefit from exactly the same work.

But accessibility and agent compatibility are related, not equivalent. A screen reader user brings judgment and patience, and can explore their way out of an ambiguous flow. An agent may take the first plausible match and keep going. Hover-only menus, steps that depend on timing, and bot protection (sometimes there on purpose) can stop an agent on an otherwise accessible site. And agents that work from screenshots still depend on visual clarity, so none of this is an argument for caring less about how things look.


Beyond the rendered page

There are more layers beneath and around the page, at very different stages of maturity.

LayerStatusWhat it's good for
Semantic HTML and ARIAEstablished standardsAnything that reads the page, including assistive technology
Structured data (schema.org)Established, widely used by search enginesFacts like price, availability and location
Clear hierarchy and stable URLsEstablished practiceGoing straight to a show, a filter or a state
Markdown versions of pagesEmerging conventionClean, cheap text for models to read
llms.txtProposed in 2024, little evidence that major AI products read itAn index for developer docs read by coding tools
WebMCPDraft community report, early preview in ChromeLetting a page expose its actions as named tools

The first three are where I'd spend effort today. The rest are worth trying cheaply, with realistic expectations. Google has said it doesn't support llms.txt, and log studies so far show very few fetches. WebMCP is a more interesting idea: a page registers tools with a name, a description and an input schema, so an agent can call "add to saved list" instead of hunting for the button. It's also explicitly not a standard yet.1


Two versions of one page

To see how much structure matters in practice, I built the two calendar pages at the top of this note. Version A uses the patterns from the first table. Version B keeps the CSS and fixes the markup: an article and heading per show, real links and buttons, labeled filters, prices with visually hidden context, and structured data.

Here's roughly what an accessibility snapshot looks like for one show in each:

# Version A
- generic:
  - text: Pale Hours
  - text: with Static Bloom
  - text: Sat, Nov 14
  - text: $25 $34
  - generic  # the save icon

# Version B
- article:
  - heading "Pale Hours" [level=3]:
    - link "Pale Hours"
  - paragraph: with Static Bloom
  - list:
    - listitem: Sat, Nov 14
    - listitem: All ages
  - paragraph: Early bird $25 sold out, now $34 per ticket
  - button "Save Pale Hours" [pressed=false]

The agent test

To compare the two versions, give each agent the same task against both: find the cheapest all-ages Saturday show at The Hollis and add it to the saved list. Run it several times per version, and record whether it succeeds, how many steps it takes, and where it goes wrong when it doesn't.

A few honest limits. Agents change from week to week, so a result from this month may not hold next month. The wording of the task affects the outcome. And two pages I built myself are cleaner than most real sites, in both versions. I'd treat any result as a signal about structure, not a benchmark of any agent.


Designing for both readers

It's tempting to treat semantics as an implementation detail, something to tidy up after the design is done. But structure decides who, or what, can use a product. Most of the work it takes is work we owed people anyway, and the newer layers will keep shifting. The page is still the interface both readers share, and its structure is the part I can control today.

Footnotes

  1. It's being drafted by engineers at Google and Microsoft in the W3C's Web Machine Learning Community Group. ↩︎