Skip to main content
Knowledge mapWhat does the agent see behind a button?ArcBlock

You are here. See how this question connects to other ideas.

Select a node to open its page · Expand to read within the map

Knowledge mapFollow a connection. Understand a question.
← AFS and interfaces

One question

What does the agent see behind a button?

A visual control can correspond to named structure, state and actions, rather than only a region of pixels.

Imagine a task shown as a card. A person recognizes its title and status. An agent can work with the corresponding address and supported operations, without first inferring those known facts from a screenshot.

Other systems already expose structure through the DOM, accessibility trees and APIs. The AFS-UI question is how that structure participates in the same resource and operation model as the rest of the application.

The correspondence is semantic. It does not require a separate path for every shadow or decorative pixel. Visual inspection remains useful for layout, charts and appearance.

Teaching model · example paths, local state

Try the idea

Change the task status and switch its presentation. The object address stays the same.

What a person sees

Prepare the keynote

In progress

What an agent can address · Example workspace

/work/tasks/keynote/title
Prepare the keynote
/work/tasks/keynote/status
in-progress

Same object. Different representations.

This model illustrates semantic correspondence. It does not inspect this browser’s live AFS session.

Check your understanding

Would a pixel-level path for every shadow help an agent operate a task?

Usually not. Identify the object, meaningful state and permitted actions first.

Continue along a learning path