[{"data":1,"prerenderedAt":2852},["ShallowReactive",2],{"/blog/human-approval-checkpoints-for-ai-agents":3,"related-/blog/human-approval-checkpoints-for-ai-agents":414},{"id":4,"title":5,"authorId":6,"body":7,"category":372,"created":373,"description":374,"extension":375,"faqs":376,"featurePriority":389,"head":390,"hideCta":391,"landingPath":390,"meta":392,"navigation":403,"ogImage":390,"path":404,"robots":390,"schemaOrg":390,"seo":405,"sitemap":406,"stem":407,"tags":408,"__hash__":413},"blog/blog/1051.human-approval-checkpoints-for-ai-agents.md","Human Approval Checkpoints for AI Agents (2026)","salome-koshadze",{"type":8,"value":9,"toc":362},"minimark",[10,14,23,26,60,65,68,74,77,105,108,111,114,119,123,126,130,133,138,141,144,161,164,168,171,175,178,181,216,219,225,228,232,235,239,242,268,271,274,277,281,284,288,359],[11,12,13],"p",{},"AI agents now run multi-step work across enterprise web apps, which gives software write access to production systems. One unverified action can send a wire transfer, change a live configuration, or edit a customer record. A checkpoint sits in front of those actions: the agent's proposed change waits until a person approves it.",[15,16],"nuxt-picture",{":height":17,":width":18,"alt":19,"loading":20,"src":21,"provider":22},"680","900","Diagram titled \"Where the Checkpoint Lives in the Stack\" showing four layers where an agent's action can be paused - the browser layer where a reviewer joins the same DOM the agent sees (Webfuse), the application layer where a proposal sits pending until a webhook writes (Velt), the development workflow where the pause happens on a design document before code is written (HumanLayer), and the orchestration runtime where a developer marks any node as an interrupt (LangGraph) - above a production systems band for wire transfers, live configuration and customer records, with an arrow showing that an unchecked action falls straight through every layer","lazy","/blog/human-approval-checkpoints-for-ai-agents/1.svg","none",[11,24,25],{},"Building that checkpoint means pausing agent execution, showing the proposed action to a reviewer, and resuming once the reviewer approves. Webfuse, HumanLayer, LangGraph and Velt each do this at a different layer of the stack - a browser proxy, a coding IDE, an orchestration framework, and a UI SDK.",[27,28,30,36,42,48,54],"tldr-box",{"title":29},"TL;DR",[11,31,32],{},[33,34,35],"strong",{},"Pick the tool by where the risky action happens.",[11,37,38,41],{},[33,39,40],{},"Webfuse"," (our own product) pauses inside the user's live browser session. Built for web apps with no API.",[11,43,44,47],{},[33,45,46],{},"HumanLayer"," puts the review on the design document, before any code is written.",[11,49,50,53],{},[33,51,52],{},"LangGraph"," pauses at a node you mark in the graph. You build the reviewer experience yourself.",[11,55,56,59],{},[33,57,58],{},"Velt"," puts approval inside your app's own UI. Proposals sit apart from live data until someone signs off.",[61,62,64],"h2",{"id":63},"webfuse-pausing-live-browser-sessions-for-a-human-to-join","Webfuse: pausing live browser sessions for a human to join",[11,66,67],{},"Webfuse runs as a proxy between the browser and the origin server. When a session starts, it intercepts the site's HTTP responses and renders the page inside a virtualized container, without touching the target app's source code. The agent reads and acts on the same DOM the person sees, in the same browser tab, in real time.",[15,69],{":height":70,":width":71,"alt":72,"loading":20,"src":73},"1508","2880","The Webfuse homepage, headlined \"Enable AI Agents to scroll on any Web App\" with the word scroll shown as a selected page element, and a subtitle describing connecting an agent to the live web through the Model Context Protocol to give an LLM eyes to see the DOM and hands to act on the user's behalf with no install required","/blog/human-approval-checkpoints-for-ai-agents/webfuse.png",[11,75,76],{},"When an agent hits a high-stakes step, Webfuse pulls a reviewer into that live session. The session becomes shared, and the reviewer sees the page itself rather than a summary of it. The pieces that make this work:",[78,79,80,87,93,99],"ul",{},[81,82,83,86],"li",{},[33,84,85],{},"Co-browsing escalation"," - a reviewer joins the live session to approve or correct what the agent is doing, in the same DOM state.",[81,88,89,92],{},[33,90,91],{},"Session recordings and audit logs"," - every agent action gets logged with a video replay, tying the model's reasoning to what happened on screen.",[81,94,95,98],{},[33,96,97],{},"Element-level rules"," - admins set allow and deny lists for specific URLs or DOM elements, so an agent can read a support ticket but not touch the billing field on the same page.",[81,100,101,104],{},[33,102,103],{},"Data masking"," - the proxy strips personal information, a card number or a password, before it reaches the model.",[11,106,107],{},"A bank employee approving a wire transfer an agent has drafted: the agent fills in the form fields, an admin rule marks the step as one that needs sign-off, and the employee gets pulled into the same session to check the numbers before clicking confirm. Both sides see the identical page state, and the video log captures the whole exchange for compliance review later.",[11,109,110],{},"Webfuse fits customer-facing workflows running on legacy web apps with no API, where the graphical interface is the only way in. It connects over MCP, so agent frameworks such as LangChain or AutoGen can drive a live session, and it supports a direct WebSocket bridge for voice agents.",[11,112,113],{},"Webfuse works at the browser layer, so a third-party proxy sits inline with production traffic. It also can't gate work that happens outside a browser: a background job, a database trigger, an internal API call.",[115,116],"article-signup-cta",{"heading":117,"subtitle":118},"Put a human in the loop of a live browser session","Webfuse runs any web app through a proxy, so an agent and a person can work in the same page at the same time. Pull a reviewer into the session before the agent submits a form, with element-level rules and a video log of every action.",[61,120,122],{"id":121},"humanlayer-stopping-the-agent-before-it-writes-code","HumanLayer: stopping the agent before it writes code",[11,124,125],{},"HumanLayer is an IDE and cloud platform built for teams running AI coding agents against large codebases. The checkpoint comes before any code gets written.",[15,127],{":height":70,":width":71,"alt":128,"loading":20,"src":129},"HumanLayer's workflow section, headlined \"Do not outsource the thinking\", with the line \"Structured workflows give the agent every opportunity to show you what it's wrong about before moving to implementation\" above a progress strip of the six QRSPI phases - Questions and Research marked complete, Design currently active, and Structure, Plan and Implement still greyed out ahead - over a panel showing the agent gathering codebase context and research findings","/blog/human-approval-checkpoints-for-ai-agents/humanlayer.png",[11,131,132],{},"The workflow is called QRSPI: Questions, Research, Design, Structure, Plan, Implement. Questions come first, before anything is written. During Research, the agent maps the codebase, its dependencies and the patterns around it. Design produces a document the team comments on, Structure breaks that into phased, verifiable steps, and Plan adds file paths, test cases and acceptance criteria. Only then does the agent move into Implement.",[15,134],{":height":135,":width":18,"alt":136,"loading":20,"src":137,"provider":22},"570","Diagram titled \"Shifting the Review Left: QRSPI\" plotting the cost to correct an error as a rising curve across HumanLayer's six phases - Questions, Research, Design, Structure, Plan and Implement - with the curve cheap and flat at the start where nothing is built yet and steepest at the point marked \"code starts here\", contrasting a roughly 200-line design document where a wrong assumption surfaces immediately against a 1,000-plus-line finished diff where the same error is already load-bearing","/blog/human-approval-checkpoints-for-ai-agents/4.svg",[11,139,140],{},"HumanLayer's stated principle: \"Do not outsource the thinking. Every phase is a place to push back.\" The design phase is where that pushback happens as a group, and the aim, in their words, is \"your whole team - humans and Agents - commenting on the design before a single line is written.\"",[11,142,143],{},"Beyond the workflow itself:",[78,145,146,152,158],{},[81,147,148,151],{},[33,149,150],{},"Task workspaces"," group agent sessions, files, and worktrees under one shared environment, so a team can see which agent is touching which part of the codebase.",[81,153,154,157],{},[33,155,156],{},"Bring-your-own-model"," support means teams plug in their own Claude, Codex, or Copilot subscription instead of paying a separate token bill.",[81,159,160],{},"A local daemon runs parallel agent sessions on a developer's machine, and the cloud side handles longer background tasks and syncs sessions across a team, so a reviewer working from a different device still sees the current state of a session.",[11,162,163],{},"HumanLayer hasn't open-sourced the QRSPI implementation, so the mechanics above come from how the team describes it publicly rather than from a spec. The scope is narrow: this is for software teams working in a codebase, not for approving agent actions in a business app.",[61,165,167],{"id":166},"langgraph-pausing-a-program-at-a-set-point-in-its-logic","LangGraph: pausing a program at a set point in its logic",[11,169,170],{},"LangGraph is a low-level framework from LangChain for building agent workflows as graphs, where nodes are functions or model calls and edges control the flow between them. It ships no review interface, only the primitives to build one.",[15,172],{":height":70,":width":71,"alt":173,"loading":20,"src":174},"The LangGraph documentation page on interrupts, explaining that interrupts pause graph execution at specific points and wait for external input before continuing, and that when an interrupt is triggered LangGraph saves the graph state through its persistence layer and waits indefinitely until execution resumes - with an on-this-page sidebar listing the common patterns, including approve or reject, review and edit state, and validating human input","/blog/human-approval-checkpoints-for-ai-agents/langgraph.png",[11,176,177],{},"A single call to a language model is stateless and forgets everything the moment it returns a response. Wrapping those calls in a graph with a persisted state object gives the agent memory across a long workflow, and it gives a pause something concrete to save and restore. That state object is typed, a dictionary or a Pydantic model in most setups, and each node records how it changed the state as the run continues.",[11,179,180],{},"The relevant primitive is the interrupt. A developer marks a node in the graph, a payment step for example, as one that should pause and wait. What that gets you:",[78,182,183,189,204,210],{},[81,184,185,188],{},[33,186,187],{},"The run stops and hands back a value."," Per the documentation, the interrupt function pauses graph execution and returns a value to the caller, then waits for you to resume with input.",[81,190,191,194,195,199,200,203],{},[33,192,193],{},"The state goes to a database."," A checkpointer writes it, ",[196,197,198],"code",{},"SqliteSaver"," for local work and ",[196,201,202],{},"PostgresSaver"," in production.",[81,205,206,209],{},[33,207,208],{},"Nothing runs while it waits."," No compute is consumed, so a graph can sit paused for minutes or days.",[81,211,212,215],{},[33,213,214],{},"Time travel rewinds a run."," A reviewer returns to an earlier checkpoint, edits a value, and branches down a different path instead of restarting from the beginning.",[11,217,218],{},"A support workflow shows the shape of it: a graph holds a customer's account ID, conversation history, and a proposed refund amount as it moves through nodes for data lookup, sentiment check, and refund calculation. A conditional edge routes any refund over fifty dollars to an interrupt node before the final execution step. The graph stops there, and a manager reviewing the state sees the agent's full reasoning trace before approving the refund through an API call that resumes the graph from that exact point.",[15,220],{":height":221,":width":222,"alt":223,"loading":20,"src":224,"provider":22},"600","940","Diagram titled \"Pausing a Graph at a Marked Node\" following a refund workflow through nodes for looking up an account, checking sentiment and drafting a refund, then a conditional edge asking whether the refund is over fifty dollars - routing under-threshold refunds automatically to execution and over-threshold refunds to an interrupt that pauses the graph, showing the state written to disk through a PostgresSaver checkpointer with an account ID, conversation history and a proposed refund of $84.00, a note that no compute runs while the run sits paused for minutes or days, and a manager reading the full reasoning trace with approve and reject controls that resume the graph through an API call","/blog/human-approval-checkpoints-for-ai-agents/3.svg",[11,226,227],{},"LangGraph suits teams building custom, often cyclical workflows who want full control over where the pause happens and what the reviewer sees. You build the review dashboard, the notification system and the resume logic yourself.",[61,229,231],{"id":230},"velt-routing-ai-proposals-through-an-approval-interface-inside-the-app","Velt: routing AI proposals through an approval interface inside the app",[11,233,234],{},"Velt is an SDK for embedding review and approval into a web product's own UI, so the reviewer stays in the app instead of moving to Slack or email. It targets apps where an agent proposes a change and a person signs off on it before the change reaches production.",[15,236],{":height":70,":width":71,"alt":237,"loading":20,"src":238},"Velt's approval flows page, headlined \"Add an approval workflow builder to your product\", with a workflow builder panel showing a Q3 forecast approval chain of three stages - an FP&A Lead marked mandatory and passed, a committee stage requiring a two-of-three quorum and still pending, and a mandatory CFO stage waiting - plus an option to add a further step","/blog/human-approval-checkpoints-for-ai-agents/velt.png",[11,240,241],{},"An agent's suggestion - a document change, a report figure, a database row - sits in a pending state, separate from live app data, until someone reviews it. The pieces developers wire up:",[78,243,244,250,256,262],{},[81,245,246,249],{},[33,247,248],{},"Approval flows"," with routing rules: a single sign-off for one type of content, a multi-step chain for another, based on role or document type.",[81,251,252,255],{},[33,253,254],{},"Contextual suggestions"," shown as inline diffs or comment threads attached to the exact UI element they affect.",[81,257,258,261],{},[33,259,260],{},"Immutable audit logs"," recording every proposal, edit, and approval with a timestamp and the approver's identity.",[81,263,264,267],{},[33,265,266],{},"Memory ingestion",", where the system indexes past approvals so future agent proposals line up with what reviewers have accepted before.",[11,269,270],{},"A financial analyst reviewing an AI-generated report sees the agent's proposed numbers highlighted in the dashboard, the analyst opens a side panel to see the agent's reasoning, edits a figure directly in the suggestion view, and clicks approve. Only then does Velt fire a webhook to the backend, which writes the change to the production database. If the change needs sign-off from more than one person, Velt routes it to the next approver in the chain automatically, and the underlying record stays untouched until every required approval comes in.",[11,272,273],{},"Velt also ships a review agent, where a model checks AI-generated content against a set of stored rules - a brand guideline, a compliance policy - and drops a comment flagging an issue before a person even opens the document. It's a first-pass check rather than a final decision, and it cuts what the human reviewer has to catch manually.",[11,275,276],{},"Velt bills on activity rather than seats, so a large but mostly passive user base does not drive the cost. Their materials describe the billing unit in two ways, as monthly active collaborators in one place and as monthly active documents - \"a unique document that has CRUD operations performed on it by any Velt feature during the month\" - in another. Velt states that its products are \"SOC 2 Type II and HIPAA compliant\", with multi-region hosting and a self-hosting option for sensitive data. It fits fintech dashboards, creative review tools, and any app where a person needs to see and edit an AI proposal in the interface they already use. Like Webfuse, it works at the app layer, so the agent logic still lives somewhere else.",[61,278,280],{"id":279},"what-these-four-have-in-common","What these four have in common",[11,282,283],{},"All four separate the proposal from the live data it would change, and all four keep a record of who saw what and when. The reviewer gets something concrete to act on: a page, a design document, a state payload, a diff. The record carries as much weight as the pause. A wire transfer approved with no trace of who clicked confirm, and on what basis, is still an audit gap.",[61,285,287],{"id":286},"which-one-fits-your-setup","Which one fits your setup",[289,290,291,307],"table",{},[292,293,294],"thead",{},[295,296,297,301,304],"tr",{},[298,299,300],"th",{},"Tool",[298,302,303],{},"Best for",[298,305,306],{},"Where it pauses",[308,309,310,323,335,347],"tbody",{},[295,311,312,317,320],{},[313,314,315],"td",{},[33,316,40],{},[313,318,319],{},"Legacy web apps with no API, customer-facing sessions",[313,321,322],{},"Inside a live browser session",[295,324,325,329,332],{},[313,326,327],{},[33,328,46],{},[313,330,331],{},"AI coding agents working in large codebases",[313,333,334],{},"Before code gets written",[295,336,337,341,344],{},[313,338,339],{},[33,340,52],{},[313,342,343],{},"Custom, often cyclical agent workflows needing full control",[313,345,346],{},"At a developer-defined graph node",[295,348,349,353,356],{},[313,350,351],{},[33,352,58],{},[313,354,355],{},"In-app review of AI-generated content, with a UI already built",[313,357,358],{},"Before a webhook writes to your database",[11,360,361],{},"The four are not mutually exclusive. A team can run LangGraph as the orchestration layer and use Webfuse for the steps that happen in a browser, or pair HumanLayer's design review with Velt's in-app sign-off on the shipped feature. The build effort differs more than the outcome: Velt and Webfuse ship a reviewer experience, LangGraph ships primitives, and HumanLayer ships a workflow you follow.",{"title":363,"searchDepth":364,"depth":364,"links":365},"",2,[366,367,368,369,370,371],{"id":63,"depth":364,"text":64},{"id":121,"depth":364,"text":122},{"id":166,"depth":364,"text":167},{"id":230,"depth":364,"text":231},{"id":279,"depth":364,"text":280},{"id":286,"depth":364,"text":287},"ai-agents","2026-08-14","Four platforms that pause an AI agent before a high-stakes action and hand the decision to a person. Compare Webfuse, HumanLayer, LangGraph and Velt on where the pause happens, what the reviewer sees, and what each one leaves you to build.","md",[377,380,383,386],{"question":378,"answer":379},"What is a human-in-the-loop checkpoint for an AI agent?","It is a deliberate pause before an agent performs an action that is hard to reverse, such as moving money, changing a live configuration or editing a customer record. The agent's proposed action is held separately from the production data until a person reviews it and approves, edits or rejects it.",{"question":381,"answer":382},"Where should the approval step sit?","At whichever layer the high-stakes action actually happens. Webfuse pauses inside a live browser session, Velt inside the application's own interface, HumanLayer before any code is written, and LangGraph at a node the developer marks in the workflow graph.",{"question":384,"answer":385},"Which option requires the least engineering work?","Velt and Webfuse ship a reviewer experience. LangGraph supplies the pause and resume primitives but leaves the review dashboard, notifications and resume logic to you, so it takes the most build effort in exchange for the most control.",{"question":387,"answer":388},"Do these tools remove the need for human review?","No. They change where and how the review happens, and they make the record of it auditable. A person still makes the decision on the actions that matter.",0,null,false,{"shortTitle":393,"relatedLinks":394},"AI Agent Approval Checkpoints",[395,399],{"text":396,"href":397,"description":398},"5 Best Tools to Let AI Agents Guide Users Through a Website (2026)","/blog/5-best-tools-to-let-ai-agents-guide-users-through-a-website-2026","A comparison of the tools that let an agent act on a live page alongside the person using it.",{"text":400,"href":401,"description":402},"How to Connect an AI Agent to the Live Web","/blog/how-to-connect-an-ai-agent-to-the-live-web","The five architectures for giving an agent an execution layer on the web, and how they differ.",true,"/blog/human-approval-checkpoints-for-ai-agents",{"title":5,"description":374},{"loc":404},"blog/1051.human-approval-checkpoints-for-ai-agents",[372,409,410,411,412],"human-in-the-loop","enterprise-ai","web-agents","governance","J5y9YguOHhFLYkGVLCdBms93PA0OP12RVSFcNifSgn0",[415,2091],{"id":416,"title":417,"authorId":418,"body":419,"category":372,"created":2054,"description":2055,"extension":375,"faqs":2056,"featurePriority":390,"head":390,"hideCta":391,"landingPath":390,"meta":2069,"navigation":403,"ogImage":390,"path":2081,"robots":390,"schemaOrg":390,"seo":2082,"sitemap":2083,"stem":2084,"tags":2085,"__hash__":2090},"blog/blog/1012.dom-downsampling-for-llm-based-web-agents.md","DOM Downsampling for LLM-Based Web Agents","thassilo-schiepanski",{"type":8,"value":420,"toc":2039},[421,426,451,455,462,467,483,487,493,497,515,541,544,548,551,562,568,599,603,623,635,640,656,670,673,677,697,701,709,721,725,728,1121,1127,1134,1298,1305,1396,1403,1475,1484,1490,1499,1503,1509,1519,1531,1765,1782,1804,1810,1853,1857,1869,1878,1883,1888,1891,1895,1901,1906,1944,1948,1954,1958,1968,1972,1975,2035],[15,422],{":width":18,"alt":423,"format":424,"loading":20,"src":425},"Downsampling visualised for digital images and HTML","webp","/blog/dom-downsampling-for-web-agents/1.png",[11,427,428,435,436,435,441,446,447,450],{},[429,430,434],"a",{"href":431,"rel":432},"https://operator.chatgpt.com",[433],"nofollow","Operator (OpenAI)",", ",[429,437,440],{"href":438,"rel":439},"https://www.director.ai",[433],"Director (Browserbase)",[429,442,445],{"href":443,"rel":444},"https://browser-use.com",[433],"Browser Use"," – we are currently witnessing the rise of ",[33,448,449],{},"web AI agents",". The first iteration of serviceable web agents was enabled by frontier LLMs, which act as instantaneous domain model backends. The domain, hereby, corresponds to the landscape of web application UIs.",[61,452,454],{"id":453},"what-is-a-snapshot","What is a Snapshot?",[11,456,457,458,461],{},"Web agents provide an LLM with a task, and serialised runtime state of a currently browsed web application (e.g., a screenshot). The LLM is ought to suggest relevant actions to perform in the web application. Serialisation of such runtime state is referred to as a ",[33,459,460],{},"snapshot",". And the snapshot technique primarily decides the quality of LLM interaction suggestions.",[463,464,466],"h3",{"id":465},"gui-snapshots","GUI Snapshots",[11,468,469,470,473,474,478,479,482],{},"Screenshots – for consistency reasons referred to as ",[33,471,472],{},"GUI snapshots"," – resemble how humans visually perceive web application UIs. LLM APIs subsidise the use of image input through upstream compression. Compresssion, however, irreversibly affects image dimensions, which takes away pixel precision; no way to suggest interactions like ",[475,476,477],"em",{},"“click at 100, 735”",". As a workaround, early web agents used ",[475,480,481],{},"grounded"," GUI snapshots. Grounding describes adding visual cues to the GUI, such as bounding boxes with numerical identifiers. Grounding lets the LLM refer to specific parts of the page by identifier, so the agent can trace back interaction targets.",[15,484],{":width":18,"alt":485,"format":424,"loading":20,"src":486},"Grounded GUI snapshot as implemented by Browser Use","/blog/dom-downsampling-for-web-agents/2.png",[11,488,489],{},[490,491,492],"small",{},"Grounded GUI snapshot as implemented by Browser Use.",[463,494,496],{"id":495},"dom-snapshots","DOM Snapshots",[11,498,499,500,510,511,514],{},"LLMs arguably are much better at understanding code than images. Research supports they excel at describing and classifying HTML, and also navigating an inherent UI",[501,502,503],"sup",{},[429,504,509],{"href":505,"ariaDescribedBy":506,"dataFootnoteRef":363,"id":508},"#user-content-fn-1",[507],"footnote-label","user-content-fnref-1","1",". The DOM (document object model) – a web browser's runtime state model of a web application – translates back to HTML. For this reason, ",[33,512,513],{},"DOM snapshots"," offer a compelling alternative to GUI snapshots. DOM snapshots offer a handful of key advantages:",[516,517,518,521,524,527,530],"ol",{},[81,519,520],{},"DOM snapshots connect with LLM code (HTML) interpretation abilities.",[81,522,523],{},"DOM snapshots can be compiled from deep clones, hidden from supervision (unlike GUI grounding).",[81,525,526],{},"DOM snapshots render text input that on average consume less bandwidth than screnshots.",[81,528,529],{},"DOM snapshots allow for exact programmatic targeting of elements (e.g., via CSS selectors).",[81,531,532,533,536,537,540],{},"DOM snapshots are available with the ",[196,534,535],{},"DOMContentLoaded"," event (whereas the GUI completes initial rendering with ",[196,538,539],{},"load",").",[11,542,543],{},"Yet, DOM snapshots have a major problem: potentially exhaustive model context. Whereas GUI snapshot commonly cost four figures of tokens, a raw DOM snapshot can cost into hundreds of thousands of tokens. To connect with LLM code interpretation abilities, however, developers have used element extraction techniques – picking only (likely) important elements from the DOM. Element extraction flattens the DOM tree, which disregards hierarchy as a potential UI feature (how do elements relate to each other?).",[61,545,547],{"id":546},"dom-downsampling-a-novel-approach","DOM Downsampling: A Novel Approach",[11,549,550],{},"To enable DOM snapshots for use with web agents, it requires client-side pre-processing – similar to how LLM vision APIs process image input. Downsampling is a fundamental signal processing technique that reduces data that scales out of time or space constraints under the assumption that the majority of relevant features is retained. Picture JPEG compression as an example: put simply, a JPEG image stores only an average colour for patches of pixels. The bigger the patches, the smaller the file. Although some detail is lost, key image features – colours, edges, objects – keep being recognisable – up to a large patch size.",[11,552,553,554,557,558,561],{},"We transfer the concept of ",[33,555,556],{},"downsampling"," to ",[33,559,560],{},"DOMs",". Particularly, since such an approach retains HTML characteristics that might be valuable for an LLM backend. We define UI features as concepts that, to a substantial degree, facilitate LLM suggestions on how to act in the UI in order to solve related web-based tasks.",[61,563,565],{"id":564},"d2snap",[475,566,567],{},"D2Snap",[11,569,570,571,579,587,595,596,598],{},"We recently proposed ",[429,572,575],{"href":573,"rel":574},"https://arxiv.org/abs/2508.04412",[433],[33,576,577],{},[475,578,567],{},[501,580,581],{},[429,582,586],{"href":583,"ariaDescribedBy":584,"dataFootnoteRef":363,"id":585},"#user-content-fn-2",[507],"user-content-fnref-2","2",[501,588,589],{},[429,590,594],{"href":591,"ariaDescribedBy":592,"dataFootnoteRef":363,"id":593},"#user-content-fn-3",[507],"user-content-fnref-3","3"," – a first-of-its-kind downsampling algorithm for DOMs. Herein, we'll briefly explain how the ",[475,597,567],{}," algorithm works, and how it can be utilised to build efficient and performant web agents.",[463,600,602],{"id":601},"how-it-works","How it works",[11,604,605,606,608,609,435,612,615,616,619,620,540],{},"There are basically three redundant types of DOM nodes, and HTML concepts: elements, text, and attributes. We defined and empirically adjusted three node-specific procedures. ",[475,607,567],{}," downsamples at a variable ratio, configured through procedure-specific parameters  ",[196,610,611],{},"k",[196,613,614],{},"l",", and ",[196,617,618],{},"m"," (",[196,621,622],{},"∈ [0, 1]",[624,625,626],"blockquote",{},[11,627,628,629,634],{},"We used ",[429,630,633],{"href":631,"rel":632},"https://openai.com/index/hello-gpt-4o/",[433],"GPT-4o"," to create a downsampling ground truth dataset by having it classify HTML elements and scoring semantics regarding relevance for understanding the inherent UI – a UI feature degree.",[636,637,639],"h4",{"id":638},"procedure-elements","Procedure: Elements",[11,641,642,644,645,648,649,652,653,655],{},[475,643,567],{}," downsamples (simplifies) elements by merging container elements like ",[196,646,647],{},"section"," and ",[196,650,651],{},"div"," together. A parameter ",[196,654,611],{}," controls the merge ratio depending on the total DOM tree height. For competing concepts, such as element name, the ground truth determines which element's characterisitics to keep – comparing UI feature scores.",[11,657,658,659,435,661,663,664,669],{},"Elements in content elements (",[196,660,11],{},[196,662,624],{},", ...) are translated to a more comprehensive ",[429,665,668],{"href":666,"rel":667},"https://www.markdownguide.org/basic-syntax/",[433],"Markdown"," representation.",[11,671,672],{},"Interactive elements, definite interaction target candidates, are kept as is.",[636,674,676],{"id":675},"procedure-text","Procedure: Text",[11,678,679,681,682,685,693,694,696],{},[475,680,567],{}," downsamples text by dropping a fraction. Natural units of text are space-separated words, or punctuation-separated sentences. We reuse the ",[475,683,684],{},"TextRank",[501,686,687],{},[429,688,692],{"href":689,"ariaDescribedBy":690,"dataFootnoteRef":363,"id":691},"#user-content-fn-4",[507],"user-content-fnref-4","4"," algorithm to rank sentences in text nodes. The lowest-ranking fraction of sentences, denoted by parameter ",[196,695,614],{},", is dropped.",[636,698,700],{"id":699},"procedure-attributes","Procedure: Attributes",[11,702,703,705,706,708],{},[475,704,567],{}," downsamples attributes by dropping those with a name that, according to ground truth, holds a UI feature degree below a threshold. Parameter ",[196,707,618],{}," denotes this threshold.",[624,710,711],{},[11,712,713,714,720],{},"Check out the ",[429,715,717,719],{"href":573,"rel":716},[433],[475,718,567],{}," paper"," to learn about the algorithm in-depth.",[463,722,724],{"id":723},"example-of-a-downsampled-dom","Example of a Downsampled DOM",[11,726,727],{},"Consider a partial DOM state, serialised as HTML:",[729,730,734],"pre",{"className":731,"code":732,"language":733,"meta":363,"style":363},"language-html shiki shiki-themes catppuccin-latte night-owl","\u003Csection class=\"container\" tabindex=\"3\" required=\"true\" type=\"example\">\n  \u003Cdiv class=\"mx-auto\" data-topic=\"products\" required=\"false\">\n    \u003Ch1>Our Pizza\u003C/h1>\n    \u003Cdiv>\n      \u003Cdiv class=\"shadow-lg\">\n        \u003Ch2>Margherita\u003C/h2>\n        \u003Cp>\n          A simple classic: mozzarela, tomatoes and basil.\n          An everyday choice!\n        \u003C/p>\n        \u003Cbutton type=\"button\">Add\u003C/button>\n      \u003C/div>\n      \u003Cdiv class=\"shadow-lg\">\n        \u003Ch2>Capricciosa\u003C/h2>\n        \u003Cp>\n          A rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\n          A true favourite!\n          \u003C/p>\n        \u003Cbutton type=\"button\">Add\u003C/button>\n      \u003C/div>\n    \u003C/div>\n  \u003C/div>\n\u003C/section>\n","html",[196,735,736,803,846,869,878,899,918,927,933,939,949,978,988,1007,1025,1034,1040,1046,1056,1083,1092,1102,1112],{"__ignoreMap":363},[737,738,741,745,748,752,755,759,763,765,768,770,772,774,776,779,781,783,786,788,791,793,795,798,800],"span",{"class":739,"line":740},"line",1,[737,742,744],{"class":743},"s9rnR","\u003C",[737,746,647],{"class":747},"sY2RG",[737,749,751],{"class":750},"swkLt"," class",[737,753,754],{"class":743},"=",[737,756,758],{"class":757},"sbuKk","\"",[737,760,762],{"class":761},"sfrMT","container",[737,764,758],{"class":757},[737,766,767],{"class":750}," tabindex",[737,769,754],{"class":743},[737,771,758],{"class":757},[737,773,594],{"class":761},[737,775,758],{"class":757},[737,777,778],{"class":750}," required",[737,780,754],{"class":743},[737,782,758],{"class":757},[737,784,785],{"class":761},"true",[737,787,758],{"class":757},[737,789,790],{"class":750}," type",[737,792,754],{"class":743},[737,794,758],{"class":757},[737,796,797],{"class":761},"example",[737,799,758],{"class":757},[737,801,802],{"class":743},">\n",[737,804,805,808,810,812,814,816,819,821,824,826,828,831,833,835,837,839,842,844],{"class":739,"line":364},[737,806,807],{"class":743},"  \u003C",[737,809,651],{"class":747},[737,811,751],{"class":750},[737,813,754],{"class":743},[737,815,758],{"class":757},[737,817,818],{"class":761},"mx-auto",[737,820,758],{"class":757},[737,822,823],{"class":750}," data-topic",[737,825,754],{"class":743},[737,827,758],{"class":757},[737,829,830],{"class":761},"products",[737,832,758],{"class":757},[737,834,778],{"class":750},[737,836,754],{"class":743},[737,838,758],{"class":757},[737,840,841],{"class":761},"false",[737,843,758],{"class":757},[737,845,802],{"class":743},[737,847,849,852,855,858,862,865,867],{"class":739,"line":848},3,[737,850,851],{"class":743},"    \u003C",[737,853,854],{"class":747},"h1",[737,856,857],{"class":743},">",[737,859,861],{"class":860},"s2kId","Our Pizza",[737,863,864],{"class":743},"\u003C/",[737,866,854],{"class":747},[737,868,802],{"class":743},[737,870,872,874,876],{"class":739,"line":871},4,[737,873,851],{"class":743},[737,875,651],{"class":747},[737,877,802],{"class":743},[737,879,881,884,886,888,890,892,895,897],{"class":739,"line":880},5,[737,882,883],{"class":743},"      \u003C",[737,885,651],{"class":747},[737,887,751],{"class":750},[737,889,754],{"class":743},[737,891,758],{"class":757},[737,893,894],{"class":761},"shadow-lg",[737,896,758],{"class":757},[737,898,802],{"class":743},[737,900,902,905,907,909,912,914,916],{"class":739,"line":901},6,[737,903,904],{"class":743},"        \u003C",[737,906,61],{"class":747},[737,908,857],{"class":743},[737,910,911],{"class":860},"Margherita",[737,913,864],{"class":743},[737,915,61],{"class":747},[737,917,802],{"class":743},[737,919,921,923,925],{"class":739,"line":920},7,[737,922,904],{"class":743},[737,924,11],{"class":747},[737,926,802],{"class":743},[737,928,930],{"class":739,"line":929},8,[737,931,932],{"class":860},"          A simple classic: mozzarela, tomatoes and basil.\n",[737,934,936],{"class":739,"line":935},9,[737,937,938],{"class":860},"          An everyday choice!\n",[737,940,942,945,947],{"class":739,"line":941},10,[737,943,944],{"class":743},"        \u003C/",[737,946,11],{"class":747},[737,948,802],{"class":743},[737,950,952,954,957,959,961,963,965,967,969,972,974,976],{"class":739,"line":951},11,[737,953,904],{"class":743},[737,955,956],{"class":747},"button",[737,958,790],{"class":750},[737,960,754],{"class":743},[737,962,758],{"class":757},[737,964,956],{"class":761},[737,966,758],{"class":757},[737,968,857],{"class":743},[737,970,971],{"class":860},"Add",[737,973,864],{"class":743},[737,975,956],{"class":747},[737,977,802],{"class":743},[737,979,981,984,986],{"class":739,"line":980},12,[737,982,983],{"class":743},"      \u003C/",[737,985,651],{"class":747},[737,987,802],{"class":743},[737,989,991,993,995,997,999,1001,1003,1005],{"class":739,"line":990},13,[737,992,883],{"class":743},[737,994,651],{"class":747},[737,996,751],{"class":750},[737,998,754],{"class":743},[737,1000,758],{"class":757},[737,1002,894],{"class":761},[737,1004,758],{"class":757},[737,1006,802],{"class":743},[737,1008,1010,1012,1014,1016,1019,1021,1023],{"class":739,"line":1009},14,[737,1011,904],{"class":743},[737,1013,61],{"class":747},[737,1015,857],{"class":743},[737,1017,1018],{"class":860},"Capricciosa",[737,1020,864],{"class":743},[737,1022,61],{"class":747},[737,1024,802],{"class":743},[737,1026,1028,1030,1032],{"class":739,"line":1027},15,[737,1029,904],{"class":743},[737,1031,11],{"class":747},[737,1033,802],{"class":743},[737,1035,1037],{"class":739,"line":1036},16,[737,1038,1039],{"class":860},"          A rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\n",[737,1041,1043],{"class":739,"line":1042},17,[737,1044,1045],{"class":860},"          A true favourite!\n",[737,1047,1049,1052,1054],{"class":739,"line":1048},18,[737,1050,1051],{"class":743},"          \u003C/",[737,1053,11],{"class":747},[737,1055,802],{"class":743},[737,1057,1059,1061,1063,1065,1067,1069,1071,1073,1075,1077,1079,1081],{"class":739,"line":1058},19,[737,1060,904],{"class":743},[737,1062,956],{"class":747},[737,1064,790],{"class":750},[737,1066,754],{"class":743},[737,1068,758],{"class":757},[737,1070,956],{"class":761},[737,1072,758],{"class":757},[737,1074,857],{"class":743},[737,1076,971],{"class":860},[737,1078,864],{"class":743},[737,1080,956],{"class":747},[737,1082,802],{"class":743},[737,1084,1086,1088,1090],{"class":739,"line":1085},20,[737,1087,983],{"class":743},[737,1089,651],{"class":747},[737,1091,802],{"class":743},[737,1093,1095,1098,1100],{"class":739,"line":1094},21,[737,1096,1097],{"class":743},"    \u003C/",[737,1099,651],{"class":747},[737,1101,802],{"class":743},[737,1103,1105,1108,1110],{"class":739,"line":1104},22,[737,1106,1107],{"class":743},"  \u003C/",[737,1109,651],{"class":747},[737,1111,802],{"class":743},[737,1113,1115,1117,1119],{"class":739,"line":1114},23,[737,1116,864],{"class":743},[737,1118,647],{"class":747},[737,1120,802],{"class":743},[11,1122,1123,1124,1126],{},"Here are some ",[475,1125,567],{}," downsampling results, which are based on different parametric configurations. A percentage denotes the reduced size.",[636,1128,1130,1133],{"id":1129},"k3-l3-m3-55",[196,1131,1132],{},"k=.3, l=.3, m=.3"," (55%)",[729,1135,1137],{"className":731,"code":1136,"language":733,"meta":363,"style":363},"\u003Csection tabindex=\"3\" type=\"example\" class=\"container\" required=\"true\">\n  # Our Pizza\n  \u003Cdiv class=\"shadow-lg\">\n    ## Margherita\n    A simple classic: mozzarela, tomatoes, and basil.\n    \u003Cbutton type=\"button\">Add\u003C/button>\n    ## Capricciosa\n    A rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\n    \u003Cbutton type=\"button\">Add\u003C/button>\n  \u003C/div>\n\u003C/section>\n",[196,1138,1139,1187,1192,1210,1215,1220,1246,1251,1256,1282,1290],{"__ignoreMap":363},[737,1140,1141,1143,1145,1147,1149,1151,1153,1155,1157,1159,1161,1163,1165,1167,1169,1171,1173,1175,1177,1179,1181,1183,1185],{"class":739,"line":740},[737,1142,744],{"class":743},[737,1144,647],{"class":747},[737,1146,767],{"class":750},[737,1148,754],{"class":743},[737,1150,758],{"class":757},[737,1152,594],{"class":761},[737,1154,758],{"class":757},[737,1156,790],{"class":750},[737,1158,754],{"class":743},[737,1160,758],{"class":757},[737,1162,797],{"class":761},[737,1164,758],{"class":757},[737,1166,751],{"class":750},[737,1168,754],{"class":743},[737,1170,758],{"class":757},[737,1172,762],{"class":761},[737,1174,758],{"class":757},[737,1176,778],{"class":750},[737,1178,754],{"class":743},[737,1180,758],{"class":757},[737,1182,785],{"class":761},[737,1184,758],{"class":757},[737,1186,802],{"class":743},[737,1188,1189],{"class":739,"line":364},[737,1190,1191],{"class":860},"  # Our Pizza\n",[737,1193,1194,1196,1198,1200,1202,1204,1206,1208],{"class":739,"line":848},[737,1195,807],{"class":743},[737,1197,651],{"class":747},[737,1199,751],{"class":750},[737,1201,754],{"class":743},[737,1203,758],{"class":757},[737,1205,894],{"class":761},[737,1207,758],{"class":757},[737,1209,802],{"class":743},[737,1211,1212],{"class":739,"line":871},[737,1213,1214],{"class":860},"    ## Margherita\n",[737,1216,1217],{"class":739,"line":880},[737,1218,1219],{"class":860},"    A simple classic: mozzarela, tomatoes, and basil.\n",[737,1221,1222,1224,1226,1228,1230,1232,1234,1236,1238,1240,1242,1244],{"class":739,"line":901},[737,1223,851],{"class":743},[737,1225,956],{"class":747},[737,1227,790],{"class":750},[737,1229,754],{"class":743},[737,1231,758],{"class":757},[737,1233,956],{"class":761},[737,1235,758],{"class":757},[737,1237,857],{"class":743},[737,1239,971],{"class":860},[737,1241,864],{"class":743},[737,1243,956],{"class":747},[737,1245,802],{"class":743},[737,1247,1248],{"class":739,"line":920},[737,1249,1250],{"class":860},"    ## Capricciosa\n",[737,1252,1253],{"class":739,"line":929},[737,1254,1255],{"class":860},"    A rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\n",[737,1257,1258,1260,1262,1264,1266,1268,1270,1272,1274,1276,1278,1280],{"class":739,"line":935},[737,1259,851],{"class":743},[737,1261,956],{"class":747},[737,1263,790],{"class":750},[737,1265,754],{"class":743},[737,1267,758],{"class":757},[737,1269,956],{"class":761},[737,1271,758],{"class":757},[737,1273,857],{"class":743},[737,1275,971],{"class":860},[737,1277,864],{"class":743},[737,1279,956],{"class":747},[737,1281,802],{"class":743},[737,1283,1284,1286,1288],{"class":739,"line":941},[737,1285,1107],{"class":743},[737,1287,651],{"class":747},[737,1289,802],{"class":743},[737,1291,1292,1294,1296],{"class":739,"line":951},[737,1293,864],{"class":743},[737,1295,647],{"class":747},[737,1297,802],{"class":743},[636,1299,1301,1304],{"id":1300},"k4-l6-m8-27",[196,1302,1303],{},"k=.4, l=.6, m=.8"," (27%)",[729,1306,1308],{"className":731,"code":1307,"language":733,"meta":363,"style":363},"\u003Csection>\n  # Our Pizza\n  \u003Cdiv>\n    ## Margherita\n    A simple classic:\n    \u003Cbutton>Add\u003C/button>\n    ## Capricciosa\n    A rich taste:\n    \u003Cbutton>Add\u003C/button>\n  \u003C/div>\n\u003C/section>\n",[196,1309,1310,1318,1322,1330,1334,1339,1355,1359,1364,1380,1388],{"__ignoreMap":363},[737,1311,1312,1314,1316],{"class":739,"line":740},[737,1313,744],{"class":743},[737,1315,647],{"class":747},[737,1317,802],{"class":743},[737,1319,1320],{"class":739,"line":364},[737,1321,1191],{"class":860},[737,1323,1324,1326,1328],{"class":739,"line":848},[737,1325,807],{"class":743},[737,1327,651],{"class":747},[737,1329,802],{"class":743},[737,1331,1332],{"class":739,"line":871},[737,1333,1214],{"class":860},[737,1335,1336],{"class":739,"line":880},[737,1337,1338],{"class":860},"    A simple classic:\n",[737,1340,1341,1343,1345,1347,1349,1351,1353],{"class":739,"line":901},[737,1342,851],{"class":743},[737,1344,956],{"class":747},[737,1346,857],{"class":743},[737,1348,971],{"class":860},[737,1350,864],{"class":743},[737,1352,956],{"class":747},[737,1354,802],{"class":743},[737,1356,1357],{"class":739,"line":920},[737,1358,1250],{"class":860},[737,1360,1361],{"class":739,"line":929},[737,1362,1363],{"class":860},"    A rich taste:\n",[737,1365,1366,1368,1370,1372,1374,1376,1378],{"class":739,"line":935},[737,1367,851],{"class":743},[737,1369,956],{"class":747},[737,1371,857],{"class":743},[737,1373,971],{"class":860},[737,1375,864],{"class":743},[737,1377,956],{"class":747},[737,1379,802],{"class":743},[737,1381,1382,1384,1386],{"class":739,"line":941},[737,1383,1107],{"class":743},[737,1385,651],{"class":747},[737,1387,802],{"class":743},[737,1389,1390,1392,1394],{"class":739,"line":951},[737,1391,864],{"class":743},[737,1393,647],{"class":747},[737,1395,802],{"class":743},[636,1397,1399,1402],{"id":1398},"k-l0-m-35",[196,1400,1401],{},"k→∞, l=0, ∀m"," (35%)",[729,1404,1406],{"className":731,"code":1405,"language":733,"meta":363,"style":363},"# Our Pizza\n## Margherita\nA simple classic: mozzarela, tomatoes, and basil.\nAn everyday choice!\n\u003Cbutton>Add\u003C/button>\n## Capricciosa\nA rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\nA true favourite!\n\u003Cbutton>Add\u003C/button>\n",[196,1407,1408,1413,1418,1423,1428,1444,1449,1454,1459],{"__ignoreMap":363},[737,1409,1410],{"class":739,"line":740},[737,1411,1412],{"class":860},"# Our Pizza\n",[737,1414,1415],{"class":739,"line":364},[737,1416,1417],{"class":860},"## Margherita\n",[737,1419,1420],{"class":739,"line":848},[737,1421,1422],{"class":860},"A simple classic: mozzarela, tomatoes, and basil.\n",[737,1424,1425],{"class":739,"line":871},[737,1426,1427],{"class":860},"An everyday choice!\n",[737,1429,1430,1432,1434,1436,1438,1440,1442],{"class":739,"line":880},[737,1431,744],{"class":743},[737,1433,956],{"class":747},[737,1435,857],{"class":743},[737,1437,971],{"class":860},[737,1439,864],{"class":743},[737,1441,956],{"class":747},[737,1443,802],{"class":743},[737,1445,1446],{"class":739,"line":901},[737,1447,1448],{"class":860},"## Capricciosa\n",[737,1450,1451],{"class":739,"line":920},[737,1452,1453],{"class":860},"A rich taste: mozzarella, ham, mushrooms, artichokes, and olives.\n",[737,1455,1456],{"class":739,"line":929},[737,1457,1458],{"class":860},"A true favourite!\n",[737,1460,1461,1463,1465,1467,1469,1471,1473],{"class":739,"line":935},[737,1462,744],{"class":743},[737,1464,956],{"class":747},[737,1466,857],{"class":743},[737,1468,971],{"class":860},[737,1470,864],{"class":743},[737,1472,956],{"class":747},[737,1474,802],{"class":743},[11,1476,1477,1478,1480,1481,1483],{},"Asymptotic ",[196,1479,611],{}," (kind of 'infinite' ",[196,1482,611],{},") completely flattens the DOM, that is, leads to a full content linearisation similar to reader views as present in most browsers. Notably, it preserves all interactive elements like buttons – which are essential for a web agent.",[463,1485,1487],{"id":1486},"adaptived2snap",[475,1488,1489],{},"AdaptiveD2Snap",[11,1491,1492,1493,1495,1496,1498],{},"Fixed parameters might not be ideal for arbitrary DOMs – sourced from a landscape of web applications. We created ",[475,1494,1489],{}," – a wrapper for ",[475,1497,567],{}," that infers suitable parameters from a given DOM in order to hit a certain token budget.",[463,1500,1502],{"id":1501},"implementation-integration","Implementation & Integration",[11,1504,1505,1506,1508],{},"Picture an LLM-based weg agent that is premised on DOM snapshots. Implementing ",[475,1507,567],{}," is simple: Deep clone the DOM, and feed it to the algorithm. Now, take the snapshot; this is, serialise the resulting DOM. Done.",[624,1510,1511],{},[11,1512,1513,1514,1518],{},"Read our ",[429,1515,1517],{"href":1516},"/blog/a-gentle-introduction-to-ai-agents-for-the-web","gentle introduction to AI agents for the web"," to get started with high-level web agent concepts.",[11,1520,1521,1522,1524,1525,1530],{},"The open source ",[475,1523,567],{}," API, provided as a ",[429,1526,1529],{"href":1527,"rel":1528},"https://github.com/webfuse-com/D2Snap",[433],"package on GitHub"," provides the following signature:",[729,1532,1536],{"className":1533,"code":1534,"language":1535,"meta":363,"style":363},"language-ts shiki shiki-themes catppuccin-latte night-owl","type DOM = Document | Element | string;\ntype Options = {\n  assignUniqueIDs?: boolean; // false\n  debug?: boolean;           // true\n};\n\nD2Snap.d2Snap(\n  dom: DOM,\n  k: number, l: number, m: number,\n  options?: Options\n): Promise\u003Cstring>\n\nD2Snap.adaptiveD2Snap(\n  dom: DOM,\n  maxTokens: number = 4096,\n  maxIterations: number = 5,\n  options?: Options\n): Promise\u003Cstring>\n\n","ts",[196,1537,1538,1571,1583,1602,1616,1621,1626,1641,1653,1671,1681,1697,1701,1712,1720,1733,1745,1753],{"__ignoreMap":363},[737,1539,1540,1544,1548,1551,1555,1558,1561,1563,1567],{"class":739,"line":740},[737,1541,1543],{"class":1542},"s76yb","type",[737,1545,1547],{"class":1546},"sXbZB"," DOM ",[737,1549,754],{"class":1550},"s-_ek",[737,1552,1554],{"class":1553},"s-DR7"," Document",[737,1556,1557],{"class":743}," |",[737,1559,1560],{"class":1553}," Element",[737,1562,1557],{"class":743},[737,1564,1566],{"class":1565},"scrte"," string",[737,1568,1570],{"class":1569},"scGhl",";\n",[737,1572,1573,1575,1578,1580],{"class":739,"line":364},[737,1574,1543],{"class":1542},[737,1576,1577],{"class":1546}," Options ",[737,1579,754],{"class":1550},[737,1581,1582],{"class":1569}," {\n",[737,1584,1585,1589,1592,1595,1598],{"class":739,"line":848},[737,1586,1588],{"class":1587},"swl0y","  assignUniqueIDs",[737,1590,1591],{"class":743},"?:",[737,1593,1594],{"class":1565}," boolean",[737,1596,1597],{"class":1569},";",[737,1599,1601],{"class":1600},"sDmS1"," // false\n",[737,1603,1604,1607,1609,1611,1613],{"class":739,"line":871},[737,1605,1606],{"class":1587},"  debug",[737,1608,1591],{"class":743},[737,1610,1594],{"class":1565},[737,1612,1597],{"class":1569},[737,1614,1615],{"class":1600},"           // true\n",[737,1617,1618],{"class":739,"line":880},[737,1619,1620],{"class":1569},"};\n",[737,1622,1623],{"class":739,"line":901},[737,1624,1625],{"emptyLinePlaceholder":403},"\n",[737,1627,1628,1630,1634,1638],{"class":739,"line":920},[737,1629,567],{"class":860},[737,1631,1633],{"class":1632},"s5FwJ",".",[737,1635,1637],{"class":1636},"sNstc","d2Snap",[737,1639,1640],{"class":860},"(\n",[737,1642,1643,1646,1650],{"class":739,"line":929},[737,1644,1645],{"class":860},"  dom: ",[737,1647,1649],{"class":1648},"sqxXB","DOM",[737,1651,1652],{"class":1569},",\n",[737,1654,1655,1658,1661,1664,1666,1669],{"class":739,"line":935},[737,1656,1657],{"class":860},"  k: number",[737,1659,1660],{"class":1569},",",[737,1662,1663],{"class":860}," l: number",[737,1665,1660],{"class":1569},[737,1667,1668],{"class":860}," m: number",[737,1670,1652],{"class":1569},[737,1672,1673,1676,1678],{"class":739,"line":941},[737,1674,1675],{"class":860},"  options",[737,1677,1591],{"class":1550},[737,1679,1680],{"class":860}," Options\n",[737,1682,1683,1686,1690,1692,1695],{"class":739,"line":951},[737,1684,1685],{"class":860},"): ",[737,1687,1689],{"class":1688},"s8Irk","Promise",[737,1691,744],{"class":1550},[737,1693,1694],{"class":860},"string",[737,1696,802],{"class":1550},[737,1698,1699],{"class":739,"line":980},[737,1700,1625],{"emptyLinePlaceholder":403},[737,1702,1703,1705,1707,1710],{"class":739,"line":990},[737,1704,567],{"class":860},[737,1706,1633],{"class":1632},[737,1708,1709],{"class":1636},"adaptiveD2Snap",[737,1711,1640],{"class":860},[737,1713,1714,1716,1718],{"class":739,"line":1009},[737,1715,1645],{"class":860},[737,1717,1649],{"class":1648},[737,1719,1652],{"class":1569},[737,1721,1722,1725,1727,1731],{"class":739,"line":1027},[737,1723,1724],{"class":860},"  maxTokens: number ",[737,1726,754],{"class":1550},[737,1728,1730],{"class":1729},"sZ_Zo"," 4096",[737,1732,1652],{"class":1569},[737,1734,1735,1738,1740,1743],{"class":739,"line":1036},[737,1736,1737],{"class":860},"  maxIterations: number ",[737,1739,754],{"class":1550},[737,1741,1742],{"class":1729}," 5",[737,1744,1652],{"class":1569},[737,1746,1747,1749,1751],{"class":739,"line":1042},[737,1748,1675],{"class":860},[737,1750,1591],{"class":1550},[737,1752,1680],{"class":860},[737,1754,1755,1757,1759,1761,1763],{"class":739,"line":1048},[737,1756,1685],{"class":860},[737,1758,1689],{"class":1688},[737,1760,744],{"class":1550},[737,1762,1694],{"class":860},[737,1764,802],{"class":1550},[11,1766,1767,1768,1770,1771,1776,1777,1781],{},"Moreover, ",[475,1769,567],{}," it is available on the ",[429,1772,1775],{"href":1773,"rel":1774},"https://dev.webfuse.com/automation-api",[433],"Webfuse Automation API",". ",[429,1778,40],{"href":1779,"rel":1780},"https://www.webfuse.com",[433]," essentially is a proxy to seamlessly serve any existing web application with custom augmentations, such as a web agent widget.",[729,1783,1787],{"className":1784,"code":1785,"language":1786,"meta":363,"style":363},"language-js shiki shiki-themes catppuccin-latte night-owl","const domSnapshot = await browser.webfuseSession\n    .automation\n    .take_dom_snapshot({ modifier: 'downsample' })\n","js",[196,1788,1789,1794,1799],{"__ignoreMap":363},[737,1790,1791],{"class":739,"line":740},[737,1792,1793],{},"const domSnapshot = await browser.webfuseSession\n",[737,1795,1796],{"class":739,"line":364},[737,1797,1798],{},"    .automation\n",[737,1800,1801],{"class":739,"line":848},[737,1802,1803],{},"    .take_dom_snapshot({ modifier: 'downsample' })\n",[11,1805,1806,1807,1809],{},"Need precise control over the underlying ",[475,1808,567],{}," invocation? Configure it exactly how you want:",[729,1811,1813],{"className":1784,"code":1812,"language":1786,"meta":363,"style":363},"const domSnapshot = await browser.webfuseSession\n    .automation\n    .take_dom_snapshot({\n        modifier: {\n            name: 'D2Snap',\n            params: { hierarchyRatio: 0.6, textRatio: 0.2, attributeRatio: 0.8 }\n        }\n    })\n",[196,1814,1815,1819,1823,1828,1833,1838,1843,1848],{"__ignoreMap":363},[737,1816,1817],{"class":739,"line":740},[737,1818,1793],{},[737,1820,1821],{"class":739,"line":364},[737,1822,1798],{},[737,1824,1825],{"class":739,"line":848},[737,1826,1827],{},"    .take_dom_snapshot({\n",[737,1829,1830],{"class":739,"line":871},[737,1831,1832],{},"        modifier: {\n",[737,1834,1835],{"class":739,"line":880},[737,1836,1837],{},"            name: 'D2Snap',\n",[737,1839,1840],{"class":739,"line":901},[737,1841,1842],{},"            params: { hierarchyRatio: 0.6, textRatio: 0.2, attributeRatio: 0.8 }\n",[737,1844,1845],{"class":739,"line":920},[737,1846,1847],{},"        }\n",[737,1849,1850],{"class":739,"line":929},[737,1851,1852],{},"    })\n",[463,1854,1856],{"id":1855},"performance-evaluation","Performance Evaluation",[11,1858,1859,1860,1862,1863,1865,1866,1868],{},"Now for the moment of truth: How does ",[475,1861,567],{}," stack up against the industry standard? We evaluated ",[475,1864,567],{}," in comparison to a grounded GUI snapshot baseline close to those used by ",[475,1867,445],{}," – coloured bounding boxes around visible interactive elements.",[11,1870,1871,1872,1877],{},"To evaluate snapshots isolated from specific agent logic, we crafted a dataset that spans all UI states that occur while solving a related task. We sampled our dataset from the existing ",[429,1873,1876],{"href":1874,"rel":1875},"https://github.com/OSU-NLP-Group/Online-Mind2Web",[433],"Online-Mind2Web"," dataset.",[15,1879],{":width":1880,"alt":1881,"format":424,"loading":20,"src":1882},"800","Exemplary solution UI state trajectory of a defined web-based task","/blog/dom-downsampling-for-web-agents/3.png",[11,1884,1885],{},[490,1886,1887],{},"Exemplary solution UI state trajectory for the task: “View the pricing plan for 'Business'. Specifically, we have 100 users. We need a 1PB storage quota and a 50 TB transfer quota.”",[11,1889,1890],{},"These are our key findings...",[636,1892,1894],{"id":1893},"substantial-success-rates","Substantial Success Rates",[11,1896,1897,1898,1900],{},"The results exceeded our expectations. Not only did ",[475,1899,567],{}," meet the baseline's performance – our best configuration outperformed it by a significant margin. Full linearisation matches performance, and estimated model input token size order of the baseline.",[15,1902],{":width":1903,"alt":1904,"format":424,"loading":20,"src":1905},"550","Success rate per web agent snapshot subject evaluated across the dataset","/blog/dom-downsampling-for-web-agents/4.png",[490,1907,1908,1909,1916,1917,1919,1920,1923,1924,1927,1928,1931,1932,1935,1936,1939,1940,1943],{},"\n  Success rate per web agent snapshot subject evaluated across the dataset.\n  Labels: ",[196,1910,1911,1912],{},"GUI",[1913,1914,1915],"sub",{}," gr.",": Baseline, ",[196,1918,1649],{},": Raw DOM (cut-off at ~8K tokens), ",[196,1921,1922],{},"k( l m)",": Parameter values; e.g., ",[196,1925,1926],{},".9 .3 .6",", or ",[196,1929,1930],{},".4"," if equal). ",[196,1933,1934],{},"∞",": Linearisation,  ",[196,1937,1938],{},"8192 / 32768",": via token-limited (resp.) ",[1941,1942,1489],"i",{},".\n",[636,1945,1947],{"id":1946},"containable-token-and-byte-size","Containable Token and Byte Size",[11,1949,1950,1951,1953],{},"Even light downsampling delivers dramatic size reductions. Most ",[475,1952,567],{}," configurations average just one token order above the baseline – a massive improvement over raw DOM snapshots. Better yet, most DOMs from the dataset could actually be downsampled to the baseline order. And while image data balloons in file size, our text-based approach stays lean and efficient.",[15,1955],{":width":1880,"alt":1956,"format":424,"loading":20,"src":1957},"Comparison of mean input size across and per subject","/blog/dom-downsampling-for-web-agents/5.png",[490,1959,1960,1961,1964,1965,1967],{},"\n  Left: Comparison of mean input size (tokens vs bytes) across and per subject.",[1962,1963],"br",{},"\n  Right: Estimated input token size across the dataset created by a single ",[1941,1966,567],{}," evaluation subject.\n",[636,1969,1971],{"id":1970},"hierarchy-actually-matters","Hierarchy Actually Matters",[11,1973,1974],{},"Which UI feature matters most for LLM web agent backend performance? We alternated parameter configurations to find out. Interestingly, hierarchy reveals itself as the strongest of the three assessed features. Element extraction throws away hierarchy, which suggests that downsampling is a superior technique.",[647,1976,1979,1984],{"className":1977,"dataFootnotes":363},[1978],"footnotes",[61,1980,1983],{"className":1981,"id":507},[1982],"sr-only","Footnotes",[516,1985,1986,2001,2012,2023],{},[81,1987,1989,1993,1994],{"id":1988},"user-content-fn-1",[429,1990,1991],{"href":1991,"rel":1992},"https://arxiv.org/abs/2210.03945",[433]," ",[429,1995,2000],{"href":1996,"ariaLabel":1997,"className":1998,"dataFootnoteBackref":363},"#user-content-fnref-1","Back to reference 1",[1999],"data-footnote-backref","↩",[81,2002,2004,1993,2007],{"id":2003},"user-content-fn-2",[429,2005,573],{"href":573,"rel":2006},[433],[429,2008,2000],{"href":2009,"ariaLabel":2010,"className":2011,"dataFootnoteBackref":363},"#user-content-fnref-2","Back to reference 2",[1999],[81,2013,2015,1993,2018],{"id":2014},"user-content-fn-3",[429,2016,1527],{"href":1527,"rel":2017},[433],[429,2019,2000],{"href":2020,"ariaLabel":2021,"className":2022,"dataFootnoteBackref":363},"#user-content-fnref-3","Back to reference 3",[1999],[81,2024,2026,1993,2030],{"id":2025},"user-content-fn-4",[429,2027,2028],{"href":2028,"rel":2029},"https://aclanthology.org/W04-3252",[433],[429,2031,2000],{"href":2032,"ariaLabel":2033,"className":2034,"dataFootnoteBackref":363},"#user-content-fnref-4","Back to reference 4",[1999],[2036,2037,2038],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s9rnR, html code.shiki .s9rnR{--shiki-default:#179299;--shiki-dark:#7FDBCA}html pre.shiki code .sY2RG, html code.shiki .sY2RG{--shiki-default:#1E66F5;--shiki-dark:#CAECE6}html pre.shiki code .swkLt, html code.shiki .swkLt{--shiki-default:#DF8E1D;--shiki-default-font-style:inherit;--shiki-dark:#C5E478;--shiki-dark-font-style:italic}html pre.shiki code .sbuKk, html code.shiki .sbuKk{--shiki-default:#40A02B;--shiki-dark:#D9F5DD}html pre.shiki code .sfrMT, html code.shiki .sfrMT{--shiki-default:#40A02B;--shiki-dark:#ECC48D}html pre.shiki code .s2kId, html code.shiki .s2kId{--shiki-default:#4C4F69;--shiki-dark:#D6DEEB}html pre.shiki code .s76yb, html code.shiki .s76yb{--shiki-default:#8839EF;--shiki-dark:#C792EA}html pre.shiki code .sXbZB, html code.shiki .sXbZB{--shiki-default:#DF8E1D;--shiki-default-font-style:italic;--shiki-dark:#D6DEEB;--shiki-dark-font-style:inherit}html pre.shiki code .s-_ek, html code.shiki .s-_ek{--shiki-default:#179299;--shiki-dark:#C792EA}html pre.shiki code .s-DR7, html code.shiki .s-DR7{--shiki-default:#DF8E1D;--shiki-default-font-style:italic;--shiki-dark:#FFCB8B;--shiki-dark-font-style:inherit}html pre.shiki code .scrte, html code.shiki .scrte{--shiki-default:#8839EF;--shiki-dark:#C5E478}html pre.shiki code .scGhl, html code.shiki .scGhl{--shiki-default:#7C7F93;--shiki-dark:#D6DEEB}html pre.shiki code .swl0y, html code.shiki .swl0y{--shiki-default:#4C4F69;--shiki-default-font-style:italic;--shiki-dark:#D6DEEB;--shiki-dark-font-style:inherit}html pre.shiki code .sDmS1, html code.shiki .sDmS1{--shiki-default:#7C7F93;--shiki-default-font-style:italic;--shiki-dark:#637777;--shiki-dark-font-style:italic}html pre.shiki code .s5FwJ, html code.shiki .s5FwJ{--shiki-default:#179299;--shiki-default-font-style:inherit;--shiki-dark:#C792EA;--shiki-dark-font-style:italic}html pre.shiki code .sNstc, html code.shiki .sNstc{--shiki-default:#1E66F5;--shiki-default-font-style:italic;--shiki-dark:#82AAFF;--shiki-dark-font-style:italic}html pre.shiki code .sqxXB, html code.shiki .sqxXB{--shiki-default:#4C4F69;--shiki-dark:#82AAFF}html pre.shiki code .s8Irk, html code.shiki .s8Irk{--shiki-default:#DF8E1D;--shiki-default-font-style:italic;--shiki-dark:#C5E478;--shiki-dark-font-style:inherit}html pre.shiki code .sZ_Zo, html code.shiki .sZ_Zo{--shiki-default:#FE640B;--shiki-dark:#F78C6C}",{"title":363,"searchDepth":364,"depth":364,"links":2040},[2041,2045,2046,2053],{"id":453,"depth":364,"text":454,"children":2042},[2043,2044],{"id":465,"depth":848,"text":466},{"id":495,"depth":848,"text":496},{"id":546,"depth":364,"text":547},{"id":564,"depth":364,"text":567,"children":2047},[2048,2049,2050,2051,2052],{"id":601,"depth":848,"text":602},{"id":723,"depth":848,"text":724},{"id":1486,"depth":848,"text":1489},{"id":1501,"depth":848,"text":1502},{"id":1855,"depth":848,"text":1856},{"id":507,"depth":364,"text":1983},"2025-08-18","D2Snap is a first-of-its-kind DOM downsampling algorithm. Pre-process DOM snapshots to cut web agent token costs without losing page state.",[2057,2060,2063,2066],{"question":2058,"answer":2059},"What is D2Snap?","D2Snap is a first-of-its-kind downsampling algorithm for DOMs, used as a pre-processing technique for DOM snapshots to optimise context quality and token costs for web agents. It reduces DOM data at a variable ratio while retaining HTML characteristics valuable to an LLM backend. It is available as an open source package on GitHub and via the Webfuse Automation API.",{"question":2061,"answer":2062},"Why are DOM snapshots better than screenshots for web agents?","DOM snapshots connect with the LLM's code interpretation abilities, since LLMs are better at understanding HTML than images. They can be compiled from deep clones hidden from supervision, allow exact programmatic targeting of elements via CSS selectors, and are available with the DOMContentLoaded event. Their main drawback is that a raw DOM snapshot can cost into the hundreds of thousands of tokens.",{"question":2064,"answer":2065},"How does D2Snap reduce DOM size?","D2Snap applies three node-specific procedures to elements, text, and attributes, each controlled by a parameter between 0 and 1. It merges container elements like section and div, drops the lowest-ranking sentences using the TextRank algorithm, and removes attributes whose UI feature degree falls below a threshold. Interactive elements such as buttons are always kept as interaction target candidates.",{"question":2067,"answer":2068},"Which DOM feature matters most for web agent performance?","Hierarchy is the strongest of the three assessed features for LLM web agent backend performance. Because element extraction techniques throw away hierarchy by flattening the DOM tree, the findings suggest downsampling is a superior technique. In evaluation, the best D2Snap configuration outperformed a grounded GUI snapshot baseline by a significant margin.",{"homepage":403,"relatedLinks":2070},[2071,2075,2078],{"text":2072,"href":2073,"description":2074},"What is a Website Snapshot?","/blog/snapshots-provide-llms-with-website-state","Learn what a website snapshot is and how to utilise it for web agents",{"text":2076,"href":1516,"description":2077},"What is a Web Agent?","Learn the basics of web agents",{"text":1775,"href":2079,"external":403,"description":2080},"https://dev.webfuse.com/automation-api#take_dom_snapshot","Check out the Webfuse Automation API","/blog/dom-downsampling-for-llm-based-web-agents",{"title":417,"description":2055},{"loc":2081},"blog/1012.dom-downsampling-for-llm-based-web-agents",[372,2086,2087,2088,411,2089],"browser-agents","llms","llm-context","web-automation","O3lW3E_ur1NX1jlAAnkVjclp6x4CMntxHi8aek7NEts",{"id":2092,"title":2093,"authorId":418,"body":2094,"category":372,"created":2823,"description":2824,"extension":375,"faqs":2825,"featurePriority":390,"head":390,"hideCta":391,"landingPath":390,"meta":2838,"navigation":403,"ogImage":390,"path":1516,"robots":390,"schemaOrg":390,"seo":2847,"sitemap":2848,"stem":2849,"tags":2850,"__hash__":2851},"blog/blog/1011.a-gentle-introduction-to-ai-agents-for-the-web.md","A Gentle Introduction to AI Agents for the Web",{"type":8,"value":2095,"toc":2804},[2096,2110,2113,2120,2126,2130,2133,2148,2152,2162,2166,2170,2183,2187,2191,2194,2199,2203,2212,2216,2227,2232,2236,2254,2258,2264,2368,2371,2604,2620,2624,2627,2632,2636,2639,2643,2661,2686,2693,2697,2735,2738,2749,2753,2756,2784,2788,2796,2801],[11,2097,2098,2099,435,2103,615,2106,2109],{},"In no time, AI became a natural part of modern web interfaces. AI agents for the web enjoy a recent hype, sparked by the means of ",[429,2100,434],{"href":2101,"rel":2102},"https://openai.com/index/introducing-operator/",[433],[429,2104,440],{"href":438,"rel":2105},[433],[429,2107,445],{"href":443,"rel":2108},[433],". By now, it is within reach to automate arbitrary web-based tasks, such as booking the cheapest flight from Berlin to Amsterdam.",[61,2111,2076],{"id":2112},"what-is-a-web-agent",[11,2114,2115,2116,2119],{},"For starters, let us break down the term ",[33,2117,2118],{},"web AI agent",": An agent is an entity that autonomously acts on behalf of another entity. An artificially intelligent agent is an application that acts on behalf of a human. In contrast to non-AI computer agents, it solves complex tasks with at least human-grade effectiveness and efficiency. For a human-centric web, web agents have deliberately been designed to browse the web in a human fashion – through UIs rather than APIs.",[15,2121],{":width":2122,"alt":2123,"format":2124,"loading":20,"src":2125},"610","High-level agent description comparing human and computer agents","svg","/blog/a-gentle-introduction-to-ai-agents-for-the-web/1.svg",[463,2127,2129],{"id":2128},"the-role-of-frontier-llms","The Role of Frontier LLMs",[11,2131,2132],{},"Web agents have been a vague desire for a long time. AI agents used to rely on complete models of a problem domain in order to allow (heuristic) search through problem states. Such models would comprise the problem world (e.g., a chessboard), actors (pawns, rooks, etc.), possible actions per actor (rook moves straight), and constraints (i.a., max one piece per field). A heterogeneous space of web application UIs describes the problem domain of a web agent: how to understand a web page, and how to interact with it to solve the declared task?",[11,2134,2135,2136,2143,2144,2147],{},"Frontier LLMs disrupted the AI agent world: explicit problem domain models beyond feasibility can now be replaced by an LLM. The LLM thereby acts as an instantaneous domain model backend that can be consulted with twofold context: serialised problem state, such as a chess position code (",[475,2137,2138,2139,2142],{},"“",[737,2140,2141],{},"..."," e4 e5 2. Nc3 f5”","), and the respective task (",[475,2145,2146],{},"“What is the best move for white?”","). For web agents, problem state corresponds to the currently browsed web application's runtime state, for instance, a screenshot.",[463,2149,2151],{"id":2150},"generalist-web-agents","Generalist Web Agents",[11,2153,2154,2155,615,2158,2161],{},"Generalist web agents are supposed to solve arbitrary tasks through a web browser. Web-based tasks can be as diverse as ",[475,2156,2157],{},"“Find a picture of a cat.”",[475,2159,2160],{},"“Book the cheapest flight from Berlin to Amsterdam tomorrow afternoon (business class, window seat).”"," In reality, generalist agents still fail uncommon or too precise tasks. While they have been critically acclaimed, they mainly act as early proofs-of-concept. Tasks that are indeed solvable with a generalist agent promise great results with an according specialist agent.",[15,2163],{":width":18,"alt":2164,"format":424,"loading":20,"src":2165},"Screenshot of a generalist web agent UI (Director)","/blog/a-gentle-introduction-to-ai-agents-for-the-web/2.png",[463,2167,2169],{"id":2168},"specialist-web-agents","Specialist Web Agents",[11,2171,2172,2173,2176,2177,2182],{},"Other than generalist agents, specialist web agents are constrained to a certain task and application domain. Specialist agents bear the major share of commercial value. Most prominently, modal chat agents that provide users with on-page help. Picture a little floating widget that can be chatted to via text or voice input. In most cases, in fact, the term ",[475,2174,2175],{},"web (AI) agent"," refers to chat agents. Chat agents – text or voice – can be implemented on top of virtually any existing website. Frontier LLMs provide a lot of commonsense out-of-the-box. A ",[429,2178,2181],{"href":2179,"rel":2180},"https://docs.claude.com/en/docs/build-with-claude/prompt-engineering/system-prompts",[433],"system prompt"," can, moreover, be leveraged to drive specialist agent quality for the respective problem domain.",[15,2184],{":width":18,"alt":2185,"format":424,"loading":20,"src":2186},"Screenshots of two modal specialist web agent UIs augmenting an underlying website's UI","/blog/a-gentle-introduction-to-ai-agents-for-the-web/3.png",[61,2188,2190],{"id":2189},"how-does-a-web-agent-work","How Does a Web Agent Work?",[11,2192,2193],{},"LLM-based web agents are premised on a more or less uniform architecture. The agent application embodies a mediator between a web browser (environment), and the LLM backend (model).",[15,2195],{":width":2196,"alt":2197,"format":2124,"loading":20,"src":2198},"480","High-level web agent architecture component view","/blog/a-gentle-introduction-to-ai-agents-for-the-web/4.svg",[463,2200,2202],{"id":2201},"the-agent-lifecycle","The Agent Lifecycle",[11,2204,2205,2206,2211],{},"To reduce a user's cognitive load, solving a web-based task is usually chunked into a sequence of UI states. Consider looking for rental apartments on ",[429,2207,2210],{"href":2208,"rel":2209},"https://www.redfin.com",[433],"redfin.com",": In the first step, you specify a location. Only subsequently are you provided with a grid of available apartments for that location.",[15,2213],{":width":18,"alt":2214,"format":424,"loading":20,"src":2215},"Example of separated UI states in a rental home search application","/blog/a-gentle-introduction-to-ai-agents-for-the-web/5.png",[11,2217,2218,2219,2226],{},"Web agent logic is iterative; not least for a sequential web interaction model, but also for a conversational agent interaction model. Browsing the web, human and computer agents represent users alike. That said, Norman's well-known ",[429,2220,2223],{"href":2221,"rel":2222},"https://mitpress.mit.edu/9780262640374/the-design-of-everyday-things/",[433],[475,2224,2225],{},"Seven Stages of Action",", which hierarchically model the human cognition cycle, transfer to the web agent lifecycle. For each UI state in a web browser (environment) and web-based task (action intention); decide where to click, type, etc. (action planning), and perform those clicks, etc. (action execution). Afterwards, perceive, interpret, and evaluate the results of those actions in the web browser (state). As long as there is a mismatch between the evaluated state and the declared goal state, repeat that cycle. Potentially prompt the user with more required information.",[15,2228],{":width":2229,"alt":2230,"format":2124,"loading":20,"src":2231},"580","Donald 'Norman's Seven Stages of Action' model of the human cognition cycle that transfers to non-human agents","/blog/a-gentle-introduction-to-ai-agents-for-the-web/6.svg",[463,2233,2235],{"id":2234},"web-context-for-llms","Web Context for LLMs",[11,2237,2238,2239,2241,2242,2245,2246,2249,2250,2253],{},"The gap from an agent towards the environment, according to ",[475,2240,2225],{},", is known as the ",[475,2243,2244],{},"gulf of execution",". In real-world scenarios, how to act in the environment in respect to a planned sequence of actions might be difficult (e.g., how to actually open the trunk of a new car?). Arguably, web agents face a novel ",[475,2247,2248],{},"gulf of intention"," towards the action planning stage: how to serialise a currently browsed web page's runtime state for LLMs? ",[475,2251,2252],{},"Snapshot"," is a more comprehensive term to describe the serialisation of a web page's current runtime state. Screenshots, for instance, represent a type of snapshot that closely resembles how humans perceive a web page at a given point in time. But are they as accessible to LLMs?",[463,2255,2257],{"id":2256},"agentic-ui-interaction","Agentic UI Interaction",[11,2259,2260,2261,2263],{},"With a qualified set of well-defined actuation methods, web agents are able to close the ",[475,2262,2244],{}," quite well. HTML element types strongly afford a certain action (e.g., click a button, type to a field). Below is how an actuation schema to present the LLM backend with could look like:",[729,2265,2267],{"className":1533,"code":2266,"language":1535,"meta":363,"style":363},"interface ActuationSchema = {\n    thought: string;\n    action: \"click\"\n        | \"scroll\"\n        | \"type\";\n    cssSelector: string;\n    data?: string;\n}[];\n",[196,2268,2269,2283,2295,2312,2324,2336,2347,2358],{"__ignoreMap":363},[737,2270,2271,2274,2277,2280],{"class":739,"line":740},[737,2272,2273],{"class":1542},"interface",[737,2275,2276],{"class":1546}," ActuationSchema",[737,2278,2279],{"class":860}," = ",[737,2281,2282],{"class":1569},"{\n",[737,2284,2285,2288,2291,2293],{"class":739,"line":364},[737,2286,2287],{"class":860},"    thought",[737,2289,2290],{"class":743},":",[737,2292,1566],{"class":1565},[737,2294,1570],{"class":1569},[737,2296,2297,2300,2302,2305,2309],{"class":739,"line":848},[737,2298,2299],{"class":860},"    action",[737,2301,2290],{"class":743},[737,2303,2304],{"class":757}," \"",[737,2306,2308],{"class":2307},"sgAC-","click",[737,2310,2311],{"class":757},"\"\n",[737,2313,2314,2317,2319,2322],{"class":739,"line":871},[737,2315,2316],{"class":743},"        |",[737,2318,2304],{"class":757},[737,2320,2321],{"class":2307},"scroll",[737,2323,2311],{"class":757},[737,2325,2326,2328,2330,2332,2334],{"class":739,"line":880},[737,2327,2316],{"class":743},[737,2329,2304],{"class":757},[737,2331,1543],{"class":2307},[737,2333,758],{"class":757},[737,2335,1570],{"class":1569},[737,2337,2338,2341,2343,2345],{"class":739,"line":901},[737,2339,2340],{"class":860},"    cssSelector",[737,2342,2290],{"class":743},[737,2344,1566],{"class":1565},[737,2346,1570],{"class":1569},[737,2348,2349,2352,2354,2356],{"class":739,"line":920},[737,2350,2351],{"class":860},"    data",[737,2353,1591],{"class":743},[737,2355,1566],{"class":1565},[737,2357,1570],{"class":1569},[737,2359,2360,2363,2366],{"class":739,"line":929},[737,2361,2362],{"class":1569},"}",[737,2364,2365],{"class":860},"[]",[737,2367,1570],{"class":1569},[11,2369,2370],{},"And a suggested actions response could, in turn, look as follows:",[729,2372,2376],{"className":2373,"code":2374,"language":2375,"meta":363,"style":363},"language-json shiki shiki-themes catppuccin-latte night-owl","[\n    {\n        \"thought\": \"Scroll newsletter cta into view\",\n        \"action\": \"scroll\",\n        \"cssSelector\": \"section#newsletter\"\n    },\n    {\n        \"thought\": \"Type email address to newsletter cta\",\n        \"action\": \"type\",\n        \"cssSelector\": \"section#newsletter > input\",\n        \"data\": \"user@example.org\"\n    },\n    {\n        \"thought\": \"Submit newsletter sign up\",\n        \"action\": \"click\",\n        \"cssSelector\": \"section#newsletter > button\"\n    }\n]\n","json",[196,2377,2378,2383,2388,2412,2431,2449,2454,2458,2477,2495,2514,2532,2536,2540,2559,2577,2594,2599],{"__ignoreMap":363},[737,2379,2380],{"class":739,"line":740},[737,2381,2382],{"class":1569},"[\n",[737,2384,2385],{"class":739,"line":364},[737,2386,2387],{"class":1569},"    {\n",[737,2389,2390,2394,2398,2400,2402,2404,2408,2410],{"class":739,"line":848},[737,2391,2393],{"class":2392},"srFR9","        \"",[737,2395,2397],{"class":2396},"s30W1","thought",[737,2399,758],{"class":2392},[737,2401,2290],{"class":1569},[737,2403,2304],{"class":757},[737,2405,2407],{"class":2406},"sCC8C","Scroll newsletter cta into view",[737,2409,758],{"class":757},[737,2411,1652],{"class":1569},[737,2413,2414,2416,2419,2421,2423,2425,2427,2429],{"class":739,"line":871},[737,2415,2393],{"class":2392},[737,2417,2418],{"class":2396},"action",[737,2420,758],{"class":2392},[737,2422,2290],{"class":1569},[737,2424,2304],{"class":757},[737,2426,2321],{"class":2406},[737,2428,758],{"class":757},[737,2430,1652],{"class":1569},[737,2432,2433,2435,2438,2440,2442,2444,2447],{"class":739,"line":880},[737,2434,2393],{"class":2392},[737,2436,2437],{"class":2396},"cssSelector",[737,2439,758],{"class":2392},[737,2441,2290],{"class":1569},[737,2443,2304],{"class":757},[737,2445,2446],{"class":2406},"section#newsletter",[737,2448,2311],{"class":757},[737,2450,2451],{"class":739,"line":901},[737,2452,2453],{"class":1569},"    },\n",[737,2455,2456],{"class":739,"line":920},[737,2457,2387],{"class":1569},[737,2459,2460,2462,2464,2466,2468,2470,2473,2475],{"class":739,"line":929},[737,2461,2393],{"class":2392},[737,2463,2397],{"class":2396},[737,2465,758],{"class":2392},[737,2467,2290],{"class":1569},[737,2469,2304],{"class":757},[737,2471,2472],{"class":2406},"Type email address to newsletter cta",[737,2474,758],{"class":757},[737,2476,1652],{"class":1569},[737,2478,2479,2481,2483,2485,2487,2489,2491,2493],{"class":739,"line":935},[737,2480,2393],{"class":2392},[737,2482,2418],{"class":2396},[737,2484,758],{"class":2392},[737,2486,2290],{"class":1569},[737,2488,2304],{"class":757},[737,2490,1543],{"class":2406},[737,2492,758],{"class":757},[737,2494,1652],{"class":1569},[737,2496,2497,2499,2501,2503,2505,2507,2510,2512],{"class":739,"line":941},[737,2498,2393],{"class":2392},[737,2500,2437],{"class":2396},[737,2502,758],{"class":2392},[737,2504,2290],{"class":1569},[737,2506,2304],{"class":757},[737,2508,2509],{"class":2406},"section#newsletter > input",[737,2511,758],{"class":757},[737,2513,1652],{"class":1569},[737,2515,2516,2518,2521,2523,2525,2527,2530],{"class":739,"line":951},[737,2517,2393],{"class":2392},[737,2519,2520],{"class":2396},"data",[737,2522,758],{"class":2392},[737,2524,2290],{"class":1569},[737,2526,2304],{"class":757},[737,2528,2529],{"class":2406},"user@example.org",[737,2531,2311],{"class":757},[737,2533,2534],{"class":739,"line":980},[737,2535,2453],{"class":1569},[737,2537,2538],{"class":739,"line":990},[737,2539,2387],{"class":1569},[737,2541,2542,2544,2546,2548,2550,2552,2555,2557],{"class":739,"line":1009},[737,2543,2393],{"class":2392},[737,2545,2397],{"class":2396},[737,2547,758],{"class":2392},[737,2549,2290],{"class":1569},[737,2551,2304],{"class":757},[737,2553,2554],{"class":2406},"Submit newsletter sign up",[737,2556,758],{"class":757},[737,2558,1652],{"class":1569},[737,2560,2561,2563,2565,2567,2569,2571,2573,2575],{"class":739,"line":1027},[737,2562,2393],{"class":2392},[737,2564,2418],{"class":2396},[737,2566,758],{"class":2392},[737,2568,2290],{"class":1569},[737,2570,2304],{"class":757},[737,2572,2308],{"class":2406},[737,2574,758],{"class":757},[737,2576,1652],{"class":1569},[737,2578,2579,2581,2583,2585,2587,2589,2592],{"class":739,"line":1036},[737,2580,2393],{"class":2392},[737,2582,2437],{"class":2396},[737,2584,758],{"class":2392},[737,2586,2290],{"class":1569},[737,2588,2304],{"class":757},[737,2590,2591],{"class":2406},"section#newsletter > button",[737,2593,2311],{"class":757},[737,2595,2596],{"class":739,"line":1042},[737,2597,2598],{"class":1569},"    }\n",[737,2600,2601],{"class":739,"line":1048},[737,2602,2603],{"class":1569},"]\n",[624,2605,2606],{},[11,2607,2608,2613,2614,2619],{},[429,2609,2612],{"href":2610,"rel":2611},"https://platform.openai.com/docs/guides/function-calling",[433],"Function Calling"," and the ",[429,2615,2618],{"href":2616,"rel":2617},"https://modelcontextprotocol.io",[433],"Model Context Protocol"," represent two ends to outsource an explicit actuation model – server- and client-side, respectively.",[463,2621,2623],{"id":2622},"agentic-ui-augmentation","Agentic UI Augmentation",[11,2625,2626],{},"An agent represents yet another feature to integrate with an application and its UI. Discoverability and availability, however, are among the most fundamental requirements of a web agent. Evidently, when a user experiences UI/UX friction, at least the agent should be interactive. That said, a scrolling modal web agent UI has been the go-to approach, that is, a little floating widget on top of the underlying application's UI. It comes with a major advantage: the agent application can be decoupled from the underlying, self-contained application.",[15,2628],{":width":2629,"alt":2630,"format":2124,"loading":20,"src":2631},"360","Depiction of a web agent application augmenting an underlying application in an isolated layer","/blog/a-gentle-introduction-to-ai-agents-for-the-web/7.svg",[61,2633,2635],{"id":2634},"how-to-build-a-web-agent","How to Build a Web Agent?",[11,2637,2638],{},"Believe it or not: enhancing an existing web application with a purposeful agent is a lower-hanging fruit. The evolving agent ecosystem provides you with a spectrum of solutions: instantly use a pre-compiled agent, tweak a templated agent, or develop an agent from scratch. Either way, LLMs and web browsers exist for reuse, boiling down agent development to LLM context engineering, and UI augmentation.",[463,2640,2642],{"id":2641},"develop-a-web-agent","Develop a Web Agent",[11,2644,2645,2646,2649,2650,615,2655,2660],{},"Opting for a ",[33,2647,2648],{},"pre-compiled agent"," does not necessarily involve any actual development step. Instead, pre-compiled agents allow for high-level configuration through an agent-as-a-service provider's interface. Popular agent-as-a-service providers are, i.a., ",[429,2651,2654],{"href":2652,"rel":2653},"https://elevenlabs.io/conversational-ai",[433],"ElevenLabs",[429,2656,2659],{"href":2657,"rel":2658},"https://www.intercom.com/drlp/ai-agent",[433],"Intercom",". Serviced agents hide LLM communication and potentially interaction with a web browser behind the configuration interface.",[11,2662,2663,2664,2667,2668,2673,2674,2679,2680,2685],{},"Using a ",[33,2665,2666],{},"templated agent"," resembles the agent-as-a-service approach on a lower level. Openly sourced from a ",[429,2669,2672],{"href":2670,"rel":2671},"https://github.com/webfuse-com/agent-extension-blueprint",[433],"code repository",", templated agents allow for any kind of development tweaks. Favourably, agent templates shortcut integration with ",[429,2675,2678],{"href":2676,"rel":2677},"https://openai.com/api/",[433],"LLM APIs"," and web ",[429,2681,2684],{"href":2682,"rel":2683},"https://developer.mozilla.org/en-US/docs/Web/API",[433],"browser APIs",". Using a templated agent usually represents the preferable, best-of-both-worlds approach; common- and best-practice code snippets are available from the beginning, but everything can be customised as desired.",[11,2687,2688,2689,2692],{},"Of course, developing an ",[33,2690,2691],{},"agent from scratch"," is always an option. It is preferable whenever agent requirements deviate to a large extent from what exists in the service or template landscape.",[463,2694,2696],{"id":2695},"deploy-a-web-agent","Deploy a Web Agent",[11,2698,2699,2700,648,2705,2710,2711,2716,2717,2722,2723,2728,2729,2734],{},"When web agent code lives side-by-side with the augmented application's code, agent deployment is covered by a generic pipeline. Something like: ",[429,2701,2704],{"href":2702,"rel":2703},"https://eslint.org",[433],"linting",[429,2706,2709],{"href":2707,"rel":2708},"https://prettier.io",[433],"formatting"," agent code, ",[429,2712,2715],{"href":2713,"rel":2714},"https://esbuild.github.io",[433],"transpiling and bundling"," agent modules, ",[429,2718,2721],{"href":2719,"rel":2720},"https://www.cypress.io",[433],"testing"," agent, ",[429,2724,2727],{"href":2725,"rel":2726},"https://pages.cloudflare.com",[433],"hosting"," agent bundle, and ",[429,2730,2733],{"href":2731,"rel":2732},"https://docs.github.com/en/actions/get-started/continuous-integration",[433],"tiggering"," post deployment events. In that case, an agent represents a modular feature component in the application, no different than, for instance, a sign-up component.",[11,2736,2737],{},"Web agent source code right inside the application codebase comes at a cost:",[78,2739,2740,2743,2746],{},[81,2741,2742],{},"Agent developers can manipulate the source code of the underlying application.",[81,2744,2745],{},"Agent functionality could introduce side effects on the underlying application.",[81,2747,2748],{},"Agent changes require deployment of the entire application.",[463,2750,2752],{"id":2751},"best-practices-of-agentic-ux","Best Practices of Agentic UX",[11,2754,2755],{},"When designing user experiences for agent-enhanced applications, there are a few things to consider:",[78,2757,2758,2759,2758,2768,2758,2776],{},"\n    ",[81,2760,2761,2762,2761,2765,2767],{},"\n        ",[33,2763,2764],{},"Stream input and output to reduce latency",[1962,2766],{},"\n        LLMs (re-)introduce noticeable communication round-trip time. To reduce wait time for the human user, stream chunks of data whenever they are available.\n    ",[81,2769,2761,2770,2761,2773,2775],{},[33,2771,2772],{},"Provide fine-grained feedback to bridge high-latency",[1962,2774],{},"\n        Human attention is sensitive to several seconds of [system response time](https://www.nngroup.com/articles/response-times-3-important-limits/). Periodically provide agent _thoughts_ as feedback to perceptibly break down round-trip time.\n    ",[81,2777,2761,2778,2761,2781,2783],{},[33,2779,2780],{},"Always prompt the human user for consent to perform critical actions",[1962,2782],{},"\n        Some actions in a web application lead to irreversible or significant changes of state. Never have the agent perform such actions on behalf of the user without explicitly asking for the permission.\n    ",[463,2785,2787],{"id":2786},"non-invasive-web-agents-with-webfuse","Non-Invasive Web Agents with Webfuse",[11,2789,2790,2795],{},[429,2791,2793],{"href":1779,"rel":2792},[433],[33,2794,40],{}," is a configurable web proxy that lets you augment any web application. As pictured, web agents represent highly self-contained applications. Moreover, web agents and underlying applications communicate at runtime in the client. This does, in fact, render opportunities to bridge the above-mentioned drawbacks with Webfuse: Develop web agents with a sandbox extension methodology, and deploy them through the low-latency proxy layer. On demand, seamlessly serve users with your agent-enhanced website. Benefit from information hiding, safe code, and fewer deployments.",[115,2797],{"eyebrow":2798,"heading":2799,"highlight":2800},"Most agent stacks begin by launching a browser.","Webfuse begins in a live session.","live",[2036,2802,2803],{},"html pre.shiki code .s76yb, html code.shiki .s76yb{--shiki-default:#8839EF;--shiki-dark:#C792EA}html pre.shiki code .sXbZB, html code.shiki .sXbZB{--shiki-default:#DF8E1D;--shiki-default-font-style:italic;--shiki-dark:#D6DEEB;--shiki-dark-font-style:inherit}html pre.shiki code .s2kId, html code.shiki .s2kId{--shiki-default:#4C4F69;--shiki-dark:#D6DEEB}html pre.shiki code .scGhl, html code.shiki .scGhl{--shiki-default:#7C7F93;--shiki-dark:#D6DEEB}html pre.shiki code .s9rnR, html code.shiki .s9rnR{--shiki-default:#179299;--shiki-dark:#7FDBCA}html pre.shiki code .scrte, html code.shiki .scrte{--shiki-default:#8839EF;--shiki-dark:#C5E478}html pre.shiki code .sbuKk, html code.shiki .sbuKk{--shiki-default:#40A02B;--shiki-dark:#D9F5DD}html pre.shiki code .sgAC-, html code.shiki .sgAC-{--shiki-default:#40A02B;--shiki-default-font-style:italic;--shiki-dark:#ECC48D;--shiki-dark-font-style:inherit}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .srFR9, html code.shiki .srFR9{--shiki-default:#7C7F93;--shiki-dark:#7FDBCA}html pre.shiki code .s30W1, html code.shiki .s30W1{--shiki-default:#1E66F5;--shiki-dark:#7FDBCA}html pre.shiki code .sCC8C, html code.shiki .sCC8C{--shiki-default:#40A02B;--shiki-dark:#C789D6}",{"title":363,"searchDepth":364,"depth":364,"links":2805},[2806,2811,2817],{"id":2112,"depth":364,"text":2076,"children":2807},[2808,2809,2810],{"id":2128,"depth":848,"text":2129},{"id":2150,"depth":848,"text":2151},{"id":2168,"depth":848,"text":2169},{"id":2189,"depth":364,"text":2190,"children":2812},[2813,2814,2815,2816],{"id":2201,"depth":848,"text":2202},{"id":2234,"depth":848,"text":2235},{"id":2256,"depth":848,"text":2257},{"id":2622,"depth":848,"text":2623},{"id":2634,"depth":364,"text":2635,"children":2818},[2819,2820,2821,2822],{"id":2641,"depth":848,"text":2642},{"id":2695,"depth":848,"text":2696},{"id":2751,"depth":848,"text":2752},{"id":2786,"depth":848,"text":2787},"2025-06-15","An introduction to web agents: autonomous systems that browse on your behalf. Core methodology, key design challenges and where they fail.",[2826,2829,2832,2835],{"question":2827,"answer":2828},"What is a web AI agent?","A web AI agent is an application that autonomously acts on behalf of a human by operating a website through its user interface rather than its API. It uses a large language model to interpret the page and decide what to click, type, or scroll to complete a task. This human-like interaction lets it work on websites built for people rather than machines.",{"question":2830,"answer":2831},"What is the difference between a generalist and a specialist web agent?","A generalist web agent is built to solve arbitrary tasks across any website, while a specialist agent is constrained to one task and application domain. Generalist agents remain early proofs of concept and often fail on uncommon or highly precise tasks. Specialist agents, such as on-page chat and voice assistants, carry most of the current commercial value.",{"question":2833,"answer":2834},"How does a web agent work?","A web agent sits between a web browser and an LLM backend and runs an iterative loop. It serialises the page's current runtime state into a snapshot, sends that snapshot to the LLM along with the task, receives a planned action such as a click or keystroke, and executes it. It then evaluates the new page state and repeats the cycle until the result matches the declared goal.",{"question":2836,"answer":2837},"How do you build a web agent?","There are three main paths: use a pre-compiled agent configured through an agent-as-a-service provider, adapt an open-source templated agent, or develop one from scratch. Templated agents are usually the best balance, because they shortcut integration with LLM and browser APIs while still allowing full customisation.",{"homepage":403,"relatedLinks":2839},[2840,2841,2845],{"text":2072,"href":2073,"description":2074},{"text":2842,"href":2843,"description":2844},"Develop an AI Agent for Any Website with Webfuse","/blog/develop-an-ai-agent-for-any-website-with-webfuse","Learn how to develop and deploy a web agent for any website with Webfuse",{"text":1775,"href":2846,"external":403,"description":2080},"https://dev.webfuse.com/automation-api/",{"title":2093,"description":2824},{"loc":1516},"blog/1011.a-gentle-introduction-to-ai-agents-for-the-web",[372,2086,2087,411,2089],"lkTQmfxrQ3yBiFfGf2VuCXy0NJ3fif9aH6wJJBwoJmg",1787586691447]