REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites

Introduces REAL, a benchmark of deterministic website replicas with 112 tasks covering information retrieval and state-changing browser actions.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

What is this resource?

Resource type

Research paper

Environment

Web

Topics

Evaluation, Sandboxes, Web navigation

Record publication date

2025-04-15

Evidence retrieved

2026-11-05

Resource ID

arxiv:2504.11543

Recorded paper version

arXiv v2

Version updated: 2025-04-17T16:28:46Z.

This identifies the retained arXiv metadata snapshot; newly observed changes require separate review.

Authors

Divyansh Garg, Shaun VanWeelden, Diego Caples, Andis Draguns, Nikil Ravi, Pranav Putta, Naman Garg, Tomas Abraham, Michael Lara, Federico Lopez, James Liu, Atharva Gundawar, Prannay Hebbar, Youngchul Joo, Jindong Gu, Charles London, Christian Schroeder de Witt, Sumeet Motwani

Cite this paper

Download BibTeX · Download CSL JSON

Citations preserve the selected source edition, its authors and recorded date precision. Unrecorded venues and identifiers are omitted.

Explore related topics

Scope and limitations

The annotation and scope were reviewed against the complete retained arXiv abstract (v2). All evaluated interactions occur in controlled simulations of websites. Experiments, reported performance and artifact availability have not been independently reproduced.

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

1. arxiv api

   Retrieved 2026-10-17T07:57:47Z. Supports: name, authors, published_at, updated_at, identifiers.arxiv, details.arxiv_snapshot.

   Content fingerprint

   27144160e8981bb9f36b60c391689774913ff3f026eeeffc2923154c404a6529
2. editorial primary review

   Retrieved 2026-11-05T23:01:33Z. Supports: name, kind, summary, environments, topics, scope, relations.

   Content fingerprint

   b7001e189c17d6fa072573558661e37a7cb4c7709d33080603d72a969cc4ba39

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.

Useful? Star Awesome CUA on GitHub