MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents
Introduces a mobile-agent benchmark with tasks across open-source applications, organized by difficulty to reduce manual evaluation effort.
评测
已审阅并标注「评测」的研究与资产。
条目依据已审阅标签归类，同一主题不表示资源之间存在依赖关系。日期保留来源精度，注释保留原语言。
On the Effects of Data Scale on UI Control Agents
Studies how Android task performance changes with demonstration scale and releases AndroidControl with high- and low-level instructions.
Android
WebSuite: Systematically Evaluating Why Web Agents Fail
WebSuite organizes browser operations into an action taxonomy and tests both isolated controls and complete workflows. Its diagnostic tasks attribute failures to specific actions, enabling comparison of text-based and multimodal web agents.
网页
Large Language Models Can Self-Improve At Web Agent Tasks
Examines self-improvement by fine-tuning web agents on three mixtures of model-generated training data. WebArena experiments and trajectory-level evaluation measures study task performance, robustness, and behavior beyond aggregate success rates.
网页
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Introduces parameterized Android tasks with initialization, success checks, and cleanup for reproducible interactive evaluation.
Android
Latent State Estimation Helps UI Agents to Reason
Studies explicit textual estimates of latent UI state, including whether commanded actions occurred and how far a task has progressed. The experiments compare agents that reason with these estimates against agents that omit them across public and internal task benchmarks.
Unveiling Disparities in Web Task Handling Between Human and Web Agent
Compares human think-aloud web-task behavior with web-agent planning, action, and reflection. The analysis highlights differences in updating plans, handling ambiguity, and investigating failures.
网页
Visual Grounding Methods for Efficient Interaction with Desktop Graphical User Interfaces
Compares modular OCR-based and end-to-end methods for locating desktop GUI targets from natural-language instructions.
桌面
Navigating WebAI: Training Agents to Complete Web Tasks with Large Language Models and Reinforcement Learning
Combines supervised and reinforcement learning for MiniWoB web tasks and investigates whether agents understand HTML structure or memorize target elements.
网页
MMAC-Copilot: Multi-modal Agent Collaboration Operating Copilot
Introduces MMAC-Copilot, which coordinates agents with different expertise to operate PC applications. VIBench evaluates visual interaction with applications that lack usable APIs, spanning office, recreation and game tasks.
桌面
Benchmarking Mobile Device Control Agents across Diverse Configurations
Introduces B-MoCA, an interactive mobile-control benchmark that varies device settings, layouts, and languages to test generalization.
Android
MMInA: Benchmarking Multihop Multimodal Internet Agents
Evaluates multimodal agents on compositional tasks across changing websites and studies replaying prior trajectories as memory support.
网页
Large Language Models for Mobile GUI Text Input Generation: An Empirical Study
Studies language-model generation of mobile text inputs, comparing UI representations, execution feedback, and human assistance for app testing.
Android
Enhancing Mobile "How-to" Queries with Automated Search Results Verification and Reranking
Executes instructions from mobile how-to search results inside a controlled Android environment, then uses the observed success of those instructions to verify and rerank technical-support results.
Android
LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation
LlamaTouch executes mobile tasks on devices and evaluates whether an agent visits manually annotated essential application or system states. Screenshot and hierarchy annotations, combined with exact and fuzzy state matching, support evaluation of 496 tasks across multiple agents.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Introduces a real-computer evaluation environment with execution-based checks for tasks spanning desktop applications and operating-system state.
桌面LinuxWindowsmacOS
Autonomous Evaluation and Refinement of Digital Agents
Compares automatic trajectory evaluators for web navigation and device control against oracle judgments. The resulting feedback is used for agent fine-tuning and inference-time guidance, including experiments on WebArena.
网页
VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
Separately evaluates web-page reading, understanding, and element grounding across tasks drawn from real websites.
网页
AgentStudio: A Toolkit for Building General Virtual Agents
AgentStudio combines an interactive GUI/API environment with tools for creating tasks, annotating interface elements, and labeling actions in videos. Its online suite and GroundUI, IDMBench, and CriticBench datasets examine interaction, grounding, learning from video, and success detection.
Tur[k]ingBench: A Challenge Benchmark for Web Agents
Evaluates web agents on multimodal annotation tasks adapted from HTML interfaces originally built for crowdsourcing workers.
网页
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
Introduces enterprise web tasks on ServiceNow and BrowserGym interfaces for evaluating browser-based agents.
网页
PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion
Benchmarks robustness of API-based PowerPoint completion under sentence-level, semantic, and multilingual instruction perturbations and changes to the available API set.
相关研究
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web
Introduces a dataset and benchmark for generating executable task scripts from screenshots and visually grounded desktop or web instructions.
桌面网页
WIPI: A New Web Threat for LLM-Driven Web Agents
Studies WIPI, a black-box indirect-instruction threat in which webpage content redirects LLM-driven web agents. Evaluation spans plugin-based agents, Web GPTs, and open-source web-agent implementations.
网页
On the Multi-turn Instruction Following for Conversational Web Agents
MT-Mind2Web evaluates web navigation through sequences of conversational user instructions. Self-MAP combines memory-augmented planning with reflection to preserve context and dependencies across those interaction turns.
网页
Understanding the Weakness of Large Language Model Agents within a Complex Android Environment
Introduces AndroidArena and analyzes agent limitations in cross-application tasks, user constraints, exploration, and reflection.
Android
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue
Collects expert demonstrations of conversational web navigation and studies retrieval-based element selection for processing large pages.
网页
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
Combines multimodal web interaction with a real-website task collection and an automated evaluation protocol.
网页
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Introduces visually grounded website tasks for evaluating multimodal agents that must interpret both images and text.
网页
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Introduces AgentBoard, a multiturn agent evaluation framework with intermediate progress metrics and analysis tools. Its broader task suite includes WebShop and WebArena interactions with stepwise progress measurements.
网页相关研究
WebVLN: Vision-and-Language Navigation on Websites
WebVLN trains website navigation from question-based instructions using both rendered visual information and underlying HTML. It introduces WebVLN-v1 and a website-aware navigation network to study this digital variant of vision-and-language navigation.
网页
ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation
Introduces Windows application tasks and an actor-critic agent framework for mouse-and-keyboard automation of desktop workflows.
桌面Windows
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
CompWoB adds 50 composed web-automation tasks to study generalization from simpler interactions. The paper compares prompted and fine-tuned agents, balances task training data, and probes sensitivity to changes in instruction order.
网页
GAIA: a benchmark for General AI Assistants
Introduces questions that test assistant reasoning, multimodal understanding, browsing, and tool use beyond screen-action execution alone.
网页相关研究
PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion
Evaluates multi-turn PowerPoint creation and editing, scoring the resulting presentation files rather than requiring a fixed API sequence.
相关研究
InstructExcel: A Benchmark for Natural Language Instruction in Excel
Introduces InstructExcel, a benchmark for generating Excel OfficeScripts from natural-language requests, built using scripts recorded from user actions and a collection of existing spreadsheets.
相关研究
AllTogether: Investigating the Efficacy of Spliced Prompt for Web Navigation using Large Language Models
AllTogether standardizes prompts for HTML-based web navigation and compares prompting with instruction fine-tuning. Its analysis examines how HTML context, interaction history, and current environmental feedback affect task execution.
网页
AgentBench: Evaluating LLMs as Agents
Evaluates language agents across eight interactive environments and analyzes failures in reasoning, decisions, and instruction following.
跨平台相关研究
WebArena: A Realistic Web Environment for Building Autonomous Agents
Introduces reproducible web environments with task-completion checks for evaluating autonomous web agents.
网页
Android in the Wild: A Large-Scale Dataset for Android Device Control
Collects Android interaction demonstrations with screens, actions, and instructions across device and platform variations for robustness evaluation.
Android
Mind2Web: Towards a Generalist Agent for the Web
Introduces a web-task dataset spanning diverse websites and evaluates generalization to unseen sites and domains.
网页
Responsible Task Automation: Empowering Large Language Models as Responsible Task Automators
Presents ResponsibleTA, a coordinator-executor framework that checks command feasibility and task completion while using local memory for privacy-related controls. The paper compares prompted and learned implementations and evaluates them on UI task automation.
相关研究
SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models
Introduces SheetCopilot, an agent that steers spreadsheet software via atomic virtual APIs, along with a 221-task spreadsheet manipulation benchmark.
桌面Windows
AdaPlanner: Adaptive Planning from Feedback with Language Models
Introduces AdaPlanner, which revises code-structured plans in response to environmental feedback and reuses successful plans as exemplars. Experiments include MiniWoB++ web tasks alongside ALFWorld.
网页相关研究
Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction
Provides an Android benchmark toolkit with controlled environments, intermediate feedback, and task sets for reproducible GUI-agent evaluation.
Android
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
Introduces screenshot question answering over mobile interfaces, linking answers to screen content and bounding boxes.
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Provides a simulated shopping environment where language agents search, configure, and select products from natural-language requirements.
网页
A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility
Introduces MoTIF mobile-app interactions with feasible and infeasible commands, feasibility explanations, and follow-up questions for ambiguous requests.
WebSRC: A Dataset for Web-Based Structural Reading Comprehension
Introduces question answering over webpage text and structure, with aligned HTML, screenshots, and metadata for structural reading comprehension.
网页
Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
Compares seven traditional and learned GUI element detectors on more than 50,000 interface images. The analysis motivates a coarse-to-fine non-text detector combined with a learned text detector to address GUI-specific localization requirements.
Mapping Natural Language Commands to Web Elements
Defines language grounding on webpages as choosing the UI element targeted by a natural-language command. A collection of more than 50,000 commands and three baseline models study functional references, relationships between elements, and visual position cues.
网页
World of Bits: An Open-Domain Platform for Web-Based Agents
Introduces World of Bits, a web-task platform combining human demonstrations, keyboard and mouse actions, and cached HTTP traffic for repeatable task environments. It evaluates agents trained through behavioral cloning and reinforcement learning.
网页
End-to-End Goal-Driven Web Navigation
Introduces WebNav for converting websites into goal-directed hyperlink navigation tasks and WikiNav for evaluating language-conditioned navigation and question answering on Wikipedia.
网页
GUI Testing Using Computer Vision
Presents visual GUI test scripts that use images to specify actions and expected screen feedback. Recording both input events and screenshots can generate tests from demonstrations, supporting unit, regression, and test-driven workflows.
AgentBench
Agent evaluation suite with containerized tasks and a function-calling interface, including operating-system and shopping environments.
跨平台网页相关研究
AgentDAM
Web-agent benchmark for measuring unnecessary use of sensitive personal information during end-to-end task execution.
网页
AgentRewardBench
A benchmark and evaluation toolkit for comparing automatic web-agent judges against expert judgments of task success, side effects and repetition.
网页
AndroidLab
Reproducible Android benchmark of 138 tasks across nine apps on predefined virtual devices, run via AVD on Mac or Docker on Linux, released with the Android Instruct training set.
Android
第 541–600 条，共 723 条资源。
通过主题导出清单获取完整记录和证据。克隆目录后，可将主题记录直接交给标准 RAG 与引用导出工具。