AppSim-Bench
Simulated mobile applications and evaluation tooling for reproducible GUI-agent tasks with controlled application state.
Android
评测
已审阅并标注「评测」的研究与资产。
条目依据已审阅标签归类，同一主题不表示资源之间存在依赖关系。日期保留来源精度，注释保留原语言。
AppWorld
Simulated applications, user data, and state-based task checks for evaluating agents that interact through APIs and code.
相关研究
BrowserART
Red-teaming benchmark and test suite from Scale AI evaluating whether refusal-trained language models can be jailbroken when deployed as browser agents.
网页
ClawBench
Everyday-task benchmark on live websites that runs browser agents in containerized Chromium, blocks each final submission request for scoring by URL match and an LLM judge, and records five layers of run evidence.
网页
ColorBench
A graph-based mobile GUI benchmark that evaluates long tasks through screenshot states and multiple valid or erroneous transitions, including reflection, replanning and backtracking.
Android
CUAVerifierBench
Human-annotated browser-agent trajectories for comparing automated task-success and process-quality verifiers against reviewer judgments.
网页
EIA
Evaluation suite for Environmental Injection Attacks that embed invisible malicious HTML elements into benign websites to test privacy leakage in generalist web agents.
网页
FedMABench
A federated mobile-agent training and evaluation suite that varies user-data heterogeneity and application distribution, with released partition annotations and inference/scoring scripts.
Android
GUI-Robust
Evaluation scripts and linked data for comparing GUI action predictions and task execution under ordinary and anomalous interface conditions.
跨平台
GUIHalluBench
Measures hallucination in UI-element parsing by matching the boxes and descriptions a model lists for 1,000 English and 1,000 Chinese screenshots against human annotations.
跨平台
iOSWorld
Native iPhone-simulator benchmark with shared user data, application workflows, agent runners, and rubric-based scoring.
iOS
MacAgentBench
676 macOS tasks across 25 applications, each run in its own Docker container and scored by deterministic rule-based checks with multiple checkpoints.
桌面macOS
Mind2Web 2
Long-horizon agentic web search benchmark whose Agent-as-a-Judge framework scores time-varying, citation-backed answers, with evaluation scripts for its dev and test sets.
网页
MiniWoB++
Over 100 web-interaction environments extending OpenAI's MiniWoB, controlled from Python through a Gymnasium API that drives the browser with Selenium.
网页
MLA-Trust
Unified evaluation framework measuring truthfulness, controllability, safety, and privacy of multimodal LLM agents across interactive web and mobile GUI tasks.
网页Android跨平台
MMBench-GUI
Four-level GUI-agent benchmark across six platforms, from content understanding and element grounding to task automation and collaboration, scored with an efficiency-quality area metric.
跨平台
MMInA
Multimodal web tasks in a WebArena-based environment that chain from 1 to 10 hops across sites such as a shopping store and Wikipedia, testing long-chain reasoning.
网页
MobBench task definitions
A public set of real-device mobile task definitions used to evaluate MobA, with Chinese and English task material packaged in a versioned JSON release.
Android
Mobile-Bench-v2 released data
Offline mobile-agent task data spanning common tasks and released noisy, contaminated and ambiguous-instruction subsets.
Android
Mobile-Eval-E task definitions
Task definitions for Mobile-Agent-E’s long-horizon, multi-app Android evaluation, packaged as a 25-record test split with rubrics and human-reference operation strings.
Android
MobileAgentBench
Automated benchmark for mobile LLM agents whose default tasks use ten Simple Mobile Tools Android apps, installed as a Python library that detects task completion.
Android
MobileGUI-VBench
Mobile-app task definitions covering everyday scenarios, with annotated preconditions and clarification requirements for evaluating agent behavior.
MobileGym
A browser-hosted mobile-app simulator with programmable state, parameterized tasks and deterministic state-checking judges for parallel GUI-agent evaluation and online reinforcement learning.
网页
MobileGym++ / MobileGym++ Bench
Browser-rendered mobile simulation with resettable app state and a shared task suite for GUI-only and GUI-plus-tool agents.
网页
MobileSafetyBench
Android emulator testbed that scores mobile device-control agents on both safety and helpfulness, using ADB and Appium for observations, actions and success checks.
Android
MobileWorld (Tongyi-MAI)
A containerized Android evaluation environment with GUI-only, agent-user interaction and MCP-augmented tasks, plus checks on application and backend state.
Android
MobileWorldBench
Benchmark of semantic world modeling for mobile agents, with question-answering tasks and GPT-judged generation tasks, plus training data built on AitW and AndroidControl screens.
Android
Online Mind2Web
300 tasks on 136 live websites for evaluating web agents online, with outdated tasks replaced over time, human-reviewed submissions and the WebJudge automatic evaluator.
网页
OpenApps
Configurable browser apps and task rewards for testing computer-use agents under changes in content, layout and appearance.
网页
OS-Critic Bench
A cross-platform GUI-critic benchmark with mobile, desktop and web JSONL splits and screenshot assets, accompanied by public inference and accuracy-scoring scripts.
跨平台桌面网页
OS-Harm
Benchmark for safety of computer-use agents across misuse, prompt injection, and unintended model behavior.
桌面
OS-Map
An OSWorld-derived desktop benchmark with an Ubuntu virtual-machine environment and expanded task configurations for measuring breadth and depth of computer interaction.
桌面Linux
OSWorld
Execution-based evaluation of agents performing tasks in real desktop environments.
桌面LinuxWindowsmacOS
OSWorld-G
Computer-use grounding benchmark with original and refined instructions, released with the Jedi synthetic data pipeline, a 4-million-example Jedi dataset and Jedi-3B and Jedi-7B models.
桌面
OSWorld-Human
Human reference trajectories and scoring code for measuring the action efficiency of computer-use agents on OSWorld tasks.
桌面
PPTC
PowerPoint creation and editing benchmark with API-based execution and checks of the resulting presentation files.
相关研究
RiOSWorld
Measures risky behavior of multimodal computer-use agents in an Ubuntu virtual machine, with risk scenarios such as phishing emails and released evaluation trajectories.
桌面Linux
SaaS-Bench
106 browser-driven business workflows across 23 self-hosted SaaS apps in six domains, scored by per-task verifiers that read each app's database and give partial credit.
网页
ScienceBoard
Scientific-workflow tasks for multimodal agents in desktop software such as ChimeraX, KAlgebra, Celestia, Grass GIS and Lean 4, with apps modified to expose state for evaluation.
桌面
Screen Point-and-Read / Tree-of-Lens
Screen-reading benchmark and Tree-of-Lens implementation for interpreting user-indicated regions and their layout context.
跨平台
ScreenSpot
GUI grounding test set of 1,272 screenshot instructions from iOS, Android, macOS, Windows and web interfaces, each labeled with the target element's bounding box.
跨平台
ScreenSpot Pro
GUI grounding benchmark focused on professional high-resolution interfaces.
桌面WindowsmacOSLinux
ScreenSpot v2
Second version of the ScreenSpot grounding test set, published by OS-Copilot as separate desktop, mobile and web annotation files with a shared screenshot archive.
跨平台
ScreenSpot-v2 instruction variants
Instruction variants over ScreenSpot-v2 screenshots that distinguish original commands, explicit actions, target descriptions and impossible operations.
跨平台
SPA-Bench
340 single-app and cross-app tasks over 66 Chinese and English Android apps, with plug-in support for 11 smartphone agents and automatic coarse-to-fine success detection.
Android
ST-WebAgentBench
BrowserGym-based benchmark for evaluating the safety and policy compliance of enterprise web agents across six trustworthiness risk dimensions.
网页
TheAgentCompany
Self-contained workplace environment for evaluating agents on professional tasks involving websites, files, code, and simulated coworkers.
网页桌面
VeriWeb (formerly VeriGUI)
Verifiable web information-seeking benchmark renamed from VeriGUI; desktop evaluation is described as future work.
网页
第 601–660 条，共 723 条资源。
通过主题导出清单获取完整记录和证据。克隆目录后，可将主题记录直接交给标准 RAG 与引用导出工具。