SafeArena: Evaluating the Safety of Autonomous Web Agents
Pairs benign and harmful web tasks to assess deliberate misuse risks and categorize agent behavior by risk level.
网页
评测
已审阅并标注「评测」的研究与资产。
条目依据已审阅标签归类，同一主题不表示资源之间存在依赖关系。日期保留来源精度，注释保留原语言。
AutoEval: A Practical Framework for Autonomous Evaluation of Mobile Agents
Generates reward signals from UI-state changes and uses a judging system to evaluate mobile agents with reduced manual evaluator construction.
Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
Analyzes web-agent vulnerabilities at the component level, examining goal placement in prompts, multistep action generation, and observations as sources of increased susceptibility.
网页
Towards Sustainable Web Agents: A Plea for Transparency and Dedicated Metrics for Energy Consumption
Examines energy and carbon-accounting limitations in web-agent evaluation and argues for transparent, dedicated sustainability measurements.
网页
Towards Enterprise-Ready Computer Using Generalist Agent
Describes the CUGA enterprise-agent development process, emphasizing iterative evaluation, failure analysis, and refinement on software-task benchmarks.
网页
Programming with Pixels: Can Computer-Use Agents do Software Engineering?
Introduces a visual IDE environment and benchmark for software-engineering tasks, comparing screenshot-based control with access to editing and shell tools.
桌面
Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks
Examines active environmental injection against mobile agents, testing whether adversarial interface content redirects decisions during task execution.
Android
WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point
Introduces desktop and web tasks with varied initial states to test planning recovery and adaptability beyond canonical starting configurations.
桌面网页
CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation
Introduces CowPilot, a web-navigation framework where agents suggest actions and users can pause, reject, override, or resume them. It evaluates autonomous and collaborative use through task success and human effort.
网页
WinClick: GUI Grounding with Multimodal Large Language Models
Introduces a screenshot-based Windows GUI agent and the WinSpot benchmark for evaluating localization of actionable interface regions.
桌面Windows
WebWalker: Benchmarking LLMs in Web Traversal
Introduces WebWalkerQA and an exploration-and-critique framework for gathering information by traversing multiple levels of a website.
网页
A3: Android Agent Arena for Mobile GUI Agents with Essential-State Procedural Evaluation
Introduces Android Agent Arena with online application tasks and procedural evaluation based on reaching essential intermediate states.
Android
GUI Testing Arena: A Unified Benchmark for Advancing Autonomous GUI Testing Agent
Evaluates GUI testing through test-intent generation, execution, and defect detection using real, modified, and synthetic mobile application data.
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Simulates a software-company workplace to evaluate agents on professional tasks involving websites, programs, files, and coworker communication.
网页桌面
Large Action Models: From Inception to Implementation
Uses a Windows-agent case study to describe a development workflow for large action models, covering data collection, training, grounding, environment integration, and evaluation.
桌面Windows
The BrowserGym Ecosystem for Web Agent Research
Describes BrowserGym's shared web interaction environment and AgentLab tooling for consistent experiments across multiple agent benchmarks.
网页
Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms
Surveys virtual agents for browsers, applications and operating systems, organizing prior work by task type, observation and action spaces, model strategies and evaluation methods. It examines screen-based interaction and keyboard, mouse and touch control, alongside transferability and long-sequence decision-making challenges.
网页桌面AndroidWindows
The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use
Presents a preliminary study of Claude 3.5 Computer Use across desktop tasks and software, analyzing planning, action, and critique behavior and providing a framework for deploying GUI-automation models.
桌面
Attacking Vision-Language Computer Agents via Pop-ups
Tests whether adversarial visual pop-ups divert computer agents from intended tasks in desktop and web benchmark environments.
桌面网页
Foundations and Recent Trends in Multimodal Mobile Agents: A Survey
Surveys multimodal mobile agents, organizing prompt-based and training-based approaches alongside evaluation benchmarks, supporting technologies, and open challenges in mobile task execution.
AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents
Provides a shared Android action environment, reproducible tasks, and instruction data for training and comparing language and multimodal agents.
Android
Evaluating Cultural and Social Awareness of LLM Web Agents
Introduces CASA to assess web agents' responses to cultural and social norms in shopping and forum tasks, including misleading observations.
网页
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks
Evaluates whether web agents can retain procedural skills and factual information from long video tutorials and use them during tasks.
网页
MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control
Evaluates mobile agents on misuse, side effects, and indirect prompt-injection risks in an Android emulator environment.
Android
AdvAgent: Controllable Blackbox Red-teaming on Web Agents
Introduces a black-box web-agent red-teaming framework that learns adversarial prompts from agent feedback to study security weaknesses.
网页
Beyond Browsing: API-Based Web Agents
Compares browsing-only, API-only, and hybrid web agents on WebArena, examining how access to application APIs changes task execution alongside browser interaction.
网页
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
Introduces a smartphone-agent benchmark with English and Chinese tasks, real-time Android interaction, and completion and resource-use metrics.
Android
Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents
Introduces BrowserART and studies whether chat-model refusal behavior transfers to browser agents carrying out potentially harmful instructions.
网页
From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts
Develops a taxonomy of mobile-action consequences and annotated interaction data to evaluate whether models recognize impactful or risky UI operations.
ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
Evaluates web-task completion under explicit enterprise policies, measuring policy-compliant completion and safety violations separately.
网页
Multimodal Auto Validation For Self-Refinement in Web Agents
Studies text and visual validation for web agents and uses the resulting validator to detect failures and guide workflow self-refinement.
网页
Inferring Alt-text For UI Icons With Large Language Models During App Development
Introduces IconDesc, which generates mobile icon alt-text from partial UI context such as element identifiers, bounds, nearby text and hierarchy. Model evaluation and a user study examine the resulting accessibility descriptions.
相关研究
The Impact of Element Ordering on LM Agent Performance
Studies how the ordering of UI elements in an agent observation affects web and desktop performance. A pixel-based element detector and dimensionality-reduction ordering are evaluated on OmniACT when document hierarchy is unavailable.
网页桌面
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
Evaluates privacy-directed webpage instruction injection against generalist web agents using PII-related interaction steps from Mind2Web, including attempts to expose specific user information or entire requests.
网页
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale
Adapts real-desktop evaluation to Windows applications and supports parallel execution of benchmark tasks.
桌面Windows
WebQuest: A Benchmark for Multimodal QA on Web Page Sequences
Introduces question answering over individual screens, multiple web pages, and navigation traces to test information synthesis across views.
网页
From Grounding to Planning: Benchmarking Bottlenecks in Web Agents
Separates web-agent planning and grounding evaluations on Mind2Web to analyze which components limit task performance.
网页
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
Benchmarks visual agents across GUI, embodied, and design tasks, with training trajectories collected through solvers, agents, and human demonstrations.
跨平台相关研究
Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions
Measures how irrelevant GUI content affects multimodal agents using a simulated dataset and three perception patterns. The study also examines adversarial environment injection and possible ways to improve instruction faithfulness.
OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation
Evaluates language agents on office workflows requiring information integration, long-term planning, and switching between applications.
桌面
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
Introduces simulated application APIs and state-based task evaluation for interactive coding agents performing everyday digital workflows.
相关研究
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?
Introduces realistic open-web tasks and the SeePlanAct agent, analyzing the limitations of browsing and retrieval in time-consuming assistant work.
网页
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Benchmarks multimodal agents on data-science and engineering workflows involving enterprise applications, code, and GUI operations in real computer environments.
桌面网页
Systematic Categorization, Construction and Evaluation of New Attacks against Multi-modal Mobile GUI Agents
Develops a threat taxonomy and evaluation framework for attacks against multimodal mobile GUI agents, with case studies and dataset experiments.
AUITestAgent: Automatic Requirements Oriented GUI Function Testing
Automates mobile GUI tests from natural-language requirements, combining action generation with verification based on interaction traces.
UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset
Introduces designer-authored mobile UI critiques and quality ratings, studying how targeted examples improve automated design feedback.
相关研究
WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks
Defines compositional enterprise workflows to evaluate planning and reasoning, with generation of ground-truth interaction traces.
网页
Seeing is Believing: Vision-driven Non-crash Functional Bug Detection for Mobile Apps
Trident combines exploration, monitoring, and detection agents to identify functional bugs that do not crash a mobile app. It reasons over screenshot sequences and interface transitions to construct test oracles, evaluated on 590 non-crash bugs and additional live apps.
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
Introduces CRAB, a framework for cross-environment agent tasks with graph-based evaluation and a benchmark spanning desktop and phone interactions.
桌面跨平台
Tree Search for Language Model Agents
Applies best-first search to agent actions within interactive web environments, allowing exploration and multistep planning at inference time. WebArena and VisualWebArena experiments examine search behavior and test-time computation.
网页
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
Evaluates mobile agents using combined UI and API actions, tasks with varying app scope, and checkpoints for intermediate progress.
Identifying User Goals from UI Trajectories
Defines goal identification from observed UI trajectories and an evaluation method for comparing intent descriptions within the same interface context. Experiments reuse Android and web automation data to compare human and model inference of users’ detailed goals.
Android网页跨平台
E-ANT: A Large-Scale Dataset for Efficient Automatic GUI NavigaTion
Introduces Chinese-language GUI navigation traces and annotated screenshots collected from human interactions with mini-applications.
WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks
Introduces WONDERBREAD, a benchmark pairing human WebArena workflow demonstrations with screen recordings, action logs, and written procedures. Six evaluation tasks cover documentation, knowledge transfer, and process improvement beyond end-to-end execution.
网页
Dissecting Adversarial Robustness of Multimodal LM Agents
Introduces adversarial tasks on VisualWebArena and an evaluation framework that tracks how manipulated information passes between agent components. The analysis examines vulnerabilities introduced by reflection and tree-search evaluators as well as direct action generation.
网页
WebCanvas: Benchmarking Web Agents in Online Environments
Introduces WebCanvas and Mind2Web-Live, evaluating online task progress through meaningful intermediate states on changing websites.
网页
GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
Introduces video-based GUI questions and annotations for studying temporal interface understanding across varied interaction settings.
跨平台
VideoGUI: A Benchmark for GUI Automation from Instructional Videos
Evaluates GUI automation from instructional videos at separate levels of high-level planning, action narration, and primitive execution.
桌面网页
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning
Introduces GuardAgent, which translates safety requests into executable guardrail code using plans and retrieved demonstrations. Mind2Web-SC supplies a web-agent safety-policy evaluation alongside a separate healthcare access-control benchmark.
网页相关研究
Practical, Automated Scenario-based Mobile App Testing
ScenTest builds scenario-specific event knowledge graphs from crowdsourced test reports containing GUI and action information. It uses the current app state and testing context to generate scenario tests across applications.
第 481–540 条，共 723 条资源。
通过主题导出清单获取完整记录和证据。克隆目录后，可将主题记录直接交给标准 RAG 与引用导出工具。