AUITestAgent
Generates mobile GUI interactions from natural-language test requirements and checks application behavior against those requirements.
Automation
Source-reviewed research and assets tagged with Automation.
Membership follows reviewed tags; it does not establish a dependency between neighboring records. Dates preserve source precision. Annotations retain their original language.
COLA demo
Research demonstration of collaborative Windows GUI automation with dynamic task scheduling and recovery mechanisms.
DesktopWindows
GUI-HARVEST
Optimizes GUI-agent execution code from repeated rollout evidence and checks proposed changes against fixed validation gates.
Desktop
Lumen browser agent
A screenshot-driven browser agent over Chrome DevTools Protocol, with resumable sessions, context compression, action replay and configurable completion checks.
Web
Mobile Harness
Portable agent instructions and integration tooling for local or cloud phone control through mobilerun-core.
AndroidiOS
Mobile Next MCP
An MCP server exposing mobile accessibility snapshots, screenshots and device interactions across real devices, simulators and emulators.
AndroidiOS
Nanobrowser
A browser-extension agent that reads pages, interacts with elements and moves between tabs from a side-panel task interface.
Web
Nut.js
Desktop automation library for keyboard, mouse, and screen interaction.
DesktopAdjacent research
OculiX (SikuliX continuation)
A maintained continuation of SikuliX that locates on-screen targets with image matching and OCR, drives desktop or Android interfaces, and exposes scripting and optional MCP interfaces.
DesktopWindowsmacOSLinuxAndroid
OpenAgents Web Agent
The OpenAgents browser-control component, connecting a Chrome extension to a web agent and local chat platform for navigating sites and completing interface tasks.
Web
PC-Agent-E training and deployment
Tools for refining and augmenting human computer-use trajectories, preparing supervised training data, and deploying a native PC-Agent-E scaffold.
DesktopWindowsLinux
Playwright MCP
MCP server that lets LLM clients drive Playwright browsers through structured accessibility snapshots instead of screenshots, with opt-in vision, PDF and devtools capabilities.
Web
PyAutoGUI
Cross-platform Python module for scripted mouse and keyboard control, message boxes and screenshots with on-screen image search, working on the primary monitor under Windows, macOS and Linux.
DesktopAdjacent research
ShowUI-Aloha
Desktop agent that records demonstrations, derives semantic action traces, and uses them to plan and execute related workflows.
DesktopWindowsmacOS
Sikuli legacy research implementation
The original Sikuli source distribution combines screenshot targets with a Jython scripting library and IDE for visual GUI automation and testing.
DesktopmacOSLinuxWindows
STEVE / STEVE-R1 research code
Research code and linked models for step-verified computer-use training; the current repository describes STEVE-R1 and its trajectory releases.
DesktopWindowsLinux
Universe legacy GUI interaction client
An archived Gym-compatible client that exposes screen observations and keyboard/mouse actions over VNC for remote application and browser environments, including the historical World of Bits integration.
WebDesktopAdjacent research
Windows-MCP
A Windows MCP bridge exposing application, window, keyboard and mouse operations plus native UI-state inspection for agents.
WindowsDesktopWeb
Showing 121–141 of 141 resources.
Get full records and evidence through the topic export manifest. Source records can be passed to the standard RAG and citation exporters after cloning the catalog.