从 GUI 定位到智能体训练、评测与安全，每篇都可追溯到原始论文。
阅读英文研究综述 ↗ · 方法、评测与人的控制权；英文工作稿附有逐篇阅读记录。
条目经过 AI 辅助的一手来源编辑审阅，实验未独立复现。发表日期保留来源记录的精度。下方资源链接来自已审阅的介绍或文档关系。 收录方法 ↗
Introduces Rousillon, which generalizes a demonstration of collecting one hierarchical data row into a browser-automation program that traverses related pages. The paper develops relation-selection and generalization methods and evaluates users building such scripts.
Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology ↗
Introduces Rico, collecting Android interface designs and interaction traces through crowdsourced use followed by automated exploration. The paper demonstrates UI-layout retrieval using learned representations and describes visual, textual, structural, and interaction data.
UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology ↗
Introduces World of Bits, a web-task platform combining human demonstrations, keyboard and mouse actions, and cached HTTP traffic for repeatable task environments. It evaluates agents trained through behavioral cloning and reinforcement learning.
Proceedings of the 34th International Conference on Machine Learning ↗
Introduces SUGILITE, which combines spoken instructions, demonstrated Android actions, and UI hierarchies to generate reusable automation scripts. Its interface includes context checking and repairs for unfamiliar situations and changing app interfaces.
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems ↗
Introduces Ringer, which converts demonstrated webpage interactions into scripts combining actions, target selectors, and waiting conditions. It evaluates replay robustness on tasks recorded from real websites.
Introduces WebNav for converting websites into goal-directed hyperlink navigation tasks and WikiNav for evaluating language-conditioned navigation and question answering on Wikipedia.
Presents visual GUI test scripts that use images to specify actions and expected screen feedback. Recording both input events and screenshots can generate tests from demonstrations, supporting unit, regression, and test-driven workflows.
Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI '10) ↗
Introduces screenshot-based GUI search and visual scripting, using image patterns to locate interface targets and direct mouse and keyboard events. It evaluates screenshot queries and demonstrates automation and interactive-help tasks.
Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09) ↗
Introduces Koala, a collaborative web automation system that records browser interactions as editable pseudo-natural-language scripts. Scripts are shared through a wiki and instantiated with personal or corporate data for reuse by other users.
Describes PLOW, which learns executable web-task models from a session combining demonstrations, natural-language explanations, and dialogue. Browser instrumentation connects user actions with DOM structure to support task generalization.
Twenty-Second AAAI National Conference on Artificial Intelligence ↗
Describes Eager, which detects repeated HyperCard interactions, generalizes them into loops, and previews predicted actions by highlighting interface objects. Users can then ask it to execute the inferred repetitive task.
第 1,381–1,391 篇，共 1,391 篇，按发表时间由新到旧排列。