# FormFactory: An Interactive Benchmarking Suite for Multimodal Form-Filling Agents

Introduces FormFactory, an interactive web-based benchmark for agents filling online forms, with varied field formats and evaluation of instruction-to-field alignment.

- Catalog URL: https://awesomecua.com/resources/arxiv-2506-01520/
- Original source: https://arxiv.org/abs/2506.01520
- Kind: paper
- Review: curated
- Scope: core
- Environments: web
- Topics: evaluation, screen-understanding
- Evidence checked: 2026-11-05T23:01:45Z
- ID: arxiv:2506.01520

## Limitations

The annotation and scope were reviewed against the complete retained arXiv abstract (v1). The evaluated MLLMs did not exceed 5% accuracy on this benchmark. Experiments, reported performance and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.com/resources/arxiv-2506-01520/record.json

## Authors

Bobo Li, Yuheng Wang, Hao Fei, Juncheng Li, Wei Ji, Mong-Li Lee, Wynne Hsu

## Citation exports

- BibTeX: https://awesomecua.com/resources/arxiv-2506-01520/citation.bib
- CSL JSON: https://awesomecua.com/resources/arxiv-2506-01520/citation.csl.json

## Recorded arXiv snapshot

- Version: 1
- Version source: https://arxiv.org/abs/2506.01520v1
- First submission: 2025-06-02T10:34:57Z
- Version updated: 2025-06-02T10:34:57Z
- Abstract SHA-256: d76a3740a32d528b3ee123b1b504be0ade628c6c44b842d835fddaa68ba649f7
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2506.01520&max_results=1 (retrieved 2026-11-01T06:06:08Z; SHA-256 3bdedda702491a341bc9a2809baf59f66fd76898b3632d9f4d6f921e07c7005e)
- https://export.arxiv.org/api/query?id_list=2505.05262%2C2505.05849%2C2505.05895%2C2505.06507%2C2505.07003%2C2505.07062%2C2505.07473%2C2505.10468%2C2505.10887%2C2505.10978%2C2505.11717%2C2505.12053%2C2505.12490%2C2505.12728%2C2505.13291%2C2505.13887%2C2505.14030%2C2505.14454%2C2505.14646%2C2505.14963%2C2505.15801%2C2505.15816%2C2505.15859%2C2505.17020%2C2505.17629%2C2505.17735%2C2505.18746%2C2505.19713%2C2505.20023%2C2505.20075%2C2505.20732%2C2505.20795%2C2505.21055%2C2505.22304%2C2505.22429%2C2505.22648%2C2505.23450%2C2505.23559%2C2505.23678%2C2505.24298%2C2505.24876%2C2506.00520%2C2506.00641%2C2506.01423%2C2506.01520%2C2506.01952%2C2506.02153%2C2506.03231%2C2506.03569%2C2506.04301&max_results=50 (retrieved 2026-11-05T23:01:45Z; SHA-256 72d781966615fe4da127a9c089d5d6ab85838feb4205ef1ebd446ef97189bf70)