# Agentic Reinforcement Learning with Implicit Step Rewards

Introduces iStar, a general agentic RL method that learns implicit step rewards; WebShop is one of several interactive benchmarks used to evaluate it.

- Catalog URL: https://awesomecua.com/resources/arxiv-2509-19199/
- Original source: https://arxiv.org/abs/2509.19199
- Kind: paper
- Review: curated
- Scope: adjacent
- Environments: unspecified
- Topics: reinforcement-learning, reward-models, evaluation
- Evidence checked: 2026-11-05T23:02:00Z
- ID: arxiv:2509.19199

## Limitations

The annotation and scope were reviewed against the complete retained arXiv abstract (v3). WebShop is one evaluation domain alongside VisualSokoban and SOTOPIA; the method is not specific to web agents. Experiments, reported performance and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.com/resources/arxiv-2509-19199/record.json

## Authors

Xiaoqian Liu, Ke Wang, Yuchuan Wu, Fei Huang, Yongbin Li, Junge Zhang, Jianbin Jiao

## Citation exports

- BibTeX: https://awesomecua.com/resources/arxiv-2509-19199/citation.bib
- CSL JSON: https://awesomecua.com/resources/arxiv-2509-19199/citation.csl.json

## Recorded arXiv snapshot

- Version: 3
- Version source: https://arxiv.org/abs/2509.19199v3
- First submission: 2025-09-23T16:15:42Z
- Version updated: 2025-09-28T06:47:55Z
- Abstract SHA-256: e305addce9afbc29d17a0a5ffe2cf8968286ed9102a8975d4d0cdf348d353327
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2508.20410%2C2508.20890%2C2508.21456%2C2509.00088%2C2509.00531%2C2509.00969%2C2509.01684%2C2509.03536%2C2509.04018%2C2509.04908%2C2509.05368%2C2509.05831%2C2509.06917%2C2509.07098%2C2509.07553%2C2509.08820%2C2509.10345%2C2509.10540%2C2509.11253%2C2509.12543%2C2509.12987%2C2509.13615%2C2509.14547%2C2509.14744%2C2509.15061%2C2509.15237%2C2509.17336%2C2509.19185%2C2509.19199%2C2509.21072%2C2509.21291%2C2509.21766%2C2509.21799%2C2509.22715%2C2509.22830%2C2509.23738%2C2509.24704%2C2509.25455%2C2509.25693%2C2510.00172%2C2510.00515%2C2510.02190%2C2510.04491%2C2510.05096%2C2510.05156%2C2510.05788%2C2510.07284%2C2510.07915%2C2510.08787%2C2510.09517&max_results=50 (retrieved 2026-10-17T07:57:56Z; SHA-256 3429cc2fc7737050cac964c976293fc4793e52c5913636c1f67d04243fbefb6a)
- https://export.arxiv.org/api/query?id_list=2509.14547%2C2509.14744%2C2509.15061%2C2509.15237%2C2509.17336%2C2509.19185%2C2509.19199%2C2509.21072%2C2509.21291%2C2509.21733%2C2509.21766%2C2509.21799%2C2509.22644%2C2509.22715%2C2509.22830%2C2509.23694%2C2509.23738%2C2509.24704%2C2509.25301%2C2509.25455%2C2509.25693%2C2510.00078%2C2510.00172%2C2510.00507%2C2510.00515%2C2510.02190%2C2510.02204%2C2510.02608%2C2510.03204%2C2510.03485%2C2510.03612%2C2510.04452%2C2510.04491%2C2510.04536%2C2510.04673%2C2510.04791%2C2510.05096%2C2510.05137%2C2510.05156%2C2510.05788%2C2510.06223%2C2510.06607%2C2510.07284%2C2510.07532%2C2510.07915%2C2510.08787%2C2510.09116%2C2510.09517%2C2510.10049%2C2510.10587&max_results=50 (retrieved 2026-11-05T23:02:00Z; SHA-256 2343ea50b1761631c89f41f46494ffa8b9c95a6d6b1cf95d58720bf26271ce29)