The field of reinforcement learning is moving towards more effective training of world models and web agents. Recent developments have focused on improving the utility of generative models by leveraging reinforcement learning with verifiable rewards. This approach has shown substantial performance gains in both language- and video-based world models across various domains. Additionally, there is a growing interest in developing specialized reward models for web navigation, which can be utilized during both training and test-time. These models have the potential to significantly improve the speed and cost-effectiveness of web agents. Furthermore, researchers are exploring new methods for planning and model-based reinforcement learning, including the use of temporally-extended actions and hierarchical planners. These advancements are enabling more efficient and effective planning, and improving the performance of web agents in complex tasks. Noteworthy papers include: RLVR-World, which presents a unified framework for training world models with reinforcement learning, and Web-Shepherd, which proposes a process reward model for web navigation that achieves state-of-the-art performance. Other notable papers include WebAgent-R1, which introduces an end-to-end multi-turn RL framework for training web agents, and Gaze Into the Abyss, which proposes a novel approach for planning to seek entropy when reward is scarce.