The field of reinforcement learning is moving towards a greater emphasis on fairness and causality. Researchers are recognizing the importance of considering long-term fairness in dynamic decision-making systems, rather than just focusing on immediate bias in static contexts. This involves developing algorithms that can balance competing fairness notions and avoid discriminatory decision outcomes.
Causality is also becoming a key aspect of reinforcement learning, as researchers seek to develop agents that can understand the underlying causes of their actions and outcomes. This includes identifying and mitigating spurious correlations between rewards and observations, as well as developing causal state representations that can improve out-of-trajectory performance.
Furthermore, there is a growing trend towards more efficient and effective reward modeling, including approaches that eliminate the need for extensive human-annotated preference data.
Noteworthy papers include:
- A Causal Lens for Learning Long-term Fair Policies, which proposes a framework for measuring long-term fairness in dynamic decision-making systems.
- Breaking Habits: On the Role of the Advantage Function in Learning Causal State Representations, which shows that the advantage function can mitigate the effects of policy confounding.
- Fake it till You Make it: Reward Modeling as Discriminative Prediction, which proposes an efficient reward modeling framework that eliminates manual preference annotation.
- Efficient and Generalizable Environmental Understanding for Visual Navigation, which introduces a causal framework for visual navigation and proposes a Causality-Aware Navigation approach.
