Diversity in Large Language Models

The field of large language models is moving towards increasing diversity in generated outputs, with a focus on developing novel decoding strategies and reinforcement learning frameworks. Researchers are exploring ways to modify token logits, introduce diversity signals, and optimize for semantic diversity, leading to significant improvements in output diversity and quality. Notable papers in this area include: Avoidance Decoding for Diverse Multi-Branch Story Generation, which proposes a decoding strategy that penalizes similarity to previously generated outputs, and Jointly Reinforcing Diversity and Quality in Language Model Generations, which introduces a framework that jointly optimizes for response quality and semantic diversity. Other noteworthy papers, such as VendiRL and Enhancing Diversity in Large Language Models via Determinantal Point Processes, demonstrate the effectiveness of self-supervised reinforcement learning and determinantal point processes in learning diverse skills and improving semantic diversity. Outcome-based Exploration for LLM Reasoning also presents a promising approach to mitigating diversity collapse in reinforcement learning. Overall, these advancements have the potential to significantly improve the usefulness of large language models in creative and exploratory tasks.

Sources

Avoidance Decoding for Diverse Multi-Branch Story Generation

Jointly Reinforcing Diversity and Quality in Language Model Generations

VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skills

Enhancing Diversity in Large Language Models via Determinantal Point Processes

Outcome-based Exploration for LLM Reasoning

Built with on top of