The field of natural language processing is witnessing significant developments in preference optimization for language models. Researchers are exploring innovative approaches to improve the alignment of language models with human preferences, enabling more effective and efficient fine-tuning of these models. A key direction in this area is the development of methods that can learn from paired preference data, allowing for simpler and cheaper open recipes for state-of-the-art post-training. Another important trend is the investigation of divergence minimization perspectives for aligning diffusion models with human preferences, which has led to the introduction of novel and principled methods for preference optimization. Additionally, there is a growing interest in analyzing the theoretical properties and intrinsic limitations of direct preference optimization, leading to the proposal of bilevel optimization frameworks and selective alignment strategies. These advancements have the potential to unlock more robust and elegant pathways for preference alignment, bridging the gap between principled theory and practical performance in language models. Noteworthy papers in this area include: The Delta Learning Hypothesis, which introduces a novel hypothesis to explain the phenomenon of paired preference data enabling gains beyond the strength of individual data points. Divergence Minimization Preference Optimization for Diffusion Model Alignment, which proposes a novel method for aligning diffusion models by minimizing reverse KL divergence. Stable Preference Optimization for LLMs, which presents a theoretically grounded bilevel optimization framework for stable preference optimization. Not All Preferences are What You Need for Post-Training, which introduces a selective alignment strategy that prioritizes high-impact tokens within preference pairs.