The field of diffusion models and image generation is rapidly advancing, with a focus on improving the alignment of generated images with human preferences and safety constraints. Recent research has explored the use of reinforcement learning and reward modeling to fine-tune diffusion models, resulting in significant improvements in image quality and diversity. Noteworthy papers include DetailFusion, which proposes a dual-branch framework for composed image retrieval, and DiffusionReward, which introduces a reward feedback learning framework for blind face restoration. Other notable papers include QuARI, which explores query adaptive retrieval improvement, and Reference-Guided Identity Preserving Face Restoration, which proposes a novel approach for preserving face identity in diffusion-based image restoration.
Advances in Diffusion Models and Image Generation
Sources
Alignment and Safety of Diffusion Models via Reinforcement Learning and Reward Modeling: A Survey
DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrieval
DiffusionReward: Enhancing Blind Face Restoration through Reward Feedback Learning
QuARI: Query Adaptive Retrieval Improvement
Reference-Guided Identity Preserving Face Restoration
Cross-modal RAG: Sub-dimensional Retrieval-Augmented Text-to-Image Generation
D-Fusion: Direct Preference Optimization for Aligning Diffusion Models with Visually Consistent Samples
Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generation
ProCrop: Learning Aesthetic Image Cropping from Professional Compositions
ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models
HiGarment: Cross-modal Harmony Based Diffusion Model for Flat Sketch to Realistic Garment Image
Fine-Tuning Next-Scale Visual Autoregressive Models with Group Relative Policy Optimization
Diffusion Sampling Path Tells More: An Efficient Plug-and-Play Strategy for Sample Filtering
LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter