The field of large language models (LLMs) is witnessing significant developments in alignment and efficiency. Researchers are exploring innovative methods to improve LLMs' ability to align with human preferences, values, and intentions. One notable direction is the use of principled data selection strategies, which enable more efficient and effective alignment with limited resources. Another area of focus is the development of lightweight and scalable frameworks for post-training and fine-tuning, allowing for improved reasoning capabilities and personalized preference alignment. These advancements have the potential to unlock more accurate, helpful, and honest content generation in LLMs. Noteworthy papers include: LECTOR, which achieves a 90.2% success rate in test-oriented learning scenarios, and InfiAlign, which demonstrates strong generalization across diverse reasoning tasks while using only approximately 12% of the training data. Additionally, P-Aligner and FaST show promise in pre-aligning instructions and personalized preference alignment, respectively.
Advancements in Large Language Model Alignment and Efficiency
Sources
LECTOR: LLM-Enhanced Concept-based Test-Oriented Repetition for Adaptive Spaced Learning
Fine-tuning for Better Few Shot Prompting: An Empirical Comparison for Short Answer Grading
Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
P-Aligner: Enabling Pre-Alignment of Language Models via Principled Instruction Synthesis
FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data
SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models
Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models
RLHF Fine-Tuning of LLMs for Alignment with Implicit User Feedback in Conversational Recommenders
InfiAlign: A Scalable and Sample-Efficient Framework for Aligning LLMs to Enhance Reasoning Capabilities