The field of large language models is moving towards more efficient and effective fine-tuning methods, with a focus on in-context learning and knowledge distillation. Recent research has shown that many-shot in-context fine-tuning can significantly narrow the performance gap between few-shot and dedicated fine-tuning. Additionally, theoretical frameworks have been proposed to explain the mechanisms behind in-context learning, highlighting the importance of prompt engineering and demonstration selection. Other notable developments include the use of token-level reward guidance for direct preference optimization and the introduction of novel training objectives that treat every answer within the context as a supervised training target. Noteworthy papers include: Many-Shot In-Context Fine-tuning for Large Language Model, which proposes a novel approach to many-shot in-context fine-tuning. Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning, which provides a theoretical framework for understanding in-context learning as a form of knowledge distillation.
Advances in Large Language Model Fine-Tuning and Knowledge Distillation
Sources
You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Model
Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
Dataset distillation for memorized data: Soft labels can leak held-out teacher knowledge
TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
SFT-GO: Supervised Fine-Tuning with Group Optimization for Large Language Models
Sequential Policy Gradient for Adaptive Hyperparameter Optimization
A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals