Advances in Low-Data Regimes and Efficient Fine-Tuning of Large Language Models

The field of large language models (LLMs) is moving towards addressing the challenges of low-data regimes and efficient fine-tuning. Researchers are exploring innovative methods to adapt LLMs to new distributions and domains with limited data, such as knowledge distillation and selective parameter evaluation. These approaches aim to mitigate catastrophic forgetting and enable generalization in low-data settings. Notable papers in this area include:

  • A study on grokking, which highlights the value of knowledge distillation for deployed models that must adapt to new distributions under limited data.
  • The introduction of LOREN, a curvature-aware zeroth-order optimization method for fine-tuning LLMs, which achieves higher accuracy and faster convergence while reducing peak memory usage.
  • The proposal of SPEAR-MM, a practical framework that preserves critical capabilities while enabling domain adaptation for financial LLMs, achieving 91.2% retention of general capabilities and reducing computational costs by 90%.
  • The development of GrADS, a self-adaptive gradient-aware data selection approach for supervised fine-tuning of LLMs, which enables the acquisition of representative samples and mitigates catastrophic forgetting.

Sources

When Data Falls Short: Grokking Below the Critical Threshold

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning

SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation

Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM

Built with on top of