Advances in Prompt Optimization and Continual Learning for Large Language Models

The field of natural language processing is witnessing significant advancements in prompt optimization and continual learning for large language models. Recent studies have focused on developing innovative methods to improve the performance of these models, including the use of semantic clustering, boundary analysis, and iterative refinement to select representative and diverse samples for prompt optimization. Additionally, researchers have proposed novel approaches to mitigate catastrophic forgetting in continual relation extraction, such as using task-specific prompt pools and incorporating label descriptions to provide richer context. Another area of research has explored the use of reinforcement learning to generate novel prompts that can dramatically enhance model performance. Furthermore, some studies have investigated the application of meta-learning to understand prompt tuning and in-context learning, providing insights into the fundamental limitations of prompting and the potential benefits of weight tuning. Noteworthy papers in this area include:

  • IPOMP, which achieves state-of-the-art results in prompt optimization using a two-stage approach with semantic clustering and boundary analysis.
  • PRL, which introduces a novel RL-based approach for automatic prompt generation that produces novel few-shot examples.
  • WAVE++, which proposes a task-specific prompt pool approach to capture within-task variance and mitigate catastrophic forgetting in continual relation extraction.
  • Latent Principle Discovery, which automates the process of discovering latent principles guiding model reasoning and enables smaller language models to self-improve.
  • Understanding Prompt Tuning and In-Context Learning via Meta-Learning, which provides a Bayesian view of optimal prompting and its limitations.

Sources

Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization

Temporal fine-tuning for early risk detection

Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting

PRL: Prompts from Reinforcement Learning

Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning

Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning

PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models

Latent Principle Discovery for Language Model Self-Improvement

Understanding Prompt Tuning and In-Context Learning via Meta-Learning

Built with on top of