Advancements in Large Language Models

The field of large language models is moving towards more robust and efficient integration of parametric and in-context knowledge. Researchers are exploring ways to improve the arbitration between these two types of knowledge, with a focus on developing models that can learn from retrieval and parametric knowledge in a more harmonious way. This includes investigating the effects of training conditions on model behavior and designing frameworks that can systematically understand the updating mechanism of large language models. Another area of focus is on improving the safety and efficiency of in-context learning, with approaches such as risk control and agentic workflows being proposed. Noteworthy papers in this area include: KnowledgeSmith, which proposes a unified framework to systematically understand the updating mechanism of large language models. Safe and Efficient In-Context Learning via Risk Control, which presents a novel approach to limit the degree to which harmful demonstrations can degrade model performance. ContextNav, which integrates the scalability of automated retrieval with the quality and adaptiveness of human-like curation for multimodal in-context learning. AWARE, which introduces a framework that systematically attempts to improve a transformer model's awareness for identifying cultural capital in STEM narratives. Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding, which proposes the use of quality-aware decoding to effectively extract discourse knowledge from large language models. Opt-ICL at LeWiDi-2025, which outlines a system for modeling human variation using language models' in-context learning abilities and a two-step meta-learning training procedure.

Sources

Training Dynamics of Parametric and In-Context Knowledge Utilization in Language Models

KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning

Safe and Efficient In-Context Learning via Risk Control

ContextNav: Towards Agentic Multimodal In-Context Learning

AWARE, Beyond Sentence Boundaries: A Contextual Transformer Framework for Identifying Cultural Capital in STEM Narratives

Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding

Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learning

Built with on top of