Advances in Understanding and Improving Large Language Models

The field of Large Language Models (LLMs) is rapidly evolving, with a focus on understanding their internal working mechanisms and improving their capabilities. Recent research has investigated how multimodal knowledge evolves in LLMs, and how to quantify uncertainty in their responses. There is also a growing interest in developing methods to evaluate the similarity between LLMs and detecting potential vulnerabilities. Furthermore, researchers are exploring ways to improve the robustness and reliability of LLMs, particularly in high-stakes applications such as healthcare. Notable papers include: Towards Understanding How Knowledge Evolves in Large Vision-Language Models, which provides a fresh perspective on the evolution of multimodal knowledge in LLMs. Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models, which highlights the importance of uncertainty estimation in LLMs. Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models, which proposes a novel stability measure for LLMs and identifies salient parameters and vulnerable regions in input images. The challenge of uncertainty quantification of large language models in medicine, which proposes a comprehensive framework for managing uncertainty in LLMs for medical applications. Explaining Low Perception Model Competency with High-Competency Counterfactuals, which develops novel methods to generate high-competency counterfactual images to explain low model competency. Adapting GT2-FLS for Uncertainty Quantification: A Blueprint Calibration Strategy, which proposes a blueprint calibration strategy for efficient adaptation to any desired coverage level without retraining.

Sources

Towards Understanding How Knowledge Evolves in Large Vision-Language Models

Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models

LLMSched: Uncertainty-Aware Workload Scheduling for Compound LLM Applications

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models

Do "New Snow Tablets" Contain Snow? Large Language Models Over-Rely on Names to Identify Ingredients of Chinese Drugs

Language Models Are Implicitly Continuous

A Perplexity and Menger Curvature-Based Approach for Similarity Evaluation of Large Language Models

Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest

SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models

The challenge of uncertainty quantification of large language models in medicine

Explaining Low Perception Model Competency with High-Competency Counterfactuals

Adapting GT2-FLS for Uncertainty Quantification: A Blueprint Calibration Strategy

Built with on top of