The field of large language models (LLMs) is currently focusing on improving their robustness and reliability. Researchers are exploring various methods to enhance the confidence estimation of LLMs, including data augmentation strategies and robustness evaluation techniques. Another key area of research is the development of methods to assess and improve the credibility of context documents used in LLM inference. Additionally, there is a growing interest in diagnosing data memorization in LLM-powered retrieval-augmented generation and approximating language model training data from weights. Notable papers in this area include Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions, which provides a comprehensive review of the robustness of LLMs, and CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision, which introduces a novel framework for assessing the credibility of context documents. RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation is also worth mentioning, as it presents a diagnostic method for detecting data memorization in LLMs.
Advances in Large Language Model Robustness and Reliability
Sources
The Effects of Data Augmentation on Confidence Estimation for LLMs
Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions
Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science
Intended Target Identification for Anomia Patients with Gradient-based Selective Augmentation
CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision
Context-Informed Grounding Supervision
RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation
Approximating Language Model Training Data from Weights