Advances in Large Language Model Robustness and Reliability

The field of large language models (LLMs) is currently focusing on improving their robustness and reliability. Researchers are exploring various methods to enhance the confidence estimation of LLMs, including data augmentation strategies and robustness evaluation techniques. Another key area of research is the development of methods to assess and improve the credibility of context documents used in LLM inference. Additionally, there is a growing interest in diagnosing data memorization in LLM-powered retrieval-augmented generation and approximating language model training data from weights. Notable papers in this area include Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions, which provides a comprehensive review of the robustness of LLMs, and CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision, which introduces a novel framework for assessing the credibility of context documents. RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation is also worth mentioning, as it presents a diagnostic method for detecting data memorization in LLMs.

Sources

The Effects of Data Augmentation on Confidence Estimation for LLMs

Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions

Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks

AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science

Intended Target Identification for Anomia Patients with Gradient-based Selective Augmentation

CrEst: Credibility Estimation for Contexts in LLMs via Weak Supervision

Context-Informed Grounding Supervision

RePCS: Diagnosing Data Memorization in LLM-Powered Retrieval-Augmented Generation

Approximating Language Model Training Data from Weights

Built with on top of