The field of natural language processing is witnessing significant developments in retrieval-augmented generation (RAG) and large language models (LLMs). Recent research has focused on improving the efficiency, accuracy, and robustness of RAG systems, as well as exploring new applications and defenses against potential attacks. Notably, innovations in dynamic token-level prefix augmentation, parametric-verified adaptive information retrieval, and balanced entropy engineering have enhanced the performance of RAG systems. Furthermore, the integration of multimodal knowledge graphs and graph-aware LLMs has shown promise in visual question answering and other tasks. Researchers have also investigated defenses against knowledge poisoning attacks, adversarial attacks, and privacy leakage, highlighting the importance of security and privacy in the development of RAG and LLMs. Overall, the field is moving towards more efficient, accurate, and robust models that can effectively incorporate external knowledge and mitigate potential risks. Some noteworthy papers include MMRAG-DocQA, which proposes a novel multi-modal RAG model for document question-answering, and DAEDAL, which introduces a dynamic adaptive length expansion method for diffusion large language models. Additionally, PAIRS and BEE-RAG have demonstrated significant improvements in RAG efficiency and performance.
Advancements in Retrieval-Augmented Generation and Large Language Models
Sources
GETALP@AutoMin 2025: Leveraging RAG to Answer Questions based on Meeting Transcripts
MMRAG-DocQA: A Multi-Modal Retrieval-Augmented Generation Method for Document Question-Answering with Hierarchical Index and Multi-Granularity Retrieval
Beyond Fixed: Variable-Length Denoising for Diffusion Large Language Models
Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation
Highlight & Summarize: RAG without the jailbreaks
Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation
Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation
Majority Bit-Aware Watermarking For Large Language Models
Two-dimensional Sparse Parallelism for Large Scale Deep Learning Recommendation Model Training
DTPA: Dynamic Token-level Prefix Augmentation for Controllable Text Generation
PAIRS: Parametric-Verified Adaptive Information Retrieval and Selection for Efficient RAG
A Few Words Can Distort Graphs: Knowledge Poisoning Attacks on Graph-based Retrieval-Augmented Generation of Large Language Models
Scaling Generative Recommendations with Context Parallelism on Hierarchical Sequential Transducers
Adversarial Attacks and Defenses on Graph-aware Large Language Models (LLMs)
BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generation
QA-Dragon: Query-Aware Dynamic RAG System for Knowledge-Intensive Visual Question Answering
mKG-RAG: Multimodal Knowledge Graph-Enhanced RAG for Visual Question Answering
CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generation