Advances in Large Language Models and Information Retrieval

The field of natural language processing and information retrieval is witnessing significant developments, driven by the increasing capabilities of large language models (LLMs). Researchers are exploring innovative applications of LLMs, such as semantic search, question generation, and critical thinking evaluation. A key trend is the creation of specialized datasets and benchmarks to assess the performance of LLMs in specific domains, including academic search and scientific research. These efforts aim to enhance the ability of LLMs to support complex information retrieval tasks and foster deeper reasoning. Noteworthy papers in this area include LeanExplore, which introduces a search engine for Lean 4 declarations, and ScIRGen, which develops a framework for generating realistic scientific question-answering datasets. Other notable works include AcademicBrowse, which proposes a benchmark for evaluating LLMs' academic search capabilities, and Essential-Web v1.0, which presents a large-scale, organized web dataset for pre-training LLMs.

Sources

LeanExplore: A search engine for Lean 4 declarations

ScIRGen: Synthesize Realistic and Large-Scale RAG Dataset for Scientific Research

AbsenceBench: Language Models Can't Tell What's Missing

AcademicBrowse: Benchmarking Academic Browse Ability of LLMs

Essential-Web v1.0: 24T tokens of organized web data

ELLIS Alicante at CQs-Gen 2025: Winning the critical thinking questions shared task: LLM-based question generation and selection

Built with on top of