The field of scientific language models is rapidly evolving, with a growing focus on developing models that can effectively reason over structured, human-readable knowledge. Recent research has highlighted the importance of providing high-level context to scientific language models, rather than relying solely on raw sequence data. This approach has been shown to significantly improve performance on biological reasoning tasks, and has the potential to enable the development of more powerful and generalizable models. Another key area of research is the identification of knowledge gaps in scientific literature, with large language models demonstrating a robust ability to identify both explicit and implicit gaps. This capability has significant implications for supporting early-stage research formulation, policymaking, and funding decisions. Noteworthy papers in this area include:

• Lost in Tokenization, which challenges the sequence-centric paradigm in scientific language models and proposes a context-only approach.
• GAPMAP, which introduces a novel task of inferring implicit knowledge gaps in biomedical literature and demonstrates the effectiveness of large language models in this task.