The field of information retrieval is witnessing significant advancements with the development of innovative transformer-based architectures. Researchers are exploring new mathematical frameworks for inference, leveraging optimal control approaches to derive transformer-like architectures that solve causal nonlinear prediction problems. Another direction of research focuses on optimizing retrieval models for long-tail search queries, utilizing large language models and pretraining transformer-based models on domain-specific data. Additionally, there is a growing interest in generative ranking systems, which have shown promise in improving user satisfaction and efficiency in large-scale industrial settings. Noteworthy papers in this area include:

• Dual Filter, which presents a mathematical framework for inference using transformer-like architectures.
• LONGER, which introduces a novel transformer-based architecture for scaling up long sequence modeling in industrial recommenders.
• LT-TTD, which proposes a unified transformer-based architecture for two-level ranking systems, providing theoretical guarantees for error propagation mitigation and ranking quality improvements.