The field of deep learning is moving towards incorporating more structured and algebraic principles into its models, enabling more efficient and interpretable learning in complex domains. This trend is driven by the need to improve performance in tasks that involve hierarchical or relational data, such as natural language processing, biological sequence analysis, and graph neural networks. Recent work has focused on developing new architectures and frameworks that combine geometric and algebraic principles with attention mechanisms, leading to more robust and generalizable models. Notable papers in this area include: Graded Transformers: A Symbolic-Geometric Approach to Structured Learning, which introduces a novel class of sequence models that embed algebraic inductive biases through grading transformations on vector spaces. Computational Advantages of Multi-Grade Deep Learning: Convergence Analysis and Performance Insights, which investigates the computational advantages of multi-grade deep learning and establishes convergence results for the gradient descent method. Abstractions of Sequences, Functions and Operators, which presents theoretical and practical results on the order theory of lattices of functions, focusing on Galois connections that abstract sets of functions.