The field of non-commutative monoidal structures and transformer architectures is experiencing significant growth, with a focus on developing novel frameworks that can effectively model complex data. Researchers are exploring the use of non-commutative operators and commutative generators to create directional composition operators that can preserve structural coherence. These structures have been shown to unify several well-known linear transforms in signal processing and data analysis, and can be used to derive learnable transformations tailored to specific data modalities and tasks. Additionally, the development of scalable and robust rotary position embedding methods is improving the performance of transformer models. Notable papers in this area include:

• Directional Non-Commutative Monoidal Structures with Interchange Law via Commutative Generators, which introduces a novel framework for modeling directional composition in multiple dimensions.
• ComRoPE: Scalable and Robust Rotary Position Embedding Parameterized by Trainable Commuting Angle Matrices, which proposes a method for generalizing rotary position encoding using trainable commuting angle matrices.