The field of complex system analysis and interpretability is moving towards a deeper understanding of the underlying dynamics and structures of complex systems. Recent research has focused on developing new methodologies and frameworks for analyzing and modeling complex systems, such as representing transformers as complex networks and using category theory to formalize system dynamics modeling. These advances have led to a better understanding of the self-organizing principles that drive the formation of functional circuits in large language models and the identification of universal correspondences between local connectivity and dynamic energy in complex networks. Noteworthy papers include:
- Modeling Transformers as complex networks to analyze learning dynamics, which introduces a novel methodology for representing transformers as directed, weighted graphs and reveals distinct phases of exploration, consolidation, and refinement in the network's structure.
- Compositional System Dynamics, which establishes a robust mathematical foundation for compositional system dynamics modeling and facilitates the identification of certain forms of pathways and feedback loops.
- Deterministic Frequency--Domain Inference of Network Topology and Hidden Components, which develops a deterministic, frequency-domain inference framework for reconstructing network topology directly from payoff sequences and simultaneously localizes individual hidden nodes or identifies all edges connected to multiple hidden nodes.