Advances in Complex System Analysis and Interpretability

The field of complex system analysis and interpretability is moving towards a deeper understanding of the underlying dynamics and structures of complex systems. Recent research has focused on developing new methodologies and frameworks for analyzing and modeling complex systems, such as representing transformers as complex networks and using category theory to formalize system dynamics modeling. These advances have led to a better understanding of the self-organizing principles that drive the formation of functional circuits in large language models and the identification of universal correspondences between local connectivity and dynamic energy in complex networks. Noteworthy papers include:

  • Modeling Transformers as complex networks to analyze learning dynamics, which introduces a novel methodology for representing transformers as directed, weighted graphs and reveals distinct phases of exploration, consolidation, and refinement in the network's structure.
  • Compositional System Dynamics, which establishes a robust mathematical foundation for compositional system dynamics modeling and facilitates the identification of certain forms of pathways and feedback loops.
  • Deterministic Frequency--Domain Inference of Network Topology and Hidden Components, which develops a deterministic, frequency-domain inference framework for reconstructing network topology directly from payoff sequences and simultaneously localizes individual hidden nodes or identifies all edges connected to multiple hidden nodes.

Sources

Modeling Transformers as complex networks to analyze learning dynamics

Localmax dynamics for attention in transformers and its asymptotic behavior

Compositional System Dynamics: The Higher Mathematics Underlying System Dynamics Diagrams & Practice

Deterministic Frequency--Domain Inference of Network Topology and Hidden Components via Structure--Behavior Scaling

Uncovering Graph Reasoning in Decoder-only Transformers with Circuit Tracing

Built with on top of