Hierarchical Representations and Emergent Complexity in AI Systems

The field of artificial intelligence is witnessing a significant shift towards the development of hierarchical representations and emergent complexity in AI systems. Researchers are increasingly focusing on creating models that can capture multiscale structure and organize information in a more transparent and interpretable manner. This is evident in the development of frameworks that use hierarchical prototypes, concept trees, and multiscale features to improve the robustness and scalability of AI systems. Furthermore, there is a growing interest in understanding how complex systems emerge and how they can be engineered to exhibit specific properties. Noteworthy papers in this regard include the introduction of Cobweb, a hierarchy-aware framework for neural document retrieval, and the development of MindCraft, a framework for analyzing the hierarchical emergence of concepts in deep models. Additionally, the paper on Engineering Emergence provides a mathematical framework for analyzing the causal contributions across the full multiscale structure of a system, allowing for the classification of systems as being causally top-heavy or bottom-heavy.

Sources

Hierarchical Semantic Retrieval with Cobweb

Engineering Emergence

MindCraft: How Concept Trees Take Shape In Deep Models

CoPA: Hierarchical Concept Prompting and Aggregating Network for Explainable Diagnosis

Multiplicative Turing Ensembles, Pareto's Law, and Creativity

Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy

From Segments to Concepts: Interpretable Image Classification via Concept-Guided Segmentation

Associative Memory Model with Neural Networks: Memorizing multiple images with one neuron

Explaining raw data complexity to improve satellite onboard processing

Concept Retrieval -- What and How?

Enhancing Concept Localization in CLIP-based Concept Bottleneck Models

Built with on top of