The field of AI/ML system monitoring and anomaly detection is rapidly evolving, with a focus on developing innovative methods for identifying and diagnosing performance issues. Recent research has explored the use of advanced techniques such as Gaussian Mixture Models, large language models, and physics-informed abstractions to improve the accuracy and efficiency of anomaly detection. These approaches have shown significant promise in detecting complex failure modes, hardware failures, and communication inefficiencies, and in providing actionable insights for performance optimization and fault diagnosis. Notably, the integration of large language models with graph databases and expert prompts has enabled the development of powerful tools for root cause analysis. Furthermore, the application of kernel-based approaches and statistical methods has improved the accuracy of steady-state detection in performance metric time series. Overall, these advancements have the potential to significantly enhance the reliability and performance of large-scale AI/ML systems.

Noteworthy papers include: eACGM, which presents a non-instrumented performance tracing and anomaly detection framework for machine learning systems. SynergyRCA, which introduces a novel tool for root cause analysis in Kubernetes using large language models and graph databases. LogSage, which provides an end-to-end LLM-powered framework for CI/CD failure detection and remediation. BEAR, which leverages large language models to automatically generate comprehensive reports explaining detected BGP anomaly events. KPIRoot+, which proposes an efficient integrated framework for anomaly detection and root cause analysis in large-scale cloud systems.