Advancements in Query Optimization and Storage Systems

The field of query optimization and storage systems is moving towards more adaptive, autonomous, and dynamic architectures. Researchers are exploring innovative approaches to improve query performance, such as leveraging large language models to infer workload intent and generate actionable configurations. Another key trend is the expansion of optimization scope from single queries to entire workloads, enabling more holistic and efficient decision-making. Additionally, the industry is shifting towards composable architectures that foster agility and cross-engine collaboration. Noteworthy papers in this area include:

  • Intent-Driven Storage Systems, which proposes a new paradigm for storage systems that leverages large language models to adapt to workload intent.
  • This is Going to Sound Crazy, But What If We Used Large Language Models to Boost Automatic Database Tuning Algorithms, which presents the Booster framework for assisting existing tuners in adapting to environment changes.
  • Query Optimization in the Wild: Realities and Trends, which highlights key trends in the industry, including the use of tighter feedback loops and composable architectures.

Sources

Optimizing Data Lakes' Queries

Intent-Driven Storage Systems: From Low-Level Tuning to High-Level Understanding

This is Going to Sound Crazy, But What If We Used Large Language Models to Boost Automatic Database Tuning Algorithms By Leveraging Prior History? We Will Find Better Configurations More Quickly Than Retraining From Scratch!

Query Optimization in the Wild: Realities and Trends

Built with on top of