The field of large language models is moving towards more efficient and effective methods for compressing and distilling knowledge. Researchers are exploring novel approaches to reduce the computational costs and improve the performance of these models. One key direction is the development of dynamic compressing prompts, which aim to retain essential information while adapting to context changes. Another important area of research is the application of knowledge distillation techniques to transfer knowledge from large models to smaller ones, with a focus on preserving the structural integrity of the models. Additionally, there is a growing interest in using large language models as executable agents for extracting structured information from complex websites. Noteworthy papers in this area include:

• A paper proposing a dynamic compressing prompts method that outperforms state-of-the-art techniques, especially at higher compression rates.
• A paper introducing a novel group-aware pruning strategy for compressing hybrid language models, achieving improved accuracy and inference speed.
• A paper presenting a dual-space knowledge distillation framework that unifies the prediction heads of teacher and student models, supporting knowledge distillation between models with different vocabularies.
• A paper proposing a framework for extracting structured information from complex websites using executable language models, achieving significant improvements in performance and cost reduction.