The field of large language models (LLMs) is moving towards more efficient and private models. Recent research has focused on improving knowledge distillation methods to transfer conversational abilities from larger models to smaller ones, reducing memorization risks and computational costs. Noteworthy papers include daDPO, which introduces a unified method for preference optimization and distribution-based distillation, and From Teacher to Student, which demonstrates that distilling a larger teacher model into a smaller variant reduces memorization risks. Counterfactual Influence as a Distributional Quantity and Leaner Training, Lower Leakage also highlight the importance of considering the full influence distribution of training samples and the benefits of LoRA fine-tuning in reducing memorization risks. Complexity-aware fine-tuning is another area of interest, where identifying complex data and applying targeted fine-tuning methods can lead to significant performance improvements. Notable papers in this area include: daDPO, which enables a 20% pruned model to achieve near-teacher performance. From Teacher to Student, which shows that distilling a larger teacher model into a smaller variant reduces memorization risks. Leaner Training, Lower Leakage, which demonstrates that LoRA fine-tuning significantly reduces memorization risks compared to full fine-tuning.