The field of natural language processing is shifting towards the development of more efficient and specialized models. Researchers are exploring innovative methods to prune large language models, preserving their capabilities while reducing computational requirements. This direction is driven by the need for compact, expert models that can be tailored to specific downstream tasks without sacrificing general performance. Notable approaches include the use of attention head pruning, customized pruning methods, and contrastive learning frameworks to selectively remove redundant parameters and improve reasoning capabilities. Noteworthy papers include: Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT, which achieves high accuracy in metaphor and idiom classification using a pruned model. Pruning General Large Language Models into Customized Expert Models, which proposes a method to prune large models into smaller, task-specific expert models without post-training. APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training, which enhances domain-specific performance without degrading general capabilities. Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information, which achieves a high accuracy-speedup trade-off using latency and tunability information. Structured Pruning for Diverse Best-of-N Reasoning Optimization, which proposes a contrastive learning framework to dynamically select the optimal head and layer to prune during inference.