The field of continual learning is moving towards developing more efficient and effective methods for preserving previously learned knowledge while adapting to new tasks. Recent research has focused on improving the stability and plasticity of models, enabling them to learn from a stream of data without requiring full retraining or data replay. Notable advancements include the development of novel architectures, such as dual-network designs and hierarchical layer-grouped prompt tuning, which have shown promising results in mitigating catastrophic forgetting. Additionally, researchers have explored the use of parameter-efficient tuning methods, loss-aware sampling strategies, and rehearsal enhancement mechanisms to improve the performance of continual learning models. Overall, the field is progressing towards creating more robust and adaptable models that can learn continuously without forgetting previously acquired knowledge. Noteworthy papers include RETROFIT, which achieves bounded forgetting for effective knowledge transfer, and Learning with Preserving, which maintains the geometric structure of the shared representation space to retain implicit knowledge.
Continual Learning Advances
Sources
Retrofit: Continual Learning with Bounded Forgetting for Security Applications
Learning with Preserving for Continual Multitask Learning
FSC-Net: Fast-Slow Consolidation Networks for Continual Learning
KAN/H: Kolmogorov-Arnold Network using Haar-like bases
CITADEL: A Semi-Supervised Active Learning Framework for Malware Detection Under Continuous Distribution Drift
Teaching Prompts to Coordinate: Hierarchical Layer-Grouped Prompt Tuning for Continual Learning
Convolutional Model Trees
Efficient Adversarial Malware Defense via Trust-Based Raw Override and Confidence-Adaptive Bit-Depth Reduction
Catastrophic Forgetting in Kolmogorov-Arnold Networks
Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
KANGURA: Kolmogorov-Arnold Network-Based Geometry-Aware Learning with Unified Representation Attention for 3D Modeling of Complex Structures
PolyKAN: Efficient Fused GPU Operators for Polynomial Kolmogorov-Arnold Network Variants
LFreeDA: Label-Free Drift Adaptation for Windows Malware Detection
Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance
Parameter Importance-Driven Continual Learning for Foundation Models
Learning from Mistakes: Loss-Aware Memory Enhanced Continual Learning for LiDAR Place Recognition