تقرير
Continual Learning by Modeling Intra-Class Variation
| العنوان: | Continual Learning by Modeling Intra-Class Variation |
|---|---|
| المؤلفون: | Yu, Longhui, Hu, Tianyang, Hong, Lanqing, Liu, Zhen, Weller, Adrian, Liu, Weiyang |
| سنة النشر: | 2022 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition |
| الوصف: | It has been observed that neural networks perform poorly when the data or tasks are presented sequentially. Unlike humans, neural networks suffer greatly from catastrophic forgetting, making it impossible to perform life-long learning. To address this issue, memory-based continual learning has been actively studied and stands out as one of the best-performing methods. We examine memory-based continual learning and identify that large variation in the representation space is crucial for avoiding catastrophic forgetting. Motivated by this, we propose to diversify representations by using two types of perturbations: model-agnostic variation (i.e., the variation is generated without the knowledge of the learned neural network) and model-based variation (i.e., the variation is conditioned on the learned neural network). We demonstrate that enlarging representational variation serves as a general principle to improve continual learning. Finally, we perform empirical studies which demonstrate that our method, as a simple plug-and-play component, can consistently improve a number of memory-based continual learning methods by a large margin. Comment: Published in Transactions on Machine Learning Research (25 pages, 13 figures) |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2210.05398 |
| رقم الأكسشن: | edsarx.2210.05398 |
| قاعدة البيانات: | arXiv |
كن أول من يترك تعليقا!