تقرير
United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once
العنوان: | United We Pretrain, Divided We Fail! Representation Learning for Time Series by Pretraining on 75 Datasets at Once |
---|---|
المؤلفون: | Kraus, Maurice, Divo, Felix, Steinmann, David, Dhami, Devendra Singh, Kersting, Kristian |
سنة النشر: | 2024 |
المجموعة: | Computer Science |
مصطلحات موضوعية: | Computer Science - Machine Learning |
الوصف: | In natural language processing and vision, pretraining is utilized to learn effective representations. Unfortunately, the success of pretraining does not easily carry over to time series due to potential mismatch between sources and target. Actually, common belief is that multi-dataset pretraining does not work for time series! Au contraire, we introduce a new self-supervised contrastive pretraining approach to learn one encoding from many unlabeled and diverse time series datasets, so that the single learned representation can then be reused in several target domains for, say, classification. Specifically, we propose the XD-MixUp interpolation method and the Soft Interpolation Contextual Contrasting (SICC) loss. Empirically, this outperforms both supervised training and other self-supervised pretraining methods when finetuning on low-data regimes. This disproves the common belief: We can actually learn from multiple time series datasets, even from 75 at once. |
نوع الوثيقة: | Working Paper |
URL الوصول: | http://arxiv.org/abs/2402.15404 |
رقم الأكسشن: | edsarx.2402.15404 |
قاعدة البيانات: | arXiv |
كن أول من يترك تعليقا!