تقرير
Compositional Zero-Shot Domain Transfer with Text-to-Text Models
| العنوان: | Compositional Zero-Shot Domain Transfer with Text-to-Text Models |
|---|---|
| المؤلفون: | Liu, Fangyu, Liu, Qianchu, Bannur, Shruthi, Pérez-García, Fernando, Usuyama, Naoto, Zhang, Sheng, Naumann, Tristan, Nori, Aditya, Poon, Hoifung, Alvarez-Valle, Javier, Oktay, Ozan, Hyland, Stephanie L. |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language, Computer Science - Machine Learning |
| الوصف: | Label scarcity is a bottleneck for improving task performance in specialised domains. We propose a novel compositional transfer learning framework (DoT5 - domain compositional zero-shot T5) for zero-shot domain transfer. Without access to in-domain labels, DoT5 jointly learns domain knowledge (from MLM of unlabelled in-domain free text) and task knowledge (from task training on more readily available general-domain data) in a multi-task manner. To improve the transferability of task training, we design a strategy named NLGU: we simultaneously train NLG for in-domain label-to-data generation which enables data augmentation for self-finetuning and NLU for label prediction. We evaluate DoT5 on the biomedical domain and the resource-lean subdomain of radiology, focusing on NLI, text summarisation and embedding learning. DoT5 demonstrates the effectiveness of compositional transfer learning through multi-task learning. In particular, DoT5 outperforms the current SOTA in zero-shot transfer by over 7 absolute points in accuracy on RadNLI. We validate DoT5 with ablations and a case study demonstrating its ability to solve challenging NLI examples requiring in-domain expertise. Comment: Accepted at TACL, pre-MIT Press publication version. 16 pages, 4 figures |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2303.13386 |
| رقم الأكسشن: | edsarx.2303.13386 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |