دورية أكاديمية
Topological Persistence Guided Knowledge Distillation for Wearable Sensor Data.
| العنوان: | Topological Persistence Guided Knowledge Distillation for Wearable Sensor Data. |
|---|---|
| المؤلفون: | Jeon ES; Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State, University, Tempe, 85281, AZ, USA., Choi H; Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State, University, Tempe, 85281, AZ, USA., Shukla A; Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State, University, Tempe, 85281, AZ, USA., Wang Y; Department of Epidemiology and Biostatistics, University of South Carolina, Columbia, 29208, SC, USA., Lee H; School for Engineering of Matter, Transport and Energy, Tempe, 85281, AZ, USA., Buman MP; College of Health Solutions, Arizona State University, Phoenix, 85004, AZ, USA., Turaga P; Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State, University, Tempe, 85281, AZ, USA. |
| المصدر: | Engineering applications of artificial intelligence [Eng Appl Artif Intell] 2024 Apr; Vol. 130. Date of Electronic Publication: 2023 Dec 20. |
| نوع المنشور: | Journal Article |
| اللغة: | English |
| بيانات الدورية: | Publisher: Pineridge Periodicals Country of Publication: England NLM ID: 101469722 Publication Model: Print-Electronic Cited Medium: Print ISSN: 0952-1976 (Print) Linking ISSN: 09521976 NLM ISO Abbreviation: Eng Appl Artif Intell Subsets: PubMed not MEDLINE |
| أسماء مطبوعة: | Original Publication: Swansea [England] : Pineridge Periodicals, 1988- |
| مستخلص: | Deep learning methods have achieved a lot of success in various applications involving converting wearable sensor data to actionable health insights. A common application areas is activity recognition, where deep-learning methods still suffer from limitations such as sensitivity to signal quality, sensor characteristic variations, and variability between subjects. To mitigate these issues, robust features obtained by topological data analysis (TDA) have been suggested as a potential solution. However, there are two significant obstacles to using topological features in deep learning: (1) large computational load to extract topological features using TDA, and (2) different signal representations obtained from deep learning and TDA which makes fusion difficult. In this paper, to enable integration of the strengths of topological methods in deep-learning for time-series data, we propose to use two teacher networks - one trained on the raw time-series data, and another trained on persistence images generated by TDA methods. These two teachers are jointly used to distill a single student model, which utilizes only the raw time-series data at test-time. This approach addresses both issues. The use of KD with multiple teachers utilizes complementary information, and results in a compact model with strong supervisory features and an integrated richer representation. To assimilate desirable information from different modalities, we design new constraints, including orthogonality imposed on feature correlation maps for improving feature expressiveness and allowing the student to easily learn from the teacher. Also, we apply an annealing strategy in KD for fast saturation and better accommodation from different features, while the knowledge gap between the teachers and student is reduced. Finally, a robust student model is distilled, which can at test-time uses only the time-series data as an input, while implicitly preserving topological features. The experimental results demonstrate the effectiveness of the proposed method on wearable sensor data. The proposed method shows 71.74% in classification accuracy on GENEActiv with WRN16-1 (1D CNNs) student, which outperforms baselines and takes much less processing time (less than 17 sec) than teachers on 6k testing samples. Competing Interests: Declaration of interests The authors declare that they have no known competng fnancial interests or personal relatonships that could have appeared to influence the work reported in this paper. |
| References: | Science. 1983 May 13;220(4598):671-80. (PMID: 17813860) Conf Comput Vis Pattern Recognit Workshops. 2020 Jun;2020:3639-3648. (PMID: 32995068) Annu Int Conf IEEE Eng Med Biol Soc. 2016 Aug;2016:2631-2635. (PMID: 28268862) IEEE Internet Things J. 2022 Jul 15;9(14):12848-12860. (PMID: 35813017) Annu Int Conf IEEE Eng Med Biol Soc. 2020 Jul;2020:793-797. (PMID: 33018105) Conf Rec Asilomar Conf Signals Syst Comput. 2022 Oct-Nov;2022:837-842. (PMID: 37583442) IEEE Trans Med Imaging. 2011 Oct;30(10):1760-70. (PMID: 21536520) J Neurosci Methods. 2021 Nov 1;363:109324. (PMID: 34428514) |
| معلومات مُعتمدة: | R01 AR080826 United States AR NIAMS NIH HHS; R01 GM135927 United States GM NIGMS NIH HHS |
| فهرسة مساهمة: | Keywords: Deep learning; feature orthogonality; knowledge distillation; topological data analysis; wearable sensor data |
| تواريخ الأحداث: | Date Created: 20240129 Latest Revision: 20240131 |
| رمز التحديث: | 20240131 |
| مُعرف محوري في PubMed: | PMC10810240 |
| DOI: | 10.1016/j.engappai.2023.107719 |
| PMID: | 38282698 |
| قاعدة البيانات: | MEDLINE |
Full Text Finder | تدمد: | 0952-1976 |
|---|---|
| DOI: | 10.1016/j.engappai.2023.107719 |