تقرير
The Bearable Lightness of Big Data: Towards Massive Public Datasets in Scientific Machine Learning
| العنوان: | The Bearable Lightness of Big Data: Towards Massive Public Datasets in Scientific Machine Learning |
|---|---|
| المؤلفون: | Chung, Wai Tong, Jung, Ki Sung, Chen, Jacqueline H., Ihme, Matthias |
| المصدر: | ICML 2022 2nd AI for Science Workshop |
| سنة النشر: | 2022 |
| المجموعة: | Computer Science Physics (Other) |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Physics - Fluid Dynamics |
| الوصف: | In general, large datasets enable deep learning models to perform with good accuracy and generalizability. However, massive high-fidelity simulation datasets (from molecular chemistry, astrophysics, computational fluid dynamics (CFD), etc. can be challenging to curate due to dimensionality and storage constraints. Lossy compression algorithms can help mitigate limitations from storage, as long as the overall data fidelity is preserved. To illustrate this point, we demonstrate that deep learning models, trained and tested on data from a petascale CFD simulation, are robust to errors introduced during lossy compression in a semantic segmentation problem. Our results demonstrate that lossy compression algorithms offer a realistic pathway for exposing high-fidelity scientific data to open-source data repositories for building community datasets. In this paper, we outline, construct, and evaluate the requirements for establishing a big data framework, demonstrated at https://blastnet.github.io/, for scientific machine learning. Comment: Accepted in ICML 2022 2nd AI for Science Workshop. 10 pages, 8 figures |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2207.12546 |
| رقم الأكسشن: | edsarx.2207.12546 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |