تقرير
CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data
| العنوان: | CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data |
|---|---|
| المؤلفون: | Turski, Michał, Stanisławek, Tomasz, Kaczmarek, Karol, Dyda, Paweł, Graliński, Filip |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language |
| الوصف: | In recent years, the field of document understanding has progressed a lot. A significant part of this progress has been possible thanks to the use of language models pretrained on large amounts of documents. However, pretraining corpora used in the domain of document understanding are single domain, monolingual, or nonpublic. Our goal in this paper is to propose an efficient pipeline for creating a big-scale, diverse, multilingual corpus of PDF files from all over the Internet using Common Crawl, as PDF files are the most canonical types of documents as considered in document understanding. We analysed extensively all of the steps of the pipeline and proposed a solution which is a trade-off between data quality and processing time. We also share a CCpdf corpus in a form or an index of PDF files along with a script for downloading them, which produces a collection useful for language model pretraining. The dataset and tools published with this paper offer researchers the opportunity to develop even better multilingual language models. Comment: Accepted at ICDAR 2023 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2304.14953 |
| رقم الأكسشن: | edsarx.2304.14953 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |