Duet: efficient and scalable hybriD neUral rElation undersTanding

التفاصيل البيبلوغرافية
العنوان: Duet: efficient and scalable hybriD neUral rElation undersTanding
المؤلفون: Zhang, Kaixin, Wang, Hongzhi, Lu, Yabin, Li, Ziqi, Shu, Chang, Yan, Yu, Yang, Donghua
سنة النشر: 2023
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Databases, Computer Science - Artificial Intelligence, Computer Science - Machine Learning
الوصف: Learned cardinality estimation methods have achieved high precision compared to traditional methods. Among learned methods, query-driven approaches have faced the workload drift problem for a long time. Although both data-driven and hybrid methods are proposed to avoid this problem, most of them suffer from high training and estimation costs, limited scalability, instability, and long-tail distribution problems on high-dimensional tables, which seriously affects the practical application of learned cardinality estimators. In this paper, we prove that most of these problems are directly caused by the widely used progressive sampling. We solve this problem by introducing predicate information into the autoregressive model and propose Duet, a stable, efficient, and scalable hybrid method to estimate cardinality directly without sampling or any non-differentiable process, which can not only reduce the inference complexity from $O(n)$ to $O(1)$ compared to Naru and UAE but also achieve higher accuracy on high cardinality and high-dimensional tables. Experimental results show that Duet can achieve all the design goals above and be much more practical. Besides, Duet even has a lower inference cost on CPU than that of most learned methods on GPU.
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2307.13494
رقم الأكسشن: edsarx.2307.13494
قاعدة البيانات: arXiv