On Performance Estimation in Automatic Algorithm Configuration
العنوان: | On Performance Estimation in Automatic Algorithm Configuration |
---|---|
المؤلفون: | Shengcai Liu, Ke Tang, Yunwen Lei, Xin Yao |
المصدر: | AAAI |
بيانات النشر: | Association for the Advancement of Artificial Intelligence (AAAI), 2020. |
سنة النشر: | 2020 |
مصطلحات موضوعية: | FOS: Computer and information sciences, Algorithm configuration, Computer Science - Machine Learning, Performance estimation, Computer science, 0202 electrical engineering, electronic engineering, information engineering, Estimator, 020201 artificial intelligence & image processing, 02 engineering and technology, General Medicine, Algorithm, Machine Learning (cs.LG) |
الوصف: | Over the last decade, research on automated parameter tuning, often referred to as automatic algorithm configuration (AAC), has made significant progress. Although the usefulness of such tools has been widely recognized in real world applications, the theoretical foundations of AAC are still very weak. This paper addresses this gap by studying the performance estimation problem in AAC. More specifically, this paper first proves the universal best performance estimator in a practical setting, and then establishes theoretical bounds on the estimation error, i.e., the difference between the training performance and the true performance for a parameter configuration, considering finite and infinite configuration spaces respectively. These findings were verified in extensive experiments conducted on four algorithm configuration scenarios involving different problem domains. Moreover, insights for enhancing existing AAC methods are also identified. Comment: accepted by AAAI 2020 |
تدمد: | 2374-3468 2159-5399 |
URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_dedup___::0559868b4d3d6ca8b9525c1e97ff5c7c https://doi.org/10.1609/aaai.v34i03.5618 |
حقوق: | OPEN |
رقم الأكسشن: | edsair.doi.dedup.....0559868b4d3d6ca8b9525c1e97ff5c7c |
قاعدة البيانات: | OpenAIRE |
تدمد: | 23743468 21595399 |
---|