An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering
| العنوان: | An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering |
|---|---|
| المؤلفون: | Nan Hu, Yike Wu, Guilin Qi, Dehai Min, Jiaoyan Chen, Jeff Z Pan, Zafar Ali |
| بيانات النشر: | Research Square Platform LLC, 2022. |
| سنة النشر: | 2022 |
| مصطلحات موضوعية: | FOS: Computer and information sciences, Computer Science - Computation and Language, Computer Networks and Communications, Hardware and Architecture, Computation and Language (cs.CL), Software |
| الوصف: | Large-scale pre-trained language models (PLMs) such as BERT have recently achieved great success and become a milestone in natural language processing (NLP). It is now the consensus of the NLP community to adopt PLMs as the backbone for downstream tasks. In recent works on knowledge graph question answering (KGQA), BERT or its variants have become necessary in their KGQA models. However, there is still a lack of comprehensive research and comparison of the performance of different PLMs in KGQA. To this end, we summarize two basic KGQA frameworks based on PLMs without additional neural network modules to compare the performance of nine PLMs in terms of accuracy and efficiency. In addition, we present three benchmarks for larger-scale KGs based on the popular SimpleQuestions benchmark to investigate the scalability of PLMs. We carefully analyze the results of all PLMs-based KGQA basic frameworks on these benchmarks and two other popular datasets, WebQuestionSP and FreebaseQA, and find that knowledge distillation techniques and knowledge enhancement methods in PLMs are promising for KGQA. Furthermore, we test ChatGPT, which has drawn a great deal of attention in the NLP community, demonstrating its impressive capabilities and limitations in zero-shot KGQA. We have released the code and benchmarks to promote the use of PLMs on KGQA. Comment: Accepted by World Wide Web Journal |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_dedup___::36ce30aff9f62ef8e60845a562385261 https://doi.org/10.21203/rs.3.rs-2184834/v1 |
| حقوق: | OPEN |
| رقم الأكسشن: | edsair.doi.dedup.....36ce30aff9f62ef8e60845a562385261 |
| قاعدة البيانات: | OpenAIRE |
| الوصف غير متاح. |