تقرير
Detecting Conceptual Abstraction in LLMs
| العنوان: | Detecting Conceptual Abstraction in LLMs |
|---|---|
| المؤلفون: | Regneri, Michaela, Abdelhalim, Alhassan, Laue, Sören |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language, Computer Science - Machine Learning |
| الوصف: | We present a novel approach to detecting noun abstraction within a large language model (LLM). Starting from a psychologically motivated set of noun pairs in taxonomic relationships, we instantiate surface patterns indicating hypernymy and analyze the attention matrices produced by BERT. We compare the results to two sets of counterfactuals and show that we can detect hypernymy in the abstraction mechanism, which cannot solely be related to the distributional similarity of noun pairs. Our findings are a first step towards the explainability of conceptual abstraction in LLMs. Comment: Paper accepted at the LREC-COLING 2024 Conference (Paper ID: 1968) https://lrec-coling-2024.org/list-of-accepted-papers/ |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2404.15848 |
| رقم الأكسشن: | edsarx.2404.15848 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |