Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science

التفاصيل البيبلوغرافية
العنوان: Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science
المؤلفون: Tang, Xiangru, Jin, Qiao, Zhu, Kunlun, Yuan, Tongxin, Zhang, Yichi, Zhou, Wangchunshu, Qu, Meng, Zhao, Yilun, Tang, Jian, Zhang, Zhuosheng, Cohan, Arman, Lu, Zhiyong, Gerstein, Mark
سنة النشر: 2024
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Computers and Society, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning
الوصف: Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents, called scientific LLM agents, also introduce novel vulnerabilities that demand careful consideration for safety. However, there exists a notable gap in the literature, as there has been no comprehensive exploration of these vulnerabilities. This perspective paper fills this gap by conducting a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures. We begin by providing a comprehensive overview of the potential risks inherent to scientific LLM agents, taking into account user intent, the specific scientific domain, and their potential impact on the external environment. Then, we delve into the origins of these vulnerabilities and provide a scoping review of the limited existing works. Based on our analysis, we propose a triadic framework involving human regulation, agent alignment, and an understanding of environmental feedback (agent regulation) to mitigate these identified risks. Furthermore, we highlight the limitations and challenges associated with safeguarding scientific agents and advocate for the development of improved models, robust benchmarks, and comprehensive regulations to address these issues effectively.
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2402.04247
رقم الأكسشن: edsarx.2402.04247
قاعدة البيانات: arXiv