Reactor Mk.1 performances: MMLU, HumanEval and BBH test results

التفاصيل البيبلوغرافية
العنوان: Reactor Mk.1 performances: MMLU, HumanEval and BBH test results
المؤلفون: Dunham, TJ, Syahputra, Henry
سنة النشر: 2024
المجموعة: Computer Science
مصطلحات موضوعية: Computer Science - Artificial Intelligence, Computer Science - Computation and Language
الوصف: The paper presents the performance results of Reactor Mk.1, ARCs flagship large language model, through a benchmarking process analysis. The model utilizes the Lychee AI engine and possesses less than 100 billion parameters, resulting in a combination of efficiency and potency. The Reactor Mk.1 outperformed models such as GPT-4o, Claude Opus, and Llama 3, with achieved scores of 92% on the MMLU dataset, 91% on HumanEval dataset, and 88% on BBH dataset. It excels in both managing difficult jobs and reasoning, establishing as a prominent AI solution in the present cutting-edge AI technology.
نوع الوثيقة: Working Paper
URL الوصول: http://arxiv.org/abs/2406.10515
رقم الأكسشن: edsarx.2406.10515
قاعدة البيانات: arXiv