تقرير
Maximum Causal Entropy Inverse Reinforcement Learning for Mean-Field Games
| العنوان: | Maximum Causal Entropy Inverse Reinforcement Learning for Mean-Field Games |
|---|---|
| المؤلفون: | Anahtarci, Berkay, Kariksiz, Can Deha, Saldi, Naci |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science Mathematics |
| مصطلحات موضوعية: | Electrical Engineering and Systems Science - Systems and Control, Computer Science - Machine Learning, Mathematics - Optimization and Control |
| الوصف: | In this paper, we introduce the maximum casual entropy Inverse Reinforcement Learning (IRL) problem for discrete-time mean-field games (MFGs) under an infinite-horizon discounted-reward optimality criterion. The state space of a typical agent is finite. Our approach begins with a comprehensive review of the maximum entropy IRL problem concerning deterministic and stochastic Markov decision processes (MDPs) in both finite and infinite-horizon scenarios. Subsequently, we formulate the maximum casual entropy IRL problem for MFGs - a non-convex optimization problem with respect to policies. Leveraging the linear programming formulation of MDPs, we restructure this IRL problem into a convex optimization problem and establish a gradient descent algorithm to compute the optimal solution with a rate of convergence. Finally, we present a new algorithm by formulating the MFG problem as a generalized Nash equilibrium problem (GNEP), which is capable of computing the mean-field equilibrium (MFE) for the forward RL problem. This method is employed to produce data for a numerical example. We note that this novel algorithm is also applicable to general MFE computations. Comment: 38 pages |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2401.06566 |
| رقم الأكسشن: | edsarx.2401.06566 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |