A MATLAB-based Mean-Field-Type Games Toolbox: Continuous-Time

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2019.2938191, IEEE Access
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier ACCESS.2019.DOI
A MATLAB-based Mean-Field-Type
Games Toolbox: Continuous-Time
Version
J. BARREIRO-GOMEZ, (Senior Member, IEEE), AND H. TEMBINE,(Senior Member, IEEE)
Learning & Game Theory Laboratory, New York University Abu Dhabi
Corresponding author: Julian Barreiro-Gomez (e-mail:jbarreiro@nyu.edu).
ABSTRACT In this paper, we present a user-friendly MatLab-based Mean-Field-Type Games (MFTG)

Toolbox that allows simulating a diversity of scalar-valued and matrix-valued MFTG problems for an
arbitrary number of players, e.g., non-cooperative, fully-cooperative and co-opetitive approaches. We
present details of each one of the developed tools together with the corresponding pseudo codes, and several
illustrative examples. In addition, we provide the 20 functions composing the MFTG toolbox to freely
download.
INDEX TERMS Mean-field-type games, simulations, MatLab-based toolbox
I. INTRODUCTION
EAN-FIELD-TYPE GAMES (MFTG) are a class of
M stochastic games considering, in the cost functional
and system-state dynamics, the distribution of both the sys-
tem state and control inputs (strategies). This type of games
involving distribution-dependent variables are getting special
attention because they incorporate not only mean terms,
but also variance terms within the optimization problem.
Thus, MFTG are tightly associated with the mean-variance
paradigm introduced in [4], and therefore, MFTGs become
a crucial game-theoretical tool in the field of risk-aware
engineering. Different methods to solve MFTG have been
extensively studied. Some of these methods are Stochastic
Maximum Principle [5]–[7], Dynamic Programming Princi-
ple [8], Wiener Chaos Expansion [9], and the Direct Method
[10]–[15]. Depending on the selected methodology to solve
the aforementioned game-theoretical problem, one might
obtain a solution expressed in terms of either Ordinary or
Partial Differential Equations. In this regard, there is also an
increasing interest in the appropriate tool for simulating the
MFTG, which is the main objective that this paper addresses.
A. CONTRIBUTION
The main contribution of this paper consists of an user
friendly MatLab-based MFTG toolbox composed of several
functions to solve Non-Cooperative, Fully-Cooperative and FIGURE 1: Complete functions set of the MatLab-based
MFTG toolbox
Co-opetitive Linear-Quadratic MFTG for both scalar and
matrix values (see Table 1), and considering an arbitrary
number of decision-makers. The complete set of functions
VOLUME 4, 2016 1
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
J. Barreiro-Gomez and H. Tembine : A MATLAB-based Mean-Field-Type Games Toolbox: Continuous Time Version
TABLE 1: MatLab functions of the first version of the MFTG-Toolbox

Type of Mean-Field-Type Games MatLab Functions Reported Result
Problem_setup_noncooperative_MFTG.m
LQ Scalar-Valued Non-Cooperative MFTG noncooperative_MFTG.p [1]
Figures_MFTG_ScNonCoop.p
Problem_setup_cooperative_MFTG.m
LQ Scalar-Valued Fully-Cooperative MFTG cooperative_MFTG.p [1]
Figures_MFTG_ScCoop.p
Problem_setup_coopetitive_MFTG.m
LQ Scalar-Valued Co-Opetitive MFTG coopetitive_MFTG.p [2]
Figures_MFTG_ScCo-Op.p
Problem_setup_noncoopetitive_Matrix_MFTG.m
LQ Matrix-Valued Non-Cooperative MFTG noncoopetitive_Matrix_MFTG.p [3]
Figures_MFTG_MaNonCoop.p
Problem_setup_coopetitive_Matrix_MFTG.m
LQ Matrix-Valued Cooperative MFTG coopetitive_Matrix_MFTG.p [3]
Figures_MFTG_MaCoop.p
FIGURE 2: General diagram corresponding to the main three functions of the MFTG MatLab-based toolbox.
composing the proposed MatLab-based MFTG toolbox is user-friendly toolbox to solve the main Mean-Field-Type
presented in Figure 1, which can be downloaded together Game problems neither in the scalar-valued nor the matrix-
with the numerical examples presented in this paper (see valued version. Finally, we show an example illustrating how
Section V). It is important to point out that, the solutions to perform simulations with the proposed toolbox.
for the shown problems in this paper have been already
discussed and presented in [1], [2] and [3]. We have incor-
B. SIGNIFICANCE OF THE TOOLBOX
porated throughout this paper the statement corresponding
to the main results in Mean-Field-Type Games, e.g., Non- The development of toolboxes is motivated by the need to fa-
Cooperative, Fully-Cooperative and Co-opetitive, in order to cilitate obtaining results when well-established mathematical
make the manuscript self-contained. Thus, the inputs and procedures have been already studied. Toolboxes have been
outputs of the toolbox are easy to understand and to apply. developed in several different fields and have considerably
Hence, we briefly present the problem statement and its semi- improved the research pace. Among all the different types of
explicit solution. Afterwards, we present the corresponding tools we can find the following: tools that allow designing
proposed MatLab-based function describing its inputs and or modeling in a systematic manner, e.g., the modeling of
outputs. To the best of our knowledge, there is not available joint contact mechanics [16], tools that facilitate the plot of
results, e.g., the generation of medical images [17], and/or
2 VOLUME 4, 2016
tools allowing the analysis of systems’ performance [18]. START Setting of Parameters
In particular, the toolbox that is proposed in this paper
concerns to the game, learning and automatic control theory Simulation Parameters:
as presented next. • Simulation steps
As previously mentioned, there are several alternatives to
• Initial condition estimation
solve the Mean-Field-Type Game problems addressed in this
manuscript obtaining different mathematical representations System parameters
of the solutions with diverse complexities. For instance, some Cost functional parameters
approaches allow solving the aforementioned problems in
terms of partial differential equations (PDEs). In contrast, Semi-explicit solution
Direct Method allows solving the underlying problems in a
Construction of ODEs
semi-explicit way, i.e., obtaining ordinary differential equa-
Setting boundary conditions
tions (ODEs). Depending on the complexity of the solution
Solving ODEs
and the selected method, numerical solutions are required. In
[1], the fact that the Direct Method can be used to improve
Results
numerical methods in mean-field-type game theory has been
discussed. In this regard, the toolbox presented in this paper Computing the evolution of:
significantly contributes to enhance the numerical methods. • Optimal strategies
• Expectation of optimal strategies
C. STRUCTURE OF THE PAPER
This paper is organized as follows. Section II introduces • System states
the three main functions that compose each one of the • Expectation of system states
MFTG approaches, i.e., problem setup, generation of so-
• Optimal cost
lution and presentation of results. Section III presents the
scalar-valued MFTG for non-cooperative, fully-cooperative
and co-opetitive scenarios, including numerical examples. Publications of results
Then, Section IV presents the matrix-valued MFTG for both
non-cooperative and fully-cooperative scenarios, including
numerical examples. Section V shows how to freely obtain END
the proposed MatLab-based MFTG Toolbox. Finally, con- FIGURE 3: Flow-Chart of the MAtLab functions
cluding remarks are drawn in Section VI.
II. STRUCTURE OF THE FUNCTIONS each one of the examples presented at each section through-
The MatLab-based MFTG toolbox is classified depending on out this paper. The Problem_setup_..._MFTG.m func-
the type of MFTG problem as shown in Table 1. Each classi- tion creates a structure containing all these parameters under
fication is composed by three main functions as presented the name Input_Data.
in Figure 2, i.e., a problem setup, generation of solution,
and finally a presentation of results. These three functions B. GENERATION OF SOLUTION
are described next together with some examples. We also This function is the core of the MFTG Toolbox. There
present a general flow chart/workflow in Figure 3, of the is a function in charge of computing the solution for
proposed toolbox as it has been presented for other MatLab- each one of the MFTG problems. This main functions
based tools, e.g., [16] [17]. Moreover, we also provide some receive the structure Input_Data from the function
specific and short MatLab scripts in order to illustrate how Problem_setup_..._MFTG.m, and returns a structure
user-friendly the toolbox is as in [19], and present some under the name Output_Data incorporating the evolution
respective pseudocodes. of the system states and its expectation, the evolution of the
optimal control inputs for each one of the players and its
A. PROBLEM SETUP expectation, the evolution of the Riccati equations, and the
For each one of the scenarios, there is a function under optimal costs for all the players.
the name Problem_setup_..._MFTG.m where the user
should provide information about the MFTG problem, e.g., C. PRESENTATION OF RESULTS
the number of players, the parameters corresponding to the Finally, the last main function of the MFTG Toolbox cor-
system dynamics, and the cost functions; and also informa- responds to Figures_MFTG_....p. Such function is in
tion for simulation purposes, i.e., the time steps along the charge of the presentation of results, i.e., it receives the struc-
time horizon and a guess about the initial conditions for the ture Output_Data from the core function that solves the
ordinary differential equations. All these parameters depend MFTG problem, and returns several organized plots with the
on the specific MFTG problem, and details can be found in corresponding labels. These results can be seen throughout
VOLUME 4, 2016 3
the paper, e.g., Figures 5-15.
III. SCALAR MEAN-FIELD-TYPE GAMES

We start discussing the scalar-valued MFTG problems
for three different scenarios, i.e., non-cooperative, fully-
cooperative, and co-opetitive approaches. Figure 4 presents
Algorithm 1, which is the general pseudocode corresponding
to the setting of parameters for both the scalar-valued non-
cooperative and fully-cooperative scenarios. In this section
we present 12 functions of the proposed MFTG Toolbox.
A. NON-COOPERATIVE APPROACH
The Non-Cooperative Linear-Quadratic MFTG Problem is

given by
1 1
Li (u1 , . . . , un ) = qi x(T )2 + q̄i E[x(T )]2
2 2
1 T
Z
qi x2 + q̄i E[x]2 dt

+
2 0
1 T
Z
ri u2i + r̄i E[ui ]2 dt,

+ FIGURE 4: Pseudocode corresponding to the scalar-valued
2 0 non-cooperative and/or fully-cooperative MFTG parameters
min E[Li (u1 , . . . , un )], settings.
ui ∈Ui
subject to
  for all i ∈ N , where αi , βi , γi , and δi solve the following
X differential equations:
dx = b0 + b1 x + b̄1 E[x] + b2j uj + b̄2j E[uj ]  dt
j∈N
X b22j b2
α̇i = −2b1 αi − qi + 2αi αj + 2i αi2 ,
+ σ0 dB, rj ri
j∈N \{i}
x(0) := x0 , 2
X b2j + b̄2j
β̇i = −2 b1 + b̄1 βi − qi − q̄i + 2βi βj
where qi , qi + q̄i ∈ R≥0 , ri , ri + r̄i ∈ R>0 , and (rj + r̄j )
j∈N \{i}
b0 , b1 , b̄1 , b2j , b̄2j , σ0 ∈ R, for all j ∈ N . Moreover, B 2
denotes a standard Brownian motion; and E[x] and E[uj ] b2i + b̄2i
+ β2,
denote the expected value of the system state and the j th (ri + r̄i ) i
control input, respectively. 2
b2j + b̄2j
X
γ̇i = −b0 βi − b1 + b̄1 γi + βi γj
(rj + r̄j )
j∈N \{i}
1) Semi-Explicit Solution 2 2
X b2j + b̄2j b2i + b̄2i
+ γi βj + βi (t)γi ,
The solution for the one-dimentional Non-Cooperative (rj + r̄j ) (ri + r̄i )
j∈N \{i}
MFTG is stated in Proposition 1 next. 2
1 2
X b2j + b̄2j
Proposition 1 (Result presented in [1]): The feedback- δ̇i = −b0 γi − αi σ0 + γi γj
strategies linear-quadratic mean-field-type Nash equilibrium 2 (rj + r̄j )
j∈N \{i}
for the Non-Cooperative Problem, and the optimal costs are 2
1 b2i + b̄2i
given by: + γ2,
2 (ri + r̄i ) i
b2i
u∗i = E[u∗i ] − αi (x − E[x]) , where αi (T ) = qi , βi (T ) = qi + q̄i , and γi = δi = 0.
ri
b2i + b̄2i h i
E[u∗i ] = − βi E[x] + γi , 2) MatLab-based MFTG Toolbox Example
(ri + r̄i ) We present a three-player example corresponding to
1 2 1
Li (u∗1 , . . . , u∗n ) = αi (0) (x0 − E[x0 ]) + βi (0)E[x0 ]2 the scalar-valued non-cooperative MFTG problem. No-
2 2 tice that, the MatLab script, corresponding to the ex-
+ γi (0)E[x0 ] + δi (0), ample, follows the pseudocode presented in Figure 4.
4 VOLUME 4, 2016
2.5 0 0.4
x
Optimal Strategies
1
2 Ex 2
System State
-0.2 0.3 3
1.5 u1
Eu1
1 u2
-0.4 Eu2 0.2
0.5 u3
Eu3
0 -0.6 0.1
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
time [s] time [s] time [s]
(a) (b) (c)
0.04 0.8
1 1 1
0.4 2 2 2
0.03 0.6
3 3 3
0.3
0.02 0.4
0.2
0.01 0.2
0.1
0 0 0
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
(d) (e) (f)
FIGURE 5: Results corresponding to the LQ Scalar-Valued Non-Cooperative MFTG example. These plots are generated by the
MatLab function Figures_MFTG_ScNonCoop.p
12 %%%%%%%% Cost functional Parameters %%%%%%%%

1.5 13 r = [1 2 2] ; % r = [r_1,...,r_n]
14 br = [1 2 2] ; % \bar{r} = ...
[\bar{r}_1,...,\bar{r}_n]
1 15 q = [0.5 0.5 0.5] ; % q = [q_1,...,q_n]
Cost
16 bq = [0.1 0.2 0.3] ; % \bar{q} = ...

[\bar{q}_1,...,\bar{q}_n]
0.5 17 q_T = [0.2 0.3 0.4] ; % q(T) = ...
[q_1(T),...,q_n(T)]
18 bq_T = [0.1 0.1 0.1] ; % \bar{q}(T) = ...
[\bar{q}_1(T),...,\bar{q}_n(T)]
0 19 %%%%%%%%%%%% Initial Conditions %%%%%%%%%%%%
1 2 3 20 x0 = 2.5 ; % initial condition
Player 21 Ex0 = x0 ; % initial condition
22 variables0 = [ones(4*n,1);x0;Ex0] ;
FIGURE 6: Costs of the player in the LQ Scalar-Valued Non- 23 %%%%%%%%%%%% Inputs %%%%%%%%%%%%%%%%%%%%%%%%
Cooperative MFTG example. This plot is generated by the 24 Input_Data = struct('T',T,'hh',hh,'n',n,...
MatLab function Figures_MFTG_ScNonCoop.p 25 'b0',b0,'b1',b1,'bb1',bb1,'b2',b2,...
26 'bb2',bb2,'s',s,'r',r,'br',br,...
27 'q',q,'bq',bq,'q_T',q_T,'bq_T',bq_T,...
Next, we present the MatLab script corresponding to 28 'x0',x0,'Ex0',Ex0,'variables0',variables0) ;
29 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
the problem setup from the function under the name
Problem_setup_noncooperative_MFTG.m. Once the parameters corresponding to the game problem
have been established, then the complete code for the sim-
ulation by using the proposed MatLab-based toolbox from
Listing 1: Problem Setup function for the LQ Scalar-Valued
Non-Cooperative MFTG example the function MAIN_non_cooperative_MFTG.m is as
1 %%%%%%%%%% Simulation Parameters %%%%%%%%%%% follows:
2 T = 2 ; % Simulation time
3 hh = 1000 ; % Number of simulation-steps
4 %%%%% System-State-Dynamics Parameters %%%%% Listing 2: Complete simulation of the LQ Scalar-Valued Non-
5 n = 3 ; % number of players Cooperative MFTG example
6 b0 = 1 ; 1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
7 b1 = -1 ; 2 clear all; close all ; clc
8 bb1 = -1 ; 3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
9 b2 = [1 2 3] ; % b2 = [b_{21},...,b_{2n}] 4 Problem_setup_noncooperative_MFTG
10 bb2 = [1 2 3] ; % bb2 = ... 5 [Output_Data] = ...
[\bar{b}_{21},...,\bar{b}_{2n}] noncooperative_MFTG(Input_Data) ;
11 s = 2 ; % \sigma_0 6 Figures_MFTG_ScNonCoop(Output_Data)
VOLUME 4, 2016 5
TABLE 2: Detail output data for the LQ Scalar-Valued Non-

Cooperative MFTG 2.5
Output Data Dimensions 2

Output_Data.time 1 × hh 1.5
Cost
Output_Data.alpha_r n × hh
Output_Data.beta_r n × hh 1
Output_Data.gamma_r n × hh 0.5
Output_Data.delta_r n × hh
Output_Data.x_r 1 × hh 0
1 2 3
Output_Data.Ex_r 1 × hh
Player
Output_Data.u_r n × hh
Output_Data.Eu_r n × hh FIGURE 8: Costs of the player in the LQ Scalar-Valued
Cooperative MFTG example. This plot is generated by the
Output_Data.L n MatLab function Figures_MFTG_ScCoop.p
7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1) Semi-Explicit Solution

The main function noncooperative_MFTG.p returns Proposition 2 below presents the solution for the one-
the solution of the non-cooperative MFTG problem pro- dimentional Cooperative MFTG problem.
viding the data presented in Table 2. The function Proposition 2 (Result presented in [1]): The feedback-
Figures_MFTG_ScNonCoop.p generates the plots in strategies linear-quadratic mean-field-type Nash equilibrium
Figure 5, where it can be seen, for example that, the system for the Cooperative Problem, and the optimal costs are given
state is driven to zero and that the boundary conditions are by:
met for the Riccati equations. On the other hand, Figure 6 b2i
shows the optimal costs corresponding to the three players. u∗i = E[u∗i ] − α0 (x − E[x]) ,
ri
b2i + b̄2i h i
B. COOPERATIVE APPROACH E[u∗i ] = − β0 E[x] + γ0 ,
(ri + r̄i )
On the other hand, there is a scenario in which all the
1 2
players cooperate to each other in order to pursue a common Li (u∗1 , . . . , u∗n ) = α0 (0) (x0 − E[x0 ])
objective. The Cooperative Linear-Quadratic MFTG Problem 2
1
is given by + β0 (0)E[x0 ]2 + γ0 (0)E[x0 ] + δi (0),
X 2
L0 (u1 , . . . , un ) = [Li (u1 , . . . , un )] for all i ∈ N , where α0 , β0 , γ0 , and δ0 solve the following
i∈N
" differential equations:
X 1 1 X b2
= qi x(T )2 + q̄i E[x(T )]2 α̇0 = −2b1 α0 − q0 + 2i 2
α ,
2 2 ri 0
i∈N i∈N
Z T
1 X b2i + b̄2i 2

qi x2 + q̄i E[x]2 dt

+
β2,

2 0 β̇0 = −2 b1 + b̄1 β0 − q0 − q̄0 +
# (ri + r̄i ) 0
i∈N
1 T
Z
2 2

+ ri ui + r̄i E[ui ] dt , X b2i + b̄2i 2

2 0

γ̇0 = −b0 β0 − b1 + b̄1 γ0 + β0 γ0 ,
(ri + r̄i )
min E[L0 (u1 , . . . , un )], i∈N
{ui ∈Ui }i∈N 2
α0 (t) 2 1 X b2i + b̄2i
subject to δ̇0 = −b0 γ0 − σ0 + γ2,
2 2 (ri + r̄i ) 0
  i∈N
where α0 (T ) = q0 , β0 (T ) = q0 + q̄0 , and γ0 = δ0 = 0.

X
j∈N
2) MatLab-based MFTG Toolbox Example
+ σ0 dB,
We present a three-player example corresponding to
x(0) := x0 ,
P P the scalar-valued cooperative MFTG problem. Next,
where q0 = i∈N qi , q0 + q̄0 = i∈N (qi + q̄i ) ∈ R≥0 , we present the MatLab script corresponding to the
ri , ri + r̄i ∈ R>0 , and b0 , b1 , b̄1 , b2j , b̄2j , σ0 ∈ R, for all problem setup from the function under the name
j ∈ N . Moreover, B denotes a standard Brownian motion; Problem_setup_cooperative_MFTG.m. This illus-
and E[x] and E[uj ] denote the expected value of the system trative example MatLab script follows the same procedure
state and the j th control input, respectively. established in the pseudocode in Figure 4.
6 VOLUME 4, 2016
3 0 0.6
x
Optimal Strategies
0
Ex
System State
2 u1
-1 0.4
Eu1
u2
1 Eu2
-2 u3 0.2
Eu3
0
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
(a) (b) (c)
1.2 0.08 1.5

0 0 0
1
0.06
1
0.8
0.04
0.6
0.5
0.02
0.4
0.2 0 0
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
(d) (e) (f)
FIGURE 7: Results corresponding to the LQ Scalar-Valued Fully-Cooperative MFTG. These plots are generated by the MatLab
function Figures_MFTG_ScCoop.p
Listing 3: Problem Setup function for the LQ Scalar-Valued TABLE 3: Detail output data for the LQ Scalar-Valued Fully-
Fully-Cooperative MFTG example Cooperative MFTG
1 %%%%%%%%%% Simulation Parameters %%%%%%%%%%%
2 T = 2 ; % Simulation time Output Data Dimensions
4 %%%%% System-State-Dynamics Parameters %%%%% Output_Data.time 1 × hh
5 n = 3 ; % number of players Output_Data.alpha_r 1 × hh
6 b0 = 2 ; Output_Data.beta_r 1 × hh
7 b1 = -0.5 ;
8 bb1 = -0.5 ; Output_Data.gamma_r 1 × hh
9 b2 = [1 2 3] ; % b2 = [b_{21},...,b_{2n}] Output_Data.delta_r 1 × hh
10 bb2 = [1 2 3] ; % bb2 = ... Output_Data.x_r 1 × hh
[\bar{b}_{21},...,\bar{b}_{2n}]
11 s = 2 ; % \sigma_0 Output_Data.Ex_r 1 × hh
12 %%%%%%%% Cost functional Parameters %%%%%%%% Output_Data.u_r n × hh
13 r = [3 2 1] ; % r = [r_1,...,r_n] Output_Data.Eu_r n × hh
14 br = [3 2 1] ; % \bar{r} = ...
[\bar{r}_1,...,\bar{r}_n] Output_Data.L n
15 q = [0.2 0.2 0.2] ; % q = [q_1,...,q_n]
16 bq = [0.3 0.3 0.3] ; % \bar{q} = ...
[\bar{q}_1,...,\bar{q}_n]
17 q_T = [0.1 0.2 0.3] ; % q(T) = ... main function cooperative_MFTG.p, which returns the
[q_1(T),...,q_n(T)] data presented in Table 3.
18 bq_T = [0.3 0.2 0.1] ; % \bar{q}(T) = ...
[\bar{q}_1(T),...,\bar{q}_n(T)]
19 %%%%%%%%%%%% Initial Conditions %%%%%%%%%%%%
20 x0 = 3 ; % initial condition Listing 4: Complete simulation of the LQ Scalar-Valued Fully-
21 Ex0 = x0 ; % initial condition Cooperative MFTG example
22 variables0 = [ones(4,1);x0;Ex0] ; 1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
23 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2 clear all; close all ; clc
24 Input_Data = struct('T',T,'hh',hh,'n',n,... 3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
25 'b0',b0,'b1',b1,'bb1',bb1,'b2',b2,... 4 Problem_setup_cooperative_MFTG
26 'bb2',bb2,'s',s,'r',r,'br',br,... 5 [Output_Data] = ...
27 'q',q,'bq',bq,'q_T',q_T,'bq_T',bq_T,... cooperative_MFTG(Input_Data) ;
28 'x0',x0,'Ex0',Ex0,'variables0',variables0) ; 6 Figures_MFTG_ScCoop(Output_Data)
29 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Now, we present the main MatLab script to solve Finally, the function Figures_MFTG_ScCoop.p gen-
the scalar-valued fully-cooperative MFTG problem from erates the Figures 7 and 8 where the complete solution of the
the function MAIN_cooperative_MFTG.m evoking the game can be seen. Note that the boundary conditions for the
VOLUME 4, 2016 7
1 X
λij qj x2 (T ) + q̄j E[x(T )]2

+
2
j∈N \{i}
1 T
Z
λii qi x2 + q̄i E[x]2 dt

+
2 0
1 T
Z
λii ri u2i + r̄i E[ui ]2 dt

+
2 0
1 T X
Z
λij qj x2 + q̄j E[x]2 dt

+
2 0
j∈N \{i}
1 T X
Z
λij rj u2j + r̄j E[uj ]2 dt,

+
2 0
j∈N \{i}
C
min E Li (u1 , . . . , un , Λi ) ,
ui ∈Ui
subject to
 
X
j∈N
+ σ0 dB,
x(0) = x0 ,
where qi , qi + q̄i ∈ R≥0 , ri , ri + r̄i ∈ R>0 , and
FIGURE 9: Pseudocode corresponding to the co-opetitive b0 , b1 , b̄1 , b2j , b̄2j , σ0 ∈ R, for all j ∈ N . Moreover, B
MFTG parameters settings. denotes a standard Brownian motion; and E[x] and E[uj ]
denote the expected value of the system state and the j th
control input, respectively.
Riccati equations are satisfied.
C. CO-OPETITIVE APPROACH 1) Semi-Explicit Solution

Consider that the decision-makers can interact to each other
by either cooperating or competing. In this regard, there is Proposition 3 below presents the solution for the scalar-
a co-opetitive interaction in the game since decision-makers valued co-opetitive MFTG problem.
can behave by partial/full selfishness, or partial/full altruism. Proposition 3 (Result presented in [2]): The feedback-
Such behavior is determined throughout some co-opetitive strategies linear-quadratic mean-field-type Nash equilibrium
parameters given by for the co-opetitive Problem, and the optimal costs are given
Λi = [λi1 ... λin ]> ∈ Rn , by:
b2i
for all i ∈ N , which determine the level of selfishness and u∗i = E[ui ] − αi (x − E[x]) ,
λii ri
altruism with other decision makers. The term λij indicates
the co-opetition of decision-maker i ∈ N with the decision- b2i + b̄2i h i
E[u∗i ] = − βi E[x] + γi
maker j ∈ N , i.e., λii (ri + r̄i )
∗ ∗ 1 2 1
• Partial altruism towards j: if λij > 0 the ith decision LC
i (u1 , . . . , un ) = αi (0) (x0 − E[x0 ]) + βi (0)E[x0 ]
2
maker is considering positively j 0 s playoff in this own 2 2

+ γi (0)E[x0 ] + δi (0),
behavior.
• Partial non-cooperation towards j : If λij = 0 the ith where αi , βi , γi , and δi solve the following differential
decision maker is not considering j 0 s playoff in this own equations:
behavior.
b22j
Partial Spite towards j: if λij < 0 the ith decision
X X
• α̇i = −2b1 αi − λij qj + 2αi αj
maker is considering negatively j 0 s playoff in this own λjj rj
j∈N j∈N \{i}
behavior. b22j
X b2
The Co-opetitive Linear-Quadratic MFTG Problem is intro- − λij 2 αj2 + 2i αi2 ,
λjj rj λii ri
j∈N \{i}
duced next. X
1 β̇i = −2 b1 + b̄1 βi − λij (qj + q̄j )
LC 2 2

i (u1 , . . . , un , Λi ) = λii qi x (T ) + q̄i E[x(T )] j∈N
2
8 VOLUME 4, 2016
6 b0 = 1 ;
8 7 b1 = -2 ;
8 bb1 = -2 ;
6 9 b2 = [1 2 3 4 5] ; % b2 = [b_{21},...,b_{2n}]
10 bb2 = [1 2 3 4 5] ; % bb2 = ...
Cost
[\bar{b}_{21},...,\bar{b}_{2n}]
4 11 s = 5 ; % \sigma_0
2 13 lambda = [1 0.1 0.2 0.3 0.4 ;
14 0.1 1 0.2 0.3 0.4 ;
15 0.1 0.2 1 0.3 0.4 ;
0 16 0.1 0.2 0.3 1 0.4 ;
1 2 3 4 5 17 0.1 0.2 0.3 0.4 1 ] ; % ...
Player coopetitive parameters
18 r = [1.1 1.2 1.3 1.4 1.5] ; % r = ...
FIGURE 11: Costs of the player in the LQ Scalar-Valued [r_1,...,r_n]
Co-Opetitive MFTG example. This plot is generated by the 19 br = [1.1 1.2 1.3 1.4 1.5] ; % \bar{r} = ...
MatLab function Figures_MFTG_ScCo_Op.p [\bar{r}_1,...,\bar{r}_n]
20 q = [1 2 3 4 5] ; % q = [q_1,...,q_n]
2 21 bq = [1 2 3 4 5] ; % \bar{q} = ...
X b2j + b̄2j [\bar{q}_1,...,\bar{q}_n]
+ 2βi βj 22 q_T = [0.1 0.2 0.3 0.4 0.5] ; % q(T) = ...
λjj (rj + r̄j ) [q_1(T),...,q_n(T)]
j∈N \{i}
2 2 23 bq_T = [0.1 0.1 0.1 0.1 0.1] ; % ...
X b2j + b̄2j 2 b2i + b̄2i \bar{q}(T) = ...
− λij 2 β + β2, [\bar{q}_1(T),...,\bar{q}_n(T)]
λjj (rj + r̄j ) j λii (ri + r̄i ) i 24 %%%%%%%%%%%% Initial Conditions %%%%%%%%%%%%
j∈N \{i}
25 x0 = 4 ; % initial condition
γ̇i = −b0 βi − b1 + b̄1 γi 26 Ex0 = x0 ; % initial condition
2 27 variables0 = [ones(4*n,1);x0;Ex0] ;
X b2j + b̄2j 28 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+ (βi γj + γi βj ) 29 Input_Data = struct('T',T,'hh',hh,'n',n,...
λjj (rj + r̄j ) 'b0',b0,'b1',b1,'bb1',bb1,'b2',b2,...
j∈N \{i} 30
2 2 31 'bb2',bb2,'s',s,'r',r,'br',br,...
X b2j + b̄2j b2i + b̄2i 32 'q',q,'bq',bq,'q_T',q_T,'bq_T',bq_T,...
− λij 2 βj (t)γj + β i γi , 33 'x0',x0,'Ex0',Ex0,'variables0',variables0,...
λjj (rj + r̄j ) λii (ri + r̄i ) 'lambda',lambda) ;
j∈N \{i} 34
2 35 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1 2
X b2j + b̄2j
δ̇i = −b0 γi − αi σ0 + γi γj We present the function MAIN_coopetitive_MFTG.m
2 λjj (rj + r̄j )
j∈N \{i} for the scalar-valued co-opetitive MFTG problem, which
2 2
1 X b2j + b̄2j 1 b2i + b̄2i evokes the function coopetitive_MFTG.p that is in
2
− λij 2 γ + γ2, charge of solving the game and that returns the data presented
2 λjj (rj + r̄j ) j 2 λii (ri + r̄i ) i
j∈N \{i} in Table 4.
P P
where αi (T ) = j∈N λij qj , βi (T ) = j∈N λij (qj + q̄j ),
and γi = δi = 0.
Listing 6: Complete simulation of the LQ Scalar-Valued Co-
2) MatLab-based MFTG Toolbox Example Opetitive MFTG
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Figure 9 presents Algorithm 2, corresponding to the 2 clear all; close all ; clc
pseudocode for the setting of parameters for the co- 3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
opetitive scenario. Following the previously mentioned 4 Problem_setup_coopetitive_MFTG
5 [Output_Data] = ...
pseudocode, we present a five-player example corre- coopetitive_MFTG(Input_Data) ;
sponding to the scalar-valued co-opetitive MFTG prob- 6 Figures_MFTG_ScCo_Op(Output_Data) ;
lem. Next, we present the MatLab script corresponding 7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
to the problem setup from the function under the name Finally, the function Figures_MFTG_ScCo_Op.p
Problem_setup_coopetitive_MFTG.m. Notice that generates the Figures 10 and 11, where it can be observed
the setup for this game scenario requires an extra input that the trajectories fulfill the respective boundary conditions.
corresponding to the co-opetitive parameters λ ∈ Rn×n .
IV. MATRIX-VALUED MEAN-FIELD-TYPE GAMES
Listing 5: Problem Setup function for the LQ Scalar-Valued Now, system dynamics that represent the evolution of a
Co-Opetitive MFTG matrix-valued state are considered, and considering that the
1 %%%%%%%%%% Simulation Parameters %%%%%%%%%%% players’ choices are also given by matrix-valued control
2 T = 1 ; % Simulation time inputs. Figure 12 presents the Algorithm 3, that is the general
4 %%%%% System-State-Dynamics Parameters %%%%% pseudocode corresponding to the setting of parameters for
5 n = 5 ; % number of players both the matrix-valued non-cooperative and matrix-valued
VOLUME 4, 2016 9
4 2 0.8
x u1
Optimal Strategies
1
3 Ex Eu1 2
System State
0 u2
0.6 3
2 Eu2
4
u3
-2 5
Eu3
1 u4 0.4
-4 Eu4
0 u5
Eu5
-1 -6 0.2
0 0.5 1 0 0.5 1 0 0.5 1
(a) (b) (c)
1 0.01 5
1 1
2 4 2
0.8
3 3
4 3 4
0.6 5 0.005 1 5
2 2
3
0.4 4 1
5
0.2 0 0
0 0.5 1 0 0.5 1 0 0.5 1
(d) (e) (f)
FIGURE 10: Results corresponding to the LQ Scalar-Valued Co-Opetitive MFTG Example. These plots are generated by the
MatLab function Figures_MFTG_ScCo_Op.p
TABLE 4: Detail output data for the LQ Scalar-Valued Co- Z T

Opetitive MFTG + hRi (Ui − E[Ui ]), Ui − E[Ui ]idt
0
Z T
Output Data Dimensions
+ h(Ri + R̄i )E[Ui ], E[Ui ]idt,
Output_Data.time 1 × hh 0
Output_Data.alpha_r n × hh min E [Li (X, U )] ,
Ui ∈Ui
Output_Data.beta_r n × hh
Output_Data.gamma_r n × hh subject to
Output_Data.delta_r n × hh  
Output_Data.x_r 1 × hh X
dX = B1 X + B̄1 E[X] + B2j Uj + B̄2j E[Uj ]  dt
Output_Data.Ex_r 1 × hh
j∈N
Output_Data.u_r n × hh
Output_Data.Eu_r n × hh + S0 dB,
Output_Data.L n X(0) = X0
where hA, Bi = trace(A∗ B) = trace(B ∗ A), A∗ being
the adjoint operator of A (transposition), The coefficients
Qi , Ri , Q̄i , R̄i are positive with values in Rd×d .
fully-cooperative scenarios. Here, we present eight functions
of the MFTG Toolbox. 1) Semi-Explicit Solution
Proposition 4 below presents the semi-explicit solution cor-
A. NON-COOPERATIVE APPROACH responding the matrix-valued approach in a non-cooperative
scenario.
Let us consider the following matrix-valued non-cooperative
Proposition 4 (Result presented in [3]): Assume that
MFTG problem:
Qi , Ri , Qi + Q̄i , Ri + R̄i are symmetric positive definite.
Li (X, U ) = h(Qi + Q̄i )E[X(T )], E[X(T )]i Then the matrix-valued mean-field-type (risk-neutral) Nash
+ hQi (X(T ) − E[X(T )]), X(T ) − E[X(T )]i equilibrium strategy and the (risk-neutral) equilibrium cost
Z T are given by:
+ hQi (X − E[X]), X − E[X]idt 1
0 Ui∗ = E[Ui∗ ] − Ri−1 B2i ∗
(Pi∗ + Pi )(X − E[X]),
Z T 2
h(Qi + Q̄i )E[X], E[X]idt 1
+ E[Ui∗ ] = − (Ri + R̄i )−1 (B2i + B̄2i )∗ (P̄i∗ + P̄i )E[X],
0 2
10 VOLUME 4, 2016
1 X
+ (Pi∗ + Pi )B2j Rj−1∗ B2j
∗
(Pj∗ + Pj ),
4
j∈N \{i}
P̄˙i = −Qi + Q̄i − P (B1 + B̄1 ) − (B1 + B̄1 )∗ Pi

1 1 1
+ (P̄i∗ + P̄i )(B2i + B̄2i )(Ri + R̄i )− 2 ∗ (Ri + R̄i )− 2
4
(B2i + B̄2i )∗ (P̄i∗ + P̄i )
1 X
+ (P̄j∗ + P̄j )∗ (B2j + B̄2j )(Rj + R̄j )−1∗
4
j∈N \{i}
(B2j + B̄2j )∗ (P̄i∗ + P̄i )

1 X
+ (P̄i∗ + P̄i )(B2j + B̄2j )(Rj + R̄j )−1∗
4
j∈N \{i}
(B2j + B̄2j )∗ (P̄j∗

+ P̄j ),
1
δ̇i = − h(Pi∗ + Pi )S0 , S0 i,
2
with boundary conditions Pi (T ) = Qi (T ), P̄i (T ) =
Qi (T ) + Q̄i (T ), and δi (T ) = 0, whenever these differential
equations have a unique solution that does not blow up within
[0, T ].
2) MatLab-based MFTG Toolbox Example

The following MatLab script corresponds to a two-player
matrix-valued non-cooperative MFTG example from the
function Problem_setup_noncoopetitive_Matrix
_MFTG.m, and its procedure corresponds with the pseu-
docode presented in Figure 12:
Listing 7: Problem Setup function for the LQ Matrix-Valued

Non-Cooperative MFTG Example
2 T = 3 ; % Simulation time
4 %%%%% System-State-Dynamics Parameters %%%%%
5 n = 2 ; % number of players
6 d = 2 ;
7 B1 = [-1 -2 ; -3 -4] ;
8 bB1 = [-0.1 -0.2 ; -0.3 -0.4] ;
9 B2 = zeros(d,d,n) ;
10 B2(:,:,1) = [0.1 0.1 ; 0.2 0.2] ; %
11 B2(:,:,2) = [0.1 0.2 ; 0.1 0.2] ; %
12 %
13 bB2 = zeros(d,d,n) ;
FIGURE 12: Pseudocode corresponding to the matrix-valued 14 bB2(:,:,1) = [0.4 0.5 ; 0.6 0.7] ; %
non-cooperative and/or fully-cooperative MFTG parameters 15 bB2(:,:,2) = [1.4 1.5 ; 1.6 1.7] ; %
settings.
16 %
17 S = 2 ; % \sigma_0
19 R = zeros(d,d,n) ;
∗
Li (X, U ) = EhPi (0)(X0 − E[X0 ]), X0 − E[X0 ]i 20 R(:,:,1) = eye(d) ;
21 R(:,:,2) = 2*eye(d) ;
+ EhP̄i (0)E[X0 ], E[X0 ]i + E[δi (0, s(0))], 22 %
23 bR = zeros(d,d,n) ;
for all i ∈ {1, . . . , n}, where Pi , P̄i , and δi solve the 24 bR(:,:,1) = eye(d) ;
following differential equations: 25 bR(:,:,2) = 2*eye(d) ;
26 %
Ṗi = −Qi − Pi B1 − B1∗ Pi 27 Q = zeros(d,d,n) ;
28 Q(:,:,1) = 0.1*eye(d) ;
1 −1∗ −1 ∗
+ (Pi∗ + Pi )B2i Ri 2 Ri 2 B2i (Pi∗ + Pi ) 29 Q(:,:,2) = 0.2*eye(d) ;
4 30 %
1 X 31 bQ = zeros(d,d,n) ;
+ (Pj∗ + Pj )B2j Rj−1∗ B2j
∗
(Pi∗ + Pi ) 32 bQ(:,:,1) = 0.3*eye(d) ;
4 33 bQ(:,:,2) = 0.4*eye(d) ;
j∈N \{i}
VOLUME 4, 2016 11
Optimal Strategies u1
x1 0
System State x
6 Ex1 0.1
u11 u21
x2 -0.05 Eu11 Eu21
4 Ex2 u12 0 u22
x3 -0.1 Eu12 Eu22
2 Ex3 u13 -0.1 u23
x4 Eu13 Eu23
-0.15 u14 -0.2 u24
Ex4
0 Eu14 Eu24
-0.2 -0.3
0 1 2 3 0 1 2 3 0 1 2 3
(a) (b) (c)
2 4 1
P11 P21
P12 P22
Riccati bP1
Riccati P1
Riccati P2
1 P13 2 P23 0.5
P14 P24
bP11
0 0 0 bP12
bP13
bP14
-1 -2 -0.5
0 1 2 3 0 1 2 3 0 1 2 3
(d) (e) (f)
3 20 25
1
2 2 20
15
Riccati bP2
1 15
Cost
10
0 bP21 10
bP22
5
-1 bP23 5
bP24
-2 0 0
0 1 2 3 0 1 2 3 1 2
time [s] time [s] Player
(g) (h) (i)
FIGURE 13: Results corresponding to the LQ Matrix-Valued Non-Cooperative MFTG. These plots are generated by the MatLab
function Figures_MFTG_MaNonCoop.p
Listing 8: Complete simulation of the LQ Matrix-Valued Non-

34 % Cooperative MFTG Example
35 Q_T = Q ;
36 bQ_T = bQ ; 1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
37 %%%%%%%%%%%% Initial Conditions %%%%%%%%%%%% 2 clear all; close all ; clc
38 x0 = [1 2 ; 3 4] ; % initial condition 3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
39 Ex0 = x0 ; % initial condition 4 Problem_setup_noncoopetitive_Matrix_MFTG
40 variables0 = ... 5 [Output_Data] = ...
[ones(2*n*d^2+n,1);x0(:);Ex0(:)] ; noncooperative_Matrix_MFTG(Input_Data) ;
41 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 6 Figures_MFTG_MaNonCoop(Output_Data)
42 Input_Data = struct('T',T,'hh',hh,'n',n,... 7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
43 'd',d,'B1',B1,'bB1',bB1,'B2',B2,...
44 'bB2',bB2,'S',S,'R',R,'bR',bR,... Once the MFTG problem is solved, the function
45 'Q',Q,'bQ',bQ,'Q_T',Q_T,'bQ_T',bQ_T,... Figures_MFTG_MaNonCoop.p generates the Figure 13,
46 'x0',x0,'Ex0',Ex0,'variables0',variables0) ;
47 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% where it can be verified the satisfaction of the boundary
conditions of the Riccati equations.
Therefore, next MatLab script corresponds to the simula- B. COOPERATIVE APPROACH

tion for the matrix-valued non-cooperative MFTG problem
Finally, we consider the matrix-valued cooperative MFTG
using MAIN_non_cooperative_Matrix_MFTG.m.
problem, i.e.,
The core function noncooperative_Matrix_MFTG.p
is evoked, which returns the solution as presented in Table 5. X
L0 (X, U ) = h(Qi + Q̄i )E[X(T )], E[X(T )]i
i∈N
12 VOLUME 4, 2016
TABLE 5: Detail output data for the LQ Matrix-Valued Non-

Cooperative MFTG where P0 , P̄0 , and δ0 solve the following differential equa-
tions:
hX i
Output Data Dimensions Ṗ0 = −Q0 − P0 B1 − B1∗ P0 + P0 B2i Ri−1 B2i
∗
P0 ,
Output_Data.time 1 × hh j∈N
Output_Data.P_r d × d × n × hh
Output_Data.bP_r d × d × n × hh P̄˙0 = −Q0 − Q̄0 − P̄0 (B1 + B̄1 ) − (B1 + B̄1 )∗ P̄0
 
Output_Data.delta_r n × hh X
Output_Data.x_r d × d × hh + P̄0  (B2i + B̄2i )(Ri + R̄i )−1 (B2i + B̄2i )∗  P̄0 ,
Output_Data.Ex_r d × d × hh j∈N
Output_Data.u_r d × d × n × hh δ̇0 = −hP0 S0 , S0 i,
Output_Data.Eu_r d × d × n × hh
Output_Data.L n with boundary conditions P0 (T ) = Q0 (T ), P̄0 (T ) =
Q0 (T ) + Q̄0 (T ), and δ0 (T ) = 0. Notice that these Riccati
equations have positive solution P0 , P̄0 , and δ0 and there is
no blow up in [0, T ].
X
+ hQi (X(T ) − E[X(T )]), X(T ) − E[X(T )]i 2) MatLab-based MFTG Toolbox Example
i∈N The following MatLab script is developed based on the
XZ T procedure shown in the pseudocode in Figure 12. The
+ hQi (X − E[X]), X − E[X]idt illustrative example corresponds to a two-player matrix-
i∈N 0
valued cooperative MFTG example from the function
XZ T MAIN_cooperative_Matrix_MFTG.m:
+ h(Q + Q̄ )E[X], E[X]idt
i i
i∈N 0
XZ T
Listing 9: Problem Setup function for the LQ Matrix-Valued
+ hRi (Ui − E[Ui ]), Ui − E[Ui ]idt Cooperative MFTG example
i∈N 0
XZ T 2 T = 2 ; % Simulation time
+ h(Ri + R̄i )E[Ui ], E[Ui ]idt, 3 hh = 1000 ; % Number of simulation-steps
i∈N 0 4 %%%%% System-State-Dynamics Parameters %%%%%
5 n = 2 ; % number of players
min E [L0 (X, U )] , 6 d = 2 ;
{Ui ∈Ui }i∈N
7 B1 = [-1 -2 ; -1 -2] ;
subject to 8 bB1 = [-0.2 -0.1 ; -0.2 -0.1] ;
  9 B2 = zeros(d,d,n) ;
10 B2(:,:,1) = [0.1 0.1 ; 0.2 0.2] ; %
X 11 B2(:,:,2) = [0.2 0.2 ; 0.1 0.1] ; %
dX = B1 X + B̄1 E[X] + B2j Uj + B̄2j E[Uj ]  dt 12 %
j∈N 13 bB2 = zeros(d,d,n) ;
14 bB2(:,:,1) = [1 2 ; 3 4] ; %
+ S0 dB, 15 bB2(:,:,2) = [0.1 0.2 ; 0.3 0.4] ; %
X(0) = X0 16 %
17 S = 3 ; % \sigma_0
%%%%%%%% Cost functional Parameters %%%%%%%%
P P
where Q0 = i∈N Qi , Ri , Q̄0 = i∈N Q̄i , R̄i are positive 18
19 R = zeros(d,d,n) ;
with values in Rd×d . 20 R(:,:,1) = eye(d) ;
21 R(:,:,2) = 2*eye(d) ;
22 %
23 bR = zeros(d,d,n) ;
1) Semi-Explicit Solution 24 bR(:,:,1) = eye(d) ;
25 bR(:,:,2) = 2*eye(d) ;
26 %
P n Pn in [3]): Assume that Q0 :=
Proposition 5 (Result presented 27 Q = zeros(d,d,n) ;
Q
j=1 j , Q0 + Q̄ 0 := j=1 [Qj + Q̄j ], Ri , Ri + R̄i are 28 Q(:,:,1) = 2*eye(d) ;
symmetric positive definite. P The fully cooperative solution of 29 Q(:,:,2) = 3*eye(d) ;
n 30 %
the problem inf (U1 ,...,Un ) E[ j=1 Lj ] is given by 31 bQ = zeros(d,d,n) ;
32 bQ(:,:,1) = 4*eye(d) ;
Ui∗ = − Ri−1 B2i
E[Ui∗ ] ∗
P0 (X − E[X]), 33 bQ(:,:,2) = 5*eye(d) ;
%
E[Ui∗ ] −(Ri + R̄i ) (B2i + B̄2i )∗ P̄0 E[X],
−1 34
= 35 Q_T = Q ;
∗
L0 (X, U ) = EhP0 (0)(X0 − E[X0 ]), X0 − E[X0 ]i 36 bQ_T = bQ ;
37 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+ EhP̄0 (0)E[X0 ], E[X0 ]i + E[δ0 (0)], 38 Q0 = zeros(d,d) ;
39 Q0_T = zeros(d,d) ;
i ∈ {1, . . . , n}, 40 bQ0 = zeros(d,d) ;
VOLUME 4, 2016 13
6 0
x1
0 u21
System State x
Ex1
x2 u11 Eu21
4 -0.5
Eu11
Ex2 -5 u22
x3 u12
Eu22
Ex3 Eu12
2 -10 u13 u23
x4 -1
Eu13 Eu23
Ex4
-15 u14 u24
0 Eu14 Eu24
-20 -1.5
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
(a) (b) (c)
10 15 200
0
10
5 150
Riccati bP
Riccati P
5
0 100
P01 0 bP01
P02 bP02
-5 50
P03 -5 bP03
P04 bP04
-10 -10 0
0 0.5 1 1.5 2 0 0.5 1 1.5 2 0 0.5 1 1.5 2
(d) (e) (f)
FIGURE 14: Results corresponding to the LQ Matrix-Valued Cooperative MFTG example. These plots are generated by the
MatLab function Figures_MFTG_MaCoop.p
41 bQ0_T = zeros(d,d) ;
42 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
300
43 for i = 1 : n
44 Q0 = Q0 + Q(:,:,i) ;
45 Q0_T = Q0_T + Q_T(:,:,i) ; 200
Cost
46 bQ0 = bQ0 + bQ(:,:,i) ;

47 bQ0_T = bQ0_T + bQ_T(:,:,i) ;
48 end 100
49 %%%%%%%%%%%% Initial Conditions %%%%%%%%%%%%
50 x0 = [4 2 ; 3 1] ; % initial condition
51 Ex0 = x0 ; % initial condition
52 variables0 = ... 0
[ones(2*d^2+1,1);x0(:);Ex0(:)] ; % 1 2
53 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Player
54 Input_Data = struct('T',T,'hh',hh,'n',n,...
55 'd',d,'B1',B1,'bB1',bB1,'B2',B2,... FIGURE 15: Costs of the player in the LQ Matrix-Valued
56 'bB2',bB2,'S',S,'R',R,'bR',bR,... Cooperative MFTG example. This plot is generated by the
57 'Q0',Q0,'bQ0',bQ0,'Q0_T',Q0_T,...
MatLab function Figures_MFTG_MaCoop.p
58 'bQ0_T',bQ0_T,'x0',x0,...
59 'Ex0',Ex0,'variables0',variables0) ;
60 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Finally, the function Figures_MFTG_MaCoop.p
Therefore, by using the structure Input_Data provided presents the results in Figures 14 and 15 showing the optimal
by the problem setup, the game is solved by means of trajectories for the system states, control inputs, and Riccati
the function cooperative_Matrix_MFTG.p, which re- equations; and the optimal costs, respectively.
turns a structure Output_Data as presented in Table 6.
V. HOW TO GET THE MFTG TOOLBOX?
The codes and functions composing the MatLab-based
Listing 10: Complete simulation of the LQ Matrix-Valued MFTG Toolbox (see Figure 1) can be downloaded from the
Cooperative MFTG
website of the research group Learning & Game Theory
1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2 clear all; close all ; clc Laboratory, or from the personal websites of the authors. By
3 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% the time this paper has been submitted, the MatLab-based
4 Problem_setup_coopetitive_Matrix_MFTG MFTG Toolbox is available at the following links:
5 [Output_Data] = ...
cooperative_Matrix_MFTG(Input_Data) ; • https://wp.nyu.edu/landg/
6 Figures_MFTG_MaCoop(Output_Data) • https://wp.nyu.edu/jbarreirogomez/
7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
14 VOLUME 4, 2016
TABLE 6: Detail output data for the LQ Matrix-Valued Coop-

erative MFTG [14] T. E. Duncan. Linear-exponential-quadratic gaussian control. IEEE
Transactions on Automatic Control, 58(11):2910–2911, 2013.
[15] T. E. Duncan. Linear-quadratic stochastic differential games with general
Output Data Dimensions noise processes. In F. El Ouardighi and K. Kogan, editors, In Models and
Output_Data.time 1 × hh Methods in Economics and Management Science, volume 198, pages 17–
26. Operations Research and Management Series; Springer International
Output_Data.P_r d × d × hh Publishing, 2014.
Output_Data.bP_r d × d × hh [16] I. Eskinazi and B. J. Fregly. An open-source toolbox for surrogate
Output_Data.delta_r 1 × hh modeling of joint contact mechanics. IEEE Transactions on Biomedical
Engineering, 2015. 10.1109/TBME.2015.2455510.
Output_Data.x_r d × d × hh [17] S. Queiros, P. Morais, D. Barbosa, J. C. Fonseca, J. L. Vilac a, and
Output_Data.Ex_r d × d × hh J. D’hooge. MITT: medical image tracking toolbox. IEEE Transactions
Output_Data.u_r d × d × n × hh on Medical Imaging. DOI 10.1109/TMI.2018.2840820.
[18] Y. Jiang and S. Yin. Recent advances in key-performance-indicator
Output_Data.Eu_r d × d × n × hh oriented prognosis and diagnosis with a MATLAB toolbox: DB-
Output_Data.L n KIT. IEEE Transactions on Industrial Informatics, 2018. DOI
10.1109/TII.2018.2875067.
[19] U. Picchini. SDE Toolbox: simulation and estimation of stochastic
differential equations with MATLAB, 2007.
VI. CONCLUSIONS
We have presented a user-friendly MatLab-based toolbox in
order to solve in a semi-explicit manner different MFTG
problems involving an arbitrary number of players, e.g., non-
cooperative, fully-cooperative and co-opetitive approaches.
We have also addressed both scalar-valued and matrix-valued
games. As further work, we plan to continuously add more
functions to the toolbox, e.g., incorporating regime switching
and jumps.
ACKNOWLEDGMENT
We gratefully acknowledge support from U.S. Air Force
Office of Scientific Research under grant number FA9550-
17-1-0259
REFERENCES
[1] T. Duncan and H. Tembine. Linear-quadratic mean-field-type games: A
direct method. Games, 9(7), 2018.
[2] J. Barreiro-Gomez ; T. E. Duncan ; H. Tembine. Co-opetitive linear-
quadratic mean-field-type games. IEEE Transactions on Cybernetics, DOI:
10.1109/TCYB.2019.2901006, 2019.
[3] J. Barreiro-Gomez, T. Duncan, and H. Tembine. Matrix-valued mean-
field-type games: Risk-sensitive, adversarial, and risk-neutral linear-
quadratic case. arXiv preprint arXiv:1904.11346, 2019.
[4] H. Markowitz. Portfolio selection. The Journal of Finance, 7:77–91, 1952.
[5] B. Djehiche, H. Tembine, and R. Tempone. A stochastic maximum
principle for risk-sensitive mean-field type control. IEEE Transactions on
Automatic Control, 60(10):2640–2649, 2015. JULIAN BARREIRO-GOMEZ received his B.S.
[6] J. Yong. Linear-quadratic optimal control problems for mean-field stochas- degree (cum laude) in Electronics Engineering
tic differential equations. SIAM Journal on Control and Optimization, from Universidad Santo Tomas (USTA), Bogota,
51(4):2809–2838, 2013. Colombia, in 2011. He received the MSc. degree
[7] M. Tang and Q. Meng. Linear-quadratic optimal control problems for in Electrical Engineering and the Ph.D. degree
mean-field stochastic differential equations with jumps. Asian Journal of in Engineering from Universidad de Los Andes
Control, 21(3):1–15, 2019. (UAndes), Bogota, Colombia, in 2013 and 2017,
[8] H. Pham. Linear quadratic optimal control of conditional mckean- respectively. He received the Ph.D. degree (cum
vlasov equation with random coefficients and applications. Probability, laude) in Automatic, Robotics and Computer Vi-
Uncertainty and Quantitative Risk, 1(7):1–26, 2016. sion from the Technical University of Catalonia
[9] H. Tembine. Uncertainty quantification in mean-field-type teams and (UPC), Barcelona, Spain, in 2017; the best Ph.D. thesis in control engi-
games. In Proceedings of the IEEE Control Conference on Decision and neering 2017 award from the Spanish National Committee of Automatic
Control (CDC), pages 4418–4423, Osaka, Japan, 2015. Control (CEA) and Springer; and the EECI Ph.D. Award from the European
[10] T. E. Duncan and B. Pasik-Duncan. Solvable stochastic differential games
Embedded Control Institute in recognition to the best Ph.D. thesis in Europe
in rank one compact symmetric spaces. International Journal of Control.
in the field of Control for Complex and Heterogeneous Systems 2017.
[11] T. E. Duncan. Linear exponential quadratic stochastic differential games.
IEEE Transactions on Automatic Control, 61(2016):2550–2552, 2016. He received the ISA Transactions Best Paper Award 2018 in Recognition
[12] T. E. Duncan and B. Pasik-Duncan. Linear-quadratic fractional gaussian to the best paper published in the previous year. He is currently a Post-
control. SIAM Journal of Control and Optimization, 51(2013):4604–4619, Doctoral Associate in the Learning & Game Theory Laboratory at the New
2013. York University in Abu Dhabi (NYUAD), United Arab Emirates. His main
[13] T. E. Duncan and B. Pasik-Duncan. Linear-exponential-quadratic control research interests are Mean-Field-Type Games, Risk-Aware Engineering,
for stochastic equations in a hilbert space. Dynamic Systems and Applica- Constrained Evolutionary Game Dynamics, Distributed Optimization, and
tions, 21. Distributed Predictive Control
VOLUME 4, 2016 15
HAMIDOU TEMBINE received the M.S. degree

graduated in Applied Mathematics from Ecole
Polytechnique (Palaiseau, France) and received
the Ph.D. degree in Computer Science from IN-
RIA and University of Avignon, France. He fur-
ther received his Master degree in game theory and
economics. His main research interests are learn-
ing, evolution, and games. In 2014, Dr. Tembine
received the IEEE ComSoc Outstanding Young
Researcher Award for his promising research ac-
tivities for the benefit of the society. He was the recipient of 10+ best paper
awards in the applications of game theory. Tembine holds 150+ scientific
publications including magazines, letters, journals and conferences. He is
author of the book on distributed strategic learning for engineers (published
at CRC Press, Taylor & Francis 2012) which received book award 2014,
and co-author of the book Game Theory and Learning in Wireless Networks
(Elsevier Academic Press). Dr. Tembine has been co-organizer of several
scientific meetings on game theory in networking, wireless communications
and smart energy systems. He has been TPC member and reviewer for sev-
eral international journals and conferences. He has been a visiting researcher
at University of California at Berkeley (US), University of McGill (Mon-
treal, Quebec, Canada), University of Illinois at Urbana-Champaign (UIUC,
US), Ecole Polytechnique Federale de Lausanne (EPFL, Switzerland) and
University of Wisconsin (Madison, US). He is a senior member of IEEE. He
is a Next Einstein Fellow.
16 VOLUME 4, 2016

A MATLAB-based Mean-Field-Type Games Toolbox: Continuous-Time

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A MATLAB-based Mean-Field-Type Games Toolbox: Continuous-Time

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in a future issue of this journal, but has not been

ABSTRACT In this paper, we present a user-friendly MatLab-based Mean-Field-Type Games (MFTG)

INDEX TERMS Mean-field-type games, simulations, MatLab-based toolbox

TABLE 1: MatLab functions of the first version of the MFTG-Toolbox

the paper, e.g., Figures 5-15.

III. SCALAR MEAN-FIELD-TYPE GAMES

The Non-Cooperative Linear-Quadratic MFTG Problem is

12 %%%%%%%% Cost functional Parameters %%%%%%%%

16 bq = [0.1 0.2 0.3] ; % \bar{q} = ...

TABLE 2: Detail output data for the LQ Scalar-Valued Non-

Output Data Dimensions 2

7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1) Semi-Explicit Solution

where α0 (T ) = q0 , β0 (T ) = q0 + q̄0 , and γ0 = δ0 = 0.

1.2 0.08 1.5

C. CO-OPETITIVE APPROACH 1) Semi-Explicit Solution

maker is considering positively j 0 s playoff in this own 2 2

TABLE 4: Detail output data for the LQ Scalar-Valued Co- Z T

P̄˙i = −Qi + Q̄i − P (B1 + B̄1 ) − (B1 + B̄1 )∗ Pi

(B2j + B̄2j )∗ (P̄i∗ + P̄i )

(B2j + B̄2j )∗ (P̄j∗

2) MatLab-based MFTG Toolbox Example

Listing 7: Problem Setup function for the LQ Matrix-Valued

Listing 8: Complete simulation of the LQ Matrix-Valued Non-

Therefore, next MatLab script corresponds to the simula- B. COOPERATIVE APPROACH

TABLE 5: Detail output data for the LQ Matrix-Valued Non-

46 bQ0 = bQ0 + bQ(:,:,i) ;

TABLE 6: Detail output data for the LQ Matrix-Valued Coop-

HAMIDOU TEMBINE received the M.S. degree

You might also like