Dota 2 with Large Scale Deep Reinforcement Learning

Theory

Artificial Intelligence can compete with the best players and can even surpass them in
Dota2, one of the most complicated games in the modern world.

Concepts

Artificial Intelligence, best players, one of the most complicated games in the modern
world.

Indicators of artificial intelligence

A system that can make decisions in real time and think about future consequences
based on experience about the game.

Variables Values

Reaction time Can make decisions in every 4th frame the

game runs

Future consequences Reward weights are calculated for every

decision

Game experience PFlops/s·days of computing power divided by

time needed per game

Indicators of the best players

The best players compete at a professional level and has accomplishments as a team

Variables Values

Best players Professional players in a professional team

Team accomplishments Championships won

Indicators of Dota2 as one of the most complicated games in the modern world.
The game has long time scales, has a lot of inputs to observe, and a lot of output actions
to consider, resulting in a lot of parameters to process in order to win

Variables Values

Long time scales The game lasts up to 45 mins on average

Lots of inputs to observe A team has 5 heroes each with up to around

16,000 inputs for an AI to observe each
timestep which are mostly floating point
values.

For comparison, Chess requires up to a

thousand values to observe each timestep.
Go requires up to 6000 values per timestep,
but these are binary values only.

Wide action space to consider Each hero could have up to 1, 837, 080
action combinations before checking if the
action is available or not.

A hero could have around 8,000 to 80,000

actions per time step after these checks.

Parameters A neural network may need to process

around 159 million parameters