A Naive Approach to Comparing a Classic

Deep Neural Network versus a Convolutional

Neural Network in 2048

BY: NIALL MANDAL

PROF: DR CARMEN ACEVEDO BUTCHER

April 26th, 2023

SOME BACKGROUND

What are some key terms?

● Deep Neural Network (DNN): machine learning algorithm modeled

after the structure and function of the human brain

● Convolutional Neural Network (CNN): a type of deep learning algorithm designed to

automatically learn and extract features from input images

● Reinforcement Learning (RL): a subfield of machine learning that involves training an

agent to make decisions based on feedback it receives from its environment

WHAT ARE WE TESTING?

Research Question

I want to compare the performance of a DNN versus a CNN’s performance in playing

the hit mobile-game 2048. This performance will be measured using statistical tests

(hypothesis testing)

Source: Cirulli, Gabriele. 2048, Solebon LLC, 9 Mar. 2014, https://play2048.co/.

WHAT IS 2048?

Agent Environment Overview

The player slides numbered tiles on a grid to

combine them and create a tile with the number

2048. Reinforcement learning is used here to train

Neural Networks to play the game by rewarding it

for creating higher value tiles and penalizing it for

making illegal moves.

Source: Cirulli, Gabriele. 2048, Solebon LLC, 9 Mar. 2014, https://play2048.co/.

HOW DOES REINFORCEMENT LEARNING ACTUALLY GET EXECUTED

Reinforcement Learning Background

Reinforcement learning is a fascinating field with a rich history and exciting prospects for

the future. As we continue to develop new learning algorithms and applications, we will gain

a better understanding of how to create intelligent, adaptive machines that can help us solve

complex problems and improve our lives.

Source: Galatzer-Levy, Isaac, Kelly Ruggles, and Zhe Chen. "Data Science in the Research Domain Criteria Era: Relevance of Machine
Learning to the Study of Stress Pathology, Recovery, and Resilience." Chronic Stress, vol. 2, 2018, pp. 247054701774755.
doi:10.1177/2470547017747553.
THE METHODOLOGY OF EXECUTING THIS EXPERIMENT

Methodology

● Both neural networks will be trained for 1,500,000 steps

○ Equivalent of making 1,500,000 moves in various games of 2048.

● This training period will allow each DNN/CNN to develop a “policy”, or strategy in

playing the game with theoretically optimal moves

● To measure performance, each model will play 5000 individual games of 2048

○ A hypothesis test, and a confidence interval for the difference between two means will

be calculated to determine which performs

Source: Created better
by Author (Niall Mandal)
THE DNN PERFORMED BETTER BY A SIGNIFICANT MARGIN

Results

● DNN had a higher performance than the

Convolutional Neural Network

● DNN had a higher mean and median than

the CNN

● CNN had a higher maximum score, but

the performance metric of this experiment

is mean score Source: Created by Author (Niall Mandal)

THE DNN PERFORMED BETTER BY A SIGNIFICANT MARGIN

Hypothesis Test: Difference of Two Means

Source: Created by Author (Niall Mandal)

THE DNN PERFORMED BETTER BY A SIGNIFICANT MARGIN PT. 2

Conclusion

Since our p-value of 3.5385 * 10^-89 < 0.05, we can reject the null hypothesis (H0). There is

enough statistical evidence to conclude that the mean score of games played by the DNN is

greater than the mean score of games played by the CNN.

Thank you for watching!
contact: niallmandal@berkeley.edu
Works Cited
Başar, Tamer. “Tamer Başar: Introduction.” Tamer Basar, University of Illinois Urbana-Champaign, 20 Feb. 2014,
http://tamerbasar.csl.illinois.edu/.

Dangtongdee, K. (2018). Plant Identification Using Tensorflow. California Polytechnic State University Computer Engineering Department,
1–17. https://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1271&context=cpesp

Guei, Hung, et al. “Using 2048-like Games as a Pedagogical Tool for Reinforcement Learning.” ICGA Journal, vol. 40, no. 3, 1 May 2019,
pp. 281–293., https://doi.org/10.3233/icg-180062.

Levine, Zachariah. “Learning 2048 with Deep Reinforcement Learning.” David R. Cheriton School of Computer Science, University of
Waterloo, 3 Mar. 2017, https://cs.uwaterloo.ca/~mli/zalevine-dqn-2048.pdf.

Knight, Will. “Reinforcement Learning.” MIT Technology Review, MIT Technology Review, 17 Sept. 2021,
https://www.technologyreview.com/technology/reinforcement-learning/.

McElwee, Steven, et al. “Deep Learning for Prioritizing and Responding to Intrusion Detection Alerts.” MILCOM 2017 - 2017 IEEE Military
Communications Conference (MILCOM), 11 Dec. 2017, https://doi.org/10.1109/milcom.2017.8170757.

Yamashita, Rikiya, et al. “Convolutional Neural Networks: An Overview and Application in Radiology.” Insights into Imaging, vol. 9, no. 4,
22 June 2018, pp. 611–629., https://doi.org/10.1007/s13244-018-0639-9.

Yang, Yanwei, et al. “Application of Scikit and Keras Libraries for the Classification of Iron Ore Data Acquired by Laser-Induced Breakdown
Spectroscopy (LIBS).” Sensors, vol. 20, no. 5, Mar. 2020, p. 1393. Crossref, https://doi.org/10.3390/s20051393.