“Practice with AI in Chess and its Effects in Move Accuracy”

A Thesis Presented to the

Department of Senior High School

NU Laguna

In Partial Fulfillment of the Requirements

for the course Practical Research

Submitted by:

Higuchi, Ryu

March 2024
 Chapter 1

INTRODUCTION

The sport of chess has experienced a large explosion of popularity during the pandemic,

leading a significant amount of people to take it on as a source of entertainment. However,

having little to no experience, many of them found it hard to play the game. According to

Saariluoma, P., (1992), lesser skilled players have much more simple chess positions than those

of higher skill, and tend to blunder more. Players who are advanced and above on the other

hand can calculate more moves and plan ahead as they look at possible combinations that the

amateurs cannot (Frey, P. W., 2012).

These hardships discourage beginners in pursuing to continue playing chess. Their

perception of their progress and growth crumbles at the instant they face and lose to more

experienced players. Losing repeatedly to players with a higher skill level made beginners lose

interests in continuing to play the game (Chess.com, 2015).

Beginner players suffer because they lack the time and experience to learn how each

piece and move affect the dynamics of an ongoing chess game They often overlook mistakes

due to the little time they have accumulated in playing the game. An example of this is shown

in the book of Holding, D. H. (2021), where a master, a class A player, and a beginner were

presented a chess puzzle. It only took 3 minutes for the master to solve this problem, whereas

the class A player took 7 minutes and the beginner took 25 minutes to solve. This inexperience

is also emphasized in the research of Reginold, E. M. (2001), where novices were outperformed

by experts in multiple aspects of the game including, but not limited to, chess positions,

enumeration tasks (e.g. count the number of bishops), and visual span of the board.
 This study will adapt a quasi-experimental quantitative research that will have at least

30 NU Laguna senior high school students who are beginners in chess. The data will be

collected through observation using Chess.com and Stockfish for measurement. The study will

focus and determine how training and playing with chess AIs affect the move accuracy of the

player. It will also add to the lacking literature on this specific topic.

Statement of the Problem

This study seeks to determine the effects of playing and reviewing chess games with the aid of

AI to the move accuracy of beginner chess players of National University Laguna.

The answer to the stated problem above will be answered through this questions:

1. What is the measurement of the player’s move accuracy in terms of?

a. Number of good moves
b. Number of inaccuracies
c. Number of blunders
d. Overall accuracy percentage
2. How strong is the opponent (AI) that the player played against in terms of?
a. ELO rating
b. Move accuracy percentage
3. How much games and time did the player spent on reviewing?
4. Did the game reviews have a significant effect on the player’s performance? In what
way?

Theoretical conceptual framework

In order to examine the effect of monitoring and self-regulation on skill acquisition, the

present study asked novice chess players to provide judgments of learning (JOLs) and to

select moves for restudy after studying an endgame of chess. In four groups, we varied the

JOL instruction (present versus absent) and the selection instruction (free number of move
selections versus selection of at least two moves per chess exercise). After four learning

trials, participants were required to play against a chess computer. In the learning phase,

participants who were forced to select moves for restudy outperformed those who were free

to select moves for restudy when predicting the next computer move, even after controlling

for the actual number of restudied moves. Although the groups that did provide JOLs showed

better self-regulatory behaviour, there were no or even negative performance differences

between the groups that did provide JOLs and the groups that did not provide JOLs. This

same pattern emerged in the test phase: Although no differences were found between the

groups with and without JOLs, the groups that were forced to select moves for restudy

outperformed the groups that were free in the number of move selections. These data show

that, for novice chess players, the instruction to provide JOLs possibly places a high and

ineffective load on working memory and therefore has no effect on learning a chess endgame.

To examine the relation between prior knowledge and quality of self-regulation, further

research is needed that examines the effect of the JOL and selection instruction in groups that

differ in chess experience. Copyright © 2005 John Wiley & Sons, Ltd.

De Bruin, A. B., Rikers, R. M., & Schmidt, H. G. (2005). Monitoring accuracy and self‐

regulation when learning to play a chess endgame. Applied Cognitive Psychology, 19(2),

167-181.

The paper reviews the evidence for and against the recognition-association theory and a

forward-search (SEEK) theory of chess skill. The recognition-association theory appears to

be founded on indirect evidence concerning visual short-term memory, together with

supplementary assumptions that may be questioned, and provides no role for verbal

processes. There is no direct support for the theory, which omits forward search for reasons
that are reexamined. In contrast, the SEEK theory maintains that move choice is based on

search and evaluation processes supplemented (or else supplanted) by a knowledge base.

These processes are directly evidenced by experimental findings. The objection that search

theories cannot account for speed chess is met by a review of the available evidence. It is

concluded that chess skill relies on thinking ahead rather than on pattern recognition.

Holding, D. H. (1992). Theories of chess skill. Psychological Research, 54(1), 10-16.

the recognition–action theory of chess skill holds that expertise in chess is due primarily to

the ability to recognize familiar patterns of pieces. despite its widespread acclaim, empirical

evidence for this theory is indirect. one source of indirect evidence is that there is a high

correlation between speed chess and standard chess. Assuming that there is little or no time

for calculation in speed chess, this high correlation implies that calculation is not the primary

factor in standard chess. two studies were conducted analyzing 100 games of speed chess. in

study 1, we examined the distributions of move times, and the key finding was that players

often spent considerable time on a few moves. Moreover, stronger players were more likely

than weaker players to do so. study 2 examined skill differences in calculation by examining

poor moves. the stronger players made proportionally fewer blunders (moves that a 2-ply

search would have revealed to be errors). overall, the poor moves made by the weaker players

would have required a less extensive search to be revealed as poor moves than the poor

moves made by the stronger players. Apparently, the stronger players are searching deeper

and more accurately. these results are difficult to reconcile with the view that speed chess

does not allow players time to calculate extensively and call into question the assertion that

the high correlation between speed chess and standard chess supports recognition–action

theory.
Chang, Y. H. A., & Lane, D. M. (2016). There is time for calculation in speed chess, and

calculation accuracy increases with expertise. The American journal of psychology, 129(1),

1-9.

Scope and Limitations of the Study\

This study aims to measure, and find the effects of playing against chess bots to the

accuracy of chess players, and will be delimited to NU Laguna senior high school students with

an ELO of 0-1399. The main purpose of the study is to point out whether playing against

artificial intelligences will improve their move accuracy or not. The study will only focus on

the relation of AI and chess move accuracy.

Significance of the Study

Listed below are the possible beneficiaries of this research:

Beginner chess players. This study will be proof and guide on how novice players

could use AI in chess to their advantage. It will contain information on how playing and

practicing with chess bots will improve their accuracy and control in the game.

Future researchers. The study will aid researchers studying similar topics with

relation to either chess, AI, or both. It may serve as a reference on the correlation of AI and

chess in the sense of move accuracy.

Definition of Terms
AI – Artificial Intelligence; Chess computers; Chess bots (e.g., Stockfish, Deep Blue,

AlphaZero)

ELO – Invented by Arpad Elo, it is the numerical measurement of the relative strength of a

chess player to other players (chess.com)

Inaccuracy – A move not within the top 3 moves but also not a losing move (chess.com)

Mistake – A move that considerably reduces the advantage of the winning player to even

(chess.com)

Blunder - A move that loses the game (chess.com)

Chapter 2

REVIEW OF RELATED LITERATURE & STUDIES

Overview

This chapter discusses past researches and literature on the relation of machine & AI

with chess skill. The included studies will be a guide for the researcher in the observation and

analysis of the data gathered in this research.

Topical/Theme Presentation (APA style for citation)

De Groot's experiments involved showing players unfamiliar positions and recording their

thought processes. The results indicated that Grandmasters consistently chose the objectively

correct move, while experts often missed it. The average depth of search for both groups was

surprisingly low, debunking the myth of extensive exploration. De Groot then explored the

role of perception by limiting exposure to a position for a few seconds and testing players'

ability to recall it. Grandmasters and masters demonstrated significantly higher recall

accuracy than experts and class players.

The discussion delves into the intricacies of short-term memory (STM) and long-term

memory (LTM) in the context of chess perception. The limited capacity of STM,

approximately seven chunks, is highlighted, and the importance of rehearsal and recoding for

information transfer to LTM is emphasized. The interconnected and content-addressable

nature of LTM contrasts with computer memory systems.

Simon and Gilmartin's model is introduced, suggesting that better chess players have a larger

repertoire of chess patterns stored in LTM. The simulation indicates that the size of this

pattern vocabulary correlates with chess skill. The passage concludes by addressing the

misconception that memorizing chess patterns alone does not guarantee improvement in

gameplay. Instead, the ability to associate patterns with appropriate moves, known as

productions, is proposed as a key factor in skilled chess play. (Frey, P. W., 2012).

The intersection of chess and cognitive psychology has provided a captivating arena for

understanding the nuances of human thought processes. Through a meticulous series of five

protocol-analysis experiments, this research delved into the cognitive errors embedded within

chess players' strategic, tactical, and endgame decision-making. While the conventional

explanation of cognitive errors often leans towards working-memory overload, the study
challenges this notion, asserting that such errors are only partially explicable through this

lens. Notably, the working-memory loads induced by solution paths were found to be

generally modest, prompting the need for a more nuanced examination of apperceptive

mechanisms that control information intake.

In this exploration, it becomes evident that chess players' failures can be attributed to a dual

nature: either an inability to perceive the appropriate prototypical problem space or a failure

to close it due to the oversight of crucial task-relevant cues. This twofold breakdown in

cognitive processes results in a loss of "belief in the idea" among chess players, prompting a

subsequent restructuring of their approach. The intricate dance between perception, cue

recognition, and restructuring unveils the pivotal role of apperceptive information-selection

mechanisms, further complicating the journey towards finding a solution and rendering it

increasingly improbable. The study thus advocates for a holistic understanding of chess

players' cognitive errors, transcending the conventional focus on working-memory overload.

Saariluoma, P. (1992). Error in chess: The apperception-restructuring view. Psychological

Research, 54, 17-26.

The study discusses a research study that aims to shed light on the factors influencing chess

skill, particularly the debate between the importance of pattern recognition versus detailed

analysis and evaluation of move sequences in skilled players. The study involves comparing

grandmaster tournament games played under rapid and slow conditions, with a focus on the

number and magnitude of errors. The researchers also explore the neglected aspect of mental

imagery and visualization in chess, particularly in blindfold games where players make

moves without physically seeing the board. The findings challenge the notion that pattern

recognition is the dominant factor in chess expertise, showing that grandmasters make more
and larger mistakes under time constraints, while also revealing intriguing insights into the

role of mental imagery in blindfold chess.

The research uses a natural experiment, analyzing games played in Monaco tournaments

between 1993 and 1998 by 23 grandmasters, including world champions and top-ranked

players. The study employs a powerful chess computer program to assess errors objectively,

comparing blindfold and sighted games. Surprisingly, the results indicate that grandmasters

perform equally well in blindfold and sighted conditions, challenging traditional beliefs about

the importance of seeing the board. The essay highlights the need for a more comprehensive

understanding of the cognitive processes involved in chess expertise, suggesting that both

pattern recognition and detailed analysis play crucial roles, and that mental imagery may

serve as a missing link in this complex skill.

Anderson, C. (2023). The Cognitive Science of Chess: Insights into the Theory of Expertise

and Human vs Machine learning.

In the realm of decision-making, the increasing availability of detailed trace data offers a

unique opportunity to evaluate the quality of human decisions through algorithmic analysis.

This burgeoning field seeks to understand and predict the contexts in which individuals are

prone to errors. Focusing on the decisions made by chess players, a model system entrenched

in behavioral sciences, our investigation employs large-scale datasets encompassing several

million recorded games. Leveraging chess tablebases, which provide ground truth for

positions solved by computers but challenging for human players, we aim to formulate a

general framework for predicting human errors.

Our analysis revolves around three pivotal categories of features applicable to various

decision-making scenarios: the decision-maker's skill, the time allotted for decision-making,
and the inherent difficulty of the decision. Unveiling rich structures within each category, our

research underscores compelling evidence suggesting that, in the domain of chess, features

describing the inherent difficulty of a decision outshine those based on skill or time

constraints. As we delve into the dynamics of human error prediction, our findings shed light

on the nuanced interplay of these factors and pave the way for a more comprehensive

understanding of decision-making processes across diverse domains.

Anderson, A., Kleinberg, J., & Mullainathan, S. (2017). Assessing human error against a

benchmark of perfection. ACM Transactions on Knowledge Discovery from Data

(TKDD), 11(4), 1-25.

The development of computer chess programs has been marked by significant milestones and

shifts in approach since the post-World War II era. Early efforts, inspired by groundbreaking

papers from Shannon (1950) and Turing (1953), led to the introduction of the minimax

algorithm and the classification of chess programs into type A (brute force) and type B

(selective) categories. The success of type A programs, utilizing exhaustive search,

minimaxing, alpha-beta pruning, and other enhancements, overshadowed the interest in type

B programs, which aim to emulate human-like selectivity in move consideration. Despite

some recent work by Botvinnik et al., the development of effective type B strategies has been

limited.

The historical evolution of computer chess programming can be categorized into three broad

eras: the first era (1950 - c1975) characterized by hardware limitations and the emergence of

selective type B programs, the second era (c1975 - c1985) marked by increased processor

power and the advent of dedicated chess hardware, and the third era (c1985 onwards)
witnessing algorithmic advancements and the widespread use of personal computers with

improved playing strength. Move ordering techniques, such as iterative deepening, the killer

heuristic, and history heuristic, played a crucial role in optimizing the efficiency of chess

programs. The impact of computer chess on AI has evolved from being a testbed for AI

techniques to contributing domain-independent search methodologies applicable in various

areas. The ongoing refinement of traditional algorithms and the exploration of Shannon B

type strategies present avenues for future research and development in the field

Heath, D., Allum, D., & Square, P. (1997). The Historical Development of Computer Chess

and its Impact on Artificial Intelligence. Deep Blue Versus Kasparov: The Significance for

Artificial Intelligence, 63.

Machines capable of playing chess have fascinated people since the latter half of the 18th

century, when the Turk, the first of the pseudo-automatons, began a triumphal exhibition tour

of Europe.

The ability of a machine to play chess well has taken on symbolic meaning since the first

precomputer devices more than a century ago. In 1890 a Spanish scientist, Leonardo Torres y

Quevado, introduced an electromagnetic device—composed of wire, switch, and circuit—that

was capable of checkmating a human opponent in a simple endgame, king and rook versus

king. The machine did not always play the best moves and sometimes took 50 moves to

perform a task that an average human player could complete in fewer than 20. But it could

recognize illegal moves and always delivered eventual checkmate. Torres y Quevado

acknowledged that the apparatus had no practical purpose. As a scientific toy, however, it

gained attention for his belief in the capability of machines to be programmed to follow

certain rules.
A breakthrough came in 1948, when the research scientist Claude Shannon of Bell Telephone

Laboratories in Murray Hill, New Jersey, U.S., presented a paper that influenced all future

programmers. Shannon, like Torres y Quevada and Turing, stressed that progress in

developing a chess-playing program would have a wider application and could lead, he said,

to machines that could translate from language to language or make strategic military

decisions.

Computers began to compete against humans in the late 1960s. In February 1967 MacHack

VI, a program written by Richard Greenblatt, an MIT undergraduate, drew one game and lost

four in a U.S. Chess Federation tournament. Its results improved markedly, from a

performance equivalent to a USCF rating of 1243 to reach 1640 by April 1967, about the

average for a USCF member. The first American computer championship was held in New

York City in 1970 and was won by Chess 3.0, a program devised by a team of Northwestern

University researchers that dominated computer chess in the 1970s.

Technical advances accelerated progress in computer chess during the 1970s and ’80s. Sharp

increases in computing power enabled computers to “see” much further. Computers of the

1960s could evaluate positions no more than two moves ahead, but authorities estimated that

each additional half-move of search would increase a program’s performance level by 250

rating points. This was borne out by a steady improvement by the best programs until Deep

Thought played above the 2700 level in 1988. When Deep Blue, its successor, was

introduced in 1996, it saw as far as six moves ahead. (Gary Kasparov said he normally looks

only three to five moves ahead, adding that for humans more are not needed.)

By the late 1980s the strongest machines were capable of beating more than 90 percent of the

world’s serious players. In 1988 a computer, HiTech, developed at Carnegie Mellon

University, defeated a grandmaster, Arnold Denker, in a short match. In the same year
another Carnegie Mellon program, Deep Thought, defeated a top-notch grandmaster, Bent

Larsen, in a tournament game.

Copeland, B. (2024, March 6). artificial intelligence. Encyclopedia Britannica.

https://www.britannica.com/technology/artificial-intelligence

Soltis, A. E. (2024, February 13). chess. Encyclopedia Britannica.

https://www.britannica.com/topic/chess

In the two decades since the publication of Ericsson's influential paper on deliberate practice

and expert performance, the debate surrounding the interplay of nature and nurture in skill

development has evolved significantly. The deliberate practice theory, emphasizing the

pivotal role of focused, challenging practice in achieving expertise, has been scrutinized for

its exclusion of innate abilities or 'talent.' Sternberg (1996) challenged the theory, suggesting

that studies linking deliberate practice to expert performance may confound talent and

practice, neglecting those who dropped out of skill development. Recent research (De Bruin

et al., 2008) contradicts this assumption, demonstrating that elite, adolescent chess players

who dropped out showed similar improvement with just 1 hour of deliberate practice

compared to their peers who persisted.

The crux of the debate lies in Ericsson's dismissal of innate influences, with critics arguing

for a more nuanced perspective acknowledging the interaction between genetic and

environmental factors. In the context of chess expertise, the current study adopts a

prospective approach to investigate the influence of enjoyment, practice, and intelligence on

chess performance in children newly introduced to the game. While prior research generally

found limited evidence for a connection between intelligence and chess skill, the study by

Campitelli and Gobet (2011) underlines the importance of abundant deliberate practice but
also highlights substantial inter-individual variability and the inadequacy of practice alone for

reaching grandmaster level. The present study aims to shed light on this complexity by

considering prospective measures of practice, intelligence, and motivation, thereby

addressing some of the limitations of previous research in this domain.

De Bruin, A. B., Kok, E. M., Leppink, J., & Camp, G. (2014). Practice, intelligence, and

enjoyment in novice chess players: A prospective study at the earliest stage of a chess

career. Intelligence, 45, 18-25.

The fusion of chess play and psychological research yields a compelling realm of intellectual

satisfaction for participants. The intricate dance between strategy, foresight, and decision-

making in chess mirrors the complexities of the human mind. In the mid-1980s, the

confluence of accumulated experimental results provided the foundation for a captivating

narrative on chess skill. This period marked a juncture where the chess landscape had been

explored enough to weave together a reasonably satisfying story, laying bare the structure of

chess skill and identifying the remaining gaps in knowledge. Published in 1985, a seminal

work endeavored to encapsulate the progress made at the time, offering a comprehensive

account of the research process and shedding light on how chess players think, imagine, and

decide. The resulting synthesis became a testament to the captivating synergy between the

intellectual satisfaction derived from chess play and the empirical insights gleaned from

psychological research.

As chess players embarked on a journey into the recesses of cognitive processes, the 1985

publication emerged as a pivotal document. It not only summarized the strides made in

understanding chess skill but also illuminated the intricate components of the research

process itself. The narrative unraveled the enigmatic layers of the chessplayer's cognition,
showcasing the fusion of strategic thinking and psychological mechanisms. The book not

only painted a vivid picture of the known aspects of chess play but also delineated the

contours of the unknown, pointing out the gaps in knowledge that awaited exploration. The

intersection of chess play and psychological research, as articulated in this work, presented an

engaging enterprise that transcended the boundaries of both disciplines, offering a glimpse

into the profound intellectual satisfaction derived from unraveling the mysteries of the mind

through the intricate moves of a chessboard.

Holding, D. H. (2021). The psychology of chess skill. Routledge.

In investigating the cognitive processes involved in chess expertise, a study focused on a

specific check detection task within a 5x5 section of the chessboard. The experiment involved

a King and one or two potential checking pieces, with variations in the checking status

(presence or absence of a check) and the number of attackers (one or two). Notably, the

research revealed an intriguing difference in reaction time costs between novice and expert

chess players. Novices exhibited a greater reaction time cost when a distractor was

introduced, specifically in trials without a check, whereas experts showed a more consistent

reaction time across both yes and no check trials. This disparity suggests that experts employ

automatic and parallel encoding procedures for chess relations, highlighting the efficiency

and streamlined processing in their cognitive mechanisms.

Furthermore, the study delved into the nuanced aspects of check detection by contrasting

standard trials with those where one of two attackers was cued (colored red). In this variation,

participants were tasked with determining the checking status of the cued attacker while

disregarding the other. Notably, a Stroop-like interference effect emerged in trials where a

cued nonchecking attacker appeared alongside an attacker that was indeed checking
(incongruent condition). This finding suggests that, akin to the interference observed in

classic Stroop tasks, expert chess players face challenges when selectively attending to

relevant information while suppressing irrelevant cues. These insights into automatic

encoding and interference effects shed light on the cognitive intricacies underlying chess

expertise.

Reingold, E. M., Charness, N., Schultetus, R. S., & Stampe, D. M. (2001). Perceptual

automaticity in expert chess players: Parallel encoding of chess relations. Psychonomic

Bulletin & Review, 8(3), 504-510.

Chapter 3

METHODOLOGY

This chapter discusses the following methods and procedures to be used in conducting the

study, such as the research design, sampling technique, context and participants, research

locale, data gathering instruments, and ethical considerations.

Research Design

This research will adapt a quasi-experimental quantitative research. It involves numerical

data collection and statistical analysis. Quasi-experimental research compares groups with

different circumstances or treatments to find cause-and-effect links such as how certain

variables of playing against AI affects the player’s accuracy. It draws statistical conclusions

from quantitative data. For these reasons, the researcher believes this design would be the

best choice.

Sample and Sampling Design

 The researcher has chosen a stratified sampling technique where they would pick at

least 3 students per section for this study. This allows for much precise data statistics as every

section of the senior high school students to have representatives.

Context and Participants

The respondents for this research will be NU Laguna Senior High School students with

an ELO of 0-1399 in Chess.com. This means that it is required for a participant to have a

Chess.com account.

The participants will play against 3 different chess bots with varying levels of ELO or

difficulty. After their first games against these AIs, the researcher will collect the data on the

players’ initial move accuracy. They will then be given a minimum of 15 minutes and a

maximum of 30 minutes to practice and analyze their games before having a last game against

the AIs. The time it took to practice and their final move accuracy will be also recorded.

Ethical Consideration

In this study, informed consent must be obtained transparently and comprehensively,

ensuring participants are fully aware of the study's purpose, procedures, and potential risks.

Additionally, researchers must prioritize confidentiality and anonymity to protect participants'

privacy, using secure data storage and handling procedures. Fair treatment is essential,

necessitating unbiased participant selection and equitable distribution of research benefits and

burdens. Continuous monitoring and adaptation of research protocols to emerging ethical

standards ensure the ongoing ethical integrity of the study, fostering trust and upholding the

principles of respect, justice, and beneficence in the research process.