IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL.
17, 2024
A Feasible Study of a Deep Learning Model
Supporting Human–Machine Collaborative
Learning of Object-Oriented Programming
Feng Hsu Wang
Abstract—Due to the development of deep learning technology,
its application in education has received increasing attention
from researchers. Intelligent agents based on deep learning
technology can perform higher order intellectual tasks than ever.
However, the high deployment cost of deep learning models has
hindered their widespread application in education. In addition,
there needs to be more research on applying deep learning
technology in education. In this article, we develop an intelligent
agent using a performer-based encoder–decoder neural model to
classify object-oriented programming (OOP) errors in student
code and generate hint feedback in natural language to help
students correct the code. This study investigates the feasibility of
deploying this agent in an educational setting to support the
learning of OOP. This study first examines the low-speed
inference problem of the deep learning model. A fast inference
algorithm is proposed for the model, which achieves a speedup of
eighty times. This study further explores integrating a human–
machine collaborative learning process with the deep learning
agent. Students were surveyed about their perceptions of the
agent in supporting learning. Student responses are interpreted
within the learning partnerships model (LPM) framework to show
how the agent’s technical automation and autonomy features
support student-agent learning partnerships. Finally, implications
and suggestions for educational application and research of deep
learning technology are presented.
Index Terms—Deep learning technology, human–machine col-
laborative learning, intelligent tutoring system, learning partner-
ships model (LPM), object-oriented programming (OOP).
I. INTRODUCTION
O BJECT-ORIENTED programming (OOP) is an essential
programming skill that has evolved into a programming
paradigm adopted by software industries to develop reliable
and flexible software. As an elementary programming course
for university information-engineering departments, most stu-
dents perceive the course as complex. One reason is the need
for an appropriate tool to help students learn OOP concepts
and skills. More than information about syntactic code errors
Manuscript received 7 July 2022; revised 25 September 2022 and 28
November 2022; accepted 30 November 2022. Date of publication 2 Decem-
ber 2022; date of current version 4 January 2024. This work was supported by
the Ministry of Science and Technology, Taiwan, under Grants MOST 110-
2221-E-130-009 and MOST 111-2410-H-130-023.
The author is with the Department of Computer Science and Information
Engineering, Ming Chuan University, Taoyuan City 333, Taiwan
(e-mail: fhwang@mail.mcu.edu.tw).
Digital Object Identifier 10.1109/TLT.2022.3226345
1939-1382 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See ht_tps://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
413
provided by a compiler is needed to help them solve the struc-
tural design problems of OOP programs. Another reason is
that students must wait for a teacher to help them solve these
problems due to the many students in a classroom. Therefore,
there is an urgent need for computing tools to help students
learn OOP concepts and skills effectively.
With the advancement of deep learning technology, its
practical applications have been developed with great success
in many areas. Nevertheless, its application in educational
practice is still in its infancy [1]. Deep learning technology’s
high computational power requirements hinder its widespread
application in education. Therefore, downsizing deep learning
technology is the first step to overcoming this barrier. For
example, transformer [2] is a state-of-the-art deep learning
architecture that consumes much space and time to develop
solution models. A successful downsizing effort for trans-
former is performer [3], which takes only linear time to pro-
cess a sequence of tokens to build a solution model. However,
performer still suffers from slow inference speed during the
model deployment phase. For example, on a workstation-level
computer, the inference speed of performer will take tens of
minutes to generate feedback of hundreds of lengths. It is hard
to imagine students willing to spend so long waiting for a sys-
tem to provide feedback. Therefore, improving the model
inference speed is the first step to making the performer-based
deep learning models technically feasible for educational
practice.
Furthermore, successfully integrating new technologies into
education would improve students’ motivation and allow
them to achieve high-level educational goals through human–
machine collaboration [4]. Therefore, this study explores the
integration of the deep-learning agent into a human–machine
collaborative learning process. Based on the technical automa-
tion features of an agent [5], this study proposes a human–
machine collaborative learning process in which students
were asked to solve OOP problems with the aids of the intelli-
gent agent developed using the performer-based deep learning
model. Students were surveyed to find out what they think
about the agent that supports OOP learning. Their responses
to the survey are interpreted within the learning partnerships
model (LPM) [6] to show how the agent’s technical automa-
tion and autonomy features support student–agent learning
partnerships. Finally, the contributions of the intelligent agent
 414
and the potential risks of using the agent in programming edu-
cation are discussed, and future research directions are pro-
posed. As a result, the research questions of this study are as
follows.
1) How to improve the inference speed of the performer-
based model to make it a technically feasible solution
in education?
2) How the intelligent agent can support student–agent
learning partnerships in a human–machine collaborative
learning process?
The rest of this article is organized as follows. Section II
presents previous work on deep learning, intelligent tutor-
ing systems for programming education, human–machine
collaborative learning, and the LPM. Section III presents
the performer architecture, focusing on its fast attention
via positive orthogonal random features (FAVORþ) atten-
tion operation for learning a solution model. Section IV
presents the methods and evaluation criteria to address the
two research questions. Section V presents the experiment
designs. Sections VI and VII present the results and dis-
cussions, respectively. Finally, Section VIII concludes this
article.
II. LITERATURE REVIEW
A. Deep Learning Models for Natural Language Processing
Deep learning is a branch of machine learning that uses
multilayer neural networks to achieve specific learning tasks
from large amounts of data. Deep learning has become essen-
tial for building advanced intelligent systems in many applica-
tion fields, especially in computer vision, speech recognition,
and text-based natural language processing. Regarding the
deep learning technology for natural language processing, the
sequence-to-sequence (Seq2Seq) model has been the most
widely adopted architecture. Most early Seq2Seq models used
sequential networks, such as recurrent neural network and
long short-term memory. However, training these networks
takes longer because they must wait until the state at the previ-
ous time step has been calculated before calculating the state
at the current time step.
To solve this problem, transformer based on the attention
mechanism has made significant progress in image/speech
recognition and machine translation in recent years [2]. The
self-attention mechanism in transformer can compute interde-
pendencies between arbitrary data items in a sequence in par-
allel. However, transformer consumes O(L2) space and time to
conduct the attention operation on a sequence of length L.
Among many improved versions of transformer, performer [3]
is one of the most cost effective in terms of time and space.
Performer adopts a novel attention operation called FAVORþ,
which provides scalable, low variance, and unbiased estima-
tion of attention values that can be computed by kernels repre-
sented as a set of random feature map decompositions.
Therefore, the performer architecture was chosen to build the
deep learning model in this study.
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
B. Barriers and Challenges to Human–Machine
Collaborative Learning for Programming
Intelligent agent systems for educational contexts have long
attracted the attention of researchers aiming to apply artificial
intelligence to educational research and practice. A salient fea-
ture of such systems is generating feedback in natural language
to support learning [7], [8]. Educational feedback includes for-
mative feedback intended to change students’ thinking or behav-
ior to improve learning [9]. Formative feedback is more
important in learning than superficial feedback, such as grades
[10], [11]. For programming education, formative feedback con-
veys variants of knowledge, such as knowledge about concepts
[12], [13], knowledge about mistakes [14], [15], [16], knowl-
edge about how to proceed, such as edit-based hints on adding,
deleting, and modifying text in the source code to reach a solu-
tion [17], [18], [19], [20], [21]. However, while the edit-based
hints can guide students in revising their code, they may not help
students understand why the error happened. Finally, knowledge
feedback about metacognition could contain explanations of
metacognitive strategies or guiding questions [22].
Regarding approaches to generating feedback on program-
ming, the most common method is dynamic code analysis,
which runs a program against test cases and compares the out-
put to the expected one to report testing success or failure
[23]. However, this approach requires the cost of preparing
test cases for each problem. In addition, the success or failure
feedback is of little help for learning programming. The static
analysis method analyzes a program without running it, and it
can be applied to detect misunderstood concepts, the absence
or presence of some code structures [16], [24], and generate
hints to fix the errors [25]. However, the diversity of code cre-
ates challenges for static code analysis. Several techniques
can be applied to increase the number of codes a solution
model can recognize, such as code normalization to convert a
program to another in the same or different language at the
same abstraction level [26], [27]. Intent-based diagnostics use
a knowledge base of programming goals, plans, or buggy rules
to match with student code to find out which solution strategy
the student used [28], [29]. However, this approach uses bug
templates for a limited set of predefined exercises, so it cannot
handle the wide variety of student codes.
On the other hand, AI-based approaches can handle code
diversity better. However, it was found that they only account
for about 28% of the literature [8]. Some studies use neural
networks to generate feedback for specific programming
skills, such as recurrent programming [28], [30]. However, to
the best of our knowledge, deep learning models for generat-
ing feedback in natural language for open exercises and
assignments are rare [31]. It is worth noting that the automatic
program repairs technique has been widely applied in software
engineering industries. However, it has limited contribution to
programming education because it was developed for code
written by professional programmers and, therefore, does not
suit novice students well [32].
As intelligent systems are gradually integrated into human
life, effective collaborative learning with such systems
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED
becomes an important research issue [33]. Human–machine
collaboration can generally be described by two technical fea-
tures: automation and autonomy [5]. Technical automation is
about task allocation in which a technical system performs all
(or part of) tasks previously performed by humans. On the
other hand, technical autonomy is related to four technical
characteristics used to build intelligent agents: nontranspar-
ency, nondeterminism, adaptability, and openness [5]. Such a
taxonomy can be a valuable framework to capture an intelli-
gent agent’s characteristics and interpret human and machine
interactions.
From a constructivist and student-centered learning per-
spective, in human–machine collaborative learning, intelligent
agents can do data analysis tasks and present information in
an “intelligent” way that enables students to focus on critical
thinking and to discover meaning and value behind the data.
With the advances in deep learning technology, the levels of
automation and autonomy of an intelligent agent are raised.
As a learning partner, an intelligent agent with enhanced auto-
mation and autonomy could better support human learning
[33], [34], [35]. However, little work has been conducted to
bring deep learning technology into educational contexts, and
research on the application of deep learning technology in
educational contexts is under initial development [1].
C. Learning Partnerships Model
Learning partnerships support the development of students’
self-authorship, defined as framing one’s reality based on
internal values, beliefs, and loyalty [6]. Self-authorship can be
described in epistemological, intrapersonal, and interpersonal
dimensions. The epistemological dimension of self-authorship
focuses on how one knows what one knows. The intrapersonal
dimension focuses on how one understands oneself. The inter-
personal dimension focuses on how one understands others.
From a learning partnership perspective, self-authorship repre-
sents the ability to control one’s thoughts and create one’s own
truth or reality, thereby authoring one’s experience. Self-
authorship emphasizes that students bring their own identities
and values to what they learn and believe, and participate in
the construction of knowledge [6]. As a result, students
achieve self-authorship through continuous learning and
coconstruction of knowledge with others [36].
In this study, the LPM [6], [36] was used as a conceptual
framework to model the process of human–machine collabora-
tive learning to promote student self-authorship. The LPM is
based on three critical assumptions about achieving self-
authorship in learning partnerships. The first is that knowledge
is socially constructed and complex, implying that students
negotiate what to believe with others. The second assumption
is that one’s identity is central to the construction of knowl-
edge. The third assumption is that authority and expertise are
shared in the mutual construction of knowledge, meaning that
students participate as equal partners in the mutual construc-
tion of meaning.
The LPM uses three principles to offer support to help stu-
dents develop self-authorship [36], [37]. The first is validating
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
415
Fig. 1. Encoder–decoder Java code analysis model.
students’ capacity to know, i.e., helping students see them-
selves as capable learners. The second is to situate learning in
students’ own experiences, helping them integrate identity
goals and knowledge construction. The third is to define learn-
ing as mutually constructing meaning, helping students see
that they shared authority and expertise and that learning was
about mutually constructing meaning. Finally, the LPM claims
to help students develop an internal belief system, an internal
identity/sense of self, and a capacity for mutual, interdepen-
dent relationships. In brief, the LPM can help researchers
investigate how a student develops to understand her/himself,
to understand his/her partners, and based on these understand-
ings, how she/he coconstructs knowledge with the partners.
The LPM was chosen as a lens for this study because it con-
tains features consistent with learning, support, self-reflection,
relationship maintenance, and identity development in
human–machine collaborative learning. LPM can help under-
stand how learning and self-authorship development cooccur
in collaborative human–machine relationships to inform and
enhance the practice of deep learning technology in educa-
tional settings.
III. PRELIMINARIES OF THE PERFORMER-BASED MODEL
This section presents the performer-based model [31] for
analyzing java code to generate feedback, including error type
prediction and hint generation, to support students’ learning of
OOP design principles. Error type prediction provides students
with potential error types in code, whereas hint feedback pro-
vides detailed explanations and suggestions for what to do
next. The detailed explanations of errors allow students to
learn about OOP concepts and illustrate how the code violates
the key concepts and how they can be corrected. Nevertheless,
hint feedback can add more cognitive load to students than
error-type predictions.
A. Architecture
As shown in Fig. 1, the encoder encodes a given source
code into a sequence of representation features. In this study,
we adopted a performer encoder architecture with six blocks
and four parallel attention layers or heads. The detail of the
encoder is shown in Fig. 2. The embedding layer and the
 416
Fig. 2. Detailed components of the performer-based encoder–decoder
model.
sinusoid position encoding layer transform the input sequence
of tokens, individually, and the resulting tokens are then added
together as the new embedded tokens. Next, these tokens are
propagated to the network module consisting of the multihead
self-attention layer, a dropout layer (optional), an add/norm
layer, a feedforward layer, and another add/norm layer. The
network module can be replicated and stacked one by one to
extract deeply encoded features of the input tokens. Finally,
the output of the encoder is fed into the error-type classifier
(ETC; a feed-forward network) and the performer-based
decoder.
ETC analyzes the output feature sequence of the encoder to
determine what errors may exist in the code. It outputs a 36-
long vector, indicating 35 error types and the correct one,
meaning no errors were detected. The binary cross entropy is
the loss function to train the ETC.
The decoder receives the sequence of embedded tokens
from the encoder and the error prediction vector from the ETC
module. It performs a series of transformations to generate
feedback messages for each error of the input code. The
decoder was trained using the teacher force strategy in which
the target messages are rearranged as an input sequence to the
decoder so that the tokens before (and included) the position t
can be used to predict the target token at position tþ1. During
the training phase, such as encoder, the rearranged message
tokens are propagated to the decoder’s embedding and posi-
tional encoding layers and transformed into a sequence of
embedded tokens.
Next, the embedded tokens are propagated to the masked
multihead self-attention layer that performs masked self-
attention to prevent tokens from paying attention to their fol-
lowing tokens that are not available in the inference phase.
The resulting self-attention tokens are then added to the orig-
inal tokens, normalized, and transformed through the add/
norm layer and the feed-forward layer, respectively. Then,
the embedded tokens are propagated to the multihead query
attention layer to capture the attention between the embedded
tokens and the source code embedded tokens. Again, these
network modules can be replicated and stacked to extract
deeply encoded features of the target feedback tokens.
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
Finally, an embeddingsim layer is used as the output layer to
determine the target feedback messages by computing the
similarity between the embedded tokens and target message
tokens using the token embedding weights of the decoder.
The loss function adopted for training the decoder is sparse
categorical cross entropy. The sparse categorical cross-
entropy loss is like the categorical cross-entropy loss, but it
uses an integer for each target label, which saves computa-
tion time and space.
B. Attention Operation in Performer
In transformer, the attention operation takes O(L2) to com-
pute an attention matrix encoding the relationship between
token pairs in two data sequences of length L. In performer,
the attention operation uses the FAVORþ algorithm to calcu-
late the attention matrices. It has been shown that any attention
matrix can be effectively approximated by positive random
features obtained by positive-valued nonlinear mapping func-
tions of the original queries and keys [3]. These random fea-
tures are crucial for avoiding instabilities during model
training and provide an accurate approximation of the regular
softmax-attention operation in transformer. It is worth noting
that the FAVORþ algorithm requires the entire data sequence
to ensure its correct operation. This requirement means that in
the inference phase of a performer-based model, generating a
feedback message of length L word by word will take time
complexity of O(L2), which is not feasible in practical
education.
IV. METHODS
Concerning the first research question, a cached version of
the FAVORþ attention operation is developed and evaluated
by comparing its inference time with the noncached version.
Regarding the second research question, this study proposes
and evaluates a human–machine collaborative learning pro-
cess with the intelligent agent developed using the performer
model.
A. Technical Improvement of the Performer Inference
Algorithm
1) Incremental Inference Algorithm: The performer-based
decoder model requires different behaviors in the training and
inference phases. During the training phase, the decoder needs
to specify a maximum sequence length so that it can accept
the entire target feedback prefixed by a <START> token and
learn to generate the target feedback with an ending <END>
token. To facilitate the training process, target feedback,
whose length is less than the maximum, is padded up with
<PAD> tags (usually 0) at the end. On the other hand, during
the inference phase, the decoder model attempts to generate
the feedback word by word, starting from the initial
<START> token until a maximum length is reached or the
<END> token is generated. However, since the decoder is
designed with a prespecified maximum feedback length, pad-
ding the current feedback sequence is still needed so that the
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED 417

Algorithm 1: Incremental masked self-attention FAVORþ

(1) Initialize and cache SUM ¼ 0
(2) Let l be the position of the current input token embedding.
(3) Let K l ; Ql; and V l be the transformed tokens of the token l
using corresponding linear transformations, W K ; W Q; and
W V , respectively.
0 0
(4) Let Kl and Ql be the projected tokens of K l and Ql ;
respectively; using the cachedPl1 random projection matrix fSM .
(5) Get the
Pl1 cached SUM 1 ¼ i¼1 fSM ðkÞi  V i ; and SUM2 ¼
pzffiffiffi V
m i ¼ 1 i
(6) Compute SUM 1 ¼ SUM 1 þ Kl  V l , SUM 2 ¼SUM 2 þpzffiffiffi
0
m
Vl
(7) Numerator ¼
0
f ðkÞ  Ql  SUM 1 þ SUM 2
1
max 0...l;0...m1 SM
(8) Update cached SUM 1 and SUM 2 .
Fig. 3. Transformation-level cache modules for (a) masked self-attention
and, (b) query-attention in the decoder.
(9) P S ¼ 0 //ready to compute the denominator
Initialize and cache
(10) Get cached S ¼ jl1 ½fSM ðkÞj
(11) S ¼ S þ Kl0
decoder can generate the next feedback token, causing the Denominator ¼ max 0 ffiffiffi
f ðkÞ  Ql  S þ
1 lz
p
(12) m
0...l;0...m1 SM
inference process to take more time.
(13) return Attention ¼ Numerator / Denominator
To address this performance issue, we modified the decoder
architecture to generate the next token by receiving only the
previously generated one. For this purpose, the position-embed-
ding layer is first revised to compute a token’s positional
embedding with an extra parameter indicating its position in the Algorithm 2: Incremental query-attention FAVORþ
original sequence. Second, an incremental decoder inference (1) Let K and V be the transformed tokens of the query attention.
algorithm is developed by modifying the FAVORþ algorithm (2) Let K 0 be the projected tokens of K , using the cached random
to cache the intermediate results of previously embedded tokens projection matrix fSM :
to ensure its correct operation. (3) Initialize and cache C KV ¼ K 0  V .
2) Transformation-Level Cache Mechanism: There are (4) Let l be the current input token embedding.
two levels of caches inherent in the incremental FAVORþ (5) Let Ql be the transformed tokens of token l using the linear
algorithm for the decoder, including the transformation- and transformation W Q :
(6) Let Q0l be the projected tokens of Ql ; using the cached random
algorithm-level caches. The transformation-level caches store
projection matrix fSM .
an input token’s query, key, and value transformations, (7) Get cached C KV :
whereas the algorithm-level caches store the intermediate (8) Numerator ¼ Q0l  C KV P
results produced by the incremental FAVORþ algorithm, (9) Initialize and cache SK 0 ¼ L1 0
i¼0 ½K i
which will be presented in the following section. 0 0
(10) Denominator ¼ Qt  SK
In the decoder, the masked self-attention operation must (11) return Attention ¼ Numerator / Denominator
cache all query, key, and value tensors obtained by transform-
ing the token sequence. In contrast, the query-attention opera-
tion only needs to cache the query tensors because the key and
value tensors are transformed only once from token sequences After a series of derivation steps, (1) is transformed into the
outside the query-attention module, as shown in Fig. 3. The recurrent formula, as shown in the following. The detailed der-
transformation-level cache mechanism saves time obtaining k, ivation of the recurrent formula is omitted here. Given 0  l 
q, and v values by preventing duplicate transformation opera- L  1; 0  j  d  1, m is the number of random projection
tions. However, our test showed that the transformation-level features
cache does not significantly improve the decoder’s inference
speed. X
m 1
N att ½lj ¼ Q0 ½l; r  SUM½l; rj (2)
3) Algorithm-Level Cache Mechanism: To develop the r¼0
algorithm-level cache mechanism for the FAVORþ algo-
rithm, Att(Q, K, and V), as shown in the following, needs to where
be reformulated as a recurrent formula suitable for incremen-
SUM½l; r j ¼ SUM½l  1; rj þ K 0 ½r; l  V ½lj :
T
tal implementation: (3)

Q0 ðK 0 TV Þ N att
AttðQ; K; V Þ ¼ D1 AQ;K V ¼ pffiffiffi ¼ (1) In addition,
D d N norm
X
m 1
where dimðQ0 Þ ¼ L  m, dim(K 0 T ) ¼ m  L, and dim N norm ½l ¼ Q0 ½l; r  S ½r; l (4)
(V ) ¼ L  d. r¼0

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
 418
TABLE I
EXAMPLES OF METEOR SCORES OF GENERATED HINT FEEDBACK
where
X
l
K 0 ½ r; i ¼ S ½r; l  1 þ K 0 ½r; l:
T T
S ½r; l ¼
i¼0
In practice, to improve the stability and convergence of the
FAVORþ algorithm [4], an incremental stable FAVORþ
algorithm is developed, as shown in Algorithms 1 and 2, for
masked attention and query attention, respectively.
4) Evaluation Criteria for Feedback: The type error pre-
diction is evaluated by accuracy. The hint feedback is evalu-
ated using METEOR [38]. METEOR includes synonym
matching and produces results that are highly correlated with
those of human judgment. The METEOR algorithm first cal-
culates the precision rate P and recall rate R in the case of
unigrams. Then the harmonic mean F is obtained as follows:
ða2 þ 1ÞP
F ¼
R þ aP
where a is the relative importance of P to R. This study uses a
default value of 0.9 for a. What makes METEOR unique is
that it uses a penalty formula to prevent fragmented feedback,
given as follows:
 number chunks u
p¼g
unigrams matched
where g is a parameter that determines how much the penalty
factor affects the final METEOR value, which has a default
value of 0.5, and u is a parameter that determines the weight
of the incorrect ordering rate of the unigrams, which has a
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
Fig. 4. Screenshot showing task descriptions, student codes and system
feedback.
default value of 3. The purpose of the penalty is to penalizehy-
pothetical sentences with correct words and phrases, but in a
very different order than in the references. Before the penalty
can be computed, the unigrams are first grouped into the few-
est possible chunks, defined as subsets of the unigrams adja-
cent to each other in the reference. When evaluating the
fluency of feedback, the smaller the number of chunks, the
longer the average length of each chunk is—the more consis-
tent the word order of the candidate feedback with the refer-
ence feedback. Finally, the final METEOR score is calculated
as follows:
Meteor ¼ ð1  pÞ  F: (8)
(5)
Table I tabulates the METEOR scores for feedback exam-
ples, where <BOM> and <EOM> mark the start and end of
a hint, respectively. The METEOR score prefers longer hints
even if the reference and predicted hints are the same. The
preference appears to be caused by the rounding error when
calculating the chunking penalty.
5) Design of the Agent System: To deploy the agent into
educational practice, a website was developed using the flask
package, as shown in Fig. 4. In this website, descriptions of
debugging tasks are displayed in the left window, and the sam-
ple code is displayed in the middle window for editing. Click-
ing the green diagnose button will submit the code to the
server for analysis, and the feedback will be displayed in the
right window. There are two levels of feedback: error types
and detailed hints. Students can switch the levels at will. In
addition, students can accept or refuse the feedback and sub-
(6)
mit his/her decisions.
The proposed agent system supports the learner partner-
ships model as follows. To help validate students’ capacity to
know, the agent returns feedback on the code modified and
submitted by students. To situate learning in students’ own
experiences, the agent did not try to lead the ways to correct
the code. Instead, it is entirely up to the students to correct
(7) the code based on their own experience and evaluation of the
feedback. Finally, to help students see that they can share
authority and expertise and that learning is about mutually
constructing meaning, the agent analyzes the code submitted
by students and provides feedback for informational purposes
only. The students will learn to take the agent’s feedback into
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED
TABLE II
AUTONOMY CHARACTERISTICS OF THE INTELLIGENT AGENT
Fig. 5. Screenshot showing an interactive session involving error prediction
and user responses.
account without being consumed by them. The autonomy
characteristics of the agent are given in Table II.
To illustrate the human–machine interaction supported by
the agent system, we consider the debugging task of designing
a pet system containing a pet hierarchy, including dogs and
cats, and the relations between pet and keeper objects. Fig. 5
shows that the agent successfully predicted a coding error, and
the student responded by accepting the prediction. However,
after correcting the code, the agent made the previous error
prediction again. This time the student responded by refusing
the prediction, as shown in Fig. 6. Another case showing an
interaction involving a higher level of code errors is shown in
Fig. 7, where the structural relationship between a pet object
and a keeper object is not correctly established. The agent cor-
rectly made the error prediction and delivered the correct hint
feedback. The student responded by accepting both the error
prediction and the hint feedback. In addition, the agent also
delivered a hint that helped identify a code segment that incor-
rectly built the relationship between a pet object and a keeper
object, as shown in Fig. 8. The user responded by accepting
both the error prediction and the delivered hint feedback.
B. Human-Collaborative Learning Process
In OOP education, students learn the design principles of
OOP and the ability to write OOP code. To help them achieve
the goal, they must be able to recognize and modify incorrect
OOP codes. The intelligent agent is deployed to help students
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
419
Fig. 6. Screenshot showing an interactive session involving error prediction
and user responses after the user corrected the code.
Fig. 7. Screenshot showing an interactive session involving high-level error
prediction and hint feedback.
achieve their learning goals by offering feedback to lead them
to understand and correct the code. Accordingly, a human–
machine collaborative learning process is proposed, as given
in Table III. Students must evaluate the agent’s feedback and
decide whether to accept or reject the feedback. In this way,
students learn OOP design principles through collaboration
with a machine partner.
V. EXPERIMENT DESIGNS
A. Model Development and Performance Comparison
1) Data Collection: Regarding the first research question,
this study compares the performance of the proposed inference
algorithm with the original one. A deep-learning model was
first developed based on data collected from java code submit-
ted by students for exercises, quizzes, and exams in an intro-
ductory OOP course at a university in Taiwan. A total of 330
codes were collected and manually annotated. The codes were
randomly assigned to humans, with one code annotated by
two annotators and proofread against each other. All annota-
tors adopt the same hint structure to make the model suitable
for assisted OOP learning, first describing the error type and
related OOP concepts, then explaining why the error occurred,
and finally suggesting appropriate code changes.
Since the GPU hardware device used in this study has lim-
ited storage, the code length was restricted to 1000 tokens. As
a result, we sorted out a total of 35 error types, and the maxi-
mum length of hint feedback is 738 tokens. However, the
small amount of code is far from the sample size required to
 420
Fig. 8. Screenshot showing an interactive session involving a hint that help
identify error code segments.
TABLE III
AUTOMATION DESIGN OF THE LEARNING PROCESS
train a neural model. Therefore, this study adopts the code
augmentation method proposed by Wang [39] to increase the
sample size by changing local variable names and randomly
reordering statements without affecting the original code
semantics. In the end, the sample size was increased from 330
to 11 200.
2) Model Training: The data are divided into 80% training
data and 20% testing data. The model was trained for 1000
epochs, using 20% of the training data as validation. The train-
ing process is conducted on the Window 10 operating system
with an Intel(R) Core(TM) i9-11900K 3.50 GHz processor,
128 GB RAM, and an NVIDIA RTX 3090 with 24 GB RAM.
3) Performance Comparison: First, we investigate the per-
formance of the original performer inference algorithm, which
is used as a baseline for comparison. For comparison, 1000
random samples were selected from the training, validation,
and test datasets. The average prediction accuracy and infer-
ence time for each selected dataset are reported and compared.
B. User Study
1) Data Collection: This study conducts a human evalua-
tion of the intelligent agent. Eleven students enrolled in an
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
TABLE IV
SYSTEM EVALUATION QUESTIONNAIRE
advanced Java programming course were invited to participate
in the evaluation. Four debugging tasks of different difficulty
levels are given to the students randomly, and they are
requested to complete the debugging tasks with the agent’s
help. After the experiment, they completed a seven-item sur-
vey, as given in Table IV, about their opinions of the system.
The system also recorded the students’ interactions with the
agent, including the error predictions and hints generated by
the agent and the student’s acceptance or rejection decisions.
A total of 128 log records were collected. Finally, the students
handed in a self-reflection report describing how they learned
the debugging tasks in collaboration with the agent.
2) Data Analysis: The analysis used the thematic analysis
framework of Braun and Clarke [40]. Thematic analysis was
chosen because it allows for finding meaningful patterns within
the data inductively, which helps address the research question
by finding what themes underlie the students’ experience and
perspectives of the human–machine collaborative learning sup-
ported by the intelligent agent. First, students’ responses to the
open questions are encoded using semantic and latent codes,
which are then grouped into meaningful candidate themes gen-
erated by the researcher. These candidate themes are reviewed
and further developed at the extraction level, considering the
context relevant to the entire dataset. Themes are further refined
and defined until the researcher judges that they represent a thor-
ough study understanding. Themes are then renamed according
to their content and encoding extraction.
VI. RESULTS
A. Prediction Accuracy and Inference Speed Comparison
The loss and accuracy of the training process for the error
classifier and hint generation model are shown in Figs. 9 and
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED 421

TABLE V
EXAMPLES OF HUMAN–MACHINE INTERACTIONS

Fig. 9. (a) Training loss of the classifier submodel. (b) Training accuracy of
the classifier submodel.

Fig. 10. (a) Training loss of the hint feedback submodel. (b) Training accu-
racy of the hint feedback submodel.

10, respectively. As seen in the figures, the encoder–decoder

model is well trained for training and validation data.
For error prediction, the error classifier achieves the average
binary accuracy of 1.000, 0.999, and 0.999 for the training,
validation, and testing datasets, respectively. For hint genera-
tion, the model achieves average METEOR scores of 0.922,
0.915, and 0.918 for the training, validation, and testing data-
sets, respectively. The high accuracy of error predictions, high
fluency, and accurate hints generated by the agent shows that
it performs very well in the code analysis tasks, at least for the
collected data.
Regarding the inference speed, the baseline version of the
original inference algorithm took an average of 5.74 s to gen-
erate a target token. In contrast, the proposed cached version
achieves an average of about 0.07 s per token, faster than the
baseline version by nearly 80 times, which is a significant
improvement and, to the best of our knowledge, is by far the
best in the literature on performer-based models.

B. Human–Machine Interaction
Table V tabulates two scenarios of the interactions between
the agent and students. In scenario 1, the code is about a pet
system where the pet hierarchy, including dog and cat, needs
to be adequately designed using Java’s abstraction and inheri-
keyboard reading action.” This hint is not related to
tance mechanisms. Furthermore, the relationship between the
the buggy code.
pet and keeper classes must be carefully designed.
The first interaction session has two error predictions and The second hint says
three hints. The student accepted the two error predictions and “Object data/method members are the data/methods
two of the three generated hints. The first hint says owned by an object. Each object has its own data/
“Because there is only one keyboard device, the method members, independent of other objects. Syn-
scanner object can be declared once, and there is no tactically, those data/method members not declared as
need to declare a different scanner object for each static are object data/method members. You can also

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
 422
TABLE VI
STUDENT RESPONSES TO THE QUESTIONNAIRE ITEMS
declare access privileges for object data/method mem-
bers; for example, the statement: private int x; indi-
cates that object variable x can be accessed only by
the owner object.”
The third hint says
“The relationship between two objects can often be
implemented with one object owning the other. For
example, to describe the relationship between the Pet
and the Keeper, you can declare an object data of
Keeper inside the Pet object to indicate who the
Keeper of the Pet is. The keeper object can be created
using constructors, for example, public Pet (String
type, Keeper keeper) {. . .. this.keeper;}.”
These two hints are correct for the test case. Therefore, the
student’s decisions in this interaction session were all correct.
After correcting the code, she/he resubmitted the code and
received feedback that no errors were detected.
In scenario 2, the code is about a letter system where the let-
ter hierarchy, including Chinese and English, needs to be
designed using Java’s abstraction and inheritance mecha-
nisms. Furthermore, the relationship between the Envelop
class and Letter class must be carefully designed. In the first
session, the agent predicted an abstraction error but did not
generate any hint feedback, which can happen due to the
agent’s nondeterministic characteristic. The student accepted
the error prediction but rejected the none feedback. She/he
resubmitted the code in session 2, and this time the agent
detected an extra error named object error. In addition, the
hint that did not appear in the first session was generated in
session 2, which says
“Subclasses can inherit common object data/meth-
ods by declaring them once in the parent class. Differ-
ent object data/methods can be declared in each
derived subclass.”
This case shows that accurate predictions and hints can be
obtained by repeatedly asking the agent to diagnose the same
code. In other words, the agent can help students explore the
problem space.
C. Survey Analysis
Table VI tabulates the student responses to the survey,
where all students agree that either the predicted error types or
hints help them learn in the debugging tasks. Regarding the
perceived reliability of the feedback types, five students rated
error-type predictions as more reliable, three students rated the
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
TABLE VII
STUDENT DECISIONS IN THE EVALUATION
hints as more reliable, and three students rated both feedback
types as equally reliable. In contrast, concerning the perceived
unreliability of the feedback types, four students rated error-type
predictions as less reliable, four students rated hint feedback as
less reliable, and three rated both feedback types equally unreli-
able. Although the agent achieved high performance in the train-
ing phase, it could have performed better in the human–machine
interaction phase. Nevertheless, this indicates that the agent
should play a learning partner’s role, triggering the students’
thinking and judgment instead of playing a tutor role. Regarding
the overall usefulness of the agent for learning in the debugging
tasks, six students rated it as helpful, and five students rated it as
fair. Finally, concerning the change of trust in the system before
and after the evaluation experiment, four students indicated that
their trust in the system increased, and seven students felt no
change in their trust after the learning tasks.
D. Log Analysis
Table VII tabulates the log analysis results of the students’ deci-
sions to accept or reject the feedback to investigate their trust in
the agent further. On average, the student’s acceptance rate is 67/
128 ¼ 0.523. For error predictions, the acceptance rate (0.547) is
higher than the rejection rate (0.453). As for hints, the acceptance
rate (0.5) is the same as the rejection rate (0.5). Furthermore, the
acceptance rate of error predictions (0.547) is higher than that of
hints (0.453). The rejection rate of error predictions (0.475) is
lower than that of hints (0.525). As a result, the students trust the
agent’s error predictions more than the agent’s hints, which is con-
sistent with the survey data in Table VI.
Table VIII tabulates the total counts of acceptance/rejection
decisions in each interaction session to elucidate further
changes in students’ trust in the agent across interactive ses-
sions. The maximum number of human–machine interactive
sessions is four. Fig. 11 shows the trends in the acceptance
rate of error predictions and hints in different interactive ses-
sions. It can be seen that the acceptance rates are dropping
except for the hints in the last session. The agent performs
best in the first session but could be better in subsequent ses-
sions, which suggests that the agent be retrained on two-way
interaction data rather than one-way interaction data, such as
exercises and quizzes. In addition, the students appear to be
using their judgment to accept or reject the agent’s feedback
across the interactive sessions.
E. Thematic Analysis
As given in Table IX, several themes emerged in students’
perceptions of the agent’s feedback. They are obtained based
on their positive/negative effects from the perspectives of the
3-Ds of the learner partnerships model.
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED 423

TABLE VIII
SESSIONS OF STUDENT ACCEPTANCE DECISIONS IN THE EVALUATION

1) Interpersonal Dimension: First, the theme “nondetermi-

nism” is about the predictability of the agent’s predictions given
the same input code. For example, the following is a negative
remark:
“Sometimes, the same code has different and con-
flicting prediction results.”
Fig. 11. Ratios of acceptance decisions across sessions.
TABLE IX
THEMES EMERGED FROM STUDENTS’ OPINIONS ON ERROR PREDICTIONS AND
HINTS AND THEIR RELATIONS TO LPM DIMENSIONS

The theme “responsiveness” refers to the agent’s ability to

respond to students’ actions. For example, there are some neg-
ative remarks, such as
“Sometimes, after correcting the code, the agent
still delivered the same error prediction”
and
“Sometimes, the agent still delivered the same hint
feedback after modifying the code according to the
hints.”
The theme “accuracy” refers to the agent’s ability to deliver
accurate predictions. For example, there are some positive
remarks, such as 2) Epistemological Dimension: The theme “directivity”
“The hint given is more accurate (than error predic- refers to how the delivered feedback can guide students
tions), and it feels easier to understand the problem through the debugging tasks. For example, there are some pos-
than by the error type prediction.” itive remarks, such as
“The hint given is correct most of the time.” “I can know the general direction to correct the
error (by the error predictions).”
In contrast, there are some negative remarks about the
Accuracy theme, such as “It (error prediction) can let me know where there is
a problem with the code and get to the point.”
“Sometimes the hint given did not help correct the
code.” “It (error prediction) offers positive help by giving
a general debugging direction.”
“It (the error prediction) is sometimes right and
sometimes wrong, which is quite confusing.” However, there are some negative remarks, such as

The theme “expertise” refers to the student’s evaluation of “But they (the error prediction and the hint) can
the agent’s expertise. For example, a positive remark says also be misleading.”

“The agent can analyze the relationships between
objects.”
The theme “context-sensitivity” refers to the agent’s ability
to deliver context-sensitive feedback on the input code. For
example, one student said that
“Sometimes the error prediction will mislead me to
focus on the wrong place in the code to be corrected,
resulting in an inefficient debugging process.”
The theme “specificity” refers to the level of detail in the
feedback delivered by the agent. For example,

“It is nice to get (hint) feedback with snippets about
what went wrong in the code, helping to get straight to
where the error is.”
The abovementioned themes can lead to a positive or a neg-
ative impact on students’ trust in the agent’s behavior. There-
fore, they are classified as belonging to the interpersonal
dimension.
“While the error prediction is difficult to under-
stand, the hints can give us a deeper understanding of
how to correct errors.”
“Understanding the problem (by the hints) is
beneficial.”
In contrast, there is a negative remark about error predic-
tions saying that

Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
 424
“The description of the error type prediction is a bit
vague.”
The theme “overload” refers to the cognitive load demands
of students when reading the feedback. For example,
“The (hint) narrative is slightly long and not easy to
comprehend.”
The theme “location identification” refers to how the
agent’s feedback helps students locate error codes. For
example,
“I can know the approximate error location (by the
hints).”
“I can directly find the wrong place in the code (by
the hints).”
The theme “exploration” refers to how the agent’s feedback
helps students explore the error code. For example,
“It (error prediction) can tell me code errors I did
not notice.”
The abovementioned themes can lead to a positive or a neg-
ative impact on students’ understanding of the target domain.
Therefore, they are classified as belonging to the epistemolog-
ical dimension.
3) Intrapersonal Dimension: The theme “comprehension”
refers to the students’ internal comprehension of the agent’s
feedback. For example,
“Sometimes, I do not understand what the hints
mean.”
“My Chinese is not good; if it is not vernacular, I
cannot understand it (the hint) myself.”
The theme “exertion of authority” refers to the student’s
ability to evaluate the agent’s feedback. For example,
“Sometimes the agent gives conflicting error
predictions.”
“It (the error prediction) is sometimes right and
sometimes wrong.”
The abovementioned themes can lead to a positive or a neg-
ative impact on students’ self-trust. Therefore, they are classi-
fied as belonging to the intrapersonal dimension.
VII. DISCUSSIONS
A. Technical Feasibility of the Deep Learning Agent
Advances in deep learning technology have enabled intelli-
gent agents to perform high-level intellectual analysis tasks,
such as code analysis more than ever before. However, apply-
ing deep learning technology in educational settings still has
several issues. First, deep learning models usually require
high-end computing devices, hindering their widespread
application in educational practice. This study proposes an
incremental FAVORþ algorithm for performer to reduce
token generation time from O(L) to O(1). As a result, it takes
O(L) to generate feedback of length L. The proposed
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
incremental inference algorithm achieves nearly 80 times
faster than the original one.
Improving model inference speed is essential for applying
performer-based models in educational settings. Specifically,
in the training data, the average length of the generated feed-
back is 132.90 tokens with a standard deviation of 123.67,
implying that the original inference algorithm takes an aver-
age of 12.71 min with a standard deviation of 13.83 min to
generate feedback. Such long response times make the origi-
nal model not a technically feasible solution for educational
applications. The new inference algorithm takes only an aver-
age of 9.30 s for hint generation, with a standard deviation of
8.66 s. Although the inference time can be reduced by inves-
ting in high-end devices that might cost tens or hundreds of
thousands of dollars, we can achieve satisfactory performance
on workstation-level systems that cost thousands of dollars.
On the other hand, the model’s prediction performance on test
data are quite good, with an average accuracy of 99.9% for
error prediction and 91.8% for hint feedback. Therefore, this
study helps address the cost barriers to applying deep learning
technology to education, making performer-based models a
potentially viable and accessible solution in educational
settings.
B. Educational Feasibility of the Deep Learning Agent
This study adopts the LPM to investigate how the students
perceived the agent in the human–machine collaborative
learning process. From the interpersonal perspective, we
found positive and negative factors that could affect students’
trust in the agent. Students’ trust in the agent can come from
the agent’s expertise in error prediction and generating hints
with accuracy and context sensitivity. For example, the stu-
dents found the agent to be very accurate at predicting codes
that incorrectly implemented relationships between objects.
As given in Table VII, log analysis also shows that the
student’s average acceptance rate of the agent’s feedback is
0.523, and the acceptance rate of error predictions is higher
than that of hints, which is consistent with the survey data.
However, students’ trust in the agent can be reduced due to
the agent’s nondeterministic behavior, inaccuracy of feedback,
and lack of responsiveness to students’ actions. For example,
the agent may give different diagnostic results for the same
code at different times, sometimes even contradicting each
other, leaving students needing clarification about what to do
next. In addition, the students found that sometimes after mod-
ifying the codes according to the given feedback, the agent
still gives the same feedback for the modified code. The lack
of responsiveness may be due to the improper training data
collection policy adopted to develop the neural model. As
shown in Fig. 11, students’ acceptance rates of the agent’s
feedback decreased across interactive sessions, implying that
the model did not respond well to students’ code-correcting
actions. Therefore, the model’s training data should be col-
lected from two-way interactions between teachers and stu-
dents, not just from one-way interactive activities, such as
homework and quizzes.
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED
From the intrapersonal perspective, we found that some stu-
dents showed higher self-confidence by using self-authority to
evaluate the agent’s feedback. However, we also found that
some students showed less confidence in understanding the
agent’s hints. This finding raises the issue that these students
need different learning supports during the human–machine
collaborative learning process.
From the epistemological perspective, error predictions can
provide students with general directions to correct the code
and alert them to potential bugs in their code. In addition, the
higher level of detail in the hints can help students grasp the
essentials of the concepts and skills to correct the problems. In
particular, the snippets in the hints can help students effi-
ciently identify the error code’s locations. However, students
also mentioned that the error type predictions and hints are
sometimes inaccurate, leading to an inefficient debugging pro-
cess. Students may need to evaluate the code themselves and
should not put complete trust in the agent’s decisions all the
time. Instead, they should view the agent as a learning partner
and learn to evaluate the agent’s feedback to decide which to
accept or reject. Through this peer-evaluation process, they
will have more opportunities to think and learn.
On the other hand, student cognition affects human–
machine interaction. Some students mentioned that the
descriptions of the error types needed to be more specific for
them to understand. Although students responded that the
hints are more accurate and helpful than error predictions for
understanding the problems and identifying the locations of
the error, some students mentioned that the descriptions of the
hints needed to be shorter and easier to understand. Therefore,
the agent needs to adapt the type and content of feedback to
students’ cognitive load.
In summary, the human evaluation results show that the
agent has a certain level of expertise to support students in
learning to debug OOP code, especially in the first interactive
session. In addition, the agent’s random behavior can help stu-
dents develop explorative learning strategies and promote
self-development with self-confidence. Furthermore, while the
agent’s expertise can be enhanced by retraining the model
with two-way interaction data, students should view the agent
as a learning partner rather than a learning tutor. In the part-
nership created between humans and machines, a student can
develop self-authorship by developing appropriate relation-
ships with the agent and himself and then an appropriate col-
laborative learning strategy with the agent. As one student
says in his self-reflection report:
“I see the agent’s feedback as a list of potential
problems in the code. I will check the code for every
possible error option in the list, which gives me more
confidence to make the correct changes to the code.”
VIII. CONCLUSION
A. Implications for Education, Research, and Practice
Compared with other methods of building intelligent pro-
gramming tutors, deep learning technology frees teachers
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
425
from the burden of building solution models or test cases for
problems. It can handle the diversity of student code. How-
ever, despite the advantages and growing popularity of intelli-
gent agents developed by deep learning technology, the
feasibility of using this technology as a practical solution for
education still requires considerable research. Finding a tech-
nically feasible solution by downsizing deep learning technol-
ogy is the first step toward widespread adoption in education.
The first contribution of this study is that the performer, an
efficient version of the transformer, can be improved to be a
technically feasible solution to offer intelligent learning sup-
port within a dozen seconds in the deployment phase.
This study further explores how such an intelligent agent
can support learning in educational settings. A human–
machine collaborative learning process with the intelligent
agent is proposed and evaluated based on the LPM, which
reveals the relationships between the agent’s technical auto-
mation and autonomy features and students’ development of
self-authorship. The results of the second research question
are presented in the following.
1) Nontransparency: Transparency is the ability of an
agent system to produce high-quality explanations of its
intent, performance, plans, and reasoning [41]. It was shown
that increasing the awareness of the decision-making process
of an intelligent agent improves the trust relationship between
humans and machines [42], [43], [44]. Furthermore, literature
shows that human trust in automated systems is essential to
human–machine collaboration [45]. In this study, the transpar-
ency problem is addressed by training the agent to deliver
hints that can help explain the error-type predictions, enabling
students to better understand and locate problems in code,
thereby contributing to students’ epistemological develop-
ment. Furthermore, the accuracy of the error predictions and
hints helps enhance students’ trust in the agent.
However, some students still need to know why the agent
delivered specific feedback. Therefore, other forms of feed-
back that could directly explain the agent’s decisions should
be considered. However, nontransparency is a na€ıve nature of
a neural-based agent; therefore, increasing transparency of
such agents could be challenging. The importance of the
interpretability of deep learning models has created an emerg-
ing field called eXplainable Artificial Intelligence (XAI) [46].
For example, the XAI technology can be applied to enable the
agent to display the code segments that lead to a specific type
of error. Such feedback could help improve the transparency
of the agent, thereby improving the students’ trust in the agent.
2) Nondeterminism: It is found that the agent’s nondeter-
ministic behavior negatively affects students’ trust in the
agent. Nevertheless, the nondeterministic characteristics of
the agent may help students discover novel things they did not
notice, expanding their exploration space. This nondeterminis-
tic characteristic of the agent also suggests that keeping
student autonomy is essential in the human–machine collabo-
rative learning process. Students should see the agent as a
learning partner and learn to evaluate its feedback to decide
which to accept or reject, which can help them achieve self-
authority.
 426
3) Adaptability: As can be seen from the students’
responses, human–computer interaction is dynamic and indi-
vidual. Although the current agent is adaptable to the diversity
of student code, it could have adapted better to student-modi-
fied codes across the interactive sessions. Therefore, we rec-
ommend adding a new dimension called “responsiveness” to
the characteristics of intelligent agents applied in educational
settings. Responsiveness focuses on the agent’s adaptability to
student responses. It is recommended to find appropriate data
sources that contain two-way interaction training data to
ensure that the agent learns the knowledge to generate correct
feedback for subsequent interactive sessions. For example, in
this study, retraining the model on error codes with annotated
feedback and correcting ones could improve the agent’s
responsiveness.
Furthermore, students have different preferences for the
type and content of the feedback. The agent’s adaptability
could be increased if it can adapt to student characteristics,
such as their cognitive load and trust in the agent [49]. For
example, the agent can learn from human–machine interac-
tions when and how long they spend deciding to accept or
reject the agent’s feedback, thereby adjusting its internal mod-
els of students’ cognitive load and trust in the agent and
changing its feedback behavior accordingly. Fulfillment of
extensive adaptability can help students build a positive rela-
tionship with the agent, thereby improving the human–
machine collaborative learning process and promoting stu-
dents’ self-authorship.
4) Openness: The proposed agent is currently closed to
interacting with other agents and collecting data from multiple
sources. Nonetheless, a multiagent design can be expected to
augment the agent system with the capability to expand its
original input and the flexibility in gathering source data to
improve the interaction between humans and machines. For
example, the agent’s feedback can be adaptive if it can interact
with another agent that models the student’s cognitive load
and trust states, as aforementioned.
B. Limitations and Suggestions for Future Research
First, developing deep learning models is a domain-depen-
dent task, and this study focused only on an introductory OOP
course using java programming language. Therefore, the col-
lected data may suffer from a data bias issue. In addition, a
small population of undergraduates from the programming
discipline is considered. Therefore, discretion must be exerted
to extend the results to other courses, populations, and disci-
plines. Nonetheless, the same model development process can
be replicated in other programming courses and population sit-
uations. Second, this study explored how deep learning tech-
nology can be integrated with programming education.
Therefore, large-scale controlled experiments on the impact of
human–machine collaborative learning on learning outcomes
are encouraged.
Furthermore, because the predictive power of a deep learn-
ing model may not be 100% correct, the human–machine
interaction may involve more uncertainty caused by the
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024
unstable trust between students and agents. Therefore, in the
study of human–machine collaborative learning, effective
interaction between students and agents and its impact on
student’s learning outcomes can be an essential topic for
future research [47], [48].
Future research could apply and evaluate the proposed
human–machine collaborative learning process combining deep
learning technology in different educational settings to explore
their pedagogical benefits for teachers and students. For exam-
ple, the learning process can be extended to a group-based
human–machine collaborative learning process where students
in the group learn collaboratively with the agent. In such situa-
tions, individuals will get assistance from other peers if they
have difficulty understanding the agent’s feedback alone.
REFERENCES
[1] X. Chen, H. Xie, D. Zou, and G.-J. Hwang, “Application and theory
gaps during the rise of artificial intelligence in education,” Comput.
Educ. Artif. Intell., vol. 1, 2020, Art. no. 100002.
[2] A. Vaswani et al., “Attention is all you need,” in Proc. 31st Conf. Neural
Inf. Process. Syst., 2017, pp. 6000–6010.
[3] K. Choromanski et al., “Rethinking attention with performers,” in Proc.
9th Int. Conf. Learn. Representations, 2021, pp. 1–38.
[4] P. Urlaub and E. Dessein, “From disrupted classrooms to human-
machine collaboration? The pocket calculator, Google translate, and the
future of language education,” L2 J., vol. 14, no. 1, pp. 45–59, 2022.
[5] M. Simmler and R. Frischknecht, “A taxonomy of human–machine col-
laboration: Capturing automation and technical autonomy,” AI Soc.,
vol. 36, pp. 239–250, 2021.
[6] M. B. Magolda and P. M. King, Learning Partnerships: Theory and
Models of Practice to Educate for Self-Authorship. Sterling, VA, USA:
Stylus Publishing, LLC, 2004.
[7] H. C. Lane and K. VanLehn, “Teaching the tacit knowledge of program-
ming to novices with natural language tutoring,” Comput. Sci. Educ.,
vol. 15, no. 3, pp. 183–201, 2005.
[8] H. Keuning, J. Jeuring, and B. Heeren, “A systematic literature review
of automated feedback generation for programming exercises,” ACM
Trans. Comput. Educ., vol. 19, no. 1, pp. 1–43, 2019.
[9] V. J. Shute, “Focus on formative feedback,” Rev. Educ. Res., vol. 78,
no. 1, pp. 153–189, 2008.
[10] J. Hattie and H. Timperley, “The power of feedback,” Rev. Educ. Res.,
vol. 77, no. 1, pp. 81–112, 2007.
[11] D. Boud and M. Elizabeth, Feedback in Higher and Professional Educa-
tion. Evanston, IL, USA: Routledge, 2013.
[12] J. R. Anderson and E. Skwarecki, “The automated tutoring of intro-
ductory computer programming,” Commun. ACM, vol. 29, no. 9,
pp. 842–849, 1986.
[13] S. Gross, B. Mokbel, B. Hammer, and N. Pinkwart, “Learning feedback
in intelligent tutoring systems,” K€ unstliche Intelligenz, vol. 29, no. 4,
pp. 413–418, 2015.
[14] R. Singh, S. Gulwani, and A. Solar-Lezama, “Automated feedback gen-
eration for introductory programming assignments,” ACM SIGPLAN
Notices, vol. 48, no. 6, pp. 15–26, 2013.
[15] W. Wu, G. Li, Y. Sun, J. Wang, and T. Lai, “AnalyseC: A framework
for assessing students’ programs at structural and semantic level,” in
Proc. IEEE Int. Conf. Control Automat ., 2007, pp. 742–747.
[16] N. Truong, P. Roe, and P. Bancroft, “Static analysis of students’ Java
programs,” in Proc. 6th Australas. Conf. Comput. Educ., 2004, vol. 30,
pp. 317–325.
[17] B. Hartanto and J. Reye, “CSTutor: An intelligent tutoring system that
supports natural learning,” in Proc. 4th Annu. Int. Conf. Comput. Sci.
Educ.: Innov. Technol., 2013, pp. 19–26.
[18] W. L. Johnson, “Understanding and debugging novice programs,” Artif.
Intell., vol. 42, no. 1, pp. 51–97, 1990.
[19] E. Sykes, “Design, development and evaluation of the Java intelli-
gent tutoring system,” Technol., Instruct., Cogn. Learn., vol. 8,
no. 1, pp. 25–65, 2010.
[20] W. Jin, T. Barnes, and J. Stamper, “Program representation for auto-
matic hint generation for a data-driven novice programming tutor,” in
Proc. Int. Conf. Intell. Tutoring Syst., 2012, pp. 304–309.
 WANG: FEASIBLE STUDY OF A DEEP LEARNING MODEL SUPPORTING HUMAN–MACHINE COLLABORATIVE LEARNING OF OBJECT-ORIENTED
[21] K. Rivers and K. R. Koedinger, “Data-driven hint generation in vast
solution spaces: A self-improving python programming tutor,” Int. J.
Artif. Intell. Educ., vol. 27, pp. 37–64, 2017.
[22] T. Price, R. Zhi, and T. Barnes, “Evaluation of a data-driven feedback
algorithm for open-ended programming,” in Proc. Int. Conf. Educ. Data
Mining, 2017, pp. 192–197.
[23] F. Jurado, M. Redondo, and M. Ortega, “Using fuzzy logic applied to
software metrics and test cases to assess programming assignments and
give advice,” J. Netw. Comput. Appl., vol. 35, no. 2, pp. 695–712, 2012.
[24] R. Matloobi, M. Blumenstein, and S. Green, “Extensions to generic
automated marking environment: Game-2þ,” in Proc. Interactive Com-
put. Aided Learn. Conf., 2009, vol. 1, pp. 1069–1076.
[25] M. Striewe and M. Goedicke, “A review of static analysis approaches
for programming exercises,” in Proc. Int. Conf. Comput. Assist. Assess-
ment Res. E-Assessment, 2014, pp. 100–113.
[26] H. Ueno, “A generalized knowledge-based approach to comprehend
Pascal and C programs,” IEICE Trans. Inf. Syst., vol. 83, no. 4,
pp. 591–598, 2000.
[27] S. Xu and Y. S. Chee, “Transformation-based diagnosis of student pro-
grams for programming tutoring systems,” IEEE Trans. Softw. Eng.,
vol. 29, no. 4, pp. 360–384, Apr. 2003.
[28] E. Soloway, E. Rubin, B. Woolf, J. Bonar, and W. L. Johnson, “Meno-II:
An AI-based programming tutor,” J. Comput.-Based Instruct., vol. 10,
no. 1, pp. 1–35, 1983.
[29] W. L. Johnson and E. Soloway, “Intention-based diagnosis of nov-
ice programming errors,” in Proc. AAAI Conf. Artif. Intell., 1984,
pp. 162–168.
[30] M. Day, M. R. Penumala, and J. Gonzalez-Sanchez, “Annete: An intelli-
gent tutoring companion embedded into the Eclipse IDE,” in Proc.
IEEE 1st Int. Conf. Cogn. Mach. Intell., 2019, pp. 71–80.
[31] F. H. Wang, “An efficient performer-based model for automatic feed-
back generation for teaching object-oriented programming,” in Proc.
14th Annu. Int. Conf. Educ. New Learn. Technol., 2022, pp. 2167–2174.
[32] J. Yi, U. Z. Ahmed, A. Karkare, S. H. Tan, and A. Roychoudhury, “A
feasibility study of using automated program repair for introductory pro-
gramming assignments,” in Proc. 11th Joint Meeting Foundations
Softw. Eng., 2017, pp. 740–751.
[33] A. Vizcaıno, “A simulated student can improve collaborative learning,”
Int. J. Artif. Intell. Educ., vol. 15, pp. 3–40, 2005.
[34] G. J. Hwang, H. Xie, B. W. Wah, and D. Gasevic, “Vision, challenges,
roles and research issues of artificial intelligence in education,” Comput.
Educ., Artif. Intell., vol. 1, pp. 1–5, 2020.
[35] G. J. Hwang and C.-Y. Chang, “A review of opportunities and chal-
lenges of chatbots in education,” Interactive Learn. Environ., pp. 1–14,
Jul. 2021, doi: 10.1080/10494820.2021.1952615.
[36] M. B. Magolda, “Learning partnerships model: A framework for pro-
moting self-authorship,” in Learning Partnerships: Theory and Models
of Practice to Educate for Self-Authorship, M. B. Magolda and
P. M. King Eds. Sterling, VA, USA: Stylus Publishing, LLC, 2004,
pp. 37–62.
[37] M. B. Magolda and P. M. King, “Toward reflective conversations: An advis-
ing approach that promotes self-authorship,” Peer Rev., Emerg. Trends Key
Debates Undergraduate Educ., vol. 10, no. 8, pp. 8–11, 2008.
Authorized licensed use limited to: Mukesh Patel School of Technology & Engineering. Downloaded on February 06,2024 at 02:02:13 UTC from IEEE Xplore. Restrictions apply.
427
[38] S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT
evaluation with improved correlation with human judgments,” in Proc.
ACL Workshop Intrinsic Extrinsic Eval. Measures Mach. Transl. Sum-
marization, 2005, pp. 65–72.
[39] F. H. Wang, “A semantics-preserving approach to augmentation of Java
codes for training deep learning classification models,” in Proc. 25th
TANET, 2019, pp. 277–282.
[40] V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qual-
itative Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006.
[41] J. Y. Chen, K. Procci, M. Boyce, J. Wright, A. Garcia, and M. Barnes,
“Situation awareness-based agent transparency,” United States Army
Res. Lab., Adelphi, MD, USA, Tech. Rep. ARL-TR-6905, 2014.
[42] J. E. Mercado, M. A. Rupp, J. Y. C. Chen, M. J. Barnes, D. Barber, and
K. Procci, “Intelligent agent transparency in human-agent teaming for
multi-UxV management,” Hum. Factors, vol. 58, no. 3, pp. 401–415,
2016.
[43] K. Akash, W. L. Hu, T. Reid, and N. Jain, “Dynamic modeling of trust in
human-computer interactions,” in Proc. Amer. Control Conf., 2017,
pp. 1542–1548.
[44] W.-L. Hu, K. Akash, T. Reid, and N. Jain, “Computational modeling of
the dynamics of human trust during human–machine interactions,”
IEEE Trans. Hum.-Comput. Syst., vol. 49, no. 6, pp. 485–497, Dec.
2019.
[45] M. E. G. Moe, M. Tavakolifard, and S. J. Knapskog, “Learning trust in
dynamic multiagent environments using HMMs,” in Proc. 13th Nordic
Workshop Secure IT Syst., 2008, pp. 135–146.
[46] M. Danilevsky, K. Qian, R. Aharonov, Y. Katsis, B. Kawas, and P. Sen,
“A survey of the state of explainable AI for natural language proc-
essing,” in Proc. 27th ACM SIGKDD Conf. Knowl. Discov. Data Min-
ing, 2020, pp. 447–459.
[47] K. Akash, K. Polson, T. Reid, and N. Jain, “Improving human-
machine collaboration through transparency-based feedback–Part I:
Human trust and workload model,” IFAC-PapersOnLine, vol. 51,
no. 34, pp. 315–321, 2019.
[48] K. Akash, K. Polson, T. Reid, and N. Jain, “Improving human-computer
collaboration through transparency-based feedback – Part II: Control
design and synthesis,” IFAC PapersOnLine, vol. 51, no. 34, pp. 322–
328, 2019.
[49] G. J. Hwang, H. Y. Sung, S. C. Chang, and X. C. Huang, “A fuzzy
expert system-based adaptive learning approach to improving students’
learning performances by considering affective and cognitive factors,”
Comput. Educ., Artif. Intell., vol. 1, 2020, Art. no. 100003.
Feng Hsu Wang received the M.S. and Ph.D.
degrees in computer science and information engi-
neering from National Taiwan University, Taipei,
Taiwan, in 1988 and 1993, respectively.
In 1995, he joined the Graduate School of Informa-
tion Science and Engineering, Ming Chuan Univer-
sity, Taoyuan City, Taiwan, where he is currently a
Professor. His research interests include deep learn-
ing technology, educational technology, and peda-
gogy research.