G Model

ECE 164 1–7 ARTICLE IN PRESS

Education for Chemical Engineers xxx (2018) xxx–xxx

Contents lists available at ScienceDirect

Education for Chemical Engineers

journal homepage: www.elsevier.com/locate/ece

1 Online quiz methods for remedial learning in chemical engineering

2 Q1 Arniel Ching O. Dizon a,∗ , Siyu An b , Arnold A. Lubguban a , Galen J. Suppes b
a
3 Department of Chemical Engineering and Technology, Mindanao State University – Iligan Institute of Technology, Andres Bonifacio Avenue, Tibanga,
4 Iligan City 9200, Philippines
b
5 Department of Chemical Engineering, University of Missouri-Columbia, W2033 Lafferre Hall, Columbia, Missouri, 65211, United States
6

7
21 a r t i c l e i n f o a b s t r a c t
8
9 Article history: Online quizzes with feedback and retake options can be effective as a remedial learning strategy in a
10 Received 17 April 2017 curriculum with sequential courses that rely on prerequisites. Use of groups of two to three students
11 Received in revised form 19 April 2018 on these quizzes facilitates the learning process by creating an environment where students share their
12 Accepted 20 April 2018
knowledge and understanding. Group activities can extend to sharing and building upon the students’
13 Available online xxx
strategies for locating and using free online learning resources. This paper is on methods of preparing
14
online quizzes for remedial learning including structures of questions suitable for remedial learning and
15 Keywords:
approaches to attain analysis and evaluation learning levels for a course in reaction engineering.
16 Remedial learning
17 Online quiz © 2018 Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.
18 Online quiz feedback
19 Online quiz question
20 Higher-order learning

22 1. Introduction the entire class to a common level on which lecture content can be 45

spent. 46

23 Advanced courses in sequential curriculum should require This paper presents an approach to remedial learning with the 47

24 knowledge, understanding, and skill sets which are acquired from use of online quiz questions, quiz questions’ feedback, and the 48

25 course prerequisites. It is essential that these capabilities are met, opportunity to retake the online quiz. Additionally, an emphasis 49

26 otherwise, students’ academic weaknesses can lead to systemic is placed on methods of preparing online questions for purposes of 50

27 lowering of the capabilities of a program’s graduates. Also, from remedial learning. 51

28 the perspective of teaching, it is important for all students to have Remedial learning through online quizzes can be particularly 52

29 similar capabilities so that lecture materials are presented at the effective for the following reasons: 53

30Q2 right level.

31 This paper considers courses in chemical engineering ther- • Online quiz questions can engage students in learning more effec- 54
32 modynamics and chemical reaction engineering; courses that are tively than lectures and at multiple scheduled times throughout 55
33 typically 5th to 7th semester courses in a chemical engineering cur- the week. 56
34 riculum. Typical expectations on students entering these courses • Online feedback (including correct answers and hints) for 57
35 are summarized in Table 1. Listed in this table are topics typically incorrect answers provides students the opportunity to verify 58
36 required from these courses and which should have been learned corrected methodologies before retaking quizzes, providing just- 59
37 by the students in prerequisite courses. in-time learning. 60
38 A hybrid format in which about 33% of the course includes use • Just-in-time learning consumes little time of the more-learned 61
39 of online materials is particularly conducive to online quizzes with students with the repeat-exposure providing improved reten- 62
40 feedback and retake options. Such quiz-based processes are advan- tion. 63
41 tageous because these online processes: a) do not consume lecture • Online quizzes can provide group learning environments where 64
42 time repeating content already covered in prerequisites, b) will colleagues have a common goal to help each other learn; more- 65
43 not “bore” students who successfully learned and retained con- learned students can attain higher levels of learning by helping 66
44 tent from prerequisites, and c) can be more decisive in bringing other students learn the content. 67
• Remedial quizzes can progress to higher levels learning (e.g. eval- 68

uation level) in a familiar format (a continuation of the same 69

∗ Corresponding author. format as initial quizzes intended primarily for remedial pur- 70

E-mail address: arnielchingdizon@g.msuiit.edu.ph (A.C.O. Dizon). poses). 71

https://doi.org/10.1016/j.ece.2018.04.001
1749-7728/© 2018 Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
2 A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx

Table 1 Remedial learning is simplified when only two concept nodes are 114
Q5 Example of topics for remedial learning in two undergraduate chemical engineering
involved, particularly when remedial learning is applied sequen- 115
courses.
tially. In this case, remedial learning topics are not taken from 116
Course Title (prerequisites) Prerequisite Topics (remedial learning topics) the most basic courses but only from prerequisite courses. As 117

Chem. Eng. Thermodynamics
(aka Applied Physical
Chemistry)
 Physical Chemistry
 Physics
exemplified in Table 1, remedial learning topics, as determined 118
by the teacher, were taken from prerequisite courses. To elimi- 119
nate the need for individual diagnosis of learner deficiency, online 120
• Unit conversions
quizzes are given to all students, providing remedial learning to 121
• Temperature conversions
• Calculation of Force, Work, and Energy low-achieving students and mastery learning for high-achieving 122

• Pressure (hydrostatic, ideal gas law, gauge) students. In this method, online quizzes become a form of online 123
• Antoine equation formative assessment. 124
• Application of Energy Balance
Online quizzes created in a format compatible to Learning Man- 125

Chemical Reaction Engineering agement Systems (LMS) like Blackboard, Canvas, and Moodle could 126

 General Chemistry
 Calculus and Ord. Diff. Equ.
 Mass & Energy Balances
 Chem. Eng.
Thermodynamics
 Transport Phenomena
• All Prerequisite Topics of Chem. Eng.
Thermo.
• Atomic Balances (Reaction Stoichiometry)
• Reaction conversion, selectivity, and yield
• Analytical solutions to “xn dx” integration
• Trapezoidal Rule, Euler’s Method
be used as formative assessment tool within the context of reme- 127
dial learning to develop a level of mastery of the fundamental 128
topics required in an engineering course. In a study by Marden 129
et al. (2013), the quiz model that allows multiple unsupervised 130
and untimed attempts has the strongest formative focus on stu- 131

dents in a physiology class. This quiz method was associated with 132

a significant increase in mean examination performance. Although 133

the use of multiple choice questions in the study could be limit- 134
72 • Part of the remedial learning process can involve using, and learn- ing to lower-level learning (remembering and understanding), this 135
73 ing how to effectively use, online resources (e.g., Wikipedia) that quiz type proved to be useful in a physiology course where the 136
74 are more-readily available than textbooks to the students for gen- study was applied. However, in engineering courses, higher-level 137
75 eral use in their careers. learning (application, analysis and evaluation) questions must be 138

emphasized since engineers mostly work with optimization and 139

76 The last of these is particularly important since it can be cum- design. In addition, the use of higher-level questions can provide an 140
77 bersome for students to rely too heavily on their textbooks to solve efficient way of learning both higher and lower knowledge levels, as 141
78 problems. demonstrated in a study of Jensen et al. (2014). The study, as applied 142

in a biology course, suggested that exams requiring higher-level 143

79 1.1. Background learning (application, analysis, evaluation) encouraged deeper pro- 144

cessing of information and better memory for the core information 145

80 Several studies have applied computer learning systems for (memorization and understanding). 146

81 remedial learning. These computer systems were designed to According to Gikandi et al. (2011), three fundamental issues 147

82 determine learning deficiencies and misconceptions of individual must be considered in online formative assessment to realize 148

83 students and suggest remedial learning materials based on the indi- its desired outcome: validity, reliability and dishonesty. Validity 149

84 vidual’s learning profile. is the degree at which the assessment promotes further learn- 150

85 In some studies, conceptual graphs were used as basis in deter- ing; reliability is a measure of the sufficiency of the assessment 151

86 mining learner’s deficiencies (Chen, 2011; Chu et al., 2010; Jong on the knowledge level being developed; while dishonesty could 152

87 et al., 2004; Panjaburee et al., 2013; Ting & Kuo, 2016). A con- stem from an ineffective assessment which could mean poor lev- 153

88 ceptual graph shows concepts arranged in a hierarchical structure els of validity and reliability (Gikandi et al., 2011). In this paper, 154

89 where relationship and association of concepts are shown through validity and reliability are addressed by creating quiz questions 155

90 connecting nodes. It is a schematic representation of knowledge that can be varied multiple times and by creating questions with 156

91 structure which has been traditionally used as a guide in teach- knowledge-level progressions (up to evaluation level) in a format 157

92 ing and learning complicated topics. Learner’s deficiencies are that would require a single numerical value as answer. The latter 158

93 determined by comparing their individual conceptual graphs to an is a unique approach in creating online quiz questions that has not 159

94 expert’s conceptual graph. Remedial learning path and materials been attempted in existing literature. 160

95 are then suggested based on these learning deficiencies. This paper is written for a remedial learning process that is part 161

96 While a conceptual graph is a good basis for diagnosing learner’s of a blended course where online quizzes are part of weekly activ- 162

97 deficiencies, remedial learning can be implemented without using ities throughout the semester. Initially, the online quizzes would 163

98 conceptual graphs. Other studies successfully used alternative have a high remedial learning content, while quizzes later in the 164

99 methods aside from using conceptual graph as basis for remedial semester would have higher contents of material introduced in the 165

100 learning. Chen (2008) used genetics-based personalized e-learning course and higher learning levels for the remedial content. 166

101 system to generate the learning paths according to incorrect testing

102 responses in a pre-test. Lin et al. (2016) used fuzzy expert system
103 to determine the amount of remedial materials given to students 2. Discussion 167

104 based on their test results. Hsieh et al. (2013) used a system that
105 adopted the fuzzy logic theory to create a learning path for individ- A critical aspect of using online quizzes for remedial learning is Q3 168

106 ual learners and choose the remedial materials according to learner immediate feedback with the opportunity to learn how to correctly 169

107 preference. solve the problem followed by a retake of the quiz. This process is 170

108 All of these studies required the use of computer software in the illustrated by Fig. 1. 171

109 diagnosis of an individual’s learning deficiency and in the appli- The process starts with the first attempt at the quiz. The quiz 172

110 cation of individual remedial learning solution. However, these may be an individual or group activity. Feedback is given after each 173

111 computer systems are not readily available. attempt of the quiz is submitted. The feedback shows the correct 174

112 An alternative remedial learning system is proposed in this answer for each item. Also, hints toward approaches to solving the 175

113 paper where remedial learning is pursued through online quizzes. problem may be provided as feedback when the answer is incorrect. 176

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx 3

Table 2
Examples application level questions.

Temperature Conversion Application

Question The max temperature of an oven is [X]◦ F. What is the max
temperature of the oven in Kelvin?
Answer ((X-32)*(5/9)) + 273.15; [Range] 375 < X < 500
Feedback Convert ◦ F to ◦ C using the formula that relates ◦ F to ◦ C. Then,
convert ◦ C to Kelvin using the formula that relates ◦ C to K.
Reaction Stoichiometry Application
Question For the reaction, 2NO2 + 12 O2 → N2 O5 , what is the rate of
formation of N2 O5 if the rate of disappearance of NO2 is found
to be [X] mol/L/s?
Answer X/2; [Range] 0.3 < X < 0.5;
Feedback Use the reaction stoichiometry to relate the rate of formation
and disappearance of the components.

Table 3
Examples of analysis level questions.

Equation Conversion Question

Fig. 1. Block diagram on the use of online quiz for remedial learning.
177 The student can use the feedback as a guide in revising their solu-
178 tions and verify approaches to solution. After learning the correct
179 solution, the student can then retake the quiz with another ver-
180 sion of the question toward the goal of getting a perfect score. Quiz
181 questions of important topics are repeated in later quizzes includ-
182 ing random selection of quizzes from a pool. Therefore, students
183 are encouraged to review and answer all items correctly. Also, the
184 online quiz questions may be repeated in proctored final exams.
185 From a logistics perspective, the authors have found that the
186 averaging of quiz scores from multiple attempts is preferred than
187 grading the last or highest attempt. If all attempts are not con-
188 sidered in the calculation of the score, some students or groups
189 will quickly proceed through the first attempt to get the question
190 statements with an immediate focus to the retake of the exam. In
191 addition, all first attempts of a quiz must start at the same time (as
192 soon as the quiz is made available) so that a student or a group that
193 gets graded results cannot share the results to others who have not
Question The heat capacity of a compound is of the form Cp = a + b*T.
If a = [X] and b = [Y] with Cp in J/mol/K and T in C, what is
the value of “a” if T is in K and Cp is in J/mol/K.
Answer [X] − b*273.14; [Range] 2 < X < 4, 0.001 < Y < 0.008
Feedback To change units of Cp, multiply the entire equation times
an appropriate conversion factor. To change the units of T,
perform as substitution of variable in the equation
(e.g. T(C) = T(K) − 273.14.
Analysis Level Reaction Stoichiometry Question
Question [W] moles of a coal of the formula C[X]H[Y]O[Z] undergoes
complete oxidation with [V] percent excess oxygen in the
feed to the process. How many moles of diatomic oxygen
are in the feed to this process? (Excess oxygen is defined in
terms of actual oxygen initially present divided by the
oxygen needed for complete oxidation.)
Answer (1 + V/100)*W*(X + H/4 − Z/2); [Range] 1 < V < 20, 1 < W < 20,
1 < X < 1.5, 1 < Y < 5, 0 < V < 0.2
Feedback First, solve for the amount of oxygen for complete
combustion by balancing the stoichiometric equation for
formation of carbon dioxide and water. The oxygen
specification is the stoichiometric amount plus the
indicated percent excess.
For application level questions, it is sufficiently easy for the stu- 220

194 made a first attempt. These quizzes can be taken multiple times dents to either memorize an equation or where to find the equation. 221

195 within a 24 hour duration after the first attempt. Academic dishonesty is less of an issue than with higher-level ques- 222

196 Bloom’s taxonomy identifies, progressively, the levels of tions because memorizing the equation (or its location) is easier 223

197 learning as memorization, comprehension, application, analysis, than the effort in most actions of academic dishonesty. An effective 224

198 evaluation, and creation. For upper level (junior and above) engi- deterrent to academic dishonesty is to make actions of academic 225

199 neering courses, it is reasonable to focus on application and analysis dishonesty more difficult than learning the proper methods of solv- 226

200 level questions with the prospect of introducing evaluation level ing a given problem. 227

201 questions later in the remedial learning process. Analysis Level Questions – Analysis level questions are either 228

202 This paper emphasizes an approach of using formula questions not straight forward on locating useful information to solve the 229

203 which are possible in Learning Management System (LMS) plat- question or on how to use given values in an equation. Character- 230

204 forms. Example questions at each level are provided followed by istics of an analysis question might include: 231

205 an explanation of how the formula is set up in these questions.

206 After this, approaches to evaluation level questions are introduced. • An over-abundance of quantities specified in the problem state- 232

207 Application Level Questions – Application level questions tend ment where less than about half of the quantities are used in the 233

208 to be straight forward in both identifying the pertinent information solution. 234

209 in the question and the way information is inserted into an equation • The need to make unit conversions in quantities specified in the 235

210 to arrive at a solution. The equations may be memorized, may be statement to properly apply in the available formats (e.g. a com- 236

211 obtained from textbooks, or may be located in online resources. puter program) for solution. 237

212 An example application-level quiz question is: • An application of a general equation (e.g., energy or mass balance) 238

where terms in the equation need to be specified (e.g., set to zero) 239
213 The max temperature of an oven is 420 ◦ F. What is the max tem- based on analysis of how the equation applies to the question. 240
214 perature of the oven in Kelvin?
215 In formula type questions, the input variables in a problem vary Table 3 provides example analysis level questions. The calcula- 241

216 within a range specified with the problem is set up in a LMS plat- tions involve one or more of the listed characteristics, and for these 242

217 form. Consequently, the final answer varies between students and examples, the questions use multiple variables to create variation. 243

218 attempts. Table 2 provides example application level questions, Both question formats require more than the plugging of num- 244

219 including this example question written in formula form. bers into equations. The first requires a substitution of variable and 245

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
4 A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx

246 the second requires a more-complex balancing of a reaction sto- • A problem statement that includes a complex base case spread- 311

247 ichiometry. However, there is the risk that students could either sheet calculation. 312

248 memorize the equations or set up a spreadsheet with the solution

249 ready to perform. An additional step of creating variations in the Providing base case computer code or spreadsheet calcula- 313
250 problem statement can create a deterrent from the preparation of tions provides for the opportunity to specify details of a problem 314
251 spreadsheet answers, or at least, increase the learning involved in statement more concisely than through the sole use of complete 315
252 creating/using multiple versions of a spreadsheet solution. sentences. These problems provide for the ability to present higher- 316
253 The types of variations for the equation conversion question level questions in formats that can be solved in a matter of minutes. 317
254 include: a) 20 combinations of variables X and Y, b) 20 variations in Also, the formats reinforce and expand the student’s skills in 318
255 the energy and temperature unit combinations for the heat capac- computer-based calculations. The following is an example of such 319
256 ity, c) 4 variations of the temperature units as multiplied times b, a question: 320
257 and d) a request of the value of either a or b. These variations result
258 in hundreds of different numeric values of the answers and dozens PDEPE file was set up to simulate the temperature pro- 321

259 of different possible equations. The goal of questions like these is file of a metal rod using the following specifications of the 322

260 to teach approaches to solutions. differential equation, the initial condition, and the bound- 323

261 Overcoming Issues with Academic Dishonesty – Students ary conditions: 1) c = 5; 2) f = DuDx; 3) s = 0; 4) u0 = 25; 5) 324

262 should be encouraged to collaborate in learning processes because pl = 5; 6) ql = 1; 7) pr = 0; and 8) qr = 1. How would the 325

263 of: a) high access to peers, b) the desirable social/networking envi- boundary condition equation specification change to reflect a 326

264 ronments of collaboration, and c) both the recipient and provider of metal rod with twice the heat flux of this specification as pro- 327

265 assistance benefit in the process. On the latter, the organizing and vided? Your answer is to include identifying the number (1-8) 328

266 conveying of information helps the provider achieve higher levels of the specification to change, multiplying that number times 329

267 of learning. 100, and adding this to numeric value of the new specification 330

268 In these group collaboration environments, the transition from This question involves setting up a partial differential equation 331
269 collaboration for learning to academic dishonesty is not always (PDE) in Matlab with proper use of the PDEPE function input and 332
270 clearly defined. For example, it is undesirable for the students to could be a remedial learning question if such PDE solutions were 333
271 share and create a database of correct answers. To prevent this, the part of a course prerequisite. Once a student is familiar with the 334
272 syllabus can state that students are not allowed to share or accept PDEPE function and has access to both the applicable partial dif- 335
273 answers from quiz attempts with/from others. However, because ferential equation and boundary condition equations, the problem 336
274 “exam files” of previous exams in a course are such a tradition, this is solvable in a few minutes. Also, there is sufficient variation both 337
275 can be difficult to enforce. in numeric values and how to proceed to promote the student’s 338
276 An alternative policy is that no assistance may be provided or learning of approaches to solve this type of a problem. 339
277 received to/by a student taking a quiz. Under this basis, enough Evaluation Level Questions – Engineering applications allow 340
278 variations in a question and the solution equations can be created for evaluation level questions based on comparison of results from 341
279 to make it more difficult to perform academic dishonesty than to two or more calculations where the ultimate decision includes a 342
280 learn the material. judgment as to which of multiple alternatives solves the problem. 343
281 Hence, the art of preparing online quiz questions is both one of With experience, inspection of alternative approaches (short of 344
282 creating maximum variation (with minimum effort) and creating actually calculating values) may be enough to identify the approach 345
283 question progressions that target various levels of learning, tar- that will provide the desired answer. Example types of problem 346
284 geting advanced analysis levels and evaluation levels later on the statements for evaluation level questions include: 347
285 course. An example quiz format may allow a student to repeat a quiz
286 (and specific questions) three times. In addition, that same ques- • A problem statement involving multiple unit operations with 348
287 tion may be randomly placed on a quiz another three times during
multiple possible control volumes where only one of multiple 349
288 the semester. If a question has insufficient variation, the student
control volumes will allow for solution. (The evaluation is the 350
289 can transform a higher-level question (e.g. analysis) to the lowest
comparison of different control volumes.) 351
290 level of question (memorization). • A problem statement including an optimization process such as 352
291 As applied in an undergraduate reaction engineering course,
identifying conditions that maximize a temperature, conversion, 353
292 students were required to pass the proctored individual final exam
or any of a variety of objective functions. (The evaluation is of 354
293 with over half of the questions direct repeats of the quiz questions.
multiple calculations with comparison of results to identify the 355
294 Students were informed at the start of the course that the proc-
optimum.) 356
295 tored final has a repeat of many of the quiz questions but is to be • Problems with two-part answers can be evaluation level if the 357
296 performed without internet access. This was the best way to deal
first part of the answer includes the identification of the type of 358
297 with academic dishonesty without getting proof.
calculation to be performed to meet an objective and the second 359
298 Advanced Analysis Level Questions – These questions include
part includes an answer of numeric value. 360
299 the specification of dozens of input parameters/equations, the need
300 to perform preliminary conversions before performing calcula-
301 tions, and/or an identification of how to input information into Table 4 provides an example evaluation level question. The fol- 361

302 computer program code. These advanced level analysis questions lowing steps are involved in the solution: 362

303 are borderline evaluation level.

304 Example approaches to these advanced analysis level questions • The lines of Matlab program code in the problem statement need 363

305 include: to be converted to a functioning computer program (this involves 364

adding additional code and specifying the function). 365

306 • A problem statement that includes a complex base case computer • The student must identify that the program is set up with an initial 366

307 program and a variation in the base case program where analysis temperature of 295.15 K, and that this is the value to be changed 367

308 is required to identify how the input or equations are to be varied until the objective is met. 368

309 to provide the solution. The qualifier “complex” implies multiple • The program must be run, and the output must be evaluated to 369

310 specifications (explicit or implicit) of input, equations, and units. identify the maximum temperature of the profile. 370

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx 5

Fig. 2. Average scores per attempt of groups in group quizzes.

Table 4 programs. It has the potential to help preserve the superior learning 393
Example evaluation level question.
environment of traditional campuses. 394

Equation Conversion Question Group quizzes following the algorithm of Fig. 1 have been 395
Question: A base-case near-adiabatic polymer-reaction simulation program applied in an undergraduate reaction engineering course. Students 396
(of 1-liter reacting at constant density) is based on basis set up to simulate a
were instructed to select their own group of three people and to 397
polymerization under Flory’s assumptions and includes the following
conditions code: fa (3.5), fb (3.5), k0(1E4 1E4 1E4 1E4; l/mol/s), E0(40 40 40
meet physically in a classroom setting to perform the group online 398

40), mCp = [X] (kJ/C for a 1-liter basis), Hrxn = −70 (kJ/mol, same for all quiz. A group of three is found to be optimal as the students can 399
reactions) and initial concentrations and T (1.1 1 0 fa*1.1 fb*1 295.15). What easily fit in front of a computer screen with all able to contribute 400
is the minimum preheating temperature needed to achieve a peak reaction readily. 401
temperature of 150C?
In 12 quizzes with 11–13 groups per quiz, 274 total attempts 402
Answer: T = a + b*X; 1.0 < X < 2.5.
Feedback: The preheat temperature is the initial temperature as specified in were recorded. The maximum number of attempts recorded in a 403

initial condition vector. This problem can be solved in a trial-and-error group was only up to 3 attempts. Out of the 274 attempts made 404
process where the initial temperature is varied until the maximum for the 13 quizzes, 155 attempts were first attempts, 68 attempts 405
temperature of the temperature profile generated by the program is 150C.
were second attempts, and 51 attempts were third attempts. Out 406
Note: The solution of T = a + b*X must be determined when setting up the problem. of the 155 total first attempts, 87 attempts (56.13%) were first and 407
This solution can be determined by solving the problem for multiple values of X and only attempts of a quiz, 17 first attempts (10.97%) progressed until 408
fitting a polynomial expression to those solutions.
second attempts and 51 first attempts (32.90%) progressed until 409

third attempts. Out of the 68 total second attempts, 17 attempts 410

(25%) were only up to second attempts and 51 attempts (75%) con- 411
371 • The process of selecting a new input temperature and evaluat-
tinued to third attempts. Groups were more likely to continue the 412
372 ing the program output needs to be repeated until the output attempts until the third attempt when they have reached the sec- 413
373 temperature is as specified in the problem statement. ond attempt. Fig. 2 shows average scores of groups that took 3 414

attempts of a quiz. Average scores generally improved in subse- 415

374 The Table 4 example may be modified to include heat loss as a quent attempts, for instance, in the first group quiz labeled as GQuiz 416

375 variable, targeted maximum temperature as a variable, and/or heat 1, average scores are 46.67%, 66.67%, and 100% for the first, second, 417

376 of reaction as a variable. These modifications can be made to pro- and third attempts, respectively. 418

377 vide a high degree of robustness against the student transforming None of the groups continued past the third attempt. Students 419

378 the question to a memorization process (based on earlier attempts). chose to stop at the third attempt and may be satisfied when they 420

were able to answer 5 out of 6 questions, which is the usual number 421

of questions per quiz. Hence, the average quiz scores of the third 422
379 2.1. Group quizzes for Remedial Learning attempt are above 80% as shown in Fig. 2. Also, students may have 423

determined three attempts as their duration limit for a group quiz. 424
380 Group-based quizzes with feedback and retake options offer All quizzes and all attempts per quiz were given a one hour time 425
381 what is potentially one of the most efficient learning processes by limit. The average duration on each attempt per quiz, for groups 426
382 putting students in a situation where they can apply their collec- that took three attempts of the quiz, is shown in Fig. 3. The bar 427
383 tive intellect to rapidly understand the problem, apply methods, chart shows a decrease in the average time taken in succeeding 428
384 and verify their abilities. On remedial topics, quiz questions can attempts for all quizzes. For group quiz 1, labeled as GQuiz 1, the 429
385 be presented without formal preparation as a method to bring all average durations are 19.1, 10.4, 6.1 minutes for the first, second, 430
386 members of a class to the same level of learnedness so that lectures and third attempts, respectively. This chart is an indication that 431
387 can be presented at a more-optimal level than otherwise possible. students were becoming efficient in solving succeeding attempts, 432
388 An additional advantage of quizzes in which students physically getting more correct answers in shorter durations. 433
389 gather in front of a single computer is the social and networking The group online quizzes were followed by the individual online 434
390 environment it can create. In an era where college education is quizzes a few days later where the individual quizzes are to be 435
391 rapidly evolving, these group-type of environments could emerge taken outside the classroom setting. As most of the quiz questions 436
392 as one of the greatest values of traditional campuses over online

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
6 A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx

Fig. 3. Average attempt duration of group quizzes.

Table 5
Survey Questions.

Code Survey Question

Learning Your learning in this course is better than other

Fig. 4. Survey Results for 11 students of Chemical Reaction Engineering class
with supplementary online activities. SA-Strongly Agree, A-Agree, N-Neither
Agree/Disagree, D-Disagree, SD-Strongly Disagree, N/A-Not Applicable.
437 in the individual quiz are repeats from the group quiz, it is the
438 responsibility of each individual to learn from the group quiz if
439 they are not able to be a leader in contribution.
440 Group quizzes following the algorithm of Fig. 1 have also
441 been implemented for two semesters over a two-year period in
442 a graduate-level core-curriculum chemical engineering course on
443 reaction engineering. A survey was conducted in a group where
444 6 out of the 11 students had taken and passed an undergraduate
445 course in chemical reaction engineering. The results of the survey
graduate courses you have taken or are learning.
Time The time required in this course is high relative to
other graduate course you have taken or are taking
(strongly agree indicates high relative time
requirement, strongly disagree indicates low relative
time requirement).
Remedial Learning If you have needed remedial learning (topics put
forward as review, on quizzes, but not necessarily
covered in class) the group approach was effective for
efficiently achieving the remedial learning (please
mark N/A if you have needed minimal remedial
learning).
Group Quizzes Of the learning methods used in class (lectures,
homework, group quizzes, individual quizzes, reading,
in-class quizzes/worksheets), group quizzes were
more effective for learning (strongly agree means most
effective, agree means effective but not most effective
relative to other aspects).
Expectation to learn You had good expectations to learn in this class and
the class is meeting these expectations.
for the small sample size, substantiating the value of group quizzes 460
using the approach of Fig. 1 algorithm. 461
3. Conclusion 462

446 are shown in a stacked bar chart in Fig. 4.

447 When asked, “What is the most effective learning method you Improved remedial learning methods can greatly expand the 463

448 have used in this class (lecture, group quizzes, online quizzes, inde- efficiency, availability, and quality of college education. Remedial 464

449 pendent reading, in-class activities other than lecture)?”, group learning based on a process involving the retaking of online quizzes 465

450 quizzes obtained a 5.5 score while independent reading and in class with feedback and ability to verify approaches (by being provided 466

451 activities obtained 1.0 and 0.5, respectively. This is supported by the correct answers to past quiz attempts) has been shown to be partic- 467

452 result of the survey under the code ‘Group Quizzes’ where 6 out of ularly effective as a learning method. In this process, it is critical that 468

453 11 found group quizzes to be more effective for learning than other the quiz questions can be retaken multiple times while retaining 469

454 methods and 5 out of the 11 found it to be effective for learning demands for “approaches” to be used to solve the problem versus 470

455 method. The survey questions are presented in Table 5. “memorization” (robustness criterion). In addition, repeating ques- 471

456 The results of the survey do not merely indicate that remedial tions of critical topics throughout the course, with progressive 472

457 quizzes with feedback and retake can provide efficient learning, increases in the targeted learning levels, will solidify the learning 473

458 the results indicate that this process is one of the most effective of applications and analyses levels while fostering the attaining of 474

459 learning methods available. The strong signal tends to compensate evaluation-level capabilities (improvement criterion). 475

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001
 G Model
ECE 164 1–7 ARTICLE IN PRESS
A.C.O. Dizon et al. / Education for Chemical Engineers xxx (2018) xxx–xxx 7

476 Examples have been provided that illustrate that online ques- Chen, C.-M., 2008. Intelligent web-based learning system with personalized learning 492

477 tions for engineering problems can be prepared to meet both path guidance. Comput. Educ. 51 (1), 787–814. 493
Gikandi, J.W., Morrow, D., Davis, N.E., 2011. Online formative assessment in higher 494
478 robustness and improvement criterion. It is possible, and per- education: a review of the literature. Comput. Educ. 57 (4), 2333–2351. 495
479 haps likely, that group-based online remedial quiz processes will Hsieh, T.C., Lee, M.C., Su, C.Y., 2013. Designing and implementing a personalized 496
480 emerge as one of the major advantages of traditional campuses remedial learning system for enhancing the programming learning. Educ. Tech- 497
nol. Soc. 16 (4), 32–46. 498
481 in a rapidly evolving environment of higher education. Effective Jensen, J.L., McDaniel, M.A., Woodard, S.M., Kummer, T.A., 2014. Teaching to the 499
482 remedial learning can both improve the capabilities of a program’s test. . .or testing to teach: exams requiring higher order thinking skills encourage 500
483 graduates and reduce the time and cost of college education. greater conceptual understanding. Educ. Psychol. Rev. 26 (2), 307–329. 501
Jong, B., Lin, T., Wu, Y., Chan, T., 2004. Diagnostic and remedial learning strategy 502
based on conceptual graphs. J. Comput. Assist. Learn. 20 (5), 377–386. 503
484 Acknowledgement Lin, C.-C., Guo, K.-H., Lin, Y.-C., 2016. A simple and effective remedial learning system 504
with a fuzzy expert system. J. Comput. Assist. Learn. 32 (6), 647–662. 505
Marden, N.Y., Ulman, L.G., Wilson, F.S., Velan, G.M., 2013. Online feedback assess- 506
Q4
485 This work was supported by the USAID-STRIDE [grant number
ments in physiology: effects on students’ learning experiences and outcomes. 507
486 0213997-G-2016-017-00]. Adv. Physiol. Educ. 37 (2), 192–200. 508
Panjaburee, P., Triampo, W., Hwang, G.-J., Chuedoung, M., Triampo, D., 2013. Devel- 509
opment of a diagnostic and remedial learning system based on an enhanced 510
487 References
concept – effect model. Innov. Educ. Teach. Int. 50 (1), 72–84. 511
Ting, M.Y., Kuo, B.C., 2016. A knowledge-structure-based adaptive dynamic assess- 512
488 Chen, L.H., 2011. Enhancement of student learning performance using personalized ment system for calculus learning. J. Comput. Assist. Learn. 32 (2), 105–119. 513
489 diagnosis and remedial learning system. Comput. Educ. 56 (1), 289–299.
490 Chu, H.-C., Hwang, G.-J., Huang, Y.-M., 2010. An enhanced learning diagnosis model
491 based on concept-effect relationships with multiple knowledge levels. Innov.
Educ. Teach. Int. 47 (1), 53–67.

Please cite this article in press as: Dizon, A.C.O., et al., Online quiz methods for remedial learning in chemical engineering. Educ. Chem.
Eng. (2018), https://doi.org/10.1016/j.ece.2018.04.001