Representing Examination Question Knowledge into Genetic Algorithm

Noor Hasimah Ibrahim Teo, Nordin Abu Bakar, Mohamad Rezduan Abd Rashid
Faculty of Computer and Mathematical Sciences
University Technology MARA
Selangor, Malaysia
shimateo@tmsk.uitm.edu.my, nordin@tmsk.uitm.edu.my, naudzer90@gmail.com

Abstract— Examination is a medium of knowing the IQ of a
person and how far is the understanding of certain subjects.
Normally, the questions are prepared by course instructor in
sets and become a burden if the subject is new to them as they
needs to prepare it in a given time. This paper describes a
method of auto generating a new set of final exam questions.
The objectives of this paper are: to optimize selection of final
examination question based on the Cognitive Level of Blooms
Taxonomy; to design and develop a prototype of auto-
generator examination question using Genetic Algorithm and
to evaluate performance of this tool. There are many types of
questions in an examination; but this system chooses to analyze
the structured question only. The newly populated
examination questions are based on the fittest value of the
fitness function calculated. This technique can be upgraded
and be used by other type of question too.
Keywords- examination question; knowledge extraction;
Genetic Algorithm; cognitive knowledge ;
1. INTRODUCTION
Through examination, students’ progression in their
programmes can be determined and enables them to
demonstrate that they have achieved the intended learning
outcomes. It is also important to relate it with the
examination questions.
In this study, questions are extracted from the final
examination question bank. The tool will give benefits to
instructor that has prepared the question. It will helps in
reducing human error and time taken to prepare a particular
set of question paper. The quality of the question paper
generated will be based on the Bloom’s Cognitive level [7]
[8] [9]. The instructor can prepare proper questions based on
Blooms Taxonomy as they don’t have to be rushed because
they can save the questions that they prepared at any time
before the question is picked [10]. They also can generate a
full set of exam questions in a very short time; through a
button-click. The quality is also being controlled before
being used in the examination.
This paper discussed the algorithm behind the
development of this tool. The tool applies Genetic
Algorithm (GA) technique for selecting the questions for the
examination paper, to design the system requirement for
auto-generator exam’s question paper and to develop and
test the tool.
Genetic Algorithms were invented to mimic some of the
processes observed in natural evolution. The idea with GA
is to use this power of evolution to solve optimization
problems.

It is time consuming in creating question and editing the
format of the exam paper. Lecturers have to manually
ensuring the level of the questions at least has the same level
with the students. Traditional test paper generating
algorithms have some shortcomings, such as slow
generating speed, low success probability and low
generating quality [1].
Existing system for selecting examination question
used different technique such as utility based agent, genetic
algorithm [2] [3] and shuffling algorithm [4]. Shuffling
algorithm only focuses on selecting the paper randomly
without repetition and duplication and does not include the
difficulty and level of the question. The types of question
selected also do not vary. Low quality of exam paper with
fewer opportunities to success and expensive costs [5].
Whereas the utility based agent only focused on a single
agent system [6].
The general algorithms of GA [11]:
i. Randomly generate an initial population M(0)
ii. Compute and save the fitness u(m) for each
individual m in the current population M(t)
iii. Define selection probabilities p(m) for each
individual m in M(t) so that p(m) is proportional to
u(m)
iv. Generate M(t+1) by probabilistically selecting
individuals from M(t) to produce offspring via
genetic operators
v. Repeat step 2 until satisfying solution is obtained.
Figure 1 show the flowchart on how the genetic algorithm
works.

978-1-4799-3190-3/14/$31.00 ©2014 IEEE 3-5 April 2014, Military Museum and Cultural Center, Harbiye, Istanbul, Turkey
2014 IEEE Global Engineering Education Conference (EDUCON)
Page 900

Figure 1: Flowchart on Genetic Algorithm Programming [3]
Scope is narrowed to the text-based question or the
structured questions and followed the cognitive level of
Blooms Taxonomy. There are lots of people who will have
benefit from the tool. Among them is an instructor. They
will get benefit from the system since they can provide the
question based on learning outcome. Next, student will
know their ability and performance in certain subjects. This
research will also bring its own benefit to any institutions or
organization that want to evaluate the quality of
examination question paper.
In this paper, the GA technique will be used to select the
questions to generate the exam paper. It is an adaptive
heuristic search based on evolutionary ideas of natural
selection and genetics. GA’s exploit random search in
optimizing the selection, it is random but have
objective/lead to the result of the search.
2. PROPERTIES FOR EXAMINATION QUESTION
GENERATOR
A typical examination question paper is divided into
several sections which indicate different type of questions.
Each question paper has multiple properties to define its
attribute [12]. These properties are required to achieve
aforementioned objectives. Following are the properties
considered for this exam generator.
• Type [13]: Indicate type of question that needs to be
created for examination paper. Type of question
considered for analysis is short answer.
978-1-4799-3190-3/14/$31.00 ©2014 IEEE 3-5 April 2014, Military Museum and Cultural Center, Harbiye, Istanbul, Turkey
2014 IEEE Global Engineering Education Conference (EDUCON)
Page 901
• Score: Indicate score for each question.
• Total Score: Indicate the overall score for the
examination paper. This total score is indicated by user.
• Level: Indicate the difficulty level for each question.
Difficulty level is determined using Bloom taxonomy
cognitive level. Each course has it predefine cognitive
level.
• Number of Question: Indicate number of question to be
generated for the examination question paper.
3. THE SYSTEM DESIGN
Fig. 1 shows the activities undertaken to complete this
project. It applies a standard System Development Lifecycle
(SDLC).
Preliminary
investigation Question Bank
Extract information
Data Collection
Question Information Table
System design
Genetic Algorithm generator
Development and Examination
testing
question paper
generated
Documentation
Figure 2: System design
Question Bank will store all questions that have been
created by the instructors. Questions are organized according
to the set of paper for each semester and by subject.
Properties of these question will be extracted and keep in
information table. The information from this table will be the
input for the GA generator. The GA generator will process
the input to obtain new population with fittest. New set of
examination question paper will be produced from this step.
1. Database Structure
The examination question collected in .pdf format will
be converted to .txt format. Database contains information
of the examination question that is extracted from the .txt
format of exam question and will be written into
information table. Fig. 2 shows the extraction from question
bank into information table. Information table contain
information such as question number, question, score and
keywords from each question respectively.

Figure 3: Extraction from exam question to innformation table
2. Populating Questions using GA
Two initial populations created from thhe questions that
are selected randomly according to the num mber of question
input by user. Each population reppresented by a
chromosome in genetic algorithm. Fig. 3 shows the
representation of examination question insiide chromosome.
Each cell in the chromosome will representt each question in
the selected question paper.
Figure 4: Example of chromosom
mes
Fitness value for each chromosome w
will be calculated
according to the formula in (1).
௎ூି௉ூ
x100. (1)
௎ூ
UI is the value of user input and PI is the value within
the population. The smaller value of fitness function
indicates smaller error in the population. Errror is defined as
the difference between user input and geneerated population
requirement.
Selection: The best population is indicatted by the lowest
value of fitness function between two iniitial populations.
The Roulette wheel selection [13] is used tto select the best
978-1-4799-3190-3/14/$31.00 ©2014 IEEE 3-5 April 2014, Military Museum and Cultural Center, Harbiye, Istanbul, Turkey
2014 IEEE Global Engineering Education Conference (EDUCON)
Page 902
population. Each of the questions also has corresponding
score and level information.
Single crossover technique is used with crossover point
is chosen randomly. The process is i to enhance the fitness
value of generated chromosome.
The mutation process will take out
o one question from the
chromosome and replace it with one new question from
question bank randomly.
At the mutation point, the bit iss mutated and number of
the question is changed with their reespective score and level.
Step selection to mutation will be repeated until the
optimal objective function or max ximum loop is obtained.
All fitness value will be compared d and the lowest will be
chosen. Optimal fitness is evaluatedd based on user input and
the maximum loop is set to 1000 tim mes.
4. RESUL
LT
Fig. 4 shows the input interface for the tool support. The
input indicates the instructor reequirement in order to
construct a question paper. The inpput consists of number of
question needed, percentage of eachh level and total score.
Figure 5: User in
nput
User has to input the num mber of questions and the
percentage of each level of Bloom ms Cognitive Level. The
values will be the reference of all caalculations. For example,
if user inserts 10 for the number ofo questions, 15% of it is
from cognitive level 1, 15% of it from cognitive level 2,
10% of it from cognitive level 3, 10%1 of it from cognitive
level 4, 20% of it from cognitive levvel 5, and 30% of it from
cognitive level 6. The percentage will
w determine how many
questions from the 10 questions inttended for each cognitive
level. Result button will show the questions
q generated to the
user.
 The fitness value is calculated based on user input.
In this case, the number of questions, and the percentage of
questions for each level of cognitive level from level one
until level six is the boundary in calculating the fitness of
the population of questions sets. The sums of the percentage
of each level must be 100. The questions had been extracted
from pdf files to text files using software. After that, the
questions are divided according to Blooms cognitive levels
using keyword matching. Fig. 5 shows the new generated
question paper.
Figure 6: New question generated
The .txt file show the list of question generated according to
the input user requested. The prototype has been run 100
times. In order to do the analysis, the fitness value of each
run was captured. Each run consist of different number of
question and percentage of each cognitive level.
Fig. 6 shows the graph analysis showing the fitness value of
each trial. As shown below the fitness can reach as high as
97%.
Figure 7: Graph analysis of fitness value
5. CONCLUSION
From creating examination paper manually that can make
the panels become stressed, this paper proposed tool support
for populating examination question paper based on
Cognitive Levels of Blooms Taxonomy and to develop an
auto-generation of examination question prototype.
978-1-4799-3190-3/14/$31.00 ©2014 IEEE 3-5 April 2014, Military Museum and Cultural Center, Harbiye, Istanbul, Turkey
2014 IEEE Global Engineering Education Conference (EDUCON)
Page 903
There are two main techniques used; text matching, to
match the keywords of cognitive domain in Blooms
Taxonomy with the question so that the questions can be
sorted accordingly and genetic algorithm, to process the
sorted questions to become a new set of better combination
of questions.
This project only focused on the structured question on
the examination question. The number of question for each
trial also affects this project end result because now there
are fewer questions for each cognitive level. The question
should be retrieved from a text file of bank question so that
there are more options to be selected; thus increase the
fitness values. The marks of the question also should be
used to calculate the fitness value.
For future research, the scope can be opened to other
Computer Science and Mathematics papers too.
Furthermore, the question types in examination papers are
not only the short answer question. The other types are
multiple choice questions, true or false, fill in the blanks and
essay. This tool support can be enhanced to include the
other question types so that a perfect set of examination
questions can be made.
ACKNOWLEDGMENT
This research is funded by Ministry of Higher Education
(Malaysia) and Universiti Teknologi MARA (UiTM) under
Research Acculturation Grants Scheme (RAGS)
(RAGS/2012/UITM/SG05/2).
REFERENCES
[1] Xiong, L., & Shi, J. (2010). Automatic Generating Test Paper
System Based On Genetic Algorithm. Second International
Workshop on Education Technology and Computer Science
[2] NHI Teo, NA Bakar, S Karim (2012). Designing GA-Based
Auto-Generator of Examination Questions. Computer
Modeling and Simulation (EMS), 2012 Sixth UKSim/AMSS
European Symposium.
[3] Genetic Algorithm. (2010). Genetic Algorithm. Retrieved
October 4, 2012, from http://www.myreaders.info/09-
Genetic_Algorithms.pdf
[4] Sutan, N. S. (2010). Shuffling Algorithms for Automatic
Generator Question Paper System. Computer and Information
Science (CCSECIS) , (pp. 3(2):244-251.).
[5] Huang, W., & Wang, Z.-h. (2008). Design of Examination
Paper Generating System from Item Bank by Using Genetic
Algorithm. International Conference on Computer Science
and Software Engineering.
[6] Memoona Naz, M. A.-u.-h. (2010). Utility Based Agent for
Test Paper Generation. International Journal of
Multidisciplinary Sciences and Engineering, (pp. Vol. 1, No.
1, September 2010.).
[7] Bjorn, S. (2012). Bloom's Taxonomy. Retrieved October 20,
2012, from http://www.coun.uvic.ca/learning/exams/blooms-
taxonomy.html.
[8] Bloom, B. (. (1956). Taxonomy of educational objectives: The
classification of educational goals: Handbook I, cognitive
domain. New York.
 [9] Clark, D. (2010). Bloom's Taxonomy of Learning Domains. Jones, J. H. (2009). Relationship Between Examination
Retrieved October 20, 2012, from Questions and Bloom's Taxonomy. ASEE/IEEE Frontiers in
http://www.nwlink.com/~donclark/hrd/bloom.html Education Conference W1G-6. San Antonio, Texas
[10] Huitt, W. (2011). Bloom et al.'s taxonomy of the cognitive [12] University of Waterloo. (2012). Centre for Teaching
domain. Retrieved October 20, 2012, from Educational Excellence. Retrieved October 20, 2012, from Exam
Psychology Interactive. Valdosta, GA: Valdosta State Questions : Types, Characteristics and Suggestions:
University: http://cte.uwaterloo.ca/teaching_resources/tips/exam_question
http://www.edpsycinteractive.org/topics/cognition/bloom.htm s.html
l
[11] Kumara Sastry, D. G. (2004). Genetic Algorithms. University
of Illionis, USA, University of Nottingham, UK.Karl O.
[13] NeuroDimension, Inc. (2012). Optigen Library – Selection
Operators. Retrieved October 20, 2012 ,from
http://www.nd.com/genetic/selection.html

978-1-4799-3190-3/14/$31.00 ©2014 IEEE 3-5 April 2014, Military Museum and Cultural Center, Harbiye, Istanbul, Turkey
2014 IEEE Global Engineering Education Conference (EDUCON)
Page 904