562 International Medical Journal Vol. 21, No. 6, pp.

562 - 566 , December 2014

MEDICAL EDUCATION

Comparative Study of Different Formats of MCQs: Multiple

True-False and Single Best Answer Test Formats, in a New
Medical School of Malaysia

Nordin Bin Simbak, Myat Moe Thwe Aung, Salwani Binti Ismail,
Norhasiza Binti Mat Jusoh, Tarik Ibrahem Ali, Wisam Abdul Kadder Yassin,
Mainul Haque, Husbani Mohd Amin Rebuan

ABSTRACT
Introduction: Multiple choice questions (MCQs) alone or in conjunction with other test instruments are extensively used in
high-stakes examination in medical institutions. Amongst numerous genres of MCQ style of exam with five options, it seems that
examinees have greater chance to get correct answers for multiple true-false (MTF) than single best answer (SBA). Moreover,
there has been no previous literature review revealed prediction and correlation of MCQ test format with overall student's per-
formance.
Objectives: This study, therefore, examined students' performance between these two evaluation techniques and correlated
them with other assessment outcomes.
Materials and Methods: The calculated contribution results of SBA orthopedic and negative no-carry-over marked MTF sur-
gery questions of Year 3 students enrolled in Session (2011/12) and Session (2012/13) in UniSZA were analyzed separately for
each year. In addition, the correlation coefficient (r) was figured out to determine the linear relationship between each exam
style and other performance results.
Results: Both academic year results pointed out that the students got higher marks in SBA questions than MTF quiz. SBA
test results were found to be well correlated with clinical marks (objective structure clinical examination (OSCE) and long case
examination and modified essay question (MEQ).
Conclusion: The findings of this study encouraged the use of SBA quiz in place of MTF test format for examinations.
However, the diversity of marking systems, competence of lecturers, teaching techniques and question difficulty indexes between
these two different subject matters may lead to biased conclusion. Thus, further studies of comparison between these two exam-
ination styles conducting in the same subject are requisite to get more reliable and unbiased inferences.

KEY WORDS
comparison, multiple true-false, single best answer, correlation

INTRODUCTION
Human resources development especially for health professionals
which definitely include medical doctor as first, are the key indicator to
determine and ensure 'economic and social prosperity' for any nation
(Gajjar et al., 2014). "Assessment plays a major role in the process of
medical education, in the lives of medical students, and in society by
certifying competent physicians who can take care of the public"(Shum-
way and Harden, 2003). It is in reality 'assessment' which energies stu-
dent to learn and also make them aware for the development 'competen-
cies'; thus to accomplish in exam (Gajjar et al., 2014; Cohen-Schotanus,
1999). Assessment is a critical element and 'major challenge' of medical
education, and if it is properly designed, it can help accomplish key
goals of curriculum (Case and Swanson, 2002; Reinert, 2013). Medical
education has a number of test formats. The variety testing style has
been basic characteristic of any medical school. Essentially these
diverse testing ensures standard of the education and to give license a
newly graduate medical doctor to treat mankind (Norcini and McKinley,
2007). One assessment method will not ensure to assess all domain of
medical education (Al-Wardy, 2010). There are several styles of test for-
mats such as quizzes, essay forms, case presentations, etc., evaluating
the quality of student performance. The way of assessment used by fac-
ulty may have a huge impact on the student learning. It affects on how
the student should spend the amount of time on the imperative aspects
of a subject(s) (Norcini and McKinley, 2007; Piontek, 2008).
The examiners should construct an assessment format to get infor-
mation of students' ability rather than to make them stressful. Again
there are multiple reports that medical students are globally including
Malaysia under stress (Yusoff et al., 2009; Sani et al., 2012; Salam et
al., 2013a; Rahman et al., 2013). Unintentional stress will generate a
negative impact for the development of professional attitude in medical
students (Salam et al., 2012; Salam et al., 2013b; Salam et al., 2013c;
Salam et al., 2013d). To generate high features of educational assess-

Received on March 15, 2014 and accepted on July 8, 2014

Faculty of Medicine and Health Sciences (FPSK), Universiti Sultan Zainal Abidin (UniSZA)
Kampus Kota, Jalan Sultan Mahmud 20400, Kuala Terengganu, Malaysia
Correspondence to: Nordin Bin Simbak
(e-mail: nordinsimbak@unisza.edu.my)

C 2014 Japan Health Sciences University

& Japan International Cultural Exchange Foundation
 Comparison between Multiple True-False and Single Best
Table 1. Correlation between SBA, MTF marks & student's
other performance results of students enrolled in
Session 2011/2012 (n = 30)
Variables r P-value
SBA marks and Continuous assessment marks 0.229 0.223
SBA marks and Clinical marks 0.441 0.015
SBA marks and MEQ marks 0.430 0.018
MTF marks and Continuous assessment marks - 0.119 0.532
MTF marks and Clinical marks - 0.257 0.170
MTF marks and MEQ marks 0.115 0.544
Figure 1. Comparison between SBA and MTF marks of students enrolled in Session 2011/2012 (n = 30)
Figure 2. Comparison between SBA and MTF marks of students enrolled in Session 2012/2013 (n = 52)
ment necessitates both the way of artistically engaging students in fair
and sensible examinations that provide the complete picture of student
competency, and the way of creating test design, item writing and grad-
ing procedures (Piontek, 2008).
An assessment or examination is considered ideal when learner is
precisely examined his or her 'knowledge, comprehension, application,
analysis and evaluation of material pertinent to the subject being exam-
ined (Lumley et al., 2004, Crossley et al., 2002; Begum 2012). Each
and every appraisal method has distinct characteristics and its own pros
and cons. There is no perfect method and therefore, the assortment of
evaluating formats is required to compensate the shortcomings of one
by the advantages of others (Al-Wardy, 2010).
Multiple choice questions (MCQs), also called objective test ques-
tions, are a popular appraising tool in medical and health-related disci-
plines (McCoubrie, 2004). The MCQ answers are easier to mark
because of 'logistical advantage' with 'minimum human interventions'
than other assessment processes by using optical mark reader technolo-
gy (McCoubrie, 2004). Single best answer (SBA) or one best answer
563
Table 2. Correlation between SBA, MTF marks & student's
other performance results of students enrolled in
Session 2012/2013 (n = 52)
Variables r P-value
SBA marks and Continuous assessment marks 0.263 0.060
SBA marks and Clinical marks 0.416 0.002
SBA marks and MEQ marks 0.352 0.011
MTF marks and Continuous assessment marks 0.164 0.246
MTF marks and Clinical marks 0.237 0.091
MTF marks and MEQ marks - 0.138 0.331
(OBA), complex multiple choice (CMC), multiple true-false (MTF),
multiple response (MR) and extended matching items (EMI) stay under
the same heading of MCQs format.
MTF items are similar to SBA items in their appearance. In SBA
format, students have to choose the best answer to each question where-
as the students have to mark each sub-item as true or false in MTF for-
mat (Frisbie and Sweeney, 1982). The chance of selecting the correct
choice is 20% in the SBA question type with a stem and five options (4
distracters) while the odds of guessing the correct answer is obviously
50% to each sub-item of a five-item MTF question form. Hence, there is
strong possibility that examinees can score higher marks in MTF format
than the SBA format (Chandratilake et al., 2011). Therefore, the guess-
ing actually reduces the reliability and validity of assessment done with
MTF format (McCoubrie, 2004; Chandratilake et al., 2011; Nnodim,
1992). Although it is reported that there is strong correlation between
with overall competence and performs with cognitive knowledge with
MCQs especially extended matching question (McCoubrie, 2004).
"Traditional MTF questions are useful where it is important to
564 Simbak N. et al.
assess factual recall of essential knowledge, for example, in manage-
ment protocols for emergency situations. However, this style of question
does not allow facts to be placed in context, or lend it to testing the
application of knowledge and problem solving that is so essential in
clinical practice. This is where SBAs can be invaluable; they allow the
candidate to demonstrate that they 'know how' rather than simply
'know', and this is a fundamental principle of the assessment of clinical
skill" (Brennan, 2009).
The University of Sultan Zainal Abidin (Formerly known as
Universiti Darul Iman Malaysia) was established in late 2005 as part of
a development and expansion of higher education that has been planned
by the Ministry of Higher Education Teaching, based on the Cabinet
decision. Faculty of Health Sciences has been recommended the approv-
al of its establishment on January 29, 2007 by the Ministry of Higher
Education. However, meetings of the Board of Directors dated 18
September 2007 agreed to change the name of Faculty of Health
Sciences to Faculty of Medicine and Health Sciences (FPSK) is consis-
tent with current faculty which will offer more courses related to these
areas. Faculty of Medicine and Health Sciences currently has offers four
diploma programs and two undergraduate programs, namely: Diploma
in Nursing Science; Diploma in Radiography; Diploma in Medical
Laboratory Technology; Diploma in Physiology; Bachelor of Medicine
and Surgery (Honours); Bachelor in Dietetics. Only five batches of stu-
dents are admitted in Bachelor of Medicine and Surgery for MBBS
degree. First batch supposed to pass and become medical doctor by the
third quarter of 2014. MBBS programme of FPSK, UniSZA is very new
course. It is still in embryonic stage. Therefore, curriculum need lot
more amendment to find the best for the medical students. The present
study curiously explored the comparison of students' performance
between true-false and single best answer formats, and the correlation
between these two exam techniques and student's other performance
outcomes.
MATERIALS AND METHODS
At the end of each clinical posting period, all the lecturer-construct-
ed questions were first vetted and selected by the clinical vetting com-
mittee of University Sultan Zainal Abidin (UniSZA) for content accura-
cy. The finalized vetted questions then got the seal of approval from the
Dean of FPSK for the examinations.
This study analyzed the data of five-item MTF surgery questions
and five-option SBA orthopaedic questions of Year 3 students enrolled
in Session (2011/12) and Session (2012/13) in UniSZA.
Both question formats had the total 20 questions completed in 40
minutes and used different marking systems. "Negative no carry-over
marking system" was applied in MTF question type format. With this
system, a correct answer for sub-item gains a mark, a wrong one loses
half mark and no answer gets not at all mark, but the negative marks of
one question were not carried over to other questions. As for SBA for-
mat, a correct choice was awarded five marks and wrong answers or
no-attempt (blank response) resulted in no mark for each question.
The total marks of both these two types of quizzes contributed 20%
to overall 100 marks of end-posting exam for Session 2011/12 students,
but only 10% for Session 2012/13 students. The rest of the total marks
were distributed as follows: 40% from continuous assessment, 10%
from Modified Essay Question (MEQ) exam and 30% or 40% from
clinical marks (Objective structure clinical examination (OSCE) and
long case examination) according to the Session of the enrolment.
The calculated contribution marks of both exam formats were ana-
lyzed separately for each year by using Statistical Package for the Social
Sciences (SPSS) version 20. Paired T test was used to compare the
marks between these two exam formats, and Pearson's correlation coef-
ficient (r) was used to determine the correlation between the marks of
MCQs and student’s other performance results. The level of significance
(α) was set at < 0.05 for this study.
The study had obtained the ethical approval from University Human
Research Ethics (UHREC), UniSZA: UniSZA.N/1/TD2/628-1 (31).
RESULTS
Year 3 Students Enrolled in Session 2011/12
There were altogether 30 students registered in Session 2011/12.
Out of overall 100 marks, the mean marks (SD) of SBA exam was 12.8
(2.0) and of MTF exam was 10.5 (1.5). The study results indicated that
significant difference of the mean marks existed between these two
exam formats (p < 0.001) (Figure 1).
According to the correlation analysis, the SBA marks were not sig-
nificantly correlated with continuous assessment marks (p = 0.229), but
significantly positive correlated with clinical marks (p = 0.015) and with
MEQ marks (p = 0.018). Concerning the MTF marks, these marks were
not significantly correlated with any other parts of exam marks (Table 1).
Year 3 Students Enrolled in Session 2012/13
There were altogether 52 students registered in Session 2012/13.
The mean marks (SD) of SBA exam was 6.9 (1.7) and of MTF exam
was 5.6 (1.0). There was also significant difference of the mean marks
between these two exam formats (p < 0.001) (Figure 2).
As a result of the correlation analysis, the study found out the same
correlation of SBA marks as the previous year. SBA marks were directly
significant correlated with clinical (p = 0.002) and MEQ marks (p =
0.011). Regarding the MTF marks, these marks were also not signifi-
cantly correlated with any other parts of exam marks (Table 2).
DISCUSSION
It is over more than five decades MCQs are used as an examination
tool globally as an instrument of assessment in different fields of educa-
tion which also include medicine. (Ananthakrishnan et al., 1993; Sood
and Adkoli, 2000; Times Higher Education,2001; Ramakrishnan and
Singhal, 2004; Swanson et al., 2005; Kartha, 2005; Shizuka et al., 2006;
Collins, 2006; Swanson et al., 2006). To develop proper MCQs is still
ongoing challenge for teachers and also for medical school. Although
there are multiple guidelines has been develop for constructing MCQs
but it requires through 'knowledge of the content and understanding of
the objectives of the assessment'. It also reported that many times teach-
ers do not follow the guideline properly and resulted in poor construc-
tion of MCQs (Cheung and Bucat, 2002; Haladyna et al., 2002;
Haladyna, 2004a; Haladyna, 2004b; Downing, 2005; Rodriguez et al.,
2005; Walsh et al., 2005a; Walsh et al., 2005b; Tarrant et al., 2006;
Epstein, 2007; Huang et al., 2007; Hutchinson and Wells, 2013). In
UniSZA, MTF type and SBA type of MCQ quizzes are administered in
assessment of clinical subjects in Year 3 MBBS course.
The result of this study found out that there were significantly dif-
ferent mean scores of these two formats in surgery and orthopaedics.
The SBA marks in orthopaedic subject exam were higher than MTF
marks in surgery.
Although it seems that SBA test items are more difficult to choose
correctly because of its less chance of selecting the best choice than that
of MTF test items, students often see that the latter format is trickly
constructed to make the wrong answer (University of Guelph Library,
2013). There were some studies providing a wealth of evidences that
MTF quiz are more difficult to answer correctly than SBA format (Gates
and Hoyer, 1986; Oosterhof and Glasnapp, 1974). Another study
(Pinckney et al., 1993) revealed that students were more likely to mark
false questions true than to mark true questions false.
In SBA quiz, the candidates are sure that one answer is correct and
they can pick the best answer by the deduction process of elimination
(Pinckney et al., 1993). On the contrary, examinees have to attempt
every sub-item meticulously under one question. Consequently, there is
higher chance to commit wrong answers in MTF format than SBA test
items within the same examination period.
Another clarification is that due to having 50% chances to get a cor-
rect answer for each sub-item of MTF test; it seems to encourage exam-
inees to answer the questions by guessing. However, owing to negative
marking system, random guessing can lead to total scores towards zero
irrespective of the number of questions (Hammond et al., 1998).
Some students may have no clear idea about negative marking and
they try to answer almost every question. Subsequently, the students
 Comparison between Multiple True-False and Single Best 565

may fail or get the lower marks in MTF exam than in SBA/OBA exam. from https://www.rcoa.ac.uk/sites/default/files/EXM-SBA-MCQs.pdf (Accessed on 5th
A further study (Gross, 1982) reported that without penalty for guessing March 2013)
MTF marks would exceed SBA test result. Case SM, Swanson DB. (2002). Constructing Written Test Questions For the Basic and
Regarding the correlation between SBA, MTF test formats and Clinical Sciences. Philadelphia, PA: National Board of Medical Examiners; Available
other exam-format results, the findings of this study explicated that only from: http://www.medbev.umontreal.ca/gtea/NBME_MCQ.pdf (Accesed on 10th
SBA test scores were significantly positive correlated with clinical January 2014)
marks and MEQs marks. It implies that those students who did well on Chandratilake M, Davis M, Ponnamperuma G. (2011). Assessment of medical knowledge:
the OBA questions had good marks on student's other performance. the pros and cons of using true/false multiple choice questions. Natl Med J India,
As mentioned earlier, there is no one and only exam style to test 24(4), 225-8. Available from http://www.nmji.in/archives/Volume-24/Issue-4/Medical-
student's competency and clinical skill. Different exam tools are avail- Education.pdf (Accessed on 5th February 2014)
able for different levels of Miller's hierarchical (Al-Wardy, 2010) clini- Cheung D, Bucat R. (2002). How can we construct good multiple-choice items? Paper pre-
cal competence assessment model. MCQ and MEQ test formats perform sented at the Science and Technology Education conference, Hong Kong, Available
evaluation of cognition (knows, knows how). OSCE, long and short from http://www3.fed.cuhk.edu.hk/chemistry/files/constructMC.pdf (Accessed on 5th
case examination format are for assessment of performance (shows March 2014)
how) while portfolio or log book is used to appraisal of performance in Cohen-Schotanus J. (1999). Student assessment and examination rules. Medical Teacher,
practice (does) (Al-Wardy, 2010). 21(3), 1999. Available from http://informahealthcare.com.libproxy.dundee.ac.uk/doi/
Amongst numerous genres of MCQ style of exam, albeit MTF style pdf/10.1080/01421599979626 (Accessed on 3rd March 2014).
exam can test a large proportion of curriculum and is relatively easy to Collins J. (2006). Education techniques for lifelong learning: Writing multiple-choice ques-
be coined, previous studies and literature review revealed that it mostly tions for continuing medical education activities and self-assessment modules.
assesses factual recall of knowledge (declarative knowledge) rather than Radiographics, 26, 543-51. Available from http://pubs.rsna.org/doi/pdf/10.1148/
application of knowledge (procedural knowledge) or problem-solving rg.262055145 (Accessed on 4th March 2014)
ability (Brennan, 2009). Crossley J, Humphris G, Jolly B. (2002). Assessing health professionals. Medical Education
On the contrary, well-constructed SBA style format is more reliable 36, 800-804. Available from http://onlinelibrary.wiley.com.libproxy.dundee.ac.uk/
(Chandratilake et al.,2011; Oosterhof and Glasnapp, 1974; Tan and doi/10.1046/j.1365-2923.2002.01294.x/pdf (Accessed from 4th March 2014)
McAleer, 2008; Green, 1979) and lends itself for the candidates to Downing SM. (2005). The effects of violating standard item writing principles on tests and
demonstrate that they 'know how' rather than simply 'know' (Brennan, students: The consequences of using flawed test items on achievement examinations in
2009). Thus, SBA quiz results are trustworthy in discrimination between m e d i c a l e d u c a t i o n . A d v H e a l t h S c i E d u c , 1 0 , 1 3 3 - 4 3 . Av a i l a b l e f r o m
competent students and poor-performance students (Tan and McAleer, h t t p : / / 1 9 9 . 8 7 . 2 2 5 . 6 3 / f a c u l t y / d e v e l o p m e n t / c o u r s e s R e s o u rc e s / d o c u m e n t s /
2008). SteveDowningHandout2.pdf (Accessed on 5th March 2014)
To answer the test format of factual recall like MTF quiz, students Epstein RM. (2007). Assessment in medical education. N Engl J Med, 356, 387-96.
just have to memorize the isolated facts whereas for application ques- Available from http://www.nejm.org/doi/pdf/10.1056/NEJMra054784 (Accessed on
tions, students have to engage in different learning style in which facts 3rd March 2014)
have to be grasped in a clinical, logical and relevant fashion. In this Frisbie DA, Sweeney DC. (1982). The relative merits of multiple true-false achievement
way, students could integrate and evaluate the medical information for tests. Journal of Educational Measurement, 19(1), 29-35. Available from https://www.
catering human lives. As a result of all good merits of SBA test items, re s e a rc h g a t e . n e t / p u b l i c a t i o n / 2 2 7 9 9 5 0 9 2 _ T H E _ R E L AT I V E _ M E R I T S _ O F _
that assessment tool may have good significant relation with other com- MULTIPLE_TRUEFALSE_ACHIEVEMENT_TESTS (Accessed on 12th January 2014)
petency toolkits. Gajjar S, Sharma R, Kumar P, Rana M. (2014). Item and test analysis to identify quality
multiple choice questions (MCQS) from an assessment of medical students of
Ahmedabad, Gujarat. Indian J Community Med, 39(1), 17-20. Available from: http://
CONCLUSION www.ijcm.org.in/text.asp?2014/39/1/17/126347 (Accessed 3rd February 2014)
Gates FR, Hoyer WD. (1986). Measuring Miscomprehension: a Comparison of Alternate
Formats. Advances in Consumer Research, 13, 143-146. Available from http://www.
The findings of this study offer support for the use of SBA quiz in acrwebsite.org/search/view-conference-proceedings.aspx?Id=6481 (Accessed on 12th
place of MTF test format for examinations. However, the diverse teach- January 2014)
ing techniques, variation of competency of lecturers, dissimilar marking Green K. (1979). Multiple Choice Versus True-False: A Comparison of Reliability and
systems and different question difficulty indexes between these two dif- Validity. The Journal of Experimental Education, 48(1), 42-44. Available from http://
ferent subjects may contribute unequal scores and can lead to bias in www.jstor.org/discover/10.2307/20151314?uid=369334011&uid=2&uid=3&uid=67&
comparison of MTF and OBA assessment formats. Hence, further study uid=363017341&uid=62&sid=21103629193093 (Accessed on 3rd Ferbruary 2014)
of comparing these two test items applying in the same subject matter Gross LJ. (1982). Scoring Multiple True/False Tests Some Considerations. Eval Health
should be conducted to get the more reliable and unbiased conclusion. Prof, 5(4), 459-68. doi:10.1177/016327878200500407 Available from http://ltj.sage-
Finally, more research is advocated in medical education especially in pub.com.libproxy.dundee.ac.uk/content/23/2/198.full.pdf+html (Accessed on 3rd March
assessment should be conducted to achieve the best curriculum for 2014)
MBBS students of UniSZA to achieve the goal 'Vision 2020' of Haladyna TM, Downing SM, Rodriguez MC. (2002). A review of multiple-choice item
Malaysia. writing guidelines for classroom assessment. Applied Meas Educ, 15, 309-34.
Available from http://www.tandfonline.com/doi/pdf/10.1207/S15324818AME1503_5#.
UxbTtcFWHmQ (Accessed on 5th March 2014)
ACKNOWELEDGEMENT Haladyna TM. (2004b). Guidelines for developing MC items. In Haladyna TM (Ed.),
Developing and validating multiple-choice test items 3rd edn. Mahwah, New Jersey:
Lawrence Erlbaum Associates, 97-126.
Authors are much grateful to ethical committee for the approval of Haladyna TM. (2004a). MC formats. In: Haladyna TM (Ed.), Developing and validating
the study. This study obtained no funding. Authors do not have any con- multiple-choice test item. 3rd ed. Mahwah, New Jersey: Lawrence Erlbaum
flict of interest. Associates, 67-96.
Hammond EJ, McIndoe AK, Sansome AJ, Spargo PM. (1998). Multiple-choice examina-
tions: adopting an evidence-based approach to exam technique. Anaesthesia, 53(11),
REFERENCES 1105-8. Available from http://onlinelibrary.wiley.com/doi/10.1046/j.1365-
2044.1998.00583.x/pdf (Accessed on 10th February 2014)
Huang Yi-M, Trevisan M, Storfer A. (2007). The impact of the 'all-of-the-above' option and
Al-Wardy NM. (2010). Assessment Methods in Undergraduate Medical Education. Sultan student ability on multiple choice tests. Int J Scholarship Teach Learn, 1(2), 1-13.
Qaboos Univ Med J, 10(2), 203-09. Available from http://www.ncbi.nlm.nih.gov/pmc/ Article 11. Available from http://digitalcommons.georgiasouthern.edu/ij-sotl/vol1/
articles/PMC3074721/pdf/squmj-10-2-203.pdf (accessed on 11th January 2014) iss2/11 (Accessed on 3rd March 2014)
Ananthakrishnan N, Ananthakrishnan S, Oumachigui A. (1993). MBBS examinations--are Hutchinson D, Wells J. (2013). An Inquiry into the Effectiveness of Student Generated
we asking the right questions? J Postgrad Med, 39, 31-2. Available from: http://www. MCQs as a Method of Assessment to Improve Teaching and Learning. Creative
jpgmonline.com/text.asp 1993/39/1/31/654 (Accessed on 5th March 2014) E d u c a t i o n , 4 ( 7 A 2 ) , 11 7 - 1 2 5 . Av a i l a b l e a t h t t p : / / w w w. s c i r p . o rg / j o u r n a l /
Begum T. (2012). A Guideline on Developing Effective Multiple Choice Questions and PaperDownload.aspxpaperID=34889 (Accessed on 5th 2014).
Construction of Single Best Answer format. J Bangladesh Coll Phys Surg, 230, 159- Kartha CC. (2005). Anguish of a Medico. Curr Sci, 89, 725-6.
1 6 6 . Av a i l a b l e f r o m h t t p : / / w w w. b a n g l a j o l . i n f o / i n d e x . p h p / J B C P S / a r t i c l e / Lumley JSP, Craven JL, editors. (2004). MCQs in Anatomy. 3rd ed. New York: Churchill
view/12466/9081 (Accessed on 4th March 2014) Livingstone, 2-4.
Brennan L. (2009). "Single best answer MCQs." RCoA Bulletin, 2009; 57: 39-41. Available McCoubrie P. (2004). Improving the fairness of multiple-choice questions: a literature
566 Simbak N. et al.
review. Med Teach, 26(8), 709-12. Available from http://informahealthcare.com.lib-
proxy.dundee.ac.uk/doi/pdf/10.1080/01421590400013495 (Accessed 10th February
2014)
Nnodim JO. (1992). Multiple-choice testing in Anatomy. Med Edu, 26(4), 301-9. Avalaible
from http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2923.1992.tb00173.x/pdf
(Accessed on 5th March 2014)
Norcini JJ, McKinley DW. (2007). Assessment methods in medical education. Teaching and
Teacher Education, 23, 239-250. Available from http://www.famecourse.org/pdf/
NorciniandMcKinley.pdf (Accessed on 3rd March 2014)
Oosterhof AC, Glasnapp DR. (1974). Comparative reliablilities and difficulties of the mul-
tiple-choice and true-false formats. Journal of Experimental Education, 42(3), 62-64.
Available from http://psycnet.apa.org/psycinfo/1974-33272-001 (Accessed on 11th
January 2014)
Pinckney BA, Borcher GM, Clemens ET. (1993). Comparative Studies of True/false,
Multiple Choice and Multiple-multiple Choice. NACTA Journal, 21-24. Available
from http://www.nactateachers.org/attachments/article/815/Pinckney_Pages_from_
NACTA_Journal_March_1993-7.pdf (Accessed on 10th January 2014)
Piontek ME. (2008). Best practices for designing and grading exams. Center for Research
on Learning and Teaching. The University of Michigan. 1071 Palmer Commons. 100
Washtenaw Avenue. Ann Arbor, MI 48109-2218. Available from: http://www.crlt.
umich.edu/sites/default/files/resource_files/CRLT_no24.pdf (Accesed on 10th January
2014
Rahman NIA, Ismail S, Seman TNABT, Rosli NFAB, Jusoh SABM, Dali WPEW, Zakirul
Islam MZ, Haque M. (2013). Stress among Preclinical Medical Students of University
Sultan Zainal Abidin. Journal of Applied Pharmaceutical Science, 3(11): 76-081.
Available from http://www.japsonline.com/admin/php/uploads/1110_pdf.pdf (Accessed
on 4th March 2014)
Ramakrishnan K, Singhal K. (2004). Training needs of international medical graduates
seeking residency training: Evaluation of medical training in India and the United
States. Internet J Fam Pract, 39(1), 1. Available from http://ispub.com/IJFP/3/1/8127
(Accessed on 5th March 2014)
Reinert A. Assessment in Medical education: A Premier on Methodology. Perspectives.
Class of 2013. College of Physicians and Surgeons. Columbia University. Available
from http://ps.columbia.edu/education/sites/default/files/student_life/Assessment%20
in%20Medical%20Education%20-%20A%20Primer%20on%20
Methodology%5B1%5D%20copy.pdf (Accessed on 3rd February 2014)
Rodriguez MC. (2005). Three options are optimal for multiple-choice items: A meta-analy-
sis of 80 years of research. Educ Meas: Issues Pract, 24, 3-13. Available from http://
onlinelibrary.wiley.com/doi/10.1111/j.1745-3992.2005.00006.x/pdf (Accessed on 5th
March 2014)
Salam A, Song CO, Mazlan NF, Hassin H, Lee LS, Abdullah MH. (2012). Professionalism
of Future Medical Professionals in Universiti Kebangsaan Malaysia (UKM) Medical
Centre. Int Med J, 19(3), 224-8.
Salam A, Haque M, Helali AM, Islam MZ, Mohsena M, Rahman Z, Yesmin F, Rahman
NIA, , Alattaraqchi AG, Yousuf R. (2013d). Comparative Study on Professionalism of
fourth coming Medical Doctors between Two Private Medical Colleges in Savar,
Bangladesh. International Journal of Pharmacy and Pharmaceutical Sciences, 5(3),
659-665. http://www.ijppsjournal.com/Vol5Suppl3/7441.pdf (Accessed on 4th March
2014).
Salam A, Islam MZ, Helali AM, Yousuf R, Yesmin F, Alattraqchi AG, Rao USM, Haque M.
(2013c). Comparative Study of Professionalism of Future Medical Professionals
among Three Private Medical Colleges of Bangladesh. Asian J Pharm Clin Res, 6(3),
170-9. Available from http://www.ajpcr.com/Vol6Issue3/157.pdf (Accessed on 4 th
March 2014)
Salam A, Yousuf R, Bakar SMA, Haque M. (2013a). Stress among Medical Students in
Malaysia: A Systematic Review of Literatures. Int Med J, 20(6), 649-655.
Salam A, Yousuf R, Islam MZ, Yesmin F, Helali AM, Alattraqchi AG, Rao USM, Haque M.
(2013b). Professionalism of future medical professionals in Universiti Sultan Zainal
Abidin, Malaysia. Bangladesh J Pharmacol, 8(2), 124-30. doi: http://dx.doi.
org/10.3329/bjp.v8i2.14158b Available from http://www.banglajol.info/index.php/
BJP/article/download/14158/10108 (Accessed on 4th March 2014)
Sani M, Mahfouz MS, Bani I, Alsomily AH, Alagi D, Alsomily NY, Madkhaly FM,
Madkhali R, Hakami AAM, Hakami A, Shaqraa Shaabi, Al Ebrahim S, Mashiakhi SH,
Ageel B, Asiri S. (2012). Prevalence of stress among medical students in Jizan
University, Kingdom of Saudi Arabia. Gulf Medical Journal, 1(1), 19-25. Available
from http://gulfmedicaljournal.com/download/Prevalence_of_stress_among_medical_
students_in_Jizan_University_Kingdom_of_Saudi_Arabia.pdf (Accessed on 4th March
2014)
Shizuka T, Takeuchi O, Yashima T, Yoshizawa K. (2006). A comparison of three- and four
option English tests for university entrance selection purposes in Japan. Language
Testing, 23, 35-57. Available from http://ltj.sagepub.com/content/23/1/35.full.pdf+ht-
ml (Accessed on 5th March 2014)
Shumway JM, Harden RM. (2003). AMEE Guide No. 25: The assessment of learning out-
comes for the competent and reflective physician. Medical Teacher, 25(6), 569-58.
Av a i l a b l e f r o m h t t p : / / i n f o r m a h e a l t h c a re . c o m . l i b p ro x y. d u n d e e . a c . u k / d o i /
pdf/10.1080/0142159032000151907 (Accessed on 4th March 2014)
Sood R, Adkoli BV. (2000). Medical education in India: Problems and prospects. J Indian
Acad Clin Med, 1(3), 210-12. Available from http://medind.nic.in/jac/t00/i3/jac-
t00i3p210.pdf (Accessed on 5th March 2014)
Swanson DB, Holtzman KZ, Albee K, Clauser BE. (2006). Psychometric characteristics and
response times for content-parallel extended-matching and one-best-answer items in
relation to number of options. Acad Med, 81, S52-S55. Available from http://journals.
lww.com/academicmedicine/Fulltext/2005/10001/Psychometric_Characteristics_and_
Response_Times.25.aspx (Accessed on 5th March 2014)
Swanson DB, Holtzman KZ, Clauser BE, Sawhill AJ. (2005). Psychometric characteristics
and response times for one-best-answer questions in relation to number and sources of
options. Acad Med, 80, S93-S96. Available from http://journals.lww.com/academic-
medicine/Fulltext/2005/10001/Psychometric_Characteristics_and_Response_
Times.25.aspx (Accessed on 5th March 2014)
Tan LT, McAleer JJA. (2008). The introduction of single best answer questions as a test of
knowledge in the final examination for the fellowship of the Royal College of
Radiologists in Clinical Oncology. Clin Oncol, 20(8), 571-6. Available from http://
www.sciencedirect.com/science/article/pii/S0936655508002616 (Accessed on 2nd
February 2014)
Tarrant M, Knierim A, Hayes SK, Ware J. (2006). The frequency of item writing flaws in
multiple-choice questions used in high stakes nursing assessments. Nurse Educ Today,
26, 662-71. Available from http://www.sciencedirect.com/science/article/pii/
S0260691706001067 (5th March 2014)
Times Higher Education. What is the right choice? At the Heart of the Higher Education
Debate. 2001. Available from http://www.timeshighereducation.co.uk/159581.article
(Accessed on 10th January 2014)
University of Guelph Library. Handouts-Multiple Choice Exams. 2013. Guelph, ON,
Canada N1G 2W1 519-824-4120 x53617 Available from http://www.lib.uoguelph.ca/
assistance/learning_services/handouts/multiple_choice_exams.cfm (Accessed on 10th
January 2014)
Walsh K. (2005a). Advice on writing multiple choice questions (MCQs). BMJ Careers.
Available from http://careers.bmj.com/careers/advice/view-article.html?id=616
(Accessed on 5th March 2014)
Walsh K. (2005b). Answering multiple choice questions. BMJ Careers. Available at http://
careers.bmj.com/careers/advice/view-article.html?id=891 (Accessed on 5 th March
2014)
Yusoff MSB, Rahim AFA, Yaacob MJ. (2010). Prevalence and Sources of Stress among
Universiti Sains Malaysia Medical Students. Malaysian J Med Sci, 17(1), 30-37.
Available from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3216143/pdf/mjms-17-
1-030.pdf (Accessed on 4th March 2014)