Test Analysis Report Slides

Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Test Analysis
Report
Detailed information regarding the statistical
performance of an test
Slide 2
Overview
Detailed information regarding the statistical
performance of an test
Report Sections
First section presents information in a tabular format
Stats including mean, standard deviation and reliability
Second section: Distribution of tests scores using a
frequency distribution table
lists all scores obtained by examinees
histogram graphically displays the range of scores
Slide 3
Test Analysis Report
First section presents information in a tabular format
Stats including mean, standard deviation and reliability
Slide 4
Test Analysis Report
Second section: Distribution of tests scores using a
frequency distribution table
lists all scores obtained by examinees
histogram graphically displays the range of scores
Slide 5
Report Management and Distribution
Distribute via Browser, PDF, Word, Excel, Email,
or Print
My Reports
Slide 6
Internal Consistency
Questionmark uses Cronbachs Alpha Test
Reliability
Reason its the gold standard:
Highly documented and very well researched (60+
years)
Can be used with a wide variety of question types
(partial credit and right/wrong)
Most widely used method in high and medium stakes
Slide 7
Cronbachs Alpha
Internal consistency test
reliability: A measure of how
well the assessment questions
relate to one another to
measure the same topic
Crucial information for legal
defensibility is the test
reliably measuring what is was
designed to measure?
Speaks to the quality of
assessment questions: Are they
of high statistical quality?
) (
1
2
2 2
T
p T
S
S S
k
k

Slide 8
Other Measures
Skew
Mode
Kurtosis
Mean (arithmetic)
Median
Mode
Standard Deviation
Variance
Standard Error of measurement
Standard Error of the Mean
Slide 9
Skew
A measure of the symmetry of the distribution
of scores
are scores are pushed or skewed to one side or
the other?
Ranges from about -2 to +2
0
2
4
6
8
10
12
14
16
18
20
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Part icipant scores
Positive skew
0
2
4
6
8
10
12
14
16
18
20
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant scores
Negative skew
0
2
4
6
8
10
12
14
16
18
20
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 20% 40% 60% 80% 100%
Participant scores
Normal distribution (no skew)
Slide 10
Kurtosis
A measure of the symmetry of a distribution of
scores
How peaked/pointed versus flat are the distribution of
scores? What is happening at the tails?
Normal range from about -3 to +3
0
2
4
6
8
10
12
14
16
18
20
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant scores
Normal distribution (zero kurtosis: mesokurtic)
0
2
4
6
8
10
12
14
16
18
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant scores
Negative kurtosis (flat: platykurtic)
0
2
4
6
8
10
12
14
16
18
20
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant scores
Positive kurtosis (pointed: leptokurtic)
( )
) 3 )( 2 (
) 1 ( 3
) 3 )( 2 )( 1 (
) 1 (
2
4

n n
n
z
n n n
n n
Slide 11
Mean (arithmetic)
The most commonly used measure of central tendency
(refers to the middle of a distribution of scores)
Range of values depends on scores
Slide 12
Median
Another measure of central tendency, less sensitive than
the mean to outliers
Where 50% of participants obtained higher scores and
50% of participants obtained lower scores
Slide 13
Mode
A third measure of central tendency, used a great deal in
survey analysis
The most common score in a distribution of scores
34% 43% 56% 56% 56% 63% 67% 76% 88%
Mode = 56%
Slide 14
Standard deviation
The spread or variation of scores between participants
Are the scores spread out (e.g., 0 to 100%) or clustered
together (e.g., all scores between 55 and 62%)
Ricks test
score = 75%
Sallys test
score = 83%
Marks test
score = 53%
Ellas test
score = 91%
Standard deviation = 16.36%
Slide 15
Variance
Another measure of variation
The first step in calculating a standard
deviation: The standard deviation is the
square root of the variance
Used in some advanced calculations (e.g.,
Analysis of variance: ANOVA, Multiple analysis
of variance: MANOVA)
Slide 16
Standard Error of
measurement
Theoretical test score = 66.4%
Product knowledge test
Theoretical
standard
deviation =
1.26%
1.26% of
error
surrounding
Ricks observed
score
Ricks observed
score = 66.1%
observed test score = theoretical true score + error
Slide 17
Standard Error of the Mean
0
5
10
15
20
25
30
35
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Part icipant scores
Sample (153 participants)
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
#

o
f

p
a
r
t
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Parti ci pant scores
Population (87,000 participants)
Sample mean = 56.78%
Sample standard deviation = 15.21%
Standard error of mean = 1.23%
___________________________
within plus or minus 1 standard error
of the mean value, 68 times out of 100,
the true population mean will reside
Typically the population information
is not known, but if you could see
the information
Population true mean = 57. 81%
More information:
www.questionmark.com

Test Analysis Report Slides

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Test Analysis Report Slides

Uploaded by

Copyright:

Available Formats

Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.

You might also like