Theory-Based Content

Two Types of inferences based on test scores

1. Inferences on an individual’s underlying constructs/attributes

- Are the individual’s underlying characteristics accounted for? (measurement is
done)
2. Inferences on an individual’s future behaviour based on their test score
 Directly linked to purpose of the test
 Does the test predict job performance? (analysing the measurements)

Inferences on Attributes (Measurement)

 What do the test scores provide an indication of?
 Are you able to compare individuals to each other?
 Does the test measure what it is meant to
 Is the test valid? (does variations in test score relate to the hypothesized
outcome – smart people have a higher IQ)
 Evidential basis
 Is there a relationship with other measures of the same factor of interest?

Using Inferences to make decisions

 Making decisions about people based on their test score
 Can it be applied to a situation the determines future success? (Higher IQ = greater
chance of passing a degree)
 Is the test valid? (does variations in test score relate to the hypothesized outcome –
people with high IQ pass their degree)

Predictor VS Criterion
1. Predictor = the test conducted
2. Criterion = the outcome as a result of the predictors used (A criterion is a measure
that could be used to determine the accuracy of a decision, slide 8)

Establishing Criterion-Related Validity

Definitions
 Criterion-related validity = Extent to which inferences (derived from predictor
measures) about criterion are justified (directly linked to test accuracy)
 Invalid test = ineffective & unfair decisions
Establishment
 Correlate test scores with measures of success or outcomes of decisions

Analysing a Correlation Mix to Determine the Best Predictor

 Phase 3, pg.5 (which test/predictor has the strongest correlation with first year
averages?)

Calculation-Based Content
Interpreting confidence intervals obtained from SEM

 X ji ±( Z × [ S EM ] ) , where SEM =S [ X ji ] √ [ 1−r ttX ] ; S [ X ji ] =Std .dev of X

Calculating reliability: Split-half Method

even-numbered items

uneven-numbered items

S^2 = variance
Convergence & Discrimination
1. Convergence = measures of constructs that should and are related to each other

2. Discrimination = measures of constructs that shouldn’t and aren’t related to each other.

 Self-esteem and Locus of control SHOULD NOT converge with each other (which is
shown) but with its own constructs (which is shown) – therefore, they support both
convergence and discrimination = CONSTRUCTS VALIDITY

Utility Analysis
Method where average job performance is known
 Average Increase per Person=Mean Performance of SELECTED group−Mean Performance of E


Average MONETARY increase= Average increase per person׿ appointed applicants × years

 Costs=¿ APPLICANTS × cost per applicant ∈selection procedure

 Utility=Increase−Costs

Method without average job performance

 .
Average increase per person=Std . dev of job performance × standardized score of selected group


Total MONETARY increase=std . dev × standardized score × ¿ selected applicants × years

 Costs=¿ APPLICANTS × cost per applicant ∈selection procedure

 Utility=Increase−Costs
Utility in Testing
1. In reality: not given Sy ∴ Sy = 0.40 x average salary (40% rule)

2. Not given Zys ∴ Zys = Zxs(m) x rxy

Z ‾ y s =Z ‾ xs × r xy

SR = given ∴ match with corresponding m(Zxs)

 Average increase per person=Sy ×m ×r xy

 Total MONETARY increase=Sy × m× r xy × ¿ appointed applicants × years

 Costs=¿ APPLICANTS × cost per applicant ∈selection procedure

 Utility=Increase−Costs

Utility: Brogden-Cronbach-Gleser utility formula

 Return on investment = (n)(t)(r)(SDy)(m) – Cost of testing

Regression Equation: Y’ = bx + m
 b = std.dev(y)/[std.dev(x) x rxy]

 m = b x mean(x) x mean(y)

 Y’ = [(rxy)(Sy/Sx)(X – Xmean)] + Ymean

 Calculating confidence with Standard Error of Estimate
. =S 1−r 2
y√

Sest xy

 Confidence interval=Y ' ±(z × Sest )

 Provides upper and lower bounds

Success & Failure Ratios

Chart Title
FALSE NEGATIVE TRUE POSTIVES
Success = rightfully accepted
= Falsely rejected
I II
7 TRUE NEGATIVE
= rightfully rejected
IV III

FALSE POSITIVES
= falsely accepted
Failure

Reject Accept
(PREDICTED FAILURE) (PREDICTED SUCCESS)

 Overall Accuracy Ratio = (II + IV)/(I + II + III + IV)

 Success Ratio = (II)/(II + III)

Calculated/given

(r) - given Determined with

SR & r