Comparison of Schools in

Inner London Education

Authorities (ILEA):
A Multilevel Analysis Of School
Examination Results

PRESENTED BY: BUTALID CELIZ DALUMPINES PUEBLO TAPICAN TORRE

Introduction
 Objectives of the Study
 Significance of the Study
Objective of this Analysis

To establish a model that predicts the

normalized exam scores of students based on
the resulting significant predictors
Significance Of This Paper Analysis

This paper will act as a supplementary basis in

comparing schools and other educational
institutions in terms of their students’ achievement.
Methodology
Procedure

 The data was analyzed using SAS University Edition

 Model building was executed
 Comparison of Fit Statistics
 Choosing of the Best Model
 Diagnostic Checking
“ Data are examination results from
 4,059 students
 65
schools in Inner London Education
Authorities
”
 Variable Description
Exam Score exam scores were normalized
London Reading (LR) Test Score standardized LR Test Score
Verbal Reasoning (VR) Score student level VR score band at intake
1 = bottom 25% 2 = mid 50% 3 = top 25%
Student ID
Student Gender 0 = boys 1 = girls
School ID
School Gender 1 = mixed 2 = all boys 3 = all girls
School Average school average of intake score
Band of Students Intake Score 1 = bottom 25% 2 = mid 50% 3 = top 25%

Table1. List of Variables & Attributes

Results & Discussion
Null Model (Fully Unconditional Model)
Table 2. Results from the Null Model
Random Effect Coefficient ෢se 𝛽0𝑗 =
𝑉𝑎𝑟 ෢ 𝑢0𝑗 Pr=> 𝜏ZƸ 00
Z Value
𝑉𝑎𝑟
School mean, 𝑢0𝑗 0.1716 0.03363
෢ 𝑟𝑖𝑗5.10
𝑉𝑎𝑟 = 𝜎
ො 2
<.0001
The estimated variability
Level 1 effect, 𝑟𝑖𝑗 0.8478
The
in
0.01897 theestimated
school
44.69 mean<.0001
variability of students
within school

 Proportion of variance in within

65 schools
schools
𝜏Ƹ 00 0.1716
𝜌ො = 0.83
𝜌ො = 2
= = 0.17
𝜏Ƹ 00 + 𝜎ො 0.1716 + 0.8478
Null Model (Fully Unconditional Model)

Table 3. Fit Statistics of the Null Model

-2 Res Log Likelihood 11014.70
AIC (Smaller is Better) 11018.70
AICC (Smaller is Better) 11018.70
BIC (Smaller is Better) 11023.00
 Final Model (Intercept and Slope as Outcome)
 Level – 1 Model

𝑦𝑖𝑗 = β0𝑗 + β1𝑗 𝐿𝑜𝑛𝑑𝑜𝑛 𝑅𝑒𝑎𝑑𝑖𝑛𝑔 𝑇𝑒𝑠𝑡 𝑆𝑐𝑜𝑟𝑒 + β2𝑗 𝐿𝑜𝑛𝑑𝑜𝑛 𝑅𝑒𝑎𝑑𝑖𝑛𝑔 𝑇𝑒𝑠𝑡 𝑆𝑐𝑜𝑟𝑒 ∗ 𝑉𝑒𝑟𝑏𝑎𝑙 𝑅𝑒𝑎𝑠𝑜𝑛𝑖𝑛𝑔 𝑆𝑐𝑜𝑟𝑒 + 𝑟𝑖𝑗

 Level – 2 Model
β0𝑗 = 𝛾00 + 𝛾01 𝑠𝑐ℎ𝑜𝑜𝑙 𝑔𝑒𝑛𝑑𝑒𝑟 + 𝛾02 𝑠𝑐ℎ𝑜𝑜𝑙 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 + 𝛾03 𝑏𝑎𝑛𝑑 𝑜𝑓 𝑠𝑡𝑢𝑑𝑒𝑛𝑡𝑠 𝑖𝑛𝑡𝑎𝑘𝑒 𝑠𝑐𝑜𝑟𝑒 + μ0𝑗
β1𝑗 = 𝛾10 + μ1𝑗
β2𝑗 = 𝛾20

 Combined Model
𝑦𝑖𝑗 = 𝛾00 + 𝛾01 𝑠𝑐ℎ𝑜𝑜𝑙 𝑔𝑒𝑛𝑑𝑒𝑟 + 𝛾02 𝑠𝑐ℎ𝑜𝑜𝑙 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 + 𝛾03 𝑏𝑎𝑛𝑑 𝑜𝑓 𝑠𝑡𝑢𝑑𝑒𝑛𝑡𝑠 𝑖𝑛𝑡𝑎𝑘𝑒 𝑠𝑐𝑜𝑟𝑒 +
𝛾10 𝐿𝑜𝑛𝑑𝑜𝑛 𝑅𝑒𝑎𝑑𝑖𝑛𝑔 𝑇𝑒𝑠𝑡 𝑆𝑐𝑜𝑟𝑒 + 𝛾20 𝐿𝑜𝑛𝑑𝑜𝑛 𝑅𝑒𝑎𝑑𝑖𝑛𝑔 𝑇𝑒𝑠𝑡 𝑆𝑐𝑜𝑟𝑒 ∗ 𝑉𝑒𝑟𝑏𝑎𝑙 𝑅𝑒𝑎𝑠𝑜𝑛𝑖𝑛𝑔 𝑆𝑐𝑜𝑟𝑒 +
μ0𝑗 + μ1𝑗 𝐿𝑜𝑛𝑑𝑜𝑛 𝑅𝑒𝑎𝑑𝑖𝑛𝑔 𝑇𝑒𝑠𝑡 𝑆𝑐𝑜𝑟𝑒 + 𝑟𝑖𝑗
 Final Model (Intercept and Slope as Outcome)
Table 4. Results from the Final Model
Fixed Effect Coefficient
The se
average t value
normalized p-value
There is a 0.4667 increaseexam
in the
Intercept, γ00 -0.2813 0.07069 -3.98 0.0001
London Reading Test, γ10 0.4667
score
average (intercept)
normalized
0.03818 12.23
is -0.2813.
exam score for
<0.0001
Interaction between LRT Score and
every unit
Similarly, theincrease
interaction in LRT score LRT
between
Interaction of LRT and VR Score, γ20
VR Score
holding
Score of
and students’
other VR variables
Score standing
constant
of intake
students’
London Reading Test * Verbal Reasoning (1) -0.1824 0.05607 -3.25 0.0019
London Reading Test * Verbal Reasoning (2) -0.09262 that falls at the
standing
0.04382 intake bottom
-2.11 that 25%
falls
0.0389 is lower
at the
London Reading Test * Verbal Reasoning (3) 0 .
middlecompared .
50% isnormalized
lower to the top
.
compared 25%. to the
The average exam score
School Gender Differentiation, γ01 top 25%.from mixed
School Gender (1) -0.2584
of students coming
0.07344 -3.52 0.0009
School Gender (2) -0.1395
There
gender
0.1019
is a 0.3504
school-1.37 increase
is lower compared
0.1765
in theto
School Gender (3) 0 average
students
. normalized
coming fromexam
. students girls score for
school.
. belonging
The proportion of
School Average of Intake Score, γ02 0.3504 every unit3.39
0.1032 increase 0.0012in school
Theto proportion
bottom 25% of students
or have belonging
lowest
Band of Students Intake Score Differentiation, γ03 average intake score holding other
to middle 50%
performance or have
in terms average
of normalized
Band of Students Intake Score (1) 0.7748 0.05348variables14.49 constant <0.0001
0.3702
performance
exam score isin9.08
0.04079
termscompared
higher of <0.0001
normalized to
Band of Students Intake Score (2)
Band of Students Intake Score (3) 0 exam
the . score is of
proportion .higher compared
students . belonging to
Random Effect Coefficientthe proportion
se toof Z students
top 25% p-value
value belonging
Level-1 effect, rij 0.5239 0.01183 to44.28 top 25% <0.0001
Level-2 effect on the intercept, u0j 0.06526 0.01417 4.61 <0.0001
Level-2 effect on London Reading Test, u1j 0.01247 0.00426 2.93 0.0017
 Final Model (Conditional)

 Proportion of variance in 65 schools

𝜏Ƹ 00 0.06526
𝜌ො = = = 0.111
𝜏Ƹ 00 + 𝜎ො 2 0.06526 + 0.5239

 Proportion of variance explained at Level 1 when LRT score is set as a predictor of exam
scores
𝜎ො 2 𝑛𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 − 𝜎ො 2 𝑓𝑖𝑛𝑎𝑙 𝑚𝑜𝑑𝑒𝑙
𝑃𝑟𝑜𝑝𝑜𝑟𝑡𝑖𝑜𝑛 𝑜𝑓 𝑣𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑒𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑎𝑡 𝑙𝑒𝑣𝑒𝑙 1 =
𝜎ො 2 𝑛𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙
0.8478−0.5239
=
0.8478
= 0.382
Final Model

Table 5. Fit Statistics of the Final Model

-2 Res Log Likelihood 9109.0
AIC (Smaller is Better) 9117.0
AICC (Smaller is Better) 9117.0
BIC (Smaller is Better) 9125.7
 Level 1 • Homogeneity of Variance
• Normality

• Homogeneity of Variance
Level 2 • Normality
• Influential Observations

Diagnostic Checking
Diagnostic Checking

 Table 6. Homogeneity of Variance at Level 1

_TYPE_ _FREQ_ h p_value df Heterogeneity of
0 65 138.633 .000000198 64 variance is observed
at Level 1

 Table 7. Test of Normality at Level 1

Test Statistic p Value
Errors at Level 1
Kolmogorov-Smirnov 0.014097 0.0487 are not normal
 Figure 1. Distribution and Probability Plot of Residuals

Test of Normality
at Level 1

 Histogram
 Normal Quantiles
Residuals do not
totally depart from
being symmetric
Diagnostic Checking

 Table 8. Homogeneity of Variance at Level 2

Method Num DF Den DF F Value Pr > F Homogeneity
of variance is
Folded F 54 74 1.22 0.4183 observed at
Level 2

 Table 9. Test of Normality at Level 2

Test Statistic p Value

Shapiro-Wilk 0.962426 0.0012
Errors at Level 2
Kolmogorov-Smirnov 0.086804 <0.0174 are not normal
 Figure 2. Distribution and
Distribution Probability
and Probability Plot of Residuals
Plot for Estimate

0.4500

0.3000

0.1500

Estimate
0

-0.1500

Test of Normality -0.3000

-0.4500

at Level 2 -0.6000

0 20 40 60

 Histogram Count

 Normal Quantiles 0.4

Residuals do not
0.2
totally depart from
being symmetric
Estimate
0.0

-0.2

-0.4

-0.6

-3 -2 -1 0 1 2 3

Normal Quantiles
Diagnostic Checking

 Figure 3. Influential Observations Cook’s D values lie

between -1 to 1
which indicates the
absence of influential
observations
 Table 10. Robust VS Model-Based Standard Errors

Coefficient HLM Coefficient HLM Model-Based se HLM Robust se

Intercept, γ00 0.4897 0.08700 0.1030
London Reading Test, γ10 0.1864 0.06092 0.06015
London Reading Test * Verbal Reasoning, γ20 0.09395 0.02739 0.02928
School Gender, γ01 0.1290 0.03606 0.03529
School Average of Intake Score, γ02 0.3513 0.1018 0.1114
Band of Students Intake Score, γ03 -0.3917 0.02595 0.03077
Summary and Conclusions

 The data contained 4,059 observations and 10

variables

 Final Model includes:

 Level-1 predictors: LRT and interaction between LRT and VR

 Level-2 predictors: school gender, school average of intake

score and band of students intake score
 Summary and Conclusions

 For both level 1 and level 2:

 intercept and LRT were treated as random

 LRT, School average and band of students’ intake score had a

positive direction or relationship with normalized exam score

 school gender and all significant interactions had an inverse

effect or relationship with normalized exam score
Summary and Conclusions

 Diagnostic Checking:
 Homogeneity assumption was satisfied at level 2

 No influential observations

 Test of Normality was violated for both levels, however a symmetric

distribution appears in the plot of the residuals

 Generally, we can say that the model fits the data well
References

 Goldstein, H., Rasbash, J., et al (1993). A multilevel analysis of

school examination results. Oxford Review of Education, 19:
425-433
 University of Bristol. Centre for Multilevel Modelling. © 2002-2017
http://www.bristol.ac.uk/cmm/learning/mmsoftware/data-
rev.html#exam
THE END