You are on page 1of 11

MiMa Function Tutorial (version 1.

4) 1

MiMa: An S-Plus/R Function to ﬁt Meta-Analytic Mixed-,

Random-, and Fixed-Eﬀects Models
Wolfgang Viechtbauer
University of Maastricht
http://www.wvbauer.com/

March 8, 2006

Contents
1 Meta-Analytic Models 2
1.1 Mixed-Eﬀects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Random-Eﬀects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Fixed-Eﬀects with Moderators Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Fixed-Eﬀects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

4 Example 4

5 Predicted/Estimated Eﬀect Sizes 7

6 Categorical Moderators 8

11 How to Cite the Function (APA style) 11

12 References 11
MiMa Function Tutorial (version 1.4) 2

1 Meta-Analytic Models

1.1 Mixed-Eﬀects Model

Given are i = 1, . . . , k independent eﬀect size estimates, each estimating a corresponding eﬀect
size. This can be expressed algebraically by

Yi = θi + i , (1)

where i is the sampling error for the ith study. The sampling errors are assumed to be nor-
mally distributed with mean zero and variance vi . Now assume that p moderator variables are
inﬂuencing the eﬀect sizes in a linear fashion. Let Xij denote the value of the j th moderator
variable for the ith eﬀect size. We then write

θi = β0 + β1 Xi1 + . . . + βp Xip + ui , (2)

where ui is assumed to follow a normal distribution with mean zero and variance τ 2 . The
average eﬀect size is given by β0 +β1 Xi1 +. . .+βp Xip , while the study-speciﬁc eﬀect size is given
by (2). Therefore, τ 2 denotes the amount of residual heterogeneity, that is, the amount of excess
or unexplained variability in the eﬀect sizes after accounting for the amount of heterogeneity
we would expect based on the moderator variables that are included in the model. Combining
Equations 1 and 2 yields the mixed-eﬀects model, which is given by

Yi = β0 + β1 Xi1 + . . . + βp Xip + ui + i . (3)

Therefore, βj denotes by how much the average eﬀect size changes for a one unit increase in
the corresponding moderator variable when holding all other moderator values constant, while
β0 denotes the average eﬀect size when all moderator variables are equal to zero.

1.2 Random-Eﬀects Model

When no moderators are included in the model, then the mixed-eﬀects model simpliﬁes to

Yi = µ + ui + i , (4)

where β0 has been replaced with µ. According to the random-eﬀects model, all of the hetero-
geneity among the eﬀect sizes is random and cannot be accounted for by moderators. There-
fore, τ 2 in the random-eﬀects model simply denotes the amount of total heterogeneity, that is,
the amount of variability in the eﬀect sizes.

1.3 Fixed-Eﬀects with Moderators Model

When moderators are included in the model and the moderators can account for all of the
heterogeneity among the eﬀect sizes, then the ﬁxed-eﬀects with moderators model applies,
which is given by
Yi = β0 + β1 Xi1 + . . . + βp Xip + i . (5)
Therefore, this model is again just a special case of the mixed-eﬀects model, where τ 2 = 0.
MiMa Function Tutorial (version 1.4) 3

1.4 Fixed-Eﬀects Model

When all of the eﬀect sizes are homogeneous (i.e., no moderators are operating and there is no
random heterogeneity among the eﬀect sizes), then θ1 = θ2 = . . . = θk = θ and the appropriate
model is given by
Yi = θ + i , (6)
using θ in place of β0 .

2 Hypothesis Tests in the Mixed-Eﬀects Model Context

Once estimates of β0 through βp and τ 2 have been obtained (i.e., once the model has been
ﬁtted), we can conduct a variety of hypothesis tests:

1. We can test whether the amount of residual heterogeneity (τ 2 ) is signiﬁcantly diﬀerent

from zero (i.e., H0 : τ 2 = 0). For this, we use the QE statistic, which follows a chi-square
distribution under the null hypothesis with degrees of freedom equal to k − p − 1.

2. We can also test whether any of the β1 , . . . , βp parameters in the model are signiﬁcantly
diﬀerent from zero, or in other words, whether at least one moderator is inﬂuencing the
eﬀect sizes (i.e., H0 : β1 = β2 = . . . = βp = 0). For this, we use the QME statistic, which is
distributed chi-squared with p degrees of freedom (i.e., equal to the number of parameters
tested) under the null hypothesis.

3. We can also test individual parameters, by dividing the parameter estimate by its stan-
dard error. This ratio follows a standard normal distribution under the null hypothesis.
Conﬁdence intervals for each parameter are also easily obtained by inverting these tests.

Notes: (1) The sampling variances (i.e., the vi values) are usually not exactly known. In-
stead, the vi values are replaced with corresponding estimates (v̂i ). Therefore, the distribu-
tions of the test statistics above are only exact when the within-study sample sizes are large
(i.e., when the error in the sampling variance estimates is negligible). (2) Moreover, when ﬁt-
ting a mixed/random-eﬀects model, then τ 2 must be estimated (τ̂ 2 ). The estimate is then
simply treated as a known constant. Therefore, the QME statistic and the individual param-
eter tests assume that k is suﬃciently large, such that τ 2 is estimated with high precision
and the error in the estimate is negligible. (3) Finally, most eﬀect size measures are not ex-
actly normally distributed, as assumed under the meta-analytic models. However, the normal
distribution approximation usually becomes more accurate as the within-study sample sizes
increase. Therefore, large within-study sample sizes are needed to be certain that the tests
perform as desired (speciﬁcally, that the Type I error rate is controlled). Reﬁned moderator
tests that provide better control of the Type I error rate when these assumptions are violated
have been developed and will be included in the MiMa function later on.
MiMa Function Tutorial (version 1.4) 4

3 MiMa Function for Fitting the Meta-Analytic Models

The S-PLUS/R function “mima” can be used for ﬁtting the various meta-analytic models.
Usage:

mima(yi, vi, mods, method="REML", threshold=0.00001, maxiter=100, alpha=0.05,

verbose="no", digits=4, fe="no", out="no")

Required Arguments:

yi -- a vector of effect size estimates

vi -- a vector of sampling variances
mods -- a vector/matrix of moderator values

Optional Arguments:

method -- method for estimating the amount of (residual) heterogeneity;

possible methods are: "REML", "ML", "HE", "DL", "SH", or "EB";
the default is "REML" (see below for more details)
threshold -- precision of the Fisher scoring algorithm when using either
ML, REML, or EB estimation; the default is 0.00001
maxiter -- maximum number of iterations of the algorithm when using either
ML, REML, or EB estimation; the default is 100
alpha -- to set the width of the confidence interval for individual
parameters (default is .05, which yields a 95% CI)
verbose -- set to "yes" to obtain information on the evolution of the
iterative algorithm when using either ML, REML, or EB estimation
digits -- number of decimals for rounding of results; the default is 4
fe -- set to "yes" to force fitting a fixed-effects model
out -- see below for explanation

Several methods for estimating the amount of (residual) heterogeneity are available. The de-
fault is restricted maximum-likelihood (REML) estimation and this is the recommended method
(Viechtbauer, 2005). Regular maximum-likelihood (ML) estimation is also available, but it is
not recommended, because the estimate of τ 2 tends to be too small (negatively biased). Two
method of moment estimators are also available (HE and DL, respectively); the ﬁrst is an ex-
tension of the estimator suggested by Hedges (1983, 1989) and the other an extension of the
estimator suggested by DerSimonian and Laird (1986). More details on these two estimators
can be found in Raudenbush (1994). Finally, the estimator suggested by Sidik and Jonkman
(2005a, 2005b) and the empirical Bayes estimator (Berkey, Hoaglin, Mosteller, & Colditz, 1995;
Morris, 1983) are also available (SH and EB, respectively). The ML, REML, and EB estimation
methods make use of the Fisher scoring algorithm, which is robust to poor starting values and
usually converges quickly (Harville, 1977; Jennrich & Sampson, 1976).

4 Example
Table 1 provides the results for k = 16 studies examining the eﬀectiveness of massage therapy
for relieving state anxiety. Listed are the sample sizes of each study (nEi and nC
i ), the eﬀect size
MiMa Function Tutorial (version 1.4) 5

estimate (Yi ), the estimated sampling variance (v̂i ), the minutes per session of massage therapy
provided in each study, whether a fully trained massage therapist or a layperson provided the
therapy, and the mean age of the sample. The data were adopted from Moyer, Rounds, and
Hannum (2004), leaving out three studies with missing data and two studies where the duration
of massage therapy provided was less than 10 minutes.

Table 1
Results for 16 Studies on the Eﬀectiveness of Massage Therapy for Reducing State Anxiety
Sample Sizes Eﬀect Size Sampling Minutes/ Trained Mean
Study nEi nCi Estimate (Yi ) Variance (v̂i ) Session Therapist Age
1 30 30 0.444 0.068 30 0 28
2 46 39 −0.495 0.049 10 0 42
3 15 15 0.195 0.134 20 1 31
4 10 10 0.546 0.207 40 1 39
5 12 12 0.840 0.181 20 1 17
6 10 10 0.105 0.200 30 1 51
7 26 24 0.472 0.082 15 1 26
8 18 14 −0.205 0.128 10 0 64
9 12 12 1.284 0.201 45 1 48
10 12 12 0.068 0.167 30 1 40
11 15 15 0.234 0.134 30 1 52
12 12 12 0.811 0.180 30 1 33
13 15 15 0.204 0.134 30 1 20
14 18 18 1.271 0.134 60 1 27
15 15 15 1.090 0.153 45 1 52
16 43 35 −0.059 0.052 10 1 61
Note. Adopted from Moyer, Rounds, and Hannum (2004). Minutes/Session = minutes of therapy provided per
session; Trained Therapist: 0 for a layperson providing the therapy and 1 for a trained therapist; Mean Age =
mean age of the sample.

The data are available in the S-PLUS dataset ﬁle “dat.sdd”. The names of the variables are:
n1, n2, yi, vi, minutes, trained, and meanage. In order to use the mima function, we must
ﬁrst create a matrix with the moderator values that we want to include in the model. For this,
load the dataset, open up the command window (click on “Window” in the menu bar and choose
“Command Window”), and type:

> attach(dat)
> mods <- cbind(minutes, trained, meanage)

The attach command makes the variables in the dataset available (technically, it adds the
dataset to the S-PLUS search path). The second command creates a matrix, called “mods”,
where the columns are composed of the minutes, trained, and meanage variables.
Next, load the mima function by double-clicking on the ﬁle called “mima.ssc” (if asked
whether to overwrite the existing script, then this means that the function is already loaded,
but to be certain that the newest version of the function is available, select ok). Once the script
MiMa Function Tutorial (version 1.4) 6

is loaded, run it (i.e., make it available) by hitting F10 (you can then close the script window).
Finally, to ﬁt the mixed-eﬀects model to the massage therapy data, simply type:

Test for (Residual) Heterogeneity:

QE = 9.1278
df = 12
p-value = 0.692

Parameter Estimates:

[,1]
intrcpt -0.2971
minutes 0.0250
trained 0.3013
meanage -0.0059

intrcpt minutes trained meanage

intrcpt 0.1218 -0.0013 -0.0112 -0.0018
minutes -0.0013 0.0000 -0.0005 0.0000
trained -0.0112 -0.0005 0.0382 -0.0001
meanage -0.0018 0.0000 -0.0001 0.0000

QME = 28.9176
df = 3
p-value = 0

estimate SE zval pval CI_L CI_U

intrcpt -0.2971 0.3490 -0.8513 0.3946 -0.9811 0.3869
minutes 0.0250 0.0067 3.7172 0.0002 0.0118 0.0381
trained 0.3013 0.1954 1.5422 0.1230 -0.0816 0.6842
meanage -0.0059 0.0063 -0.9488 0.3427 -0.0182 0.0063
MiMa Function Tutorial (version 1.4) 7

The output shows that no residual heterogeneity remains when all three moderators are
included in the model (the estimate of residual heterogeneity is zero). The test for residual
heterogeneity is also far from signiﬁcant (QE = 9.13, df = 12, p = .692). The QME statistic is
highly signiﬁcant (QME = 28.92, df = 3, p < .0001), indicating that at least one moderator vari-
able is related to the eﬀect sizes. The individual parameter tests suggest that it is the minutes
moderator that is signiﬁcant (95% conﬁdence interval bounds for this parameter: (0.01, 0.04),
z = 3.72, p = .0002).
One can also obtain a conﬁdence interval for the amount of (residual) heterogeneity in the
model with the tau2cime.r function available also on the author’s website. Simply open that
script in S-Plus and source it with F10. Then type:

> tau2cime(yi, vi, mods)

95% CI: (0, 0.144)

which shows that the bounds for a 95% conﬁdence interval for τ 2 are given by (0, 0.144). For

5 Predicted/Estimated Eﬀect Sizes

Once a model has been ﬁtted, it is easy to obtain predicted/estimated eﬀect sizes from the
output. Recall that the output for the mixed-eﬀects model ﬁtted earlier included the following
parameter estimates:

[,1]
intercept -0.2971
minutes 0.0250
trained 0.3013
meanage -0.0059

Therefore, the predicted/estimated eﬀect size for 30 minutes of therapy provided by a fully
trained therapist to a group with an average age of 40 is:

−0.2971 + 0.0250(30) + 0.3013(1) − 0.0059(40) ≈ 0.52.

We can use S-PLUS/R to carry out these calculations. We ﬁrst ﬁt the model again, using the
out="yes" option. With this argument set to yes, the estimate of residual heterogeneity, the
parameter estimates, and the variance-covariance matrix of the parameter estimates are given
in list form for further processing. Speciﬁcally, if we write:

> res <- mima(yi, vi, mods, out="yes")

then the output is put into the object “res”. Next, we select some moderator values and then
multiply the moderator values with the parameter estimates to obtain a predicted eﬀect size.
For example, for minutes = 30, trained = 1, and meanage = 40, we use:

> mvals <- cbind( c(1, 30, 1, 40) )

> t(mvals) %*% res\$b
[,1]
[1,] 0.5157254
MiMa Function Tutorial (version 1.4) 8

The “1” in front of the “30” on the line starting with mvals is needed, because of the intercept
parameter in the model. The variance/standard error of this estimate can also be obtained. To
obtain the variance, we type:

> t(mvals) %*% res\$vb %*% mvals

[,1]
[1,] 0.0103143

and for the standard error (the square root of the variance), we use:

> sqrt( t(mvals) %*% res\$vb %*% mvals )

[,1]
[1,] 0.1015593

An approximate 95% conﬁdence interval is then given by adding and subtracting 1.96 times the
standard error from the predicted eﬀect size:

0.5157 ± 1.96(0.1016) = (0.32, 0.72).

6 Categorical Moderators
Categorical moderators can be included in the model, in the same way that linear models can
include appropriately coded categorical independent variables (in the example above, the vari-
able specifying whether a layperson or a trained therapist provided the treatment was dummy
coded). See, for example, Neter, Kutner, Nachtsheim, and Wasserman (1996) for more details
on how to code categorical independent variables in the context of linear models.

7 Fitting a Random-Eﬀects Model

One can also ﬁt the random-eﬀects model with the mima function. The random-eﬀects model
is simply a special case of the mixed-eﬀects model, which does not include any moderator
variables. For this, create an empty moderator matrix and then run the mima function as
follows:

> mods <- c()

> mima(yi, vi, mods)

QE = 38.0456 df = 15 p-value = 0.0009

MiMa Function Tutorial (version 1.4) 9

Parameter Estimates:

[,1]
intrcpt 0.3772

intrcpt
intrcpt 0.0183

Omnibus Test of all Moderators:

QME = NA df = 0 p-value = NA

estimate SE zval pval CI_L CI_U

intrcpt 0.3772 0.1354 2.7855 0.0053 0.1118 0.6425

Note that the estimate of residual heterogeneity now indicates the amount of (total) het-
erogeneity in the eﬀect size estimates over and beyond that which we would expect based on
sampling variability alone. The estimate is quite large (τ̂ 2 = 0.1648). The test for residual
heterogeneity is now equivalent for the usual Q-test for heterogeneity and also reveals that
the eﬀect sizes are heterogeneous (QE = 38.05, df = 15, p = .0009). The QME statistic is
not needed here (since there are no moderators in the model), but we can test whether the
average eﬀect size (i.e., µ) is signiﬁcantly diﬀerent from zero, which turns out to be the case
(z = 2.79, p = .0053). The bounds for a 95% conﬁdence interval for µ are (0.11, 0.64). However,
given that at least one moderator appears to be inﬂuencing the eﬀect sizes, the results from
the random-eﬀects model are incomplete at best. Again, a conﬁdence interval for τ 2 can also
be obtained with the tau2cime.r function available on the author’s website:

> tau2cime(yi, vi, mods)

95% CI: (0.035, 0.524)

which shows that the bounds for a 95% conﬁdence interval for τ 2 are given by (0.035, 0.524).

8 Negative Heterogeneity Estimates

Except when using the Sidik and Jonkman estimator (SH), it is possible that the heterogeneity
estimate turns out to be negative when using any of the estimation methods. However, negative
estimates of τ 2 are outside of the parameter space. For the method of moments estimators,
negative estimates are therefore simply truncated to zero. For ML, REML, and EB estimation,
the Fisher scoring algorithm used in the mima function was modiﬁed using step halving to
guarantee a non-negative estimate (see Harville, 1977, and Jennrich & Sampson, 1976, for more
details on ML and REML estimation and penalty techniques).
MiMa Function Tutorial (version 1.4) 10

9 Possible Problems when Fitting the Mixed-Eﬀects Model

Two problems can occur when ﬁtting the mixed-eﬀects model with the mima function:

1. First of all, there is a chance that the Fisher scoring algorithm used for ML, REML, and
EB estimation does not converge. One can try increasing “maxiter” (the default is 100
iterations), but there are cases where the algorithm will cycle between several estimates
and never converge, no matter how many iterations are run (by setting verbose = "yes",
one can see how the iterative algorithm is progressing). In this case, one can try a diﬀerent
estimation method thatn the default, which is REML estimation.

2. Another problem that can occur is the following error message when running the function:
“Problem in solve.qr(a): apparently singular matrix”. This problem can occur for at least
three reasons.

(a) When there are linear relationships between the moderator variables (i.e., the values
of one moderator variable can be given as a linear function of the other moderator
variables), then this error will occur. In this case, one must remove moderators
variables from the model until such linear relationships are no longer present.
(b) When some of the moderators are highly correlated, then this error can also occur
(a correlation of exactly ±1 indicates a linear relationship between two moderators
and we are back to the ﬁrst case). Highly correlated moderators could be argued to
be redundant anyway and therefore one should probably remove one of them.
(c) The error can also occur even when the moderators are not highly correlated, but
the values of the moderators are scaled very diﬀerently (e.g., the values of one mod-
erator are between 0 and 1, while the values of another moderator are between 1
and 10,000). Such diﬀerential scaling can cause numerical problems with the matrix
inversions used as part of the algorithm (right now, “brute force inversion” is used
in the iterative procedures – in the future, this may be replaced with more elegant
methods). The solution here is to make sure that the scaling is not too dissimilar (it
does not have to be exactly the same). One could try dividing the values of a moder-
ator with very large values by 10 or 100 and re-running the model. For example, to
re-scale the meanage moderator, use:
> meanage <- meanage/10
> mods <- cbind(minutes, trained, meanage)
> mima(yi, vi, mods)
The re-scaling will change the parameter estimate and variance for this moderator,
but the z-value and p-value should remain unchanged. The other parameter esti-
mates should not be aﬀected by the re-scaling.

10 Fitting Fixed-Eﬀects Models

One can also force the mima function to ﬁt a ﬁxed-eﬀects model (with or without moderators).
To do this, set fe="yes". The estimate of (residual) heterogeneity will automatically be set
MiMa Function Tutorial (version 1.4) 11

to zero. This is not recommended, because the resulting moderator tests can have severely
inﬂated Type I error rates (i.e., moderators that are not actually signiﬁcant may appear to be
signiﬁcant). This option is available only for demonstration purposes and for conducting a
sensitivity analysis, but should not be used to actually test the moderators.

11 How to Cite the Function (APA style)

The function and this manual/tutorial should be cited (APA style) as follows:

Viechtbauer, W. (2006). MiMa: An S-Plus/R function to ﬁt meta-analytic mixed-, random-, and

com/.

12 References
Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995). A random-eﬀects regression
model for meta-analysis. Statistics in Medicine, 14, 395-411.
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials,
7, 177–188.
Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation
and to related problems. Journal of the American Statistical Association, 72, 320–338.
Hedges, L. V. (1983). A random eﬀects model for eﬀect sizes. Psychological Bulletin, 93,
388–395.
Hedges, L. V. (1989). An unbiased correction for sampling error in validity generalization
studies. Journal of Applied Psychology, 74, 469–477.
Jennrich, R. I., & Sampson, P. F. (1976). Newton-Raphson and related algorithms for maxi-
mum likelihood variance component estimation. Technometrics, 18, 11–17.
Morris, C. N. (1983). Parametric Empirical Bayes inference: Theory and applications (with
discussion). Journal of the American Statistical Association, 78, 47-65.
Moyer, C. A., Rounds, J., & Hannum, J. W. (2004). A meta-analysis of massage therapy
research. Psychological Bulletin, 130, 3–18.
Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). Applied linear statistical
models (4th. ed.). Irwin: Chicago.
Raudenbush, S. W. (1994). Random eﬀects models. In H. M. Cooper & L. V. Hedges (Eds.),
The handbook of research synthesis (pp. 301–321). New York: Russell Sage.
Sidik, K., & Jonkman, J. N. (2005a). A note on variance estimation in random eﬀects meta-
regression. Journal of Biopharmaceutical Statistics, 15, 823-838.
Sidik, K., & Jonkman, J. N. (2005b). Simple heterogeneity variance estimation for meta-
analysis. Journal of the Royal Statistical Society, Series C, 54, 367-384.
Viechtbauer, W. (2005). Bias and eﬃciency of meta-analytic variance estimators in the
random-eﬀects model. Journal of Educational and Behavioral Statistics, 30, 261-293