Professional Documents
Culture Documents
Antonius J. van Rooij, Ph.D.,1 Tim M. Schoenmakers, Ph.D.,1 Regina J.J.M. van den Eijnden, Ph.D.,2
Ad A. Vermulst, Ph.D.,3 and Dike van de Mheen, Ph.D.1,4
Abstract
The study explores the reliability, validity, and measurement invariance of the Video game Addiction Test
(VAT). Game-addiction problems are often linked to Internet enabled online games; the VAT has the unique
benefit that it is theoretically and empirically linked to Internet addiction. The study used data (n = 2,894) from a
large-sample paper-and-pencil questionnaire study, conducted in 2009 on secondary schools in Netherlands.
Thus, the main source of data was a large sample of schoolchildren (aged 13–16 years). Measurements included
the proposed VAT, the Compulsive Internet Use Scale, weekly hours spent on various game types, and several
psychosocial variables. The VAT demonstrated excellent reliability, excellent construct validity, a one-factor
model fit, and a high degree of measurement invariance across gender, ethnicity, and learning year, indicating
that the scale outcomes can be compared across different subgroups with little bias. In summary, the VAT can be
helpful in the further study of video game addiction, and it contributes to the debate on possible inclusion of
behavioral addictions in the upcoming DSM-V.
1
IVO Addiction Research Institute, Rotterdam, The Netherlands.
2
Faculty of Social and Behavioral Sciences, Utrecht University, Utrecht. The Netherlands.
3
Behavioral Science Institute, University of Nijmegen, Nijmegen, The Netherlands.
4
Department of Public Health, Erasmus Medical Centre, Rotterdam, The Netherlands.
507
508 VAN ROOIJ
secondary schools and 171 classes (n = 4,074), average per player online games, which are video games played with
class response rate 92 percent. For details on the procedure, see other people over the Internet; casual browser games, which
earlier publications.14,19 The proposed VAT items were skip- are simple games that are played online via the web browser;
ped by adolescents who did not play games at all (n = 1,024). and finally, offline games, which do not utilize the Internet.
For the purpose of scale validation, adolescents with less than Hours per week spent on these game types were obtained by
four of the items on the VAT completed were removed multiplication of two questions measuring days per week of
(n = 156). As the analyses deal with scale means, missing values gaming and average hours of gaming per day, similar to a
do not distort scores. The final dataset has 2,894 cases (62 previous study.19
percent boys), of which 99 percent completed 9 or more VAT
items. Overall average age was 14.3 (SD = 1.0) years. Psychosocial measures. The following psychosocial
measures were included: Rosenberg’s Negative Self-Esteem
Measures Scale19,22 (Chronbach’s a = 0.87), the UCLA Loneliness
Scale23,24 (Chronbach’s a = 0.87), the Depressive Mood List25–27
Compulsive Internet Use Scale. The 14-item version of
(Chronbach’s a = 0.81), and the Revised Social Anxiety Scale
the CIUS2 was used to measure CIU, and includes questions,
for Children28–30 (Chronbach’s a = 0.89). These scales have
such as ‘‘Have you unsuccessfully tried to spend less time on
been used in Dutch studies and have demonstrated good
the Internet?’’ The CIUS contains the following components:
reliability.3,11,19 For all four scales, a higher score indicates
loss of control (four items), conflict (four items), preoccupa-
more reported problems.
tion/salience (three items), coping/mood modification (two
items), and withdrawal symptoms (one item). The CIUS has
Strategy of analyses
good validity2 and internal reliability2,10,14 (current sample,
Chronbach’s a = 0.89). To confirm if the one-factor structure of the CIUS also
holds for the VAT, a 1-factor CFA was applied using Mplus
Video game Addiction Test. The items that compose version 5.1.31 On grounds of overlapping content, some error
the CIUS were rephrased into the 14 proposed VAT items, terms of items must be correlated to get an acceptable model
measured with a with a 5-point scale: 0, never; 1, seldom; 2, fit. We correlated the same error terms as reported for the
sometimes; 3, often; and 4, very often (Table 1). Conse- CIUS (item 1 with item 2, 6 with 7, 8 with 9, 11 with 12, and 13
quently, the VAT incorporates the same components as the with 14).2 Additionally, as the scale is formulated for ado-
CIUS. While we strongly suspect that online games play a lescents, the error term for item 3 was correlated with item 8
particularly important role,20 playing multiple game types is on grounds of content overlap. To deal with missing data, the
not unusual14 and problematic use of other game types can- Full Information Maximum Likelihood estimation technique
not be ruled out. Thus, the phrasing was kept universal (i.e., was used. Respondents are nested within schools and classes.
‘‘game’’ instead of ‘‘online game’’). Due to software limitations, the COMPLEXTWOLEVEL
procedure, which corrects for this nonindependency, did not
Game Addiction Scale. The seven-item version of the work. As the differences between separate COMPLEX pro-
Game Addiction Scale (GAS) was included.21 Six of the seven cedures for schools and classes were negligible, we decided to
items each measure one of Griffiths’ components of behavioral report the results for classes (most likely point of mutual in-
addiction, while one question measures ‘‘problems.’’8 The GAS fluence). The COMPLEX procedure uses a robust maximum
shows good reliability in this sample (Chronbach’s a = 0.84). likelihood estimator resulting in robust v2 values. Model fit
is reported in robust v2 values, and estimated degrees of
Weekly hours online, casual browser, and offline freedom, the Comparative Fit Index (CFI)32 and the root-
gaming. Three types of games are distinguished: multi- mean-squared error of approximation (RMSEA).33
Table 1. Factor Loadings and Item Phrasing for the 14 Video Game Addiction Test Items
Standardized
Itema How often . factor loadings
Table 2. Tests of Invariance Constraints (Metric and Scalar) for Gender, Ethnicity, and Learning Year
CFI, Comparative Fit Index; RMSEA, root-mean-squared error of approximation; k, factor loadings; s, item intercepts.
The second step was to test measurement invariance. robust v2 values were rescaled to standard v2 values to
Various forms of measurement invariance can be distin- compute a correct v2 difference.
guished,34 but we restrict ourselves to the three forms pro- As the v2 test is sensitive to sample size,36 we also exam-
posed by Steenkamp and Baumgartner.35 If the fit of the CFA ined the increment of fit indices (DCFI37 and DRMSEA) be-
model is acceptable across different groups, configural in- tween models. We follow Cheung and Rensvold’s rule of
variance is supported. Metric invariance requires that the factor thumb that a decrease in CFI of .01 or less indicates that
loadings (k) of corresponding factors are invariant (and thus invariance should not be rejected.37 Construct validity is
have the same meaning) across groups. Scalar (or strong) in- tested by cross comparisons with the GAS,21 the CIUS,2
variance requires that the item intercepts (s) of corresponding measures of psychosocial well-being19,38,39 and time spent on
factors are invariant across groups: this makes comparisons various game types.
of group means meaningful.
The three forms of measurement invariance were tested for
Results
three background variables: gender, ethnicity (Dutch or
Western European versus non-Western European ethnicity), Step 1: confirmatory factor analysis
and learning year. Learning year was operationalized as
The fit of the factor model was good: v2 (71) = 549.564,
learning one (average age 13), two, and a mixed group con-
RMSEA = 0.048, and CFI = 0.961 (Table 1), with high reliability
taining years three and four (the fourth learning year has a
(Chronbach’s a = 0.93). All loadings are > 0.62 indicating a
small sample size; n = 120). After fitting a baseline (non-
good construct validity.
invariant) model (representing configural invariance) with no
constraints, metric and scalar invariance are tested by con-
Step 2: measurement invariance
straining factor loadings (metric invariance) and intercepts
(scalar invariance) subsequently across groups. If the increase Table 2 summarizes the results for the unconstrained
in v2 (from the baseline model to k-constrained model or from model, the model with constrained factor loadings (metric
the k-constrained model to the s-constrained model) is non- invariance), and the model with constrained intercepts and
significant, metric and/or scalar invariance is supported. The constrained factor loadings (scalar invariance) for each of the
Table 3. Pearson’s Correlations Between Game and Internet Addiction Scales, Psychosocial
Outcome Measures, and Measures of Time Spent on Various Game Types
1 2 3 4 5 6 7 8 9
Calculations were performed with scale means, *p < 0.05; **p < 0.01; ***p < 0.001.
510 VAN ROOIJ
three background variables. Configural invariance is sup- are generally studied independently, while it is evident that
ported by the high values of CFI ( > 0.95) and low values of game-addiction problems are strongly linked to Internet-
RMSEA ( < 0.05). Metric invariance shows significant in- enabled online games.17,20 The high correlation between
creases in v2 for gender and ethnicity (p < 0.05), but not for the online games (in comparison with other game types) and
learning year. The decreases in the CFI-value were 0.001, video game addiction in the current study provides further
0.001, and 0.000, respectively, indicating that metric invari- support for this idea (see Table 3). The VAT has the benefit
ance is supported to a high degree. With respect to scalar that it is theoretically and empirically (r = 0.61) linked to an IA
invariance, the v2 difference values have p-values < 0.001, scale: that is, it’s source, the CIUS. As such, the VAT can be
supporting scalar noninvariance. However, the decreases in employed in further survey research to support theoretical
CFI were 0.004, 0.004, and 0.005, respectively, and are lower progress in the field of online video game addiction. More-
than the Cheung/Rensvold criterion of 0.01. Our conclusion over, findings provide a starting point for clinical validation
is that scalar invariance of the VAT should not be rejected. of the VAT.
chosocial well-being among adolescents: a longitudinal 28. La Greca AM, Dandes SK, Wick P, et al. Development of
study. Developmental Psychology 2008; 44:655–665. the social anxiety scale for children: reliability and concur-
12. Bernardi S, Pallanti S. Internet addiction: a descriptive clin- rent validity. Journal of Clinical Child Psychology 1988;
ical study focusing on comorbidities and dissociative 17:84–91.
symptoms. Comprehensive Psychiatry 2009; 50:510–516. 29. La Greca AM, Stone WL. Social anxiety scale for children–
13. Van Rooij AJ, Zinn MF, Schoenmakers TM, et al. Treating revised: factor structure and concurrent validity. Journal of
internet addiction with cognitive-behavioral therapy: a Clinical Child Psychology 1993; 22:17–27.
thematic analysis of the experiences of therapists. Interna- 30. Gross EF. Adolescent Internet use: what we expect, what
tional Journal of Mental Health and Addiction 2012; 19; teens report. Journal of Applied Developmental Psychology
10:69–82. 2004; 25:633–649.
14. Van Rooij AJ, Schoenmakers TM, Van den Eijnden RJJM, 31. Muthén LK, Muthén BO. (2007) Mplus user’s guide. Fifth ed.
et al. Compulsive internet use: the role of online gaming and Los Angeles, CA: Muthén & Muthén.
other internet applications. The Journal of Adolescent Health 32. Bentler PM. Comparative fit indexes in structural models.
2010; 47:51–57. Psychological Bulletin 1990; 107:238–246.
15. Grüsser SM, Thalemann R, Griffiths MD. Excessive com- 33. Byrne BM. (1998) Structural equation modeling with LISREL,
puter game playing: evidence for addiction and aggression? PRELIS and SIMPLIS: basic concepts, applications, and pro-
CyberPsychology and Behavior 2007; 10:290–292. gramming. Mahwah, NJ: Lawrence Erlbaum Associates.
16. Peters CS, Malesky A. Problematic usage among highly- 34. Gregorich SE. Do self-report instruments allow meaningful
engaged players of massively multiplayer online role play- comparisons across diverse population groups? Testing
ing games. CyberPsychology and Behavior 2008; 11:481–484. measurement invariance using the confirmatory factor
17. Rehbein F, Psych G, Kleimann M, et al. Prevalence and risk analysis framework. Medical Care 2006; 44:S78–S94.
factors of video game dependency in adolescence: results of 35. Steenkamp JBEM, Baumgartner H. Assessing measurement
a German nationwide survey. Cyberpsychology, Behavior, invariance in cross-national consumer research. Journal of
and Social Networking 2010; 13:269–277. Consumer Research 1998; 25:78–107.
18. West R, Hardly A. (2006) Theory of addiction. Oxford: Wiley- 36. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating
Blackwell, p. 225. the fit of structural equation models: tests of significance and
19. Van Rooij AJ, Schoenmakers TM, Vermulst AA, et al. Online descriptive goodness-of-fit measures. Methods of psycho-
video game addiction: identification of addicted adolescent logical research online [now: Methodology (Gott)] 2003;
gamers. Addiction 2011; 106:205–212. 8:23–74.
20. Council on Science and Public Health. (2007) Emotional 37. Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes
and behavioral effects, including addictive potential, of for testing measurement invariance. Structural Equation
video games. Chicago: American Medical Association. www Modeling: A Multidisciplinary Journal 2002; 9:233–255.
.ama-assn.org/resources/doc/csaph/csaph12a07-fulltext.pdf. 38. Caplan SE. Relations among loneliness, social anxiety, and
21. Lemmens JS, Valkenburg PM, Peter J. Development and problematic internet use. CyberPsychology and Behavior
validation of a game addiction scale for adolescents. Media 2007; 10:234–242.
Psychology 2009; 12:77–95. 39. Ha JH, Kim SY, Bae SC, et al. Depression and internet ad-
22. Rosenberg M. (1989) Society and the adolescent self-image, diction in adolescents. Psychopathology 2007; 40:424–430.
Revised ed. Middletown: Wesleyan University Press. 40. Ferguson CJ. An effect size primer: a guide for clinicians and
23. Ghassemzadeh L, Shahraray M, Moradi A. Prevalence of researchers. Professional Psychology: Research and Practice
internet addiction and comparison of internet addicts and 2009; 40:532–538.
non-addicts in Iranian high schools. CyberPsychology and 41. Charlton JP, Danforth IDW. Distinguishing addiction and
Behavior 2008; 11:731–733. high engagement in the context of online game playing.
24. Russell D, Peplau LA, Cutrona CE. The revised UCLA Computers in Human Behavior 2007; 23:1531–1548.
loneliness scale: Concurrent and discriminant validity evi- 42. Wölfling K, Müller KW. Pathologisches glücksspiel und
dence. Journal of Personality and Social Psychology 1980; computerspielabhängigkeit. Bundesgesundheitsblatt—
39:472–480. Gesundheitsforschung—Gesundheitsschutz 2010; 53:306–312.
25. Engels RCME, Finkenauer C, Meeus WHJ, et al. Attachment
and adolescents’ emotional adjustment: the associations
with social skills and relational competence. Journal of Address correspondence to:
Counseling Psychology 2001; 48:428–439. Dr. Antonius J. van Rooij
26. Kandel DB, Davies MNO. Epidemiology of depressive mood IVO Addiction Research Institute
in adolescents: an empirical study. Archives of General Heemraadssingel 194
Psychiatry 1982; 39:1205–1212. 3021 DM Rotterdam
27. Kandel DB, Davies MNO. Adult sequelae of adolescent de- The Netherlands
pressive symptoms. Archives of General Psychiatry 1986;
43:255–262. E-mail: rooij@ivo.nl
Copyright of CyberPsychology, Behavior & Social Networking is the property of Mary Ann Liebert, Inc. and
its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for individual use.