You are on page 1of 5
Journal of Research in Personality 46 (2012) 472–476 Contents lists available at SciVerse ScienceDirect Journal of Research in Personality journal homepage: www.elsevier.com/locate/jrp Orthogonal dreams in an oblique world: A meta-analysis of the association between attachment anxiety and avoidance Jessica J. Cameron a,⇑, Heather Finnegan b, Marian M. Morry a a b University of Manitoba, Canada University of Windsor, Canada a r t i c l e i n f o Article history: Available online 11 May 2012 Keywords: Attachment Anxiety Avoidance Meta-analysis Relationships a b s t r a c t The association between attachment dimensions (anxiety and avoidance) can influence measurement creation and statistical analyses. Our goal was to test the assumption that anxiety and avoidance was orthogonal in two popular measures: the ‘Experience in Close Relationships Scale – Revised’ (ECR-R) and the original ‘Experiences in Close Relationships Scale’ (ECR). Our meta-analysis of 242 studies revealed that despite both scales being highly reliable, the anxiety–avoidance correlation was higher for the ECR-R than the ECR. Other variables also moderated the association. Implications include methods of statistical analysis and recommendations for future measurement creation and use. Ó 2012 Elsevier Inc. All rights reserved. 1. Introduction In attachment research, like any scientific study, the quality of empirical findings is dependent on the measurement quality. Although there is a great deal of variability in the measurement of adult attachment (see, Kurdek, 2002), a growing number of researchers rely on self-report surveys of the anxiety and avoidance dimensions (see, Fraley & Waller, 1998; Mikulincer & Shaver, 2007). In many theories, these dimensions are implicitly assumed or explicitly required to be orthogonal (e.g., Mikulincer & Shaver, 2003). The association between anxiety and avoidance should reflect the theory upon which the scale was created and thus, the observed association represents a form of validity. We investigated whether anxiety and avoidance dimensions are orthogonal using two recently popular self-report adult attachment measures: ‘The Experiences in Close Relationships Scale’ (ECR; Brennan, Clark, & Shaver, 1998) and the ‘Experiences in Close Relationships Scale – Revised’ (ECR-R; Fraley, Waller, & Brennan, 2000). Attachment theory stipulates that early experiences with primary caregivers set the stage for a lifetime of expectations and learned reactions to relational events (Bowlby, 1982). Caregivers providing consistent, warm, and responsive environments for infants foster the development of secure attachments which cultivate future attachments to others. However, individuals with histories unresponsive, cruel, or neglectful caregiving will lack faith that others will help and thus, insecure attachments are formed. ⇑ Corresponding author. Address: Department of Psychology, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2. Fax: +1 204 474 7599. E-mail address: cameron2@cc.umanitoba.ca (J.J. Cameron). 0092-6566/$ - see front matter Ó 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jrp.2012.05.001 Attachment measures typically assess attachment (in)security with the use of anxiety and avoidance dimensions (see Mikulincer & Shaver, 2003, 2007). Secure individuals score low on both dimensions. However, individuals scoring high on anxiety experience a strong desire to obtain closeness with others, anxiety over a partner’s availability, and use coping strategies characterized by excessive reassurance seeking. Individuals scoring high on avoidance experience discomfort being close and dependent on an attachment figure, desire to keep emotional distance, and use coping strategies that include denying the need to obtain comfort from others. The assumption in most attachment theories is that anxious and avoidance dimensions should be orthogonal (see Mikulincer, Shaver, Sapir-Lavid, & Avihou-Kanza, 2009). For example, Mikulincer and Shaver (2003) present hyperactivating and deactivating strategies, measured by anxious and avoidance dimensions respectively, as either/or responses to attachment relevant threats and thus, the assumption is that these dimensions are unrelated. However, not all attachment theories rely on the orthogonal dimensions; in Bowlby’s (1973) original conceptualization of the stages of attachment, he posited that these two dimensions could be orthogonal in theory but oblique in actual practice. Furthermore, Fraley and Shaver (2000) suggest that anxiety could be considered a monitoring system and avoidance a behavioral orientation system. Although these theorists do not explicitly address the issue of dimensional orthogonality, the implicit message is that the independence of anxiety and avoidance is not a requirement for that framework. Thus, even though many attachment theories imply that anxiety and avoidance dimensions should be orthogonal, not all theoretical descriptions of the attachment system are based on the assumption that the dimensions are independent. J.J. Cameron et al. / Journal of Research in Personality 46 (2012) 472–476 1.1. The ECR and ECR-R To create the ECR, Brennan et al. (1998) collected all non-redundant items from available self-report adult attachment measures and conducted a factor analysis, resulting in two 18-item dimensions representing anxiety and avoidance. The ECR has become incredibly popular (Mikulincer & Shaver, 2007) yet fails to adequately measure the secure ends of the dimensions. Fraley et al. (2000) aimed to rectify this problem by utilizing item response theory on Brennan et al.’s (1998) original pool of items and revised the scale. Although many items remained similar, items yielding better item discrimination replaced items failing to tap the full range of scores. The ECR and the ECR-R exhibit comparable reliability, stability, and validity (see Mikulincer & Shaver, 2007). Although Brennan et al. (1998) and Fraley et al. (2000) used different scale creation methods, they both used techniques to pull for orthogonal factors and indeed, both found latent factors that were virtually orthogonal. However, close scrutiny of the literature reveals that these dimensions are not always orthogonal in practice. Other researchers have noted personal observations that the anxiety–avoidance association is lower in the ECR than the ECR-R (Mikulincer & Shaver, 2007). Moreover, the composite scores for each dimension were more highly correlated in the ECR-R (r = .35; R.C. Fraley, personal communication, March 3, 2010) than in the ECR (r = .11; Brennan et al., 1998). Authors finding these significant correlations often make excuses and apologize for this finding (e.g., Carnelley & Rowe, 2007) as if their results are unusual. We suggest that such findings are common-place in actual practice. Our study is the first to conduct a systematic investigation of the anxiety–avoidance association by conducting a meta-analysis on the ECR and ECR-R. We did not expect to find an anxiety–avoidance correlation that was exceedingly close to 1.0 (Crano & Brewer, 2002) as numerous investigations have supported a two or more dimensional structure (see Mikulincer & Shaver, 2007). Instead, we expected to find correlations closer to the medium (.30) to large (.50; Cohen, 1988) range. Such a finding would suggest that the ECR and ECR-R do not maintain the intended orthogonal nature (as represented by a zero to .10 or small effect size; Cohen, 1988). Furthermore, such a finding would suggest that researchers must address shared variance in their analysis; a practice that we later report is not widely adopted. Such results would spark a discussion about the utility of the ECR and ECR-R and perhaps also the theoretical assumption that anxiety and avoidance are orthogonal. In addition to our primary goal of determining the anxiety– avoidance correlation in the ECR and the ECR-R, we investigated possible moderators and tested whether one version was more influenced by potential moderators. As part of our meta-analysis, we also examined the reliability of the ECR and ECR-R. 2. Method 473 responses for 46 studies; the remaining 53 studies were excluded from the meta-analysis. Our final sample of published studies included 204 studies – 172 used the ECR and 32 used the ECR-R. To avoid the ‘‘file drawer problem’’ (Rosenthal, 1991), we posted an email on the Society of Personality and Social Psychology and the International Association of Relationship Researchers email list services requesting researchers who had used the ECR or ECR-R in any unpublished research to complete an online survey detailing their study. Through this method, we obtained complete data from 38 previously unpublished studies (23 ECR; 15 ECR-R). Altogether, 242 studies and 62,012 participants were included in the meta-analysis. 2.2. Procedure Two raters independently coded the studies for sample and scale characteristics (see Table 1). Across all codes, inter-rater reliability was high (90–100%) and all discrepancies were resolved by a third rater. 2.2.1. Sample characteristics Raters recorded the exact sample size, the proportion of the sample that was female, and the mean age. For the type of sample used, raters categorized the authors’ description of the sample as ‘‘university’’, ‘‘non-university’’, or from both sources. Next, relationship status was coded. We accommodated the various styles of reporting by recording the portion of the sample that was in a committed (i.e., married, cohabitating) or uncommitted (i.e., single, dating) relationship. Unfortunately, only 72 reported relationship status. Finally, raters indicated whether the sample was from North America or was collected outside of North American. 2.2.2. Scale characteristics Raters recorded the Pearson correlation coefficient between anxiety and avoidance dimensions and the reliability (Cronbach’s alpha) of each dimension. Raters also recorded whether the scale was translated. 2.3. Meta-analytic procedures To address our primary research question we analyzed the data using the procedures outlined in Lipsey and Wilson (2001). For the anxiety–avoidance association, Pearson correlation coefficients (i.e., the effect size) were transformed into Fisher’s z and weighted with the inverse variance. Wilson’s (2005) macros for SPSS statistical software provided analogous analyses of variance and multiple regression analyses. In reporting the results, all values were converted back into correlation coefficients. 2.1. Sample 3. Results We collected articles through two electronic databases (PsycINFO and Web of Science), using the search terms ‘‘ECR’’, ‘‘ECRR’’, and ‘‘Experiences in Close Relationships’’ (final date of retrieval was December 17, 2007). We also tried a ‘Citing Articles Search’ for articles citing Brennan et al. (1998) or Fraley et al. (2000) and we cross-referenced citations from Mikulincer and Shaver (2007) with previous searches. In total, we found over 800 citations. Inclusion criteria for the meta-analysis were (a) the use of the full, unabbreviated version of either the ECR or ECR-R, and (b) the article was published in English. Of the 257 studies meeting our criteria, 99 did not report the correlation and/or the reliability coefficients. We contacted these authors requesting the missing information and received When we encountered missing data for a moderator, the study was dropped from the analysis involving that variable. 3.1. Association between anxiety and avoidance Using Wilson’s (2005) random effects meta-analysis SPSS macros, on the 242 studies, the average random effect size was .20 (95% CI = .18 to .22), Z = 17.52, p < .001. That is, the average anxiety–avoidance correlation was .20, significant and in the small to medium range. A homogeneity analysis revealed that the sample of effect sizes was highly heterogeneous, Q(241) = 1702.73, p < .001, ranging from .22 to .68 across studies. Consequently, the sample of 474 J.J. Cameron et al. / Journal of Research in Personality 46 (2012) 472–476 Table 1 Number of ECR and ECR-R samples by coded sample and scale characteristics. Country Sample ECR North America Canada United States Outside North America Africa Australia Belgium China Germany Greece Israel Italy Lebanon Netherlands New Zealand Portugal Spain Taiwan Turkey Ukraine United Kingdom ECRR 11 107 2 8 0 6 0 1 2 3 39 3 0 4 1 6 1 1 1 1 10 1 0 9 0 0 11 0 0 1 0 8 0 0 0 2 0 5 University Non-university Mixed Translated ECR ECR-R 138 46 12 26 14 7 English Hebrew Other ECR ECRR 166 27 4 39 0 5 correlations is highly variable, suggesting that moderating variables might explain some of the variance. b ¼ :11Þ, b = .17, Z = 2.89, p = .004. Version did North America ð Y not interact with these moderators. 3.1.1. ECR vs. ECR-R To determine whether the ECR and ECR-R exhibit different anxiety–avoidance correlations, we conducted an ANOVA (maximum likelihood) with version as the only factor. Results revealed that the ECR-R had a significantly higher correlation between dimensions (r(46) = .41, p < .001; CI = .37 to .44, Z = 19.42) than did the ECR (r(196) = .15, p < .001; CI = .13 to .17, Z = 13.61), QB(1, 241) = 133.41, p < .001. The ECR and ECR-R samples significantly differed in two ways. First, the ECR was more likely to have been used in published samples (88%) than the ECR-R (68%), t(242) = 3.51, p = .001. Second, the ECR was less likely to have been used in samples collected outside of North America (40%) than the ECR-R (79%), t(242) = 4.98, p < .001. Because version was a strong predictor of the anxiety–avoidance correlation, we included version and its interaction with other possible moderators in all further tests using Wilson’s (2005) analog regression (maximum likelihood). For each moderator, we conducted two regressions. The first contained the main effects of ECR version (ECR = 0, ECR-R = 1) and one of the moderators (e.g., country). The second contained these main effects and the interaction term. Main effects were interpreted from the first regression; the interaction from the second regression. Importantly, in every regression, the main effect of version was significant (b’s = .29– .36; Z’s = 6.64–12.02, ps < .001), suggesting that controlling for the moderators did not alter its predictive strength. 3.1.3. Relationship status The percentage of the sample in a serious relationship (centered; M = 24.74, SD = 34.65) did not predict the association between dimensions. However, the interaction between relationship status and ECR version was significant, b = .23, Z = 2.36, p = .019 (see Fig. 1; N = 72). Relationship status did not influence the association in the ECR, b = .02, Z < 1, but strongly influenced the association in the ECR-R, b = .59, Z = 2.68, p = .007. Because mean age, sample source, and relationship status were highly interrelated (r’s = .66 to .79), we included the main effects for age, sample source, relationship status, and ECR version and the interaction terms for these three predictors with ECR version in a single regression analysis. Only version and the version by relationship status interaction remained significant, b = .33, Z = 1.93, p = .05, and the pattern was unchanged. Thus, this effect cannot be explained away by age or sample source. However, effects for age and sample source were eliminated by relationship status. For the meta-analysis on reliability (N = 227 studies), we used the method for summarizing Cronbach’s alphas as outlined by ** Dimension Correlation 3.1.2. Sample and scale characteristics as moderators Although we did not expect sample and scale characteristics to influence the anxiety–avoidance correlation, it was clear that such variables moderated the association. Published samples had a b ¼ :21Þ than unpublished samples somewhat higher correlation ð Y b ¼ :13Þ, b = .19, Z = 3.42, p < .001. Although gender did not ðY moderate the association, older samples had larger correlations b ¼ :19Þ than younger samples ð Y b ¼ :12Þ, b = .16, Z = 2.35, ðY p = .02. In addition, university samples had lower correlations b ¼ :14Þ than did non-university samples ð Y b ¼ :22Þ, b = .19, ðY Z = 2.07, p = .04. Finally, the correlation was higher in samples colb ¼ :17Þ than samples within lected outside of North America ð Y 3.2. The reliability of the ECR and ECR-R ** p < .01 Fig. 1. The association between anxiety and avoidance as a function of version and relationship status. J.J. Cameron et al. / Journal of Research in Personality 46 (2012) 472–476 Rodriguez and Maeda (2006) wherein each alpha was first converted to T and then weighted with the reciprocal variance of each transformed alpha. Using this method, average reliability for the anxiety dimension (original and revised ECR) was high, a = .909 (95% CI = .908 to .909). Average reliability for the avoidance dimension (original and revised ECR) was similarly high, a = .919 (95% CI = .918 to .919). In other words, the ECR (original and revised) is very reliable on both dimensions. Importantly, the ECR and ECR-R were equally reliable on both dimensions, QB’s < 1. None of the other moderators significantly influenced reliability. To determine whether reliability could explain our effects, we re-ran all regression analyses controlling for reliability. The pattern of results remained identical except for publication status which no longer significantly predicted the anxiety–avoidance correlation, Z < 1. Importantly, version remained a strong predictor in all regressions, lowest effect size was, b = .28, Z = 5.59, p < .001. 4. Discussion Anxiety and avoidance dimensions are indeed correlated, a finding discordant with the prevalent and implicit assumption that the two dimensions should be orthogonal. Moreover, the version of the ECR mattered a great deal: The anxiety–avoidance association was higher among samples using the ECR-R than the ECR. Thus, even though both scales were created using methods that pulled for orthogonal factors, neither scale, especially the ECR-R, is immune to correlated dimensions in actual practice. Samples obtained outside of North America, containing older and non-university participants, or those with more participants in committed relationships also exhibited higher anxiety–avoidance correlations. Although some moderators were artefacts (e.g., age and sample source), the effect of ECR version was robust and consistent. Interestingly, only relationship status interacted with version: The anxiety–avoidance association in the ECR-R was more strongly impacted by relationship status than in the ECR. Unfortunately, we only had complete data on relationship status for 30% of samples, making us question whether this interaction is fully representative. In fact, informal observations have suggested that the anxiety– avoidance association in both the ECR and the ECR-R is influenced by relationship variables (see Mikulincer & Shaver, 2007). In sum, our findings reveal that anxiety and avoidance dimensions are correlated in the ECR versions, especially in the ECR-R and in samples largely comprised of those in committed relationships. Ultimately, such findings call into question the largely implicit assumption that attachment dimensions are orthogonal. As such, our results may inspire researchers to revisit theoretical assumptions, measurement choices, and measure creation techniques. If researchers do not wish to seek other measurement options, they should at the very least adapt their statistical analyses to accommodate shared variance between dimensions. 4.1. Implications for statistical analysis Many researchers wish to tease apart the influences of each dimension and given the anxiety–avoidance correlation, doing so may be difficult. One method of accommodating shared variance is to include both dimensions as predictors in the same step in a regression, and thus control for shared variance. Unfortunately, this is not common practice. In our meta-analysis, only 64% of authors consistently employed such practices whereas 26% never employed any method to control for shared variance. Furthermore, samples that neglected controlling for shared variance had higher anxiety–avoidance correlations (r(73) = .25, p < .001; CI = .21 to .29, Z = 11.77) than samples that appropriately controlled for 475 shared variance (r(126) = .17, p < .001; CI = .14 to .19, Z = 10.42), QB(1, 199) = 8.39, p < .001). The present findings suggest that researchers should change their practices to accommodate shared variance. Doing so is essential for researchers using the ECR-R or samples that are prone to exhibit a higher correlation between dimensions (e.g., married samples). However, when the anxiety–avoidance association is high, such regression analyses may be skewed. A high degree of multicollinearity increases the standard errors of the individual regression coefficients, reducing significance values, and thus the standardized coefficients may become misleading (Cohen, Cohen, West, & Aiken, 2003). In such cases, when the anxiety–avoidance interaction is included in a regression analyses, the likelihood of finding a significant interaction is substantially decreased. Although the researcher can tease apart the unique effects of anxiety and avoidance by looking at their zero-order vs. unique association with the dependent variable of choice (see Cohen et al., 2003), separating the unique effects of an interaction from the individual effects of anxiety and avoidance is increasingly more complicated when dimensions are highly related (see Grömping, 2007, for further alternatives). 4.2. Differences between the ECR and ECR-R: implications for measurement Although both versions of the ECR used methods intended to pull for orthogonal factors, the ECR-R was created using a method (item response theory) that reduces measurement error and should increase reliability. However, the ECR and ECR-R were both highly reliable. Furthermore, when controlling for scale reliability, the version effect was still robust. Thus, even when measurement error is controlled, anxiety and avoidance are correlated, suggesting that this association is not a result of the ECR-R being more reliable. It stands to reason then, that the enhanced correlation between anxiety and avoidance in the ECR-R is due to the 16 unique items selected to better assess the full continuum. 4.3. Strengths and limitations Our research offers several strengths. First, we are the first to systematically investigate the anxiety–avoidance association with an impressive data set of published and unpublished studies. Second, we examined many possible moderating variables in the meta-analysis; though the effect remained heterogeneous suggesting other unmeasured moderators also influence the association. Last, but not least, we pinpoint potential problems with the use of the most common adult attachment scales and suggest readily available solutions: (1) include anxiety and avoidance simultaneously as predictors in regressions, and (2) test for and address potential multicollinearity issues. Meta-analyses are ultimately constrained by the samples available. In our sample, little research had been conducted outside of North America and thus, additional research is required to determine why the correlation between dimensions is larger outside of North America. Furthermore, the majority of the samples focused on romantic relationships, limiting generalizability. Many authors did not report important details and many did not respond to our request for information. Moreover, some moderating variables in the meta-analysis were confounded with each other, making analysis and interpretation difficult. Another limitation is that we focused our meta-analyses only on the ECR and ECR-R. The fact that the ECR and ECR-R are composed of the best items present in the most prominent self-report adult attachment measures, suggests that is it not unreasonable to assume that our results would generalize to similar, adult 476 J.J. Cameron et al. / Journal of Research in Personality 46 (2012) 472–476 attachment self-report scales. However, we recommend that researchers using all such scales look at the anxiety–avoidance correlation in their own samples and deal with potential issues of multicollinearity outlined earlier. 5. Conclusion Although some have suggested that both the ECR and ECR-R are similarly useful in predicting meaningful outcomes (see Mikulincer & Shaver, 2007), the present findings suggest that the scales are not interchangeable. Our findings highlight a discrepancy between theory and measurement. The ECR does not tap the full range of security (see Fraley et al., 2000) but benefits from more statistically distinct anxiety and avoidance dimensions, allowing the researcher to tease apart their unique contribution more confidently. The ECRR better assesses the continuum of attachment security but seems prone to higher correlations between attachment dimensions, potentially leading to problematic analysis. Researchers should be aware of these important scale differences when selecting their measures and when conducting statistical analyses. However, statistical practices alone will not rectify the larger theoretical and measurement issues. The ultimate goal of the present research is not to cast doubt on previous findings but to stimulate researchers to either create measures more in line with theory or to revisit the notion of the presumed orthogonal nature of anxiety and avoidance. Through such an open exploration of measurement and theory, our understanding of attachment systems – and we suspect – attachment security will blossom. Acknowledgments This research was funded by Social Sciences and Humanities Research Council of Canada (SSHRC) grant and University of Manitoba Research Grant to the first author and a University of Manitoba Research Grant to the third author. We thank our research assistants, Stacey Balchen, Annette Gagnon, Tara Galaugher, Natasha Stecy-Hildebrant, Christine Horbacio, Freda Howell, Michelle Keller, Simmi Mann, Tamara Sucharyna, Jessica Shultz, Cailey Strauss, and Yumiko Sakamoto. We are also grateful to Danu Anthony Stinson and John G. Holmes for providing feedback on an earlier draft. References Note. References for papers included in the meta-analysis appear at the following website: http://home.cc.umanitoba.ca/~cameron2/ECRsamples. pdf Bowlby, J. (1973). Attachment and loss (Vol. 2: Separation). London: The Hogarth Press. Bowlby, J. (1982). Attachment and loss (Vol. 1: Attachment). London: The Hogarth Press. Brennan, K. A., Clark, C. L., & Shaver, P. R. (1998). Self-report measurement of adult attachment: An integrative overview. In W. S. Rholes & J. A. Simpson (Eds.), Attachment theory and close relationships (pp. 46–76). New York: Guilford Press. Carnelley, K. B., & Rowe, A. C. (2007). Repeated priming of attachment security influences later views of self and relationships. Personal Relationships, 14, 307–320. Cohen, J. (1988). Statistical power analysis for the behavior sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/ correlation analysis for the behavioral sciences (3rd ed.). Routledge Academic. Crano, W. D., & Brewer, M. B. (2002). Principles and methods of social research (2nd ed.). Mahwah, NJ: Lawrence Erlbaum. Fraley, R. C., & Shaver, P. R. (2000). Adult romantic attachment: Theoretical developments, emerging controversies, and unanswered questions. Review of General Psychology, 4, 132–154. Fraley, R. C., Waller, N., & Brennan, K. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350–365. Fraley, R. C., & Waller, N. G. (1998). Adult attachment patterns: A test of the typological model. In J. A. Simpson & W. S. Rholes (Eds.), Attachment theory and close relationships (pp. 77–114). New York: Guilford Press. Grömping, U. (2007). Estimators of relative importance in linear regression based on variance decomposition. The American Statistician, 61, 139–147. Kurdek, L. A. (2002). On being insecure about the assessment of attachment styles. Journal of Social and Personal Relationships, 19, 811–834. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks: Sage. Mikulincer, M., & Shaver, P. (2003). The attachment behavioral system in adulthood: Activation, psychodynamics, and interpersonal processes. In M. P. Zanna (Ed.). Advances in experimental social psychology (Vol. 35, pp. 53–152). New York: Academic Press. Mikulincer, M., & Shaver, P. R. (2007). Attachment in adulthood: Structure, dynamics, and change. NY: Guilford Press. Mikulincer, M., Shaver, P. R., Sapir-Lavid, Y., & Avihou-Kanza, N. (2009). What’s inside the minds of securely and insecurely attached people? The secure-base script and its associations with attachment-style dimensions. Journal of Personality and Social Psychology, 97, 615–633. Rodriguez, M. C., & Maeda, Y. (2006). Meta-analysis of coefficient alpha. Psychological Methods, 11, 306–322. Rosenthal, R. (1991). Meta-analytic procedures for social research. Applied social research methods series (Vol. 6). Thousand Oaks, CA: Sage. Wilson, D. B. (2005). Meta-analysis macros for SAS, SPSS, and Stata. Retrieved 18.12.07.