You are on page 1of 8

ACHIEVEMENT EMOTIONS IN GERMANY AND CHINA A Cross-Cultural Validation of the Academic Emotions QuestionnaireMathematics

ANNE C. FRENZEL University of Munich TODD M. THRASH College of William & Mary REINHARD PEKRUN THOMAS GOETZ University of Munich

The aim of this study was to establish the construct comparability and cross-cultural utility of the German and Chinese versions of the Academic Emotions QuestionnaireMathematics (AEQ-M). Based on data from 312 German and 579 Chinese eighth-grade students, mean and covariance structures analysis revealed that the AEQ-M shows a high degree of measurement invariance across cultures. In addition, the emotions assessed by the AEQ-M showed similar patterns of relationships with self-reports of achievement, parental achievement expectations, and attributions of success and failure across the German and Chinese samples. Confirming earlier findings, Chinese students were found to experience higher levels of anxiety in mathematics. They were also found to experience more enjoyment, pride, and shame, and less anger, than German students. This research supports the use of the AEQ-M in cross-cultural research and provides data about a broader range of achievement emotions than has been investigated previously. Keywords: emotions; achievement; construct comparability; measurement invariance; mean and covariance structures analysis

The primary aim of this study was to establish the construct comparability of achievement emotions across samples of German and Chinese students. Specifically, we compared the original German-language version and the new Chinese-language version of the Academic Emotions QuestionnaireMathematics (AEQ-M; Pekrun, Goetz, & Frenzel, 2005), an adapted version of the Academic Emotions Questionnaire (AEQ; Pekrun, Goetz, & Perry, 2005; Pekrun, Goetz, Titz, & Perry, 2002) that assesses enjoyment, pride, anxiety, anger, and shame in the context of learning mathematics. Translating psychological content from one language to another poses many challenges, particularly with emotion-oriented content (e.g., Russel & Sato, 1995). However, as Schwarzer and Kim (1984) argued, In order to stimulate cross-cultural research these difficulties must be resolved in adapting inventories for assessment with different national groups (p. 277). To examine construct comparability, we used mean and covariance structures (MACS) analysis
AUTHORS NOTE: The Chinese version of the Academic Emotions QuestionnaireMathematics is available upon request from the first author, and the German and English versions are available from the third author. Thanks to Shijin He, Huizhen Ye, Y. T. Wan, and Rafael Buemia for their help with translation and data collection. Correspondence concerning this article should be addressed to Anne C. Frenzel, University of Munich, Department of Psychology, Leopoldstr. 13, 80802 Munich, Germany, phone +49 89 2180 6047, fax +49 89 2180 5250; e-mail: frenzel@edupsy.uni-muenchen.de. JOURNAL OF CROSS-CULTURAL PSYCHOLOGY, Vol. 38 No. 3, May 2007 302-309 DOI: 10.1177/0022022107300276 2007 Sage Publications

302

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Frenzel et al. / ACHIEVEMENT EMOTIONS IN GERMANY AND CHINA

303

(Cheung & Rensvold, 1999; Little, 1997), a statistical technique that can detect psychometric problems that result from cultural biases in item content, translation errors, acquiescence biases, and other factors (Little, 1997). If successful in establishing construct comparability, we sought to further establish the utility of the AEQ-M by examining convergent validity and mean differences in achievement emotions. Regarding convergent validity, we examined relationships between achievement emotions and several other achievement-relevant variables, including school achievement and attributions for success and failure (see Pekrun et al., 2002). Similar patterns of relationships across German and Chinese samples would support the utility of the AEQ-M in crosscultural research. Regarding mean levels of achievement emotions, differences across cultures were expected. Theory and empirical research suggest that there are differences in mean levels of achievement motivation and other achievement-related constructs across Eastern and Western countries (e.g., Chen & Stevenson, 1995; Salili, 1994). Concerning achievement emotions in particular, studies indicate that students from Eastern countries tend to experience higher levels of achievement anxiety than do students from Western countries (e.g., OECD, 2004). However, little is known about differences in achievement emotions other than anxiety. Our research extends past research by examining mean differences in a range of achievement emotions.

METHOD
PARTICIPANTS AND PROCEDURE

A total of 891 students participated in the study. Care was taken to maximize the comparability of the German and Chinese samples. The German sample consisted of 312 eighth-grade students (143 female, 165 male, and 4 unspecified) from three schools near Munich, comprising a mix of socioeconomic backgrounds. The Chinese sample consisted of 579 eighth-grade students (282 female, 263 male, and 34 unspecified) from four middle schools in Quanzhou and Xiangtan, also from diverse socioeconomic backgrounds. Mean ages in these samples were 14.99 (SD = .64) and 14.34 (SD = 1.01) years, respectively. Although students from both countries were in the eighth grade, the Chinese students were significantly younger than the German students (t = 10.14; p < .01). However, data from our previous studies indicate that there are no substantial changes in achievement-related emotions beyond the age of 13 (grade 7), at least in Germany (Pekrun et al., 2004). Therefore, the age difference is unlikely to pose a substantive threat to the validity of our findings. Participation was voluntary, and assessment occurred during regular class hours under the supervision of students mathematics teachers and with the consent of school principals.
MEASUREMENT OF EMOTIONS

The AEQ-M (Pekrun et al., 2005) was used to assess mathematics-related enjoyment (9 items), pride (6 items), anxiety (6 items), anger (8 items), and shame (8 items). For each emotion, the AEQ-M includes items related to three different situationsinstruction, homework/studying, and testingeach rated using a 5-point Likert-type scale ranging from strongly disagree to strongly agree. Sample items are as follows: enjoyment, I enjoy my math classes; pride, After doing my math homework, I am proud of myself;

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

304

JOURNAL OF CROSS-CULTURAL PSYCHOLOGY

anxiety, During a math test, I worry that I will get a bad grade; anger, I am so angry during my math classes that Id like to quit; and shame, I am ashamed that I cannot answer my math teachers questions well. For the present adaptation of the AEQ-M into Chinese, both the English and German versions were used as the basis for translation by three trilingual students of educational psychology. The translations were then blindly back-translated into German by a bilingual Chinese educational psychologist, and the wording of the items was further polished to reach the closest possible equivalence across language versions. In the current investigation, each emotion scale was found to be internally consistent in both the German and Chinese samples (enjoyment, = .89/.87; pride, = .85/.83; anxiety, = 80/.78; anger, = 85/.85; shame, = .85/.76).
VALIDITY MEASURES

Six validity variables were assessed using single items: achievement, What was your midterm grade in mathematics?; parental achievement expectations, My parents believe that I can achieve the following grade in mathematics; attributions of success and failure to ability, If I succeed [fail] in mathematics, it is usually because I am simply [not] talented in mathematics; and attributions of success and failure to effort, . . . because I have put in a lot of [not enough] effort. Achievement and parental expectations were rated on country-specific grading scales, and the attribution variables were rated using 5-point Likert-type agreement scales. These items were translated using the same translation procedures as described above. We did not examine the measurement invariance of these measures because that would have required the use of multiple-item measures. However, due to their straightforward content, the validity measures are likely to have similar measurement properties across cultures.

RESULTS
GERMAN-CHINESE MEASUREMENT INVARIANCE AND CONSTRUCT COMPARABILITY

We conducted MACS analyses using maximum likelihood estimation. For each emotion, we specified a baseline model (Model 1) that is identical in form across samples. The baseline model was a modified three-factor confirmatory factor analysis model. Instruction, homework/studying, and testing items were specified to load on separate first-order latent variables. Each first-order latent variable was specified to be fully determined by a phantom secondorder latent variable (see Little, 1997), and covariances were modeled among second-order latent variables. With this type of model, second-order latent variables may be rescaled into the standardized metric needed for the testing of equality of correlations across samples (see the following section, Convergent Validity), while allowing the variances of first-order latent variables to accurately reflect any differences in variance that exist between samples. Each baseline model was identified by standardizing the second-order latent variables in both groups, setting item uniqueness means to 0 and item uniqueness variances to 1 in both groups, setting the intercepts of the first-order latent variables to 0 and their loadings on the second-order latent variables to 1 in the German sample, and constraining the loading and intercept of one item per first-order latent variable to be equal across samples. The baseline model (Model 1) had good fit for all five emotions, indicating invariance of model form (see Table 1). This implies that the latent variables were comparable in a qualitative sense across samples.

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Frenzel et al. / ACHIEVEMENT EMOTIONS IN GERMANY AND CHINA

305

TABLE 1

Tests of Invariance of the Measurement Model and Latent Means: Model Fit Indexes
Model (1) (2) (3) Invariance of Loadings and Intercepts 237.19/60 .95 .91 130.83/18 .94 .94 61.36/18 .97 .98 149.73/44 .95 .94 155.98/44 .94 .94 (3E) and (3P) (4)

Emotion Enjoyment

Index 2 / df CFI MNCI 2 / df CFI MNCI 2 / df CFI MNCI 2 / df CFI MNCI 2 / df CFI MNCI

Invariance of Model Form 143.11/48 .97 .95 20.77/12 1.00 1.00 20.08/12 .99 1.00 119.73/34 .96 .95 106.32/34 .96 .96

Invariance of Loadings 162.77/54 .97 .94 29.27/15 .99 .99 32.97/15 .99 .98 124.17/39 .96 .95 111.57/39 .96 .96

Partial Invariance 174.85/58 .97 .95 33.18/17 .99 .99

Invariance of Latent Means 300/60 .93 .74 142/19 .94 .93 143/21 .90 .93 194/47 .94 .92 426/47 .78 .81

Pride

Anxiety

Anger

Shame

NOTE: CFI = Comparative Fit Index; MNCI = McDonalds Non-Centrality Index. The partial invariance model for enjoyment (Model 3E) includes cross-group equality constraints on instruction and testing (but not homework) intercepts, and the partial invariance model for pride (Model 3P) includes cross-group equality constraints on instruction and homework (but not testing) intercepts.

Next we tested whether the latent variables had comparable metrics by examining invariance of item loadings (slopes, i.e., the expected change in an item given a one-unit increase in the latent variable) and intercepts (i.e., the expected value of an item when the latent variable equals zero). As recommended by Little (1997), we examined invariance of measurement model parameters using a modeling rationale (i.e., indexes of practical fit). Loss of fit associated with the addition of equality constraints was considered negligible if CFI (Comparative Fit Index) and MNCI (McDonalds Non-Centrality Index) were less than or equal to .02 (Cheung & Rensvold, 1999). We used a statistical rationale (i.e., Likelihood Ratio Test) to examine invariance of latent variable parameters. Because we conducted separate analyses for each of the five emotions, we used an adjusted alpha level of .01. Constraining item loadings to be equal across groups (Model 2) was found not to decrease fit for any emotion. Additionally constraining item intercepts (Model 3) yielded a loss of fit for enjoyment and pride but not for anxiety, anger, or shame. To isolate the source of noninvariance for enjoyment, we examined three alternatives to Model 3, each with the intercept constraints for one latent variable removed. As a result, for enjoyment, Model 3 was rejected in favor of Model 3E, a partial-invariance model in which intercepts of the manifest variables representing the homework enjoyment latent variable were free to vary across samples. We followed the same procedure to isolate the source of noninvariance for pride. As a result, Model 3 was rejected in favor of Model 3P, a partial-invariance model in which intercepts of the manifest variables representing the testing pride latent variable were free to vary across samples. Models 3E and 3P fit as well as their corresponding models in Model 2. Fit indexes for all models are shown in Table 1.

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

306

JOURNAL OF CROSS-CULTURAL PSYCHOLOGY

CONVERGENT VALIDITY

To examine convergent validity, the six validity variables were added to the full or partial measurement invariance models as single-item indicators of latent variables without uniquenesses. This portion of the model was identified by standardizing the latent variables in both samples. Validity variable loadings and intercepts were unconstrained. All pairwise covariances among second-order emotion latent variables and validity latent variables were modeled and were scaled to the correlation metric as a result of our method of identification. Consistent with previous theory and research on achievement emotions (see Pekrun et al., 2002), we observed the following pattern of significant relationships, which held in both countries (except where noted) and for all situations (see Table 2). The positive emotions were positively related to mathematics grade, parental expectations, and success attributions to ability, whereas the negative emotions were negatively related to these variables. Failure attributions to lack of ability showed the opposite pattern. The positive emotions were positively related to success attributions to effort, whereas the negative emotions were not systematically related to this variable. None of the emotions except anxiety and shame among Chinese participants were related to failure attributions to lack of effort. Constraining all correlations between emotion latent variables and the validity latent variables to be equal across samples yielded a loss of fit for anxiety (2(18) = 49.74, p < .01), anger (2(18) = 47.98, p < .01), and shame (2(18) = 44.33, p < .01) but not for enjoyment or pride. Analyses of the source of noninvariance for anxiety, anger, and shame revealed that each of these emotions experienced in the homework/studying context (i.e., at home) was more negatively related to mathematics grade and parental expectations in China than in Germany. In addition, anxiety and shame, regardless of situation, were more positively related to failure attributions to lack of effort in China than in Germany. When constraints on these correlations were excluded, the constrained models fit as well as the unconstrained models.
DIFFERENCES IN THE MEAN LEVELS OF ACHIEVEMENT EMOTIONS

To examine differences in mean levels of each emotion, we used the full or partial measurement invariance models as baseline models. For anxiety, anger, and shame, means of all three latent variables (instruction, homework, and testing situations) were constrained equal across samples. For enjoyment and pride, only the means of the two latent variables having invariant intercepts were constrained equal across samples. The constraints on latent means yielded significant decrements in fit for all five emotions (p < .01; see Table 1, Model 4). Follow-up tests revealed that these overall differences were attributable to significant differences for each situation tested. Regardless of situation, Chinese students experienced higher levels of enjoyment, pride, anxiety, and shame, whereas German students experienced higher levels of anger.

DISCUSSION In this study, the AEQ-M, which assesses mathematics-related enjoyment, pride, anxiety, anger, and shame, was translated from German to Chinese and tested for measurement invariance across the two language versions. A high degree of measurement invariance across the German and Chinese versions of the AEQ-M was demonstrated, thus supporting the cross-cultural comparability of the emotion constructs assessed by the AEQ-M.

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

TABLE 2

Correlations Between Achievement Emotions and Achievement, Parental Expectations, and Attributions Within German and Chinese Samples
Enjoyment Chinese German Chinese German Chinese German Chinese Pride Anxiety Anger Shame German Chinese

Validating Construct

Latent Emotion Subconstruct (Situation)

German

Mathematics grade

Parental expectations

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Failure attribution: ability Failure attribution: effort Success attribution: ability Success attribution: effort

Test Instruction Homework Test Instruction Homework Test Instruction Homework Test Instruction Homework Test Instruction Homework Test Instruction Homework

.66** .53** .54** .62** .43** .51** .58** .55** .53** .07 .09 .00 .67** .60** .53** .06 .18** .14*

.62** .39** .48** .54** .35** .44** .52** .42** .40** .06 .09* .00 .65** .39** .38** .13* .26** .19**

.53** .53** .41** .49** .52** .36** .55** .60** .40** .08 .11 .03 .56** .62** .46** .15* .20** .27**

.64** .45** .45** .61** .44** .42** .40** .43** .27** .04 .06 .01 .51** .44** .38** .16** .26** .20**

.50** .60** .32** .47** .62** .34** .53** .70** .51** .02 .08 .08 .49** .57** .42** .05 .14 .13*

.46** .54** .55** .39** .52** .50** .41** .58** .49** .19** .11* .21** .41** .53** .44** .22** .03 .03

.51** .60** .40** .53** .54** .34** .51** .70** .47** .08 .04 .05 .49** .63** .39** .09 .02 .09

.50** .50** .54** .45** .45** .49** .37** .42** .36** .09 .06 .01 .37** .39** .28** .08 .15** .21**

.34** .16** .27** .32** .18** .34** .33** .27** .33** .05 .05 .02 .21** -.11 -.17* -.08 .13* .07

.44** .11* .65** .41** .08 .59** .38** .26** .47** .12* .15** .17* .37** .19** .51** .03 .17** .04

*p < .05. **p < .01.

307

308

JOURNAL OF CROSS-CULTURAL PSYCHOLOGY

The emotions assessed by the AEQ-M showed a theoretically meaningful pattern of relationships with achievement, parental achievement expectations, and attributions for success and failure. These relationships were found to be similar across cultures, further supporting the validity and utility of the AEQ-M in cross-cultural research. Notably, anxiety, anger, and shame were more strongly related to student achievement and parental expectations in China than in Germany, differences that emerged only with respect to emotions experienced at home. These findings support the idea that family approval plays a more important role in China than in Western European countries such as Germany (e.g., Salili, 1994). Consistent with previous findings of East-West differences in achievement anxiety, we found that Chinese students experienced higher levels of mathematics anxiety. Furthermore, we found that Chinese students experienced higher levels of mathematicsrelated enjoyment, pride, and shame, whereas German students experienced higher levels of anger. The latter finding is consistent with existing evidence that anger is avoided in collectivistic cultures but accepted in individualistic cultures (e.g., Grimm, Church, Katigbak, & Reyes, 1999; Kornadt, 1991; Matsumoto, Kudoh, Scherer, & Wallbott, 1988), a finding that researchers have attributed to differences in norms for the desirability of certain emotions (Eid & Diener, 2001) and differences in self-construals across cultures (Markus & Kitayama, 1991). Additional research is needed to identify the factors that mediate these cultural differences in achievement emotions. In sum, analyses of cross-cultural construct comparability, convergent validity, and mean differences support the validity and utility of the AEQ-M in cross-cultural research. We hope that the availability of the AEQ-M will stimulate further cross-cultural research in the achievement domain.

REFERENCES
Chen, C., & Stevenson, H. W. (1995). Culture and academic achievement: Ethnic and cross-national differences. Advances in Motivation and Achievement, 9, 119-151. Cheung, G. W., & Rensvold, R. B. (1999, August). What constitutes significant differences in evaluating measurement invariance? Paper presented at the conference of the Academy of Management, Chicago. Eid, M., & Diener, E. (2001). Norms for experiencing emotions in different cultures: Inter- and intranational differences. Journal of Personality and Social Psychology, 81, 869-885. Grimm, S. D., Church, A. T., Katigbak, M. S., & Reyes, J. A. S. (1999). Self-described traits, values, and moods associated with individualism and collectivism. Journal of Cross-Cultural Psychology, 30, 466-500. Kornadt, H.-J. (1991). Aggression motive and its developmental conditions in Eastern and Western cultures. In N. Bleichrodt & P. J. D. Drenth (Eds.), Contemporary issues in cross-cultural psychology (pp. 155-167). Amsterdam: Swets and Zeitlinger. Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32(1), 53-76. Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implication for cognition, emotion, and motivation. Psychological Review, 98, 224-253. Matsumoto, D., Kudoh, T., Scherer, K., & Wallbott, H. (1988). Antecedents of and reactions to emotions in the United States and Japan. Journal of Cross-Cultural Psychology, 19, 267-286. OECD (Organisation for Economic Co-operation and Development). (2004). Learning for tomorrows world: First results from PISA 2003. Paris: OECD. Pekrun, R., Goetz, T., & Frenzel, A. C. (2005). Academic Emotions QuestionnaireMathematics (AEQ-M): Users manual. Munich, Germany: University of Munich, Department of Psychology. Pekrun, R., Goetz, T., & Perry, R. P. (2005). Academic Emotions Questionnaire (AEQ): Users manual. Munich, Germany: University of Munich, Department of Psychology.

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Frenzel et al. / ACHIEVEMENT EMOTIONS IN GERMANY AND CHINA

309

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91-105. Pekrun, R., Goetz, T., vom Hofe, R., Blum, W., Jullien, S., Zirngibl, A., et al. (2004). Emotionen und Leistung im Fach Mathematik: Ziele und erste Befunde aus dem Projekt zur Analyse der Leistungsentwicklung in Mathematik (PALMA) [Emotions and achievement in mathematics: Goals and first results from the Project for the Analysis of Learning Mathematics (PALMA)]. In J. Doll & M. Prenzel (Eds.), Bildungsqualitt von Schule: Lehrerprofessionalisierung, Unterrichtsentwicklung und Schlerfrderung als Strategien der Qualittsverbesserung [Educational quality in schools: Professionalizing the teachers, developing instruction and advancing schools] (pp. 345-363). Mnster, Germany: Waxmann. Russel, J. A., & Sato, K. (1995). Comparing emotion words between languages. Journal of Cross-Cultural Psychology, 26, 384-391. Salili, F. (1994). Age, sex, and cultural differences in the meaning and dimensions of achievement. Personality and Social Psychology Bulletin, 2, 635-648. Schwarzer, C., & Kim, M. (1984). Adaptation of the Korean form of the Test Anxiety Inventory: A research note. In H. M. van der Ploeg, R. Schwarzer, & C. D. Spielberger (Eds.), Advances in test-anxiety research (Vol. 3, pp. 13-68). Hillsdale, NY: Erlbaum.

Anne C. Frenzel received her PhD in educational psychology and currently is an assistant professor in the Department of Psychology at the University of Munich. Her research interests concern factors that may influence students and teachers emotions related to learning and teaching. Todd M. Thrash received his PhD in social and personality psychology at the University of Rochester. He is currently an assistant professor at the College of William & Mary. He is interested in human motivation and personality, including approach-avoidance processes, implicit-explicit motive congruence, and inspiration. Reinhard Pekrun is a full professor of psychology at the University of Munich. His main research interests pertain to achievement motivation and emotion, international studies on student achievement, and the evaluation of educational systems. He is past president of the Stress and Anxiety Research Society and serves as coeditor of Anxiety, Stress and Coping: An International Journal. Thomas Goetz received his PhD in educational psychology and currently is an assistant professor in the Department of Psychology at the University of Munich. His research interests include students emotions, self-regulated learning, and self-concept.

Downloaded from http://jcc.sagepub.com at SWETS WISE ONLINE CONTENT on May 21, 2007 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.