Chameleon Effects in Homework Research the Homework Achievement Association Depends on the Measures Used and the Level of Analysis Chosen 2009 Contemporary Educational Psychology

Embed Doc
Copy Link
Readcast
Collections

CommentGo Back

Download

Chameleon effects in homework research: The homework–achievementassociation depends on the measures used and the level of analysis chosen

Ulrich Trautwein

, Inge Schnyder

, Alois Niggli

, Marko Neumann

, Oliver Lüdtke

Eberhard Karls University Tuebingen, Muenzgasse 11, 72070 Tuebingen, Germany

University of Fribourg, Fribourg, Switzerland

College of Teacher Education, Fribourg, Switzerland

Max Planck Institute for Human Development, Center for Educational Research, Berlin, Germany

Eberhard Karls University Tuebingen, Tuebingen, Germany

a r t i c l e i n f o

Article history:

Available online 19 October 2008

Keywords:

HomeworkFrequencyTime on taskEffortAchievement

a b s t r a c t

Usingadataset specificallytailoredtohomeworkresearch, withasampleof 1275studentsfrom70clas-ses in Switzerland, the association between homework and achievement in French as a second languagewastested at three levels(class level, between-student level, andwithin-student level). Thestrengthanddirection of the homework–achievement association depended on the homework indicator chosen anddiffered to some degree across analytical levels. At the class level, achievement was higher in classessetfrequenthomeworkassignmentsandinclasseswherestudentsreportedlowoveralllevelsofnegativeemotionswhendoinghomework.Atthebetween-studentlevel,highindividualhomeworkeffortandlowlevels of negative homework emotions predicted favorable developments in French achievement,whereas high homework time predicted lower achievement. At the intraindividual level, high homeworkeffort, high homework time, and low levels of negative homework emotions were statistically signifi-cantly associated with positive student evaluations of the learning gains from the specific assignment.

1. Introduction

Doeshomeworkenhancestudents’ achievement levels?Intheirrecent state-of-the-art meta-analysis,Cooper, Robinson, and Patall(2006)found mostly positive associations between homeworktime and achievement, leading them to state that ‘‘both withinand across design types, there was generally consistent evidencefor a positive influence of homework on achievement” (Cooperet al., 2006, p. 3). Althoughthis conclusionadequatelysummarizesthe existing body of research on the relationship between home-work time and achievement, it may not fully reflect the complexrelationship between homework and achievement (e.g.,Cooper,Lindsay,Nye,&Greathouse, 1998;DeJong,Westerhof, &Creemers,2000; Trautwein & Köller, 2003; Trautwein, Niggli, Schnyder, &Lüdtke, in press). In our view, definitive insights into the home-work–achievement relationship are as yet precluded by the pau-city of data collected specifically for purposes of homeworkresearch.This article has two central goals. The first is to delineate themethodological challenges inherent in the analysis of the home-work–achievement association. Weargue that homeworkresearchwill benefit from paying more attention to methodological issuesand implementing more complex designs that (a) include indica-tors of homework assignment and completion other than thetime-on-homework variable, (b) distinguish different levels of analysis (class level, between-student level, within-student level),and (c) involve repeated measurements. Our second goal is to doc-ument homework effects using a data set specifically tailored tohomework research that was obtained from a large sample of eighth-graders learning French as a second language.

2. The homework–achievement association

The majority of reviews of homework research (Cooper, 1989;Cooper & Valentine, 2001; Cooper et al., 2006; Keith, 1986; Wal-berg, 1991) and, more generally, researchoneducational effective-ness (e.g.,Hattie & Clinton, 2001) conclude that time spent onhomework is associated with achievement gains. In his classic re-view paper,Cooper (1989)used meta-analytical strategies to ana-lyze 17 experimental and quasi-experimental studies and 50nonexperimental studies. For the experimental studies, he foundan overall effect of

=.21 favoring homework conditions overno-homework conditions. Likewise, he found a positive relation-ship between time spent on homework and achievement in 43 of the 50 nonexperimental studies reviewed, most of which studies

0361-476X/$ - see front matter

Thestudy was supportedbyagrant fromthe SwissNational Science Foundation(Project No. 13DPD3-114176/1).* Corresponding author. Fax: +49 (0) 7071 295805.

E-mail address:

ulrich.trautwein@ife.uni-tuebingen.de(U. Trautwein).Contemporary Educational Psychology 34 (2009) 77–88

Contents lists available atScienceDirect

Contemporary Educational Psychology

journal homepage:www.elsevier.com/locate/cedpsych

worked with the variable ‘‘time spent on homework,” operational-ized as a global measure of homework load per week. Marked ageeffects were also documented, indicating that the association be-tween time spent on homework and achievement was weaker inelementary school than in highschool (

=.02 for grades 3 through5,

=.07 for grades 6 though 9, and

=.25 for grades 10 through12).To update this classic review,Cooper et al. (2006)recently con-ducted a meta-analysis of homework studies reported between1987 and 2003. Three types of study were distinguished. First,sixstudieshadanexperimental design. Noneofthesestudieswerepublished in peer-reviewed journals; all reported positive effectsof homework. Second, 30 studies used data from naturalistic,cross-sectionalstudiessuchastheNationalEducationLongitudinalStudyor theHighSchool andBeyonddatabase torelate homeworktime to achievement, statistically controlling for confounding vari-ables. In the large majority of these studies, homework time wasthe only homework indicator implemented, and was positivelyassociated with achievement. Third, 32 studies reported bivariatecorrelations between homework time and achievement; aweightedmeancorrelationof

=.24was calculatedfor these stud-ies. In sum, the studies reviewed seem to support the idea of a po-sitive homework–achievement association.Cooperetal. (2006)usedstate-of-the-art approachestosynthe-sizedataandprovidedabalancedaccountofthefindingsemergingfrom the U.S. studies covered in their review. The breadth anddepthof theiranalysisisimpressive. Inlight of thisseeminglycon-sistent support for a positive homework–achievement relation-ship, another homework study may seem superfluous. However,it may yet be too early to draw any definitive conclusions aboutthe positive effects of homework. One reason for this is that theCooper et al. review was restricted to research conducted in theUnited States, leaving open the question of cross-cultural general-izability. Moreover, concerns about the quality of the studies cov-eredinthereviewledCooperetal.(2006)towarnthat‘‘allstudies,regardless of type, had design flaws” (p. 3). Indeed, there are sev-eralpotentialthreatstothevalidityofthenonexperimentalstudieson the homework–achievement relationship reviewed byCooperetal.(2006).First,homeworkcanberelatedtoachievementatsev-eral analytical levels (seeTrautwein & Köller, 2003). A homeworkeffect at the

class

level (or homework assignment effect) is foundwhen students in classes with a higher quantity or quality of homework have more pronounced achievement gains than stu-dents in other classes (e.g.,De Jong et al., 2000; Trautwein, Köller,Schmitz, & Baumert, 2002). A homework effect at the

between-stu-dent

level(orhomeworkcompletioneffect)isfoundwhenstudentsinthesameclass whodifferintheirhomeworkbehavior(e.g., timespent on homework) show differential outcomes (e.g.,Cooperet al., 1998). Finally, researchers can also test whether homeworkcompliance is positively related to achievement at the

within-stu-dent

level. For instance, relative to their own baseline, do studentsunderstand the lesson better after doing their homework assign-mentsconscientiously?Homeworkisaclassicexampleofthemul-tilevel problem (e.g.,Elawar & Corno, 1985; Kreft & de Leeuw,1998;Raudenbush&Bryk, 2002), anditis essential todifferentiatebetween effects at the class level and the between-student level(and, perhaps, the within-student level) in all studies that relatehomework to achievement. Unfortunately, almost none of thestudies reviewed byCooper et al. (2006)attended to this aspect.A second major issue in several homework studies is that theyeither do not control for the role of confounding variables or over-look some potentially important confounding factors. For instance,students attending an advanced mathematics course or an eliteschool might spend more time on mathematics homework thanstudents enrolled in a basic course or a lower level school (seeKeith & Cool, 1992). Likewise, teachers in high-quality schoolsattended by students from privileged backgrounds might set morehomework. The ﬁnding of a positive relationship between home-work and achievement might thus be attributable to a commoncause (i.e., course level, school quality, or gender) rather than totime on homework per se. In a related vein, the majorityof studiesreviewed byCooper et al. (2006)are single-measurement studiesthat did not control for prior achievement. Questions pertainingto the directionality of homework effects cannot be readily an-sweredonthebasisofsuchdesigns.Doeshomeworktimeaffectla-terachievementordoesachievementaffecthomeworktime—ordobotheffectscoexist?Designsthatcontrolforpriorachievementareneeded to address this question.Third, the studies included in the meta-analysis focus almostexclusively on

time

spent on homework. This measure may in factobscure the relationship between homework and achievement,rather than elucidating it. With reference toCarroll (1963), timespent on homework is often equated with conscientious home-work behavior. However, Carroll’s model in fact predicts learningoutcomesbasedonbothtimespentandtimeneeded. Timeneededis higher in students with low cognitive abilities and/or low priorachievement. Moreover, Carroll emphasized the role played bymotivational and volitional factors (

perseverance

). In referring totime on task, Carroll in fact meant only the

active

time on task.Yet all sorts of distractions can have detrimental effects on stu-dents’ homework behavior. If a student reports spending a lot of time on his or her homework, this is not necessarily a sign of greatconscientiousness, but may reﬂect problems of motivation or con-centration (seeTrautwein & Köller, 2003;Trautwein & Lüdtke,2007, for a critical account of the time on task variable).

3. Multilevel studies on homework

A small number of European homework studies that did not ﬁtthe inclusion criteria speciﬁed byCooper et al. (2006)for their re-view (i.e., studies conducted in the United States and reported be-tween 1987 and 2003) are of relevance to the presentinvestigation. In general, these studies found only limited supportfortheassociationbetweenhomeworktimeandachievement.Sch-mitz and Skinner (1993)conducted a diary study spanning a 4-month period. Based on daily measurements, they analyzed therelationship between time spent on homework and achievementatboththebetween-studentandthewithin-studentlevel. SchmitzandSkinner also asked students to rate the ‘‘subjective effort” theyput into their homework (i.e., their perception of effort expended).In the between-student analysis, a negative correlation of

.43was found between the time needed to complete homeworkassignments and achievement. In other words, students who re-ported less study time than their peers had higher achievementscores. At the same time, subjective effort was positively relatedtoperformance.Intheirwithin-studentanalysis,SchmitzandSkin-ner again found a negative correlation between time spent onhomework and achievement—students needed more time forassignments they found difficult. When controlling for task diffi-culty, the relationship between time on homework and achieve-ment became more positive.De Jong et al. (2000)examined homework effects on the devel-opment of mathematics achievement in a sample of 1394students(56classesfrom28schools)intheirfirstyearofjuniorhighschool.Arepeatedmeasurementdesignwasused, withseparatemeasuresof prior achievement and intelligence. Homework data were ob-tained from students (homework time, homework problems,homework study tactics, and the role of parents), teachers (home-work amount), and observers (homework assignment, discussionof homework). Several variables showed statistically significantcorrelations with achievement gains. For instance, there was anegative correlation of

.15 between time spent on homework

U. Trautwein et al./Contemporary Educational Psychology 34 (2009) 77–88

and math achievement (but the correlation was no longer signifi-cant when prior achievement and intelligence were taken into ac-count). The authors interpreted this finding as indicating thatstudents with less knowledge need more time to complete theirhomework. Homework amount was the only homework variablethatexplainedasignificantamount(2.4%)ofthevariancewhenin-cluded in a multilevel analysis. De Jong et al. operationalizedhomework amount as the absolute number of tasks assigned to aclass in the school year (i.e., as a class-level variable).Another study using multilevel analyses to address homeworkeffectswasreportedbyTrautweinetal.(2002). Repeatedmeasure-ment data collected from 1976 German 7th graders in 125 classeswere analyzed to investigate the role of homework in enhancingmathematics achievement. Trautwein et al. controlled for priorachievement, intelligence, socio-economic background, motiva-tion,andtypeofsecondaryschool.Threehomeworkvariableswereused: homework frequency, frequency of teachers’ monitoring of homework completion, and time typically spent on homeworkper day. Homework frequency and frequency of monitoring wereaggregated at the class level, whereas time spent on homeworkwas introduced as both a student-level and a class-level variable(thelatterbeingdubbed‘‘homeworklength”).Whenpriorachieve-ment was controlled, the frequency of homework assignments atthe class level statistically significantly positively predicted mathachievement at the second measurement point, whereas lengthyhomework assignments had a negative, albeit nonsignificant, ef-fect. At the individual level, time spent on homework statisticallysignificantly negatively predicted achievement, again controlledfor prior achievement.Recently,Trautwein (2007)reanalyzed homework data fromtwo international student assessment studies. His study wasprompted by findings from the Programme for International Stu-dent Assessment (PISA; Organization for Economic Co-operationand Development[OECD], 2001), which suggested that longerhomework times are associated with higher achievement. Usingthe German extension of the PISA data set, Trautwein controlledfor potential common causes of homework time and achievement,and applied multilevel modeling to disentangle student-level andschool-level effects. The multilevel analyses indicated that therelationship between homework time and achievement was mod-erate at the school level and negative at the individual level. In asecond study using a German longitudinal extension to the ThirdInternational Mathematics and Science Study (TIMSS;Beatonet al., 1996) and distinguishing homework frequency from home-work time,Trautwein (2007)corroborated and extended thesefindings. First, homework frequency—but not homework time—was a significant predictor of achievement at the class level. Sec-ond, homework time was negatively related to achievement andachievementgainsatthestudentlevel.Thesefindingsindicatethatextended homework times reported by individual students weremore likelyto reflect motivational problemsor problemsof under-standing than to be a sign of high student motivation or effort.Third, when important additional predictor variables such asschool type and prior achievement were not controlled, the effectsof homework variables were artificially inflated.In sum, the studies described in this section are somewhat atodds with the overall pattern of results documented in theCooperet al. (2006)review. However, the studies reviewed above are lim-ited in numbers, and replications of their main results are war-ranted. Moreover, it seems important (Cooper et al., 2006;Trautwein & Köller, 2003) to broaden the set of homework vari-ables under scrutiny; for instance, homework effort (seeTrautw-ein, Lüdtke, Kastens, & Köller, 2006) and academic emotionswhen doing homework (Pekrun, Frenzel, Goetz, & Perry, in press;Warton,2001)havenotroutinelybeenincludedinprevioushome-work studies, despite their apparent relevance.

4. The present investigation

This article reports results from a large study that was specifi-cally designed to test homework effects. We used a multilevel re-search model as a conceptual framework to analyze the effects of homework assignment and homework completion at three levels:the class level, the between-student (or interindividual) level, andthe within-student level. Usingmultilevel analysis and pathanaly-sis,wetestedtheassociationbetweenhomeworkandachievementat thesethreelevels. Specifically, weaddressedthefollowingthreesets of research questions and hypotheses.First, looking at the class level, we tested whether classes setfrequent and/or lengthy assignments showed higher achievementgains and whether classes withhigh homeworkmorale were moresuccessful in the long run. Three aspects of our study facilitatedinvestigationofthesequestions.First,weseparatedhomeworkfre-quencyandhomeworklength;second,wedisentangledhomeworkassignment and homework completion effects; third, we con-trolled for track level, which may also be an important variable.Based on our prior studies (Trautwein, 2007; Trautwein et al.,2002), we expected to find a positive association between the fre-quencyof homeworkassignmentsandachievement at theclass le-vel. No such effect was expected for homework length. Moreover,we hypothesized that classes with high homework morale (highaverageeffortonhomework)andlowlevelsofnegativehomeworkemotions would show favorable achievement levels. Finally, weexpected that the predictive power of the other variables woulddecrease when prior achievement and school track werecontrolled.The second set of questions concerned the between-student le-vel.Comparedwiththeirclassmates, arestudentswhoinvestmoreeffort and/or time in their homework more successful? And doeshigh achievement predict high homework effort? With its longitu-dinal design, our study was well suited to answer these questions.We expected homework effort to positively predict later achieve-ment at the student level, but time on homework and high levelsof negative homework emotions to be negatively related to laterachievement. High prior achievement was expected to positivelypredict later homework effort and to negatively predict laterhomework time and negative homework emotions.Finally, the third set of research questions targeted homeworkeffectsattheintraindividuallevel.InlinewithSchmitzandSkinner(1993)and the within-student analyses they reported, we wereinterested in how homework time, homework effort, and negativehomework emotions relate to (perceived) achievement on a day-to-daybasis. Comparedwiththeir ownbaseline, whendostudentsthink they have benefited from homework assignments: whenthey put more time and/or effort into their completion? And howare negative homework emotions related to perceived achieve-ment?Adiarycomponenttoourstudyallowedustoanalyzethesewithin-person associations. We expected homework effort to pos-itively predict subjective evaluations of the learning gains fromspecific assignments and negative homework emotions to nega-tivelypredictthem.Furthermore,wespeculatedthatlongerhome-work times would be associated with less positive subjectiveevaluations of homework benefits.

5. Methods

5.1. Sample

The sample considered here derives from a large study (e.g.,Trautwein, Lüdtke, Schnyder, & Niggli, 2006) on homework inFrench as a second language conducted in collaboration betweenresearchers at the University of Teacher Education in Fribourg,

U. Trautwein et al./Contemporary Educational Psychology 34 (2009) 77–88

of 00

Category:	School Work
Rating:
Upload Date:	03/01/2009
Copyright:	Attribution Non-commercial
Tags:	Notes Notes (fewer)

Documents

People