Learning and Instruction 50 (2017) 75e84

Contents lists available at ScienceDirect

Learning and Instruction

journal homepage: www.elsevier.com/locate/learninstruc

High-structure versus low-structure cooperative learning in

introductory psychology classes for student teachers: Effects on
conceptual knowledge, self-perceived competence, and subjective
task values
Marina Supanc*, Vanessa A. Vo
€ llinger, Joachim C. Brunstein
Department of Psychology, Justus-Liebig-University, Otto-Behaghel-Strasse 10F, D-35394 Giessen, Germany

a r t i c l e i n f o a b s t r a c t

Article history: Using group presentation classes as a control condition, in nine introductory psychology classes we
Received 9 August 2016 examined the impact of high-structure versus low-structure cooperative learning on N ¼ 259 student
Received in revised form teachers' conceptual knowledge, on their self-perceived competence, and on their appraisals of task
15 March 2017
values. To vary the structure, we first created a lesson plan built upon core principles of cooperative
Accepted 31 March 2017
learning, and then eliminated from this plan critical elements structuring students' shared learning. Two-
Available online 8 April 2017
level analyses revealed that students in the two cooperative conditions (a) did better on three knowledge
tests administered throughout the course of this one-semester project, (b) developed a more favorable
Keywords:
Cooperative learning
view of their subject-specific competence, and (c) appraised the utility and intrinsic value of task as-
Structured cooperation signments more positively than did the control students. In each of the three knowledge tests, students
Higher education in high-structure groups outperformed students in low-structure groups. These findings support the
Student teachers hypothesis that structuring procedures enhance the efficaciousness of cooperative learning methods in
college classes.
© 2017 Elsevier Ltd. All rights reserved.

1. Introduction
In the literature on teaching strategies for college education,
cooperative learning (CL) is generally assessed to be a theoretically
sound and empirically backed instructional approach that en-
hances students' learning and performance (Johnson, Johnson, &
Smith, 2014; Millis, 2010; Springer, Stanne, & Donovan, 1999). In
a cooperative situation, learners work together in small groups to
accomplish common goals (Slavin, 1995). In doing so, their coop-
erative behavior is guided by two principles (see Johnson &
Johnson, 1999; Johnson et al., 2014): (a) No member can succeed
unless the other group members do (positive interdependence). (b)
Each member's contribution counts when a team's achievement is
assessed against certain criteria (individual accountability).
Although early evidence suggested that learning groups are
most successful when their cooperation is structured in accordance
with these principles (see Slavin, 1983), recent meta-analytic
* Corresponding author.
E-mail address: Marina.Supanc@psychol.uni-giessen.de (M. Supanc).
reviews (see Section 1.1) failed to show that degree of structure
moderates the efficaciousness of small-group learning methods.
Yet these results were drawn from a limited database integrating
evidence from quite different instructional experiments. Thus, our
primary aim was to analyze, within a single investigation, the ef-
fects of high-structure versus low-structure cooperation on college
students' knowledge acquisition, self-perceived competence, and
subjective task values.
Our secondary aim was to demonstrate that CL can successfully
be implemented in a college education system previously domi-
nated by traditional forms of instruction and learning. In this sit-
uation, the implementation of CL is often met with skepticism by
both students and lecturers (Phipps, Phipps, Kask, & Higgins, 2001;
Renkl, Gruber, & Mandl, 1996). Accordingly, in this study we
examined the general efficaciousness of two newly implemented
cooperative programs (high-structure CL and low-structure CL)
relative to an active control condition representing the traditional
teaching style (group presentations).
Both researchers and practitioners (e.g., Cohen, 2010; Niemi,
2002; Ruys, van Keer, & Aeltermann, 2010), working in the field
of teacher education, have persuasively argued that future teachers

http://dx.doi.org/10.1016/j.learninstruc.2017.03.006
0959-4752/© 2017 Elsevier Ltd. All rights reserved.
76 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
should, over the course of their own education, gain extensive
experience in practicing the use of CL methods. We thus addressed
the two above issues in a one-semester project, with student
teachers in introductory psychology classes as participants.
1.1. The role of structure in cooperative learning
CL denotes a family of teaching methods whose common de-
nominator is that small interactive groups are used to enhance
students' learning and interpersonal behavior (Johnson & Johnson,
1989; Slavin, 1995). Relative to the broadly defined concept of
collaboration (see Davidson & Major, 2014; Panitz, 1999), the term
CL is more strictly used to emphasize that (a) positive interde-
pendence between group members and (b) individual account-
ability for one's own learning constitute two cornerstones for
building effective teams of learners.
To translate these principles into a set of more concrete teaching
practices, instructors are advised to build their lesson plans either
on cooperative task structures or on cooperative reward systems, or
on any specific combination thereof (Slavin, 1983, 1995). A coop-
erative task structure exists when the information required to
complete an assignment is divided up into multiple segments. Each
member is accountable for one specific segment but takes re-
sponsibility for ensuring that the other members will also under-
stand the relevant aspect of the multifaceted task. To implement
cooperative incentives, rewards for good learning are assigned to
whole groups of learners but group performance is measured as the
average (or sum) of the members' individual accomplishments.
There is good evidence to suggest that relative to competitive
and individualistic learning, CL furthers primary, secondary, and
tertiary students' motivational engagement, their attitudes toward
learning, and their academic accomplishments (Johnson & Johnson,
1989; Kyndt et al., 2013; Slavin, 1995). In comparison, little is
known about the effects that structuring procedures have on the
outcomes of students' cooperative efforts.
As Slavin stated (1983, 1995), instructional interventions that
combine group goals (interdependence) with measures of indi-
vidual task performance (accountability) are most likely to produce
beneficial effects on students' learning. For instance, Archer-Kath,
Johnson, and Johnson (1994) reported that the impact of CL on
eighth-grade students' attitudes, achievements, and behavior was
most substantial when the team members' individual account-
ability was stressed through the provision of individualized feed-
back. In a similar vein, Ortiz, Johnson, and Johnson (1996) ran a CL
program in social studies for fifth-graders and found this program
to be most effective if it combined goal interdependence with
resource interdependence to promote students' shared learning.
Remarkably, this beneficial effect of structuring procedures on
students' individual achievements was significant only after group
members had worked together as a team for several weeks.
These results support the idea that a certain degree of structure
is required to make students' teamwork successful. Yet more recent
meta-analytic reviews did not confirm this view. Integrating
studies from 1995 onwards on CL in primary, secondary, and ter-
tiary education, Kyndt et al. (2013) found that, relative to studies
using only individual rewards (7 studies), studies using group re-
wards for individual learning (12 studies) did not produce superior
learning outcomes. In a review of small-group learning among
SMET undergraduates, Springer et al. (1999) compared studies
using structured CL methods (8 studies) with studies using less
structured collaborative methods (7 studies). An analysis of
achievement outcomes revealed no substantial difference between
these two approaches (ES ¼ 0.56 vs. 0.52). Similarly, using mea-
sures of learning transfer as outcome criteria, Pai, Sears, and Maeda
(2015) failed to identify any significant difference in the
efficaciousness of structured (10 studies) versus unstructured (15
studies) small-group work (ES ¼ 0.20 vs. 0.36).
Yet these meta-analytic results deserve some cautionary re-
marks: (a) Because the number of studies was relatively small, it
was difficult to draw reliable conclusions about the strength of
moderator effects associated with structuring methods. (b) For the
same reason (i.e., the limited database), it was not possible to
examine potential differences in the impact of structuring pro-
cedures for different outcome variables (e.g., performance mea-
sures and attitudinal measures) and for learners representing
different educational levels (e.g., primary, secondary, and college
students). (c) Studies that used multiple procedures to enhance
small-group learning were grouped together with studies using
only a single structure-building technique, resulting in a high de-
gree of heterogeneity within the category of structured small-
group learning. (d) To determine the overall effect of structured
cooperation, short-term studies were mixed up with long-term
interventions. (e) Comparisons across studies are no substitute
for investigations that systematically vary, within the same setting,
the presence (or absence) of certain kinds of structuring elements
in otherwise comparable small-group programs.
1.2. Cooperative learning in college classes
Meta-analytic results (see Johnson et al., 2014; Springer et al.,
1999) support the idea that CL generally improves the achieve-
ments of college students. From a review of research on the
teaching of psychology, Tomcho and Foels (2012) concluded that
beneficial effects of group activities are most substantial when
episodes of shared learning involve a high degree of participant
interdependence and thereby guarantee that all participants “are
actively engaged in the learning of all material” (p. 166).
Most studies on CL at the college level were conducted in sci-
ence classes at U.S. universities. For the impact of small-group
learning on statistics achievements, Kalaian and Kasim (2014) re-
ported that the magnitude of the observed effects was much higher
in primary studies conducted in the USA (ES ¼ 0.56) than in studies
conducted outside the USA (ES ¼ 0.13).
Studies investigating the efficaciousness of CL at German col-
leges were mostly conducted with teacher training students as
participants. Overall, these studies yielded mixed results. For
instance, in a one-session small-group study, Jurkowski and H€ anze
(2010) observed that participants working together in teams with
high goal and task interdependence did better on a conceptual
knowledge test than students working in teams with a low level of
interdependence. In a semester-long project with student teachers,
Jürgen-Lohmann, Borsch, and Giesen (2001) contrasted CL semi-
nars with traditionally organized group presentation classes.
Although CL students felt more involved in their learning activities,
they did not acquire more knowledge than control students. The
authors explained this null result by arguing that, in addition to the
lack of sensitivity of the test administered, the degree of goal
interdependence had been too weak to guarantee optimal results in
the cooperative teams.
1.3. This study
To sum up, across a broad range of academic domains and for
different groups of learners, the beneficial effects of CL are well
documented in the literature. Yet, little is known about the role
structuring procedures play in making cooperative methods suc-
cessful. Because only a few studies have systematically varied de-
gree of structure as an instructional variable, meta-analytic results
computed across a number of quite different studies are not suffi-
cient to clarify this issue. Furthermore, although CL has often been
 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
qualified as an innovative teaching method, it should not be
ignored that the implementation of CL in college classrooms may be
complicated by many obstacles, and occasionally (e.g., when stu-
dents feel reluctant to put effort into small-group projects) may
even fail to produce any visible effect on measures of learning
success (see Renkl et al., 1996). Hence, when CL is introduced in
college classrooms, it would seem advisable to scrutinize its effi-
caciousness relative to conventional teaching methods.
To address these issues, we conducted a quasi-experimental
study over the period of one term. Participants were first-year
student teachers enrolled in educational psychology classes. Our
main aim was to assess the potential benefits of highly structured
cooperation in bringing about desirable learning outcomes. Our
secondary aim was to test the added value of CL relative to tradi-
tionally organized classes. Accordingly, the study design comprised
three conditions: high-structure CL (HSCL), low-structure CL (LSCL),
and group presentation (GP) classes. The latter condition repre-
sented the (standardized) business-as-usual procedure. HSCL and
LSCL classes had in common that both were built on team learning
methods but differed from each other in the extent to which
principles of individual accountability and positive interdepen-
dence were applied to the arrangement of tasks, feedback, and
rewards provided in class to structure students' shared learning.
To test for differences between conditions, we used the
following measures as criterion variables: For each of the three
instructional units covered by this course (see Section 2.2), we
separately assessed at three measurement points how much
knowledge students had acquired about the concepts treated in
class (i.e., theories, evidence, and applied aspects of educational
psychology). To measure how students assessed their own subject-
specific competence, they were asked to indicate at the end of the
term how much they felt able to review, explain, and elaborate the
topics taught in class. At the same occasion, students were asked to
assess the intrinsic value (e.g., experiences of enjoyment) and the
utility value (e.g., the importance of their learning activities to their
future goals) of the tasks they had been working on in their classes.
Because a considerable body of evidence (see Johnson et al.,
2014; Krause, Stark, & Mandl, 2009; Tomcho & Foels, 2012) sug-
gests that CL can substantially enhance the achievements, attitudes,
and self-concepts of college students, we expected that participants
in CL classes would (a) acquire a higher level of conceptual
knowledge, (b) estimate their subject-specific competence more
favorably, and (c) appraise their learning tasks as being both
intrinsically more motivating and practically speaking more useful
than would GP control students.
As to the two cooperative programs, we expected the following:
(d) Complying with the view that cooperation is most successful
when positive interdependence is combined with individual
accountability to shape students' team learning behavior, we pre-
dicted that, relative to a low level of pre-organization, a high level
of structured cooperation would enable students to develop a more
comprehensive understanding of the scientific concepts treated.
We also explored whether the magnitude of this predicted effect
would vary as a function of students' teamwork experience (see
Ortiz et al., 1996).
(e) Extending the idea that a high degree of accountability
should strengthen the relation between learners' objective
knowledge and felt competence, we predicted that HSCL students
would estimate their subject-specific competence more favorably
than LSCL students.
(f) Although we expected that instructional procedures
prompting students to organize their group work would facilitate
experiences of competence, we suspected such procedures to
impose constraints on what learners are allowed to do and thereby
hamper experiences of autonomy (see Deci & Ryan, 2000). For
77
instance, in 12th-grade physics classes, Berger and H€ anze (2009)
observed that the same structuring method (task specialization)
produced two quite different effects on a self-report measure of
intrinsic motivation: It increased feelings of competence but
decreased feelings of autonomy. The overall effect of these opposing
influences on students' intrinsic interest was negligible (for related
findings, see Archer-Kath et al., 1994; and Moreno, 2009). This
complexity prevented us from formulating a clear-cut hypothesis
about how different degrees of structure would affect students'
perceptions of intrinsic task values. To the best of our knowledge,
estimations of utility value have never been used as a criterion
measure to evaluate the effects of structuring procedures in coop-
erative settings. Hence, similar to intrinsic task value, we did not
make a specific prediction about how differences between the two
cooperative conditions would affect utility-related task appraisals.
In brief, the following were our hypotheses: First, relative to GP
students, CL students would acquire more knowledge (Hypothesis
1.1) and HSCL students would prove to be more knowledgeable
than LSCL students (Hypothesis 1.2). Second, relative to GP stu-
dents, CL students would estimate their competence more posi-
tively (Hypotheses 2.1) and HSCL students would feel more
competent than LSCL students (Hypothesis 2.2). Third, CL students
would appraise the intrinsic value (Hypothesis 3.1) and utility value
(Hypothesis 3.2) of what they had learned more favorably than
would GP students.
2. Method
2.1. Participants and design
This study was conducted at a German university in the
2014e2015 academic year. The participants were 265 first-year
student teachers in nine introductory educational psychology
classes. Complying with the regulations specified in the academic
course guide (Module Handbook), enrollment in this course was
mandatory but course assignments remained ungraded for all
participants. Over one term, each class comprised 14 sessions (one
session per week; 90 min per session). Six students dropped out of
the class or switched to a different course offering during the first
five weeks and were thus not included in the data analysis. The
resulting sample consisted of 259 participants (185 women).
The design had one factor with three conditions: HSCL (84
students in 18 small groups), LSCL (86 students in 17 groups), and
GP seminars (89 students in 24 presentation groups). The nine
classes were conducted by three lecturers, each of whom held one
class in each of the three conditions. With this restriction, classes
were randomly assigned to conditions. When students enrolled in
one of the classes, they were unaware of the existence of different
instructional conditions. Yet students were eligible to choose one
class from a rich variety of course offerings documented in the
academic program (during the academic year, ca. 30 classes were
listed in the program, including the nine classes participating in
this study). Class size was limited to a maximum of 36 participants.
2.2. Course requirements and general procedures
In the first session, all students were informed about the general
purpose of this teaching research project. Specifically, they were
told that the project had been reviewed and approved by the uni-
versity's Teaching Innovation Fund advisory committee, that it was
intended to enhance the instructional quality of introductory psy-
chology classes, and that it involved the data-based evaluation of
different instructional approaches. Students were further told that
the data collected throughout the term would not be used in any
way to establish compliance with course requirements. Instead, to
78 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
pass the course, they would have to attend, and actively participate
in, at least 12 out of 14 sessions. Except for two students who
dropped out of their classes (see above), all participants gave their
written permission for data collection.
Learning content, background literature, and general regula-
tions for the fulfillment of course requirements were identical for
all participating classes. In the introduction phase (Session 1 and 2),
students were first informed about learning objectives and orga-
nizational issues. Based on the gender and final school grades of the
participants, students in HSCL and LSCL classes were divided into
heterogeneous small groups. In doing so, we sought to ensure that
the members within each group would be as heterogeneous as
possible (to promote the exchange of ideas between group mem-
bers representing different perspectives and ability levels; see
Johnson & Johnson, 1989), whereas differences between groups
should be as small as possible (to allow for a fair and unbiased
comparison of the achievements of the different teams; see Slavin,
1995). With one exception (two participants dropped out of a group
of five students), group size varied between four and six. On the
basis of their own topic-related interests and preferences, students
in GP classes came together to form presentation groups. In GP
classes, group size varied between two and six students, depending
on the breadth of the scope underlying the given presentation
topic.
For all participants, the learning phase (Sessions 3 to 14)
comprised three instructional units: learning motivation (Sessions
3 to 6), the psychology of teaching and learning (Sessions 7 to 10),
and learning difficulties and behavior disorders (Sessions 11 to 13).
Each unit ended with a test examining students' conceptual un-
derstanding of the topics treated in class. In Session 14, students in
all classes reviewed the essential ideas covered by this course. To
counteract the threat that instructional effects would be distorted
through confounding variables (e.g., differences in students'
learning efforts outside of the class meetings), homework assign-
ments were limited to a minimum (for GP students: preparing a
group presentation for one session; for HSCL and LSCL students:
writing a protocol for one instructional unit).
2.3. Instructional conditions
2.3.1. Commonalities and differences between the two cooperative
conditions
In both conditions, structuring procedures were aimed at three
targets: tasks (pre-organization of group work), feedback (infor-
mation about achievements and teamwork behavior), and rewards
(incentives for good work). In the HSCL condition, the rationale
guiding the design of these procedures was that each should
sharpen the team members' sense of accountability and interde-
pendence. Accordingly, in this condition we established a dense
web of cooperative tasks and incentive structures stimulating the
members of a team to share their knowledge, support each other,
reflect on their joint efforts, and take responsibility for their indi-
vidual accomplishments. To establish the LSCL condition, we used a
“dismantling strategy” (see MacKinnon, Taborga, & Morgan-Lopez,
2002) and therefore eliminated from the HSCL program several
elements that explicitly linked individual responsibility to partici-
pant interdependence. For the first instructional unit, the learning
activities assigned to HSCL students and to LSCL students are listed
in the Appendix.
Generally, HSCL and LSCL shared the following commonalities
(see Table 1): Group goals were established for all teams. All groups
received feedback as to their cooperative behavior and team
achievements. All groups were rewarded for good team
performances.
The key distinctions between the two cooperative programs
were the following: In contradistinction to the LSCL condition, in
the HSCL condition (a) the organization of group work required
each student to make a unique contribution to her or his team's task
performance; (b) the feedback given to students highlighted the
importance of each member's personal engagement in the group
process; and (c) the allocation of group rewards was contingent
upon performance improvements, making the reward system
sensitive to variations in the amount of effort a student had put into
the given team's work.
2.3.2. HSCL condition
The tasks, feedback, and rewards provided to HSCL students
were organized in the following fashion.
First, to pre-organize students' teamwork, we adopted various
well-established CL methods, such as jigsaw (Sessions 3, 4, 7, 8),
think-pair-share (Sessions 4, 5, 8, 9, 12), speed dating (Session 7),
gallery walk (Sessions 4, 9), and student team achievement division
combined with task specialization (Sessions 6, 10, 11, 12, 13).
Second, after completion of each of the three instructional units,
members of HSCL groups came together in a 20 min extra meeting
organized by an instructional assistant. Acting as a non-participant
observer during class time, this assistant had attended the previous
class sessions. In the process she had documented all learning ac-
tivities enacted in class (see below) and completed for each small
group a protocol addressing the quality of the team's cooperative
behavior as well as each individual member's participation in the
group process. An observation plan ensured that in each session,
the assistant observed each group for at least 5 min. In the feedback
meeting, the assistant provided the members of a team with two
kinds of information: First, she informed the members about how
well they had done (a) individually (each member's individual test
score) and (b) as a team (the members' mean test score) on the
unit-related knowledge test. Second, based on her observations in
class, the assistant informed the team members about the quality of
(a) each member's personal engagement (active participation) and
(b) the whole team's cooperative behavior (exchange of ideas,
seeking and giving feedback, time management) during the pre-
vious sessions. In the second and third meetings, the same assistant
provided additional information specifying how both a team's
overall performance and each member's individual contribution
had changed from one unit to the next. With respect to the
development of the team members' cooperative behavior, the as-
sistant proceeded in the same manner.
Third, in the next classroom session, two teams were rewarded
for their accomplishments: (a) one team for achieving the highest
test score, and (b) one further team for achieving the greatest in-
crease in test performance. The two teams were announced,
acknowledged, and praised in the presence of their fellow students.
2.3.3. LSCL condition
Relative to the HSCL condition, the following structure-building
elements were eliminated from the LSCL condition: First, neither
task specialization nor role assignments or external prompts to
engage in reflection were used to help LSCL students organize their
cooperative work. Second, during the feedback meetings, LSCL
students were informed about their team's achievement and
cooperative behavior but received no individualized information
about each single member's performance and engagement. Instead,
relative to HSCL teams, members of LSCL teams were given extra
time to jointly reflect on the group-level feedback provided by the
assistant. In this way, the duration of feedback meetings was held
constant across the CL conditions. Third, LSCL teams were not
rewarded for improvements in the development of their group
performance across adjacent instructional units. Instead, in each
class, the two teams that had earned the highest test scores were
 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
acknowledged for their good work. All other features (i.e., tasks,
materials, and instructional procedures) implemented in the HSCL
condition were fully applied to the sessions held in the LSCL
condition.
2.3.4. GP condition
In GP classes, instructions were delivered by student groups
using PowerPoint presentations and verbal instructions to inform
their classmates about key concepts characteristic of a circum-
scribed knowledge area. One presentation took place in each ses-
sion and each participant contributed to one presentation. In GP
classes, direct instruction prevailed, covering ca. two thirds of the
teaching time. The remaining period was filled with such activities
as silent seatwork, partner work, debating topics in ad-hoc groups,
whole-class discussions, and considerations of organizational is-
sues. Complying with the conventional procedure, the members of
each presentation group met their lecturer twice: once before (to
discuss their presentation draft) and a second time immediately
after their presentation (to receive feedback as to quality of their
presentation). Together these two meetings took approximately
60 min.
2.4. Treatment fidelity
For the two cooperative programs, all learning activities, task
materials, and time schedules were documented in a teaching
manual. Before the start of each instructional unit, these manuals
were delivered to the lecturers. During each session, one instruc-
tional assistant, who acted as non-participant observer (see above),
indicated on a notepad the instructional steps completed. These
observations were then compared to the instructions specified in
the manual. On average, 97% (HSCL) and 96% (LSCL) of the activities
were completed as planned in the manual. In GP classes, the
lecturer ensured that all topics specified in the lesson plan were
accomplished in a timely manner.
2.5. Measures
2.5.1. School grades and preferences for team work
In the first session, students were asked to report their final
school grades (Abitur). These grades could range from 1.0 to 3.9,
with low scores indicating stronger performance. We thus used
reversed scores for this variable.1
In the same session, all participants completed Kline's (1999)
Team Player Inventory. This 10-item questionnaire assesses the
degree to which individuals are positively predisposed towards
group work (sample item: “I enjoy working on team/group pro-
jects”). The response scale ranged from 1 (completely disagree) to 5
(completely agree). Coefficient alpha was .799.
2.5.2. Conceptual knowledge
Three knowledge tests were administered throughout the term.
These tests were developed by the first author and pretested in
pilot work with 57 student teachers. Each test consisted of 20 items.
To ensure a high degree of content validity, the first author pre-
pared for each instructional unit a table of specifications coordi-
nating test content (the diverse subtopics taught during an
1
Students also completed a pretest measure assessing their prior knowledge of
the scientific concepts introduced in this course. The instrument required re-
spondents to briefly define in their own words twelve key concepts. A correct
answer was awarded with one point. The portion of participants who reached in
this test only one point or no point at all was 89%. We thus decided to submit to our
inferential analyses final school grades as a covariate reflecting individual differ-
ences in participants' general academic capabilities.
79
Table 1
Structuring procedures in the two cooperative programs.
High Low
structure structure
Pre-organization of group work
- Group goals þ þ
- Assigned roles þ
- Task specialization þ
- Prompts for self-reflection þ
Provision of feedback
- Group-level feedback on achievement, behavior, and þ þ
development
- Individual-level feedback on achievement, behavior, þ
and development
Allocation of rewards
- Between-group comparison of absolute achievement þ þ
- Within-group comparison of relative achievement þ
instructional unit) with three levels of task complexity (remember,
understand, apply) adapted from Anderson and Krathwohl's (2001)
taxonomy of educational objectives. For instance, in Unit 1 entitled
as “learning motivation” sample items covering the subtopic of
“academic self-concept” were as follows: “Please specify how the
theoretical term of academic self-concept is defined in the litera-
ture” (remember). “Empirical evidences show that relative to girls,
boys often have a more positive view of their math abilities. How
would you explain this finding?” (understand). “As a teacher, what
would you do in your own class to enhance girls' self-concept in
math?” (apply). Approximately 75% of the items were presented in
an open-ended format. The remaining items were presented in a
forced-choice format. For each item, a correct answer was awarded
with one point. Each test was scored by one trained assistant who
was unaware of experimental assignments. The first author scored
one third of the material. Across the three tests, interrater agree-
ment was high (r > .90). Coefficient alpha was .691 (Test 1), .607
(Test 2), and .628 (Test 3).
2.5.3. Self-perceived competence and subjective task values
In the concluding session, participants completed a number of
self-report instruments. All responses were made on a 5-point
Likert-type scale (from 1 ¼ completely disagree to 5 ¼ completely
agree).
Participants' self-perceived competence was assessed with five
items (a ¼ .792) adopted from Braun, Gusy, Leidner, and Hannover
(2008). Two sample items read as follows: “I am able to describe the
key concepts and important issues addressed in this class.” “I am
able to illustrate quite clearly the more complex ideas we have been
dealing with during this class”.
Task values were assessed with two scales; Three items
(a ¼ .769) adopted from Braun et al. (2008) assessed the intrinsic
value inherent to the tasks given to students in class. One sample
item was: “It has been fun accomplishing the tasks given to us in
this class”. Utility value was assessed with four items (a ¼ .854)
developed by the first author to assess the degree to which par-
ticipants perceived the field of psychology in general and the
introductory course in particular to be useful for their educational
goals (“Psychology as an academic discipline will be useful for my
college education”) and their longer-term professional objectives
(“The learning content covered by this course will help me prepare
myself well for the teaching profession”).
3. Results
3.1. Data-analytic strategy
All statistical tests were run with the Mplus (Version 7.0)
80 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84

Table 2 competence), .183 (intrinsic value), and .150 (utility value). For the
Descriptive statistics for study variables. descriptive statistics, see Table 2.
Condition At Level 1, we included in the predictive model school grades
HSCL LSCL GPs
and team player scores as grand mean centered covariates. In
addition to group size, we entered at Level 2 two contrast-coded
M SD M SD M SD
variables. Contrast 1 compared cooperative students (coded as 1/
Final school grade 2.60 0.49 2.66 0.53 2.51 0.51 3 for both HSCL and LSCL) with GP students (coded as 2/3).
Team Player Inventory 3.48 0.60 3.66 0.46 3.48 0.53
Contrast 2 compared the HSCL condition (coded as 1/2) with the
Group size 4.76 0.63 5.12 0.54 3.99 0.97
LSCL condition (coded as 1/2). GP students (coded as 0) were
Acquired knowledge excluded from this comparison. The following equations specify the
Test 1 9.97 3.28 7.58 2.72 6.60 2.59 resulting model:
Test 2 6.48 2.88 5.09 2.13 3.73 1.68
Test 3 8.17 2.93 5.73 2.98 4.93 2.00 Level 1. Yij ¼ b0j þ b1jX1ij þ b2jX2ij þ εij
Level 2. b0j ¼ g00 þ g01W1j þ g02W2j þ g03W3j þ u0j, b1j ¼
Self-perceived competence 3.40 0.44 3.31 0.58 3.06 0.55
g10, b2j ¼ g20,where Yij is the outcome variable score of student i in
Intrinsic value 3.55 0.76 3.29 0.71 3.08 0.87 group j, b0j is the mean for the jth group, b1j and b2j are the effects of
Utility value 4.41 0.55 4.32 0.65 4.16 0.55 the grand mean centered Level 1 covariates (school grades and
Note. HSCL ¼ high-structure cooperative learning. LSCL ¼ low-structure cooperative
teamwork preferences), εij is the residual Level 1 error term, g00 is
learning. GPs ¼ group presentations. the common intercept across groups, g01 is the fixed effect repre-

Table 3
Correlations among study variables.

1 2 3 4 5 6 7 8

1. Knowledge Test for Unit 1 -.-

2. Knowledge Test for Unit 2 .398 -.-
3. Knowledge Test for Unit 3 .523 .535 -.-
4. Self-perceived competence .247 .206 .250 -.-
5. Intrinsic value .015 .032 .081 .448 -.-
6. Utility value -.083 .009 .032 .115 .408 -.-
7. Final school grade .289 .275 .307 .093 .037 .001 -.-
8. Team Player Inventory -.076 -.059 -.116 .091 .154 .103 .171 -.-

Note. Coefficients in italics are statistically significant at the .05 level.

software package (Muthe n & Muthe n, 1998e2015). Preliminary
analyses revealed the following (cf. Tables 2 and 3): School grades
did not differ between conditions (p ¼ .15) but were positively
related (p < .001) to each of the three knowledge tests. Group size
(p < .001) and preferences for team work (p ¼ .035) both differed
between conditions. In the inferential tests reported hereafter, we
thus included school grades, teamwork preferences, and group size
as covariates.
Across study variables, the portion of missing data varied be-
tween 0% and 20%. The main reason for missingness was that the
attendance policy of our home university required students to
attend a class regularly but allowed each student to be absent from
class twice during a term. Attendance rate did not differ between
conditions (p ¼ .303). Missing values were estimated from the
statistical model described below using the multiple imputation
procedure implemented in Mplus (see van Buuren, 2011).
To test our hypotheses, we adopted a two-level approach with
participants (Level 1) nested in small groups (Level 2). For students
in GP classes, group membership was defined in terms of the group
of students who had together prepared and delivered in class one
presentation.2 To estimate intraclass correlation coefficients (ICCs),
we ran for each set of variables an unconditional model. For
knowledge tests, ICCs were .249 (Test 1), .210 (Test 2), and .225 (Test
3). For self-report measures, ICCs were .214 (self-perceived
2
We also considered to analyze the data with a partial nesting approach (see
Sterba et al., 2014), in which random effects for nesting at the small-group level
would be estimated only for the two cooperative conditions. Yet, because students
in control classes temporarily worked together in teams to deliver a group pre-
sentation, we considered it more adequate to employ a fully nested approach.
senting the difference between CL students and GP students, g02 is
the fixed effect representing the difference between HSCL and LSCL
students, g03 is the Level 2 effect of the grand mean centered group
size covariate, u0j is the residual error term at the group level, and
g10 and g20 are pooled within groups regression slopes (equal
across groups). We ran all analyses with the “Two-Level” command
implemented in Mplus and computed the coefficients with the
maximum likelihood estimator with robust standard errors (MLR).
To determine the magnitude of the observed effects, we
computed Hedges' g (adjusted group difference divided by unad-
justed pooled within-group SD) according to the standards speci-
fied in the What Works Clearinghouse Procedures and Standards
Handbook (Version 3.0, Appendix F; U.S. Department of Education,
2013).
3.2. Conceptual knowledge
Across the three tests, students' knowledge scores were sub-
stantially interrelated (see Table 3). We thus included these inter-
test correlations in a multivariate two-level analysis of variance.
The null hypothesis that across the three tests, the coefficients for
differences between conditions were simultaneously equal to 0 was
rejected by a Wald test of parameter constraints, c2(6,
N ¼ 259) ¼ 45.934, p < .001. Together, the predictors listed in
Table 4 accounted for 24.6% (Test 1), 24.0% (Test 2), and 29.8% (Test
3) of the variance. Neither teamwork preferences nor group size
predicted participants' knowledge, but school grades did. Each of
the two coded contrasts yielded a significant effect on each of the
three knowledge tests.
One further Wald test revealed that differences between
 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84 81

Table 4
Multivariate two-level regression analysis: Results for knowledge tests.

B SE Z p¼

Test 1
Intercept b0j 7.886 0.259 30.478 .000
Final school grade b1j 1.711 0.402 4.259 .000
Team Player Inventory b2j 0.265 0.373 0.710 .478
C1 (HSCL/LSCL vs. GPs) g01 2.611 0.487 5.361 .000
C2 (HSCL vs. LSCL) g02 2.171 0.472 4.600 .000
Group size g03 1.040 0.737 1.411 .158
Test 2
Intercept b0j 4.917 0.236 20.835 .000
Final school grade b1j 1.308 0.279 4.690 .000
Team Player Inventory b2j 0.277 0.283 0.981 .327
C1 (HSCL/LSCL vs. GPs) g01 2.283 0.394 6.544 .000
C2 (HSCL vs. LSCL) g02 1.256 0.424 2.968 .003
Group size g03 0.883 0.556 1.588 .112
Test 3
Intercept b0j 6.238 0.314 19.896 .000
Final school grade b1j 1.724 0.365 4.722 .000
Team Player Inventory b2j 0.553 0.320 1.730 .084
C1 (HSCL/LSCL vs. GPs) g01 2.673 0.417 6.414 .000
C2 (HSCL vs. LSCL) g02 2.018 0.443 4.552 .000
Group size g03 0.593 1.159 0.512 .609

Residual Variance
Within
Test 1 s2rij 7.471 0.774 9.655 .000
Test 2 s2rij 4.759 0.572 8.321 .000
Test 3 s2rij 6.250 0.647 9.664 .000
Between
Test 1 s2yij 1.569 1.323 1.186 .235
Test 2 s2yij 1.380 0.936 1.474 .141
Test 3 s2yij 0.332 1.259 0.264 .792

NoteHSCL ¼ high-structure cooperative learning. LSCL ¼ low-structure cooperative learning. GPs ¼ group presentations. C1, C2 ¼ coded contrasts.

conditions did not vary significantly from Test 1 to Tests 2 and 3, Table 5
c2(4, N ¼ 259) ¼ 5.214, p ¼ .266. CL students generally out- Overview of effect size estimates (Hedge's g).

performed GP students but HSCL students did better on each test HSCL vs. GPs LSCL vs. GPs HSCL vs. LSCL
than LSCL students. Using GP students as reference group, the Knowledge test for Unit 1 1.25 0.57 0.72
magnitude of the associated effects was medium-sized for LSCL but Knowledge test for Unit 2 1.23 0.84 0.49
more substantial for HSCL (see Table 5). Knowledge test for Unit 3 1.46 0.65 0.68
Self-perceived competence 0.73 0.46 0.20
Intrinsic value 0.63 0.30 0.38
Utility value 0.51 0.31 0.16
3.3. Self-perceived competence
Note. HSCL ¼ high-structure cooperative learning. LSCL ¼ low-structure cooperative
The same set of predictors accounted for 8.9% of the variance in learning. GPs ¼ group presentations.

students' self-perceived competence. In addition to school grades,

team player scores predicted these self-ratings (see Table 6). Of the
two contrasts representing differences between conditions, only
Contrast 1 yielded a significant result. Students in the (two pooled)
CL conditions estimated their subject-specific competence more
favorably than GP control students.
3.4. Subjective task values
Because appraisals of intrinsic value and utility value were
significantly interrelated (see Table 2), we analyzed these variables
with a multivariate two-level approach. A Wald test of parameter
constraints indicated that the overall condition effect was
(marginally) significant, c2(4, N ¼ 259) ¼ 9.212, p ¼ .056. The set of
predictors displayed in Table 7 accounted for 8.8% (intrinsic value)
and 4.7% (utility value) of the variance, respectively. Teamwork
preferences were positively related to each of the two value ratings.
Different from Contrast 2, Contrast 1 had a significant effect on each
rating. Students in cooperative groups estimated both the intrinsic
value and the utility value of their learning tasks more favorably
than GP students.
4. Discussion
Generally, the reported results showed that relative to students
in the GP comparison condition, students in the two cooperative
conditions gained a better understanding of the topics treated in
class, developed a stronger sense of confidence in their subject-
specific competence, and learned to associate the tasks they had
been working on with greater enjoyment and utility value. More
specifically, the above results showed that the implementation of
structure-building elements highlighting the importance of indi-
vidual accountability and participant interdependence (coopera-
tion-fostering tasks; feedback addressing individual contributions
to group work; group rewards based on individual achievements)
in the cooperative setting (HSCL) facilitated the acquisition of new
knowledge relative to a setting in which these elements were
removed (LSCL). Yet this superiority of HSCL, relative to LSCL, did
not generalize to students' self-assessments, in the sense that it
failed to produce improvements in students' self-perceived
competence.
A multivariate analysis revealed that in each of the three
82 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84

Table 6
Two-level regression analysis: Results for self-perceived competence.

B SE Z p¼

Intercept b01 3.253 0.045 72.102 .000

Final school grade b1j 0.130 0.067 1.935 .053
Team Player Inventory b2j 0.160 0.061 2.606 .009
C1 (HSCL/LSCL vs. GPs) g01 0.317 0.115 2.755 .006
C2 (HSCL vs. LSCL) g02 0.104 0.128 0.812 .417
Group size g03 0.025 0.061 0.404 .686

Residual Variance
Within s2rij 0.273 0.029 9.351 .000
Between s2yij 0.053 0.023 2.306 .021

Note. HSCL ¼ high-structure cooperative learning. LSCL ¼ low-structure cooperative learning. GPs ¼ group presentations. C1, C2 ¼ coded contrasts.

Table 7
Multivariate two-level regression analysis: Results for subjective task values.

B SE Z p¼

Intrinsic value
Intercept b01 3.306 0.062 53.403 .000
Final school grade b1j 0.140 0.090 1.567 .117
Team Player Inventory b2j 0.308 0.096 3.193 .001
C1 (HSCL/LSCL vs. GPs) g01 0.382 0.162 2.363 .018
C2 (HSCL vs. LSCL) g02 0.279 0.160 1.738 .082
Group size g03 0.068 0.088 0.770 .442
Utility value
Intercept b01 4.298 0.045 96.263 .000
Final school grade b1j 0.039 0.089 0.438 .661
Team Player Inventory b2j 0.166 0.087 1.899 .058
C1 (HSCL/LSCL vs. GPs) g01 0.235 0.117 2.014 .044
C2 (HSCL vs. LSCL) g02 0.098 0.121 0.806 .420
Group size g03 0.046 0.057 0.814 .416
Residual Variance
Within
Intrinsic value s2rij 0.587 0.061 9.567 .000
Utility value s2rij 0.349 0.055 6.332 .000
Between
Intrinsic value s2yij 0.090 0.056 1.611 .107
Utility value s2yij 0.035 0.029 1.222 .222

Note. HSCL ¼ high-structure cooperative learning. LSCL ¼ low-structure cooperative learning. GPs ¼ group presentations. C1, C2 ¼ coded contrasts.

knowledge tests, HSCL students surpassed the achievements of
LSCL students. The same analysis yielded no evidence that the
strength of this difference varied across the three tests. These
findings indicate that structured cooperation had an instant and
continued impact on how well students understood the concepts
presented in the three instructional units. Yet, when the first
knowledge test was administered, CL students had already worked
together in their teams for several weeks. Thus, the findings of this
study do not contradict the view (see Ortiz et al., 1996) that a
certain degree of team experience is necessary before students can
derive substantial benefits from their engagement in CL activities.
Contrary to our prediction, performance differences between
HSCL and LSCL were not mirrored in the participants' estimations of
their own subject-specific competence. Notably, correlations be-
tween objective (knowledge tests) and subjective (self-reported
competence) outcome criteria were anything but substantial (for a
related finding, see Krause et al., 2009). This might have been due to
a range of factors: (a) LSCL students received no feedback as to their
individual achievements and might thus have overestimated the
degree of their knowledge. (b) As reflected in the descriptive sta-
tistics, students' test scores rarely reached the upper quartile of the
performance scale. This lack of peak performance might have
prevented at least some of the HSCL students from developing an
even stronger sense of competence. (c) The influence of third-party
variables (e.g., causal attributions, efficacy expectations, and more
broadly defined academic self-concepts) might have been one
further reason why greater achievements did not directly translate
into more favorable self-evaluations.
Relative to GP classes, participation in cooperative classes had a
positive impact on how students appraised their learning tasks. Yet
task appraisals did not distinguish HSCL from LSCL participants (see
also Berger & Ha €nze, 2009). Remarkably, Moreno (2009) reported
that students in jigsaw groups developed a lower level of situational
interest than students in less structured cooperative groups. The
results of our study do not corroborate the view that the external
structuring of cooperative behavior undermines learners' intrinsic
motives for learning. On the contrary, HSCL students experienced a
relatively greater degree of enjoyment in their course work than
did LSCL students (p ¼ .082, two-tailed).
At first glance, the reported findings are at odds with the meta-
analytic results reported in Section 1.1. However, in these reviews
each of the two cooperative programs described in this paper
would have most likely been categorized under the rubric of
“structured” cooperation. Inspection of effect sizes revealed that,
relative to LSCL, HSCL produced stronger effects, especially on
measures of knowledge acquisition. As Pai et al. (2015) noted,
studies using multiple structuring procedures to enhance students'
small-group learning are rare. We thus believe our data add new
evidence to the CL literature by showing that the implementation
of a well-organized pattern of structuring elements enhances the
efficaciousness of CL methods in promoting the acquisition of new
knowledge in college classes.
 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
4.1. Limitations and prospects
The limitations of this study should be acknowledged. First, this
study had a quasi-experimental design. To increase the internal
validity of the reported findings, it would be desirable (a) to
randomly assign individual students, instead of intact classes, to
conditions; and (b) to arrange the sequence of the instructional
units taught throughout the term in a counterbalanced fashion. Yet,
when a study is intended to take place in the complex reality of
running a college program, it might be anything but easy to meet
these requirements.
Second, we recruited a relatively small number of classes but
took care that the same three lecturers delivered the instructions in
each of the three conditions. However, an increase in the number of
participating classes would make it possible to establish in the data
analysis an even more informative three-level model, with students
nested in groups and groups nested in classes. In this way the
predictive (two-level) model we used in our present work could be
expanded to integrate, as Level-3 covariates, instructor character-
istics, class size differences, and any other contextual features
associated with the composition of classes.
Third, further research is needed to scrutinize the stability of the
reported effects in at least two respects. This research should
determine (a) whether structured cooperation makes students
more likely to generalize and apply their newly acquired knowl-
edge over extended intervals and across a broad range of tasks and
situations; and (b) whether students participating in structured CL
programs learn, at least after a while, to organize their teamwork
more independently, and thus do not depend on external prompts
and instructional regulations to make their cooperative projects
succeed.
Fourth, there is a need to test the generalizability (and limita-
tions) of the above findings across different groups of learners. For
instance, Renkl et al. (1996) argued that in a lecture-based envi-
ronment, college students often adopt a passive learning style and
thus feel skeptical that active engagement in team learning will be
worth the effort. One might speculate that exactly this group of
students could benefit most from a cooperative approach focusing
on highly structured methods. Conversely, students who are more
proficient in teamwork might be less reliant on external measures
of structuring but benefit most from a collaborative approach that
lets them decide on their own how they would like to organize
their team projects. Although our exploratory analysis of moder-
ating influences 3 did not yield any evidence that teamwork pref-
erences qualified the observed differences between the two
cooperative conditions, it would be instructive to integrate into our
experimental design (at least) one further condition in which stu-
dents receive no detailed instructions on how they should collab-
orate in the organization of their teamwork.
Fifth, although high-structure cooperation did not produce a
negative effect on students' task enjoyment, we did not directly
address in our present work the potential costs associated with
structuring methods (see Phipps et al., 2001). There is a need to
explore in greater detail whether specific kinds of structure-
building techniques can have undesired side effects on different
types of social-emotional outcome criteria. For instance, one might
3 assessing: A revision of Bloom's taxonomy of educational objectives. New York:
In follow-up analyses, we explored whether the strength of the observed
condition effects varied as a function of final school grades, participant gender, and
teamwork preferences. Using the “Two Level Random” command in Mplus, we thus
expanded our predicting model to a random intercept/random slope model. With
one exception, these tests for moderator effects failed to be significant. The
exception was that final school grades were unrelated to how much knowledge
students acquired in GP seminars but reliably (p < .05) predicted how well CL
students did at each of the three knowledge tests.
83
speculate that an overemphasis on mutual interdependence pro-
duces social pressure and thereby leads to interpersonal conflict
within a group. Similarly, an overemphasis on individual account-
ability might prevent learners from developing a sense of cohe-
siveness and personal identification with their team. Accordingly,
future researchers should carefully analyze both the benefits and
costs associated with different methods of structuring students'
teamwork in college classes.
Sixth, in the current analysis we did not address the issue of how
the observed effects of instructional conditions were transmitted to
students' achievements and learning experiences. To identify such
mediating processes, it would be fruitful to administer in differ-
ently designed cooperative settings (a) experimental measures
assessing the team members' mental effort (or “cognitive load”)
during the co-construction of new knowledge (see Moreno, 2009),
and (b) observational measures assessing critical features of the
team members' conversational behavior during the completion of
group tasks (see Khosa & Volet, 2014; and; Weinberger & Fischer,
2006). With respect to the latter, Jurkowski and Ha €nze (2010,
2015) reported that the members of highly productive teams not
only verbalize their own ideas but often also elaborate and advance
the arguments of their partners. It would thus be instructive to test
whether structuring methods make it easier for students to engage
in such “transactive” (see Berkowitz & Gibbs, 1983) forms of
knowledge communication.
5. Conclusion
Despite the noted limitations, our research shows that the
implementation of CL in college classrooms improves the acquisi-
tion of new knowledge, instills a sense of competence in learners,
and mediates the experience that one's academic work is pleasur-
able and useful. Yet this study also shows that to take full advantage
of the CL approach, it is important to integrate a coherent pattern of
structuring methods into the cooperative setting. In essence, such
methods serve to sharpen the sense, among all team members, that
even though they are mutually dependent on each other they
remain responsible for their own learning.
Acknowledgements
This research was supported by the Federal Ministry of Educa-
tion and Research (Grant 01PL12035). It was accepted at the Justus-
Liebig-University as part of the “Successful Entry into College”
program and conducted in partial fulfillment of the first author's
doctoral dissertation. The opinions expressed in this paper are
solely those of the authors and do not represent official views of the
German Education and Research Ministry or the Justus-Liebig-
University.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http://
dx.doi.org/10.1016/j.learninstruc.2017.03.006.
References
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and
Longman.
Archer-Kath, J., Johnson, D. W., & Johnson, R. T. (1994). Individual versus group
feedback in cooperative groups. The Journal of Social Psychology, 134, 681e694.
http://dx.doi.org/10.1080/00224545.1994.9922999.
€nze, M. (2009). Comparison of two small-group learning methods in
Berger, R., & Ha
12th-grade physics classes focusing on intrinsic motivation and academic per-
formance. International Journal of Science Education, 31, 1511e1527. http://
dx.doi.org/10.1080/09500690802116289.
84 M. Supanc et al. / Learning and Instruction 50 (2017) 75e84
Berkowitz, M. W., & Gibbs, J. C. (1983). Measuring the developmental features of
moral discussion. Merrill-Palmer Quarterly, 29, 399e410.
Braun, E., Gusy, B., Leidner, B., & Hannover, B. (2008). Das Berliner Eval-
uationsinstrument für selbsteingesch€ atzte, studentische Kompetenzen (BEv-
aKomp) [Berlin instrument for evaluation of students' self-rated competencies].
Diagnostica, 54, 30e42. http://dx.doi.org/10.1026/0012-1924.54.1.30.
van Buuren, S. (2011). Multiple imputation of multilevel data. In J. J. Hox, &
J. K. Roberts (Eds.), Handbook of advanced multilevel analysis (pp. 173e196).
Milton Park, England: Routledge. http://dx.doi.org/10.4324/
9780203848852.ch10.
Cohen, M. W. (2010). Cooperative learning in educational psychology: Modeling
success for future teachers. In B. J. Millis (Ed.), Cooperative learning in higher
education (pp. 69e89). Sterling, VA: Stylus Publishing.
Davidson, N., & Major, C. H. (2014). Boundary crossing: Cooperative learning,
collaborative learning, and problem-based learning. Journal on Excellence in
College Teaching, 25, 7e55.
Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human
needs and the self-determination of behavior. Psychological Inquiry, 11,
227e268. http://dx.doi.org/10.1207/S15327965PLI1104_01.
Johnson, D. W., & Johnson, R. T. (1989). Cooperation and competition: Theory and
research. Edina, MN: Interaction Book Company.
Johnson, D. W., & Johnson, R. T. (1999). Learning together and alone: Cooperative,
competitive, and individualistic learning (5th ed.). Needham Heights, MA: Allyn
and Bacon.
Johnson, D. W., Johnson, R. T., & Smith, K. A. (2014). Cooperative learning: Improving
university instruction by basing practice on validated theory. Journal on Excel-
lence in College Teaching, 25, 85e118.
Jürgen-Lohmann, J., Borsch, F., & Giesen, H. (2001). Kooperatives Lernen an der
Hochschule: Evaluation des Gruppenpuzzles in Seminaren der Pa €dagogischen
Psychologie [Cooperative learning at the university: An evaluation of jigsaw in
classes of educational psychology]. Zeitschrift für Pa €dagogische Psychologie, 15,
74e84. http://dx.doi.org/10.1024//1010-0652.15.2.74.
Jurkowski, S., & Ha €nze, M. (2010). Soziale Kompetenzen, transaktives Inter-
aktionsverhalten und Lernerfolg: Experimenteller Vergleich zweier unter-
schiedlich gestalteter Gruppenunterrichtsbedingungen und Evaluation eines
transaktivit€ atsbezogenen Kooperationsskriptes [Social competences, trans-
active interaction, and achievement: Comparison of two differently designed
cooperative learning environments and evaluation of scripting transactive
interaction]. Zeitschrift für Pa €dagogische Psychologie, 24, 241e257. http://
dx.doi.org/10.1024/1010-0652/a000020.
Jurkowski, S., & H€ anze, M. (2015). How to increase the benefits of cooperation:
Effects of training in transactive communication on cooperative learning. British
Journal of Educational Psychology, 85, 357e371. http://dx.doi.org/10.1111/
bjep.12077.
Kalaian, S. A., & Kasim, R. M. (2014). A meta-analytic review of studies of the
effectiveness of small-group learning methods on statistics achievement.
Journal of Statistics Education, 22. Retrieved from: http://www.amstat.org/
publications/jse/v22n1/kalaian.pdf.
Khosa, D. K., & Volet, S. E. (2014). Productive group engagement in cognitive activity
and metacognitive regulation during collaborative learning: Can it explain
differences in students' conceptual understanding? Metacognition and Learning,
9, 287e307. http://dx.doi.org/10.1007/s11409-014-9117-z.
Kline, T. J. B. (1999). The team player inventory: Reliability and validity of a measure
of predisposition toward organizational team-working environments. The
Journal for Specialists in Group Work, 24, 102e112. http://dx.doi.org/10.1080/
01933929908411422.
Krause, U.-M., Stark, R., & Mandl, H. (2009). The effects of cooperative learning and
feedback on e-learning in statistics. Learning and Instruction, 19, 158e170.
http://dx.doi.org/10.1016/j.learninstruc.2008.03.003.
Kyndt, E., Raes, E., Lismont, B., Timmers, F., Cascallar, E., & Dochy, F. (2013). A meta-
analysis of the effects of face-to-face cooperative learning: Do recent studies
falsify or verify earlier findings? Educational Research Review, 10, 133e149.
http://dx.doi.org/10.1016/j.edurev.2013.02.002.
MacKinnon, D. P., Taborga, M. P., & Morgan-Lopez, A. A. (2002). Mediation designs
for tobacco prevention research. Drug and Alcohol Dependence, 68, 69e83.
http://dx.doi.org/10.1016/S0376-8716(02)00216-8.
Millis, B. J. (2010). Why faculty should adopt cooperative learning approaches. In
B. J. Millis (Ed.), Cooperative learning in higher education (pp. 1e10). Sterling, VA:
Stylus Publishing.
Moreno, R. (2009). Constructing knowledge with an agent-based instructional
program: A comparison of cooperative and individual meaning making.
Learning and Instruction, 19, 433e444. http://dx.doi.org/10.1016/
j.learninstruc.2009.02.018.
Muthe n, L. K., & Muthen, B. O. (1998-2015). Mplus user's guide (7th ed.). Los Angeles,
CA: Muthe n & Muthe n.
Niemi, H. (2002). Active learning e a cultural change needed in teacher education
and schools. Teaching and Teacher Education, 18, 763e780. http://dx.doi.org/
10.1016/S0742-051X(02)00042-2.
Ortiz, A. E., Johnson, D. W., & Johnson, R. T. (1996). The effect of positive goal and
resource interdependence on individual performance. The Journal of Social
Psychology, 136, 243e249. http://dx.doi.org/10.1080/00224545.1996.9713998.
Pai, H.-H., Sears, D. A., & Maeda, Y. (2015). Effects of small-group learning on
transfer: A meta-analysis. Educational Psychology Review, 27, 79e102. http://
dx.doi.org/10.1007/s10648-014-9260-8.
Panitz, T. (1999). Collaborative versus cooperative learning: Comparing the two
definitions helps understand the nature of interactive learning. Cooperative
Learning and College Teaching, 8. Retrieved from: http://files.eric.ed.gov/fulltext/
ED448443.pdf.
Phipps, M., Phipps, C., Kask, S., & Higgins, S. (2001). University students' perceptions
of cooperative learning: Implications for administrators and instructors. The
Journal of Experimental Education, 24, 14e21. http://dx.doi.org/10.1177/
105382590102400105.
Renkl, A., Gruber, H., & Mandl, H. (1996). Kooperatives problemorientiertes Lernen
in der Hochschule [Cooperative problem-oriented learning in higher educa-
tion]. In J. Lompscher (Ed.), Lehr- und Lernprobleme im Studium. Bedingungen
und Vera €nderungsmo €glichkeiten (pp. 131e147). Bern: Huber.
Ruys, I., van Keer, H., & Aelterman, A. (2010). Collaborative learning in pre-service
teacher education: An exploratory study on related conceptions, self-efficacy,
and implementation. Educational Studies, 36, 537e553. http://dx.doi.org/
10.1080/03055691003729021.
Slavin, R. E. (1983). When does cooperative learning increase student achievement?
Psychological Bulletin, 94, 429e445. http://dx.doi.org/10.1037//0033-
2909.94.3.429.
Slavin, R. E. (1995). Cooperative learning: Theory, research, and practice (2nd ed.).
Boston: Allyn & Bacon.
Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on
undergraduates in science, mathematics, engineering, and technology: A meta-
analysis. Review of Educational Research, 69, 21e51. http://dx.doi.org/10.3102/
00346543069001021.
Sterba, S. K., Preacher, K. J., Forehand, R., Hardcastle, E. J., Cole, D. A., & Compas, B. E.
(2014). Structural equation modeling approaches for analyzing partially nested
data. Multivariate Behavioral Research, 49, 93e118. http://dx.doi.org/10.1037/
a0035628.
Tomcho, T. J., & Foels, R. (2012). Meta-analysis of group learning activities: Empir-
ically based teaching recommendations. Teaching of Psychology, 39, 159e169.
http://dx.doi.org/10.1177/0098628312450414.
U.S. Department of Education. (2013). What works clearinghouse: Procedures and
standards handbook (Version 3.0). Retrieved from: http://ies.ed.gov/ncee/wwc/
pdf/reference_resources/wwc_procedures_v3_0_standards_handbook.pdf.
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative
knowledge construction in computer-supported collaborative learning. Com-
puters & Education, 46, 71e95. http://dx.doi.org/10.1016/
j.compedu.2005.04.003.