This article was downloaded by: [Staffordshire University]

On: 06 October 2014, At: 14:23

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Educational Research and Evaluation:

An International Journal on Theory and
Practice
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/nere20

Explaining Achievement in Higher

Education
Ellen PWA Jansen & Marjon Bruinsma
a
GION/dept. COWOG , University of Groningen , The Netherlands
Published online: 16 Feb 2007.

To cite this article: Ellen PWA Jansen & Marjon Bruinsma (2005) Explaining Achievement in Higher
Education, Educational Research and Evaluation: An International Journal on Theory and Practice,
11:3, 235-252, DOI: 10.1080/13803610500101173

To link to this article: http://dx.doi.org/10.1080/13803610500101173

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 Educational Research and Evaluation
Vol. 11, No. 3, June 2005, pp. 235 – 252

Explaining Achievement in Higher

Education
Ellen P.W.A. Jansen* and Marjon Bruinsma
GION/dept. COWOG, University of Groningen, The Netherlands
Downloaded by [Staffordshire University] at 14:23 06 October 2014

(Received 21 October 2002; accepted 30 March 2004)

This research project investigated the relationship between students’ pre-entry characteristics,
perceptions of the learning environment, reported work discipline, the use of deep information
processing strategies, and academic achievement. Ability measured by grade-point average in pre-
university education was the most important predictor of achievement, followed by work discipline,
age, and gender. Involvement, work discipline, and perceived course difficulty affected the
perception of the quality of the instructor/course. Surprisingly, the perception of the quality of the
instructor/course was negatively related to perceived difficulty. The use of deep information
processing strategies did not result in higher grades.

Introduction
Educational effectiveness has always been an important focus of attention in higher
education research and policy. In The Netherlands, for example, government
financial support for institutions was based on the number of students who pass the
first-year examination as well as on the number of students who graduate. From this
point of view, studying those aspects that enhance academic achievement or prevent
students from dropping out, is important.
The question why certain students decide to drop out whereas others decide to
persist has been a starting-point for many studies in higher education (e.g., Bean,
1980; Bean & Metzner, 1985; Bijleveld, 1993; Pascarella, 1980; Prins, 1997; Tinto,
1975, 1987). These studies have shown that achievement, or rather dropout, is the
result of interactions between the student and departmental and environmental
characteristics.
Most of these studies have proceeded from theories that emphasise the integration
of students in higher education to explain academic achievement or dropout. The

*Corresponding author. *Academic Centre for Teaching and Learning, University of Groningen,
P.O. Box 800, 9700 AV Groningen, The Netherlands. E-mail: e.p.w.a.jansen@.rug.nl
ISSN 1380-3611 (print)/ISSN 1744-4187 (online)/05/030235–18
ª 2005 Taylor & Francis Group Ltd
DOI: 10.1080/13803610500101173
 236 E. P. W. A. Jansen and M. Bruinsma

first theory to focus on the interaction approach was that of Spady (1971). In his
model, Spady argued that social integration was influenced by, among other things,
study results and ‘‘normative congruence,’’ that is, the interactions between a
complex pattern of ability, aptitude, interests, goals, values and expectations, the
support of friends and mutual group values. Dropout was seen as a longitudinal
process resulting from a combination of lack of commitment and study results. Tinto
(1975, 1987) elaborated on Spady’s theory by proposing that both student
characteristics and the interactions with the social as well as the academic
environments influenced a student’s decision to persist or to withdraw from college.
Based on these theories, Pascarella (1980) emphasised the importance of informal
contact between student and teacher. Bean (1980) developed a meta-model of
dropout, which focussed on the importance of students’ background variables, those
Downloaded by [Staffordshire University] at 14:23 06 October 2014

directly influencing social and academic integration as well as environmental

variables. Webb (as cited in Prins, 1997) concluded that most of these explanatory
dropout models are not applicable to non-campus institutions where students have
fewer opportunities for social contact with peers, such as studying together, informal
parties, et cetera. Basing himself on research at a non-campus institute, he introduced
the term ‘‘perceived student/institution fit’’ as an alternative for the psychological
outcomes utility, satisfaction, goal commitment, and stress specified by Bean (1980),
and added the term ‘‘academic self-confidence’’ to the model.
In The Netherlands, these studies were soon elaborated on. Bijleveld (1993), for
example, specified a model that stressed both student and departmental character-
istics, such as the amount of problem-based learning, the number of interactive
teaching methods, the spread of the study load, and the spread of tests.
Instead of taking dropout as an outcome measure, Jansen (1996, 2004) investigated
academic achievement at six departments of the University of Groningen. In this
study, she examined the relationship between curriculum organisation, such as
planning/scheduling, instruction and assessment, and academic achievement.
Achievement was measured in terms of the number of credits obtained after 1 and
after 2 years, and passing the first-year examination. In accordance with earlier
findings (Jansen, 1993a, 1993b; Van der Drift & Vos, 1987), and based on the Carroll
model (1963) and Creemers’ model (1994), achievement measured by the number of
credits obtained after 1 year or by passing the first-year examination was influenced
by measures that affected students’ planning behaviour and encouraged a positive
attitude towards study. For instance, scheduling fewer subjects simultaneously, and
scheduling obligatory assignments during a course, which included feedback from the
instructor, stimulated students to spend more time on study activities. Prins (1997)
used Bijleveld’s model, which he extended with variables on academic and social
integration. Prins concluded that study counselling, quality of tests, spread of study
load, the number of contact hours, and student-centred education influenced the
dropout or retention rates.
De Jong, Roeleveld, Webbink, and Verbeek (1997) performed a nationwide
research on study careers in higher education in The Netherlands. In their
‘‘Amsterdam model of study careers,’’ they integrated human capital theory and
 Explaining Achievement 237

integration theory to explain academic achievement. However, they did not account
for factors on the departmental level in their analyses.
Thus, many studies have examined the factors that influence academic achieve-
ment in and dropout from higher education. Yet, these studies have paid little
attention to factors at the organisational or departmental level. Furthermore, besides
the effects of variables on organisational or departmental levels, another missing link
in these theories is the role of actual study behaviour. Berger and Milem (1999), for
example, stated that most of the existing literature on the integration model has
focussed on the perceptual components of academic and social integration while
ignoring measures of behavioural components. The results of their study confirmed
the use of a combined model, which accounted for both behavioural and perceptual
components to describe the persistence process during the 1st year of college.
Downloaded by [Staffordshire University] at 14:23 06 October 2014

According to Berger and Milem (1999), previous research into first-year retention has
underestimated the role of involvement or lack of involvement in relation to student
persistence. Related to that, another wide body of research concerns students’
approaches to learning, students’ perceptions of the learning environment and
academic performance (see e.g., Beck & Davidson, 2001; Lindblom-Ylänne &
Lonka,1999; Ramsden, 2003; Richardson, 2000; Trigwell & Prosser, 1991). Marton
and Säljö (1997) reported that students with a deeper approach to learning tend to
achieve a higher quality outcome of learning.
In our study we try to combine parts of Tinto’s (1987) theories by focussing on
student characteristics and the interactions with the academic environment, the
Dutch approaches by including departmental factors like ratings of the learning
environment, and the recommendations by Berger and Milem (1999) by
incorporating involvement and study behaviour in the theoretical model. In this
way, this study investigates the influence of factors at the student as well as at the
learning environment level that are related to study success. The present study
focusses on the question to which extent academic achievement, measured by the
grade-point average after 1 year, is influenced by students’ characteristics, students’
study behaviour, and students’ perceptions of the course environment, which, in
effect, indicates the quality of that environment. More specifically, important factors
in a course or module include the instructor, the course study load and, naturally,
students’ involvement in the course. This approach seems to match the learning and
teaching model by Trigwell and Prosser (2003, p. 207).

Research Questions
This article focusses on the question: ‘‘What is the relationship between students’
pre-entry characteristics, students’ perceptions of the learning environment, study
behaviour, and outcome measured by the mean grade-point average after 1 year?’’
This main problem is divided into the following questions:

1. What is the relationship between students’ pre-entry characteristics and their

perceptions of the learning environment?
 238 E. P. W. A. Jansen and M. Bruinsma

2. What is the relationship between students’ perceptions of the learning

environment and their actual behaviour in terms of (a) work discipline and (b)
deep information processing strategy?
3. What is the relationship between students’ pre-entry characteristics, students’
perceptions of the learning environment, and achievement in terms of grade-
point average after one year?

Theoretical Model
Figure 1 schematically illustrates the theoretical model. We expect that gender,
ability, and the-beginning-of- the year’s low work discipline directly affect
achievement. Shah and Burke (1999), Need and De Jong (1998), Smith and Naylor,
Downloaded by [Staffordshire University] at 14:23 06 October 2014

(2001), Van den Berg (2002), Richardson and Woodley (2003), and Simonite
(2003), for example, showed that women and students with higher ability achieved
higher results. Further, we expect ability to affect students’ end-of-the-year work
discipline, their perceived difficulty, and their involvement. Low work discipline
seems to be a trait that is rather difficult to change by the organisation of the
curriculum or the instruction. Students with low beginning-of-the-year work
discipline are expected to show low work discipline at the end of the year and to
be less involved with their study. We expected students with higher GPA in secondary
education to use the appropriate study strategies. Studies have shown that female
students show more discipline (cf. Macan, Shahani, Dipboye, & Phillips, 1990).
Therefore, we expect a direct effect of gender on discipline. The way age will
influence achievement is rather unclear. Age as predictor of academic attainment is
often motivated by a stereotype of older people being deficient in intellectual skills
(Richardson & King, as cited in Richardson & Woodley, 2003). Yet, we can argue
that older students often have more study experience and are therefore better

Fig. 1. The scheme of the theoretical model.

 Explaining Achievement 239

disciplined and better able to use the appropriate deep information processing
strategies.
We expect a relationship between the quality of instruction, perceived difficulty,
involvement, the end-of-the-year’s work discipline, and the outcome variable; that is,
students who rate the course and instructor positively will have better results. This
assumption is based, among other things, on the studies of Perry (for a review, see
Perry, 1997), which reported that instructor expressiveness and lecture content had a
significant effect on student ratings of instruction and on student achievement.
A similar relationship between learning approaches and the perception of the
learning environment has been demonstrated by Ramsden and Entwistle (1981) and
Trigwell and Prosser (1991), among others, who showed a relationship between
perceptions of good teaching and a deep approach to study.
Downloaded by [Staffordshire University] at 14:23 06 October 2014

Besides these effects on the deep information processing strategies, we expect

effects among the perceptions of the learning environment variables. Students who
rate the course difficulty as sufficient and who are more involved, will be more
satisfied with their study. Further, Perry (1997) indicated a relationship between
perceived control and instruction. Perceived difficulty can be seen as a proxy of
perceived control. According to Perry (1997), students who suffer loss of control are
less able to benefit from effective instruction. Therefore, we expect students who
experienced the course as difficult and time-consuming to give lower ratings of the
quality of the course and instructor and to be less involved in the course.
Students who evaluate the course and instructor positively are expected to have
a higher score for work discipline. Because deep information processing is seen as
a desired learning strategy, we expect deep information processing strategies to
affect the outcome variables. Finally, we assume that work discipline at the end of
the year is strongly related to GPA at the end of the 1st year. Work discipline is
related to time management behaviour and Macan et al., (1990), for example,
found significant relationships between time management behaviour, self-rated
performance, and GPA.

Method
Sample
Research projects on the theme of the effectiveness of the curriculum and instruction
in Higher Education have been a main focus of attention at the University of
Groningen for some years. The data used in the study reported here were collected
within this framework. Four departments, one from the Faculty of Arts, one from the
Faculty of Sciences and two from the Faculty of Economics and Business,
participated in this research project and data were collected during the academic
years 1999/2000 and 2000/2001. This article reports analyses of the students from the
department in the Faculty of Arts. Two cohorts of 1st-year students (N 1999 = 147,
male = 41, female = 106 and N2000 = 149, male = 37, female = 112) in this department
were asked to fill in the questionnaires.
 240 E. P. W. A. Jansen and M. Bruinsma

Three types of data were collected during the year: (a) student ratings of seven
courses, collected after each course; (b) data on general study behaviour, collected at
the beginning and at the end of the year, and (c) study achievement data, which we
obtained from the administration system. The students were asked to fill in their
student identification number, and because we guaranteed their anonymity many
students did so. This resulted in a unique database with data on student ratings, study
behaviour and achievement, which can be linked at the student level.

Variables and Instruments

Student pre-entry variables. At the student level, we collected data on three student-
background variables, namely gender, age, and mean grade-point average at pre-
Downloaded by [Staffordshire University] at 14:23 06 October 2014

university education as a measurement of ability. These three variables were obtained

from the student administrations.
Further, two general study behaviour variables were included in the study. The first
variable, ‘‘low work discipline,’’ was included in the analyses because time on task is
one of the most important contributors to achievement, and work discipline will
influence the time on task (e.g., Carroll, 1963; Creemers, 1994). Low work discipline
can also be seen as a motivational component. In addition to work discipline, deep
information processing strategies can be of great significance for achievement and
were included in the data collection. Deep information processing is seen as a
desirable approach to studying.Terenzini (1999, p.36), for example, states that
‘‘. . .[Learning] requires time for reflection, consolidation, and internalisation if it is to
be long lasting, ‘deeper learning’.’’ Despite this fact, in reality students content
themselves with a more surface approach to learning. This might be related to their
own conceptions or motivations towards studying, but more importantly it might be
related to the teaching or the assessment procedures.
These two variables were obtained by self-report questionnaires. Students filled in
a self-report questionnaire on two occasions, at the beginning and at the end of the 1st
year. The researcher handed out the questionnaire during lectures and tutorials.
Because not all students attended all lectures and tutorials, this procedure yielded a
total response rate of 49.8%.
The self-report questionnaire consisted of 76 items on student motivation and deep
information processing strategies. These 76 items were based on three questionnaires
from the Centre for Study Support Groningen: the ‘‘Checklist Study Motivation’’
(Schouwenburg & Stevens, 1996), the ‘‘Questionnaire for Study Problems’’
(Schouwenburg, 1996), and the ‘‘Test for Deep Information Processing’’ (Schou-
wenburg & Schilder, 1996).
We used two scales from these questionnaires in the analyses: the scale ‘‘low
work discipline’’ and the scale ‘‘deep information processing.’’ The first scale,
‘‘low work discipline,’’ consisted of 7 items, the second scale, ‘‘deep information
processing,’’ consisted of three subscales. These three subscales were based on the
24 items from the Test for Deep Information Processing, which was developed to
examine whether students use deep information processing strategies during their
 Explaining Achievement 241

examination of texts. These subscales can be considered as alternatives of three

scales in the ‘‘Leren & Studeren [Learning and Studying]’’ questionnaire by Vorst
(1993), which concern the deep approach to study. The first subscale ‘‘Critical
reading’’ consisted of 9 items, the second subscale ‘‘Broaden one’s context’’
consisted of 8 items, and the final subscale ‘‘Structuring’’ consisted of 7 items. The
answers on these three scales varied on a 5-point Likert scale from 1 ‘‘never ‘‘ to 5
‘‘always.’’ These three scales were combined in a sum score conform the
instructions in the Test for Deep Information Processing manual (Schouwenburg
& Schilder, 1996). Table 1 displays some examples of the items, including the
reliabilities per cohort and over time.

Perceptions of the learning environment. We measured the perceptions of the learning

Downloaded by [Staffordshire University] at 14:23 06 October 2014

environment using student ratings of courses. In The Netherlands, the use of standard
questionnaires to evaluate the quality of education is less widespread than, for
example, in the United States. Most institutions and even most departments use their
own questionnaires. Further, student ratings are sometimes collected with standard
items, and sometimes with course-specific items. However, even though most
questionnaires have specific subjects, the same aspects are globally present in the
majority, that is, most of the questionnaires evaluate the same underlying concepts.
Departments at the University of Groningen use different forms for evaluating their
courses. Fortunately, all departments in the Faculty of Arts use the same form. This
form, which was not designed especially for this study but is a standard departmental
questionnaire, consists of 27 items with most items on a 4-point Likert scale varying
from (4) completely agree to (1) completely disagree. The 1st year in this department,
like most other departments in The Netherlands, consists of an obligatory
programme in which an average of 10 to 15 courses are scheduled. We collected
data on 8 courses in the first cohort and on 10 courses in the second cohort.
However, only the 7 courses that were evaluated in both cohorts, were included in the
analyses. The mean response rate for these courses was 51.5%.
We analysed the data from the evaluations with a factor analysis, principal
components with varimax rotation. The evaluation form contained items that are not

Table 1. Examples of study behaviour questions and reliability coefficients for both cohorts

Reliability Reliability
Variable Example question Cohort 1 Cohort 2

Low work discipline I am always behind the planning .73 .78 .85 .89
DIP ‘‘critical reading’’ I understand the intention of the .80 .71 .70 .71
text fairly quickly
DIP ‘‘broaden one’s context’’ I think of examples myself .82 .79 .72 .75
DIP ‘‘structuring’’ I pay attention to the paragraph .78 .75 .68 .82
divisions in a text

Note. DIP = deep information processing

 242 E. P. W. A. Jansen and M. Bruinsma

applicable to all the courses, for example, ‘‘there was sufficient opportunity to
practice my oral skills.’’ We therefore only included those items with an item response
of over 76% in the factor analysis. This resulted in a six-factor solution, which
explained 58% of the variance. Only those items that loaded .35 or more on a factor
were included. The first factor, ‘‘Instructor/course,’’ indicated satisfaction with the
instructor and the course in general and consisted of six items. The second factor,
‘‘perceived difficulty,’’ indicated the feeling of being able to succeed in the course and
consisted of four items. The third factor, ‘‘Information,’’ consisted of two items and
concerned the information given in advance on subject matter, materials, and the
assessment syllabus. The fourth factor, ‘‘involvement,’’ consisted of three items
concerning students’ involvement. The fifth factor, ‘‘assessment,’’ indicated the
quality of the assessment and consisted of three items. The sixth factor consisted of
Downloaded by [Staffordshire University] at 14:23 06 October 2014

two items and covered nonspecific items. In this study, we used the factors
‘‘instructor/course,’’ ‘‘students’ involvement’’ and ‘‘perceived difficulty’’ in our
analyses to explore the theoretical model. We constructed three indices based on the
factors. The first scale, ‘‘instructor/ course,’’ had a sufficient reliability coefficient
alpha of .84; the coefficients of the scales ‘‘perceived difficulty’’ and ‘‘involvement’’
were rather low (.61 and .47, respectively). The response scale was recoded for
negative items. The indices were reduced to a scale from 1 to 4, where 4 is ‘‘good’’
and 1 is ‘‘poor’’ for the scale on course/instructor and the scale on involvement. The
scale on perceived difficulty is rated from 1 to 5, where 1 is too difficult and 5 is too
easy. The rating score for the three indices was calculated for each course. Further,
we computed a mean score of the three indices by adding up the scores for all seven
courses and dividing this score by seven. Table 2 shows the items in the three indices.

Achievement. The administration of the department provided data on the grades for
each course. These grades varied on a scale from 1 to 10, where 1 is a very bad grade

Table 2. The evaluation indices

Index Item Alpha

Course/instructor The teachers explained the subject very well.

The teachers answered questions adequately.
After the lecture, the teacher was willing to answer questions.
In general the lectures were not sufficiently structured *
In general the course was ..(bad, .., good). .84
Perceived difficulty In comparison with other courses this course was.. (too difficult.. too
easy)
The assignments were generally ..(too difficult.. too easy). .61
Involvement The course design stimulated regular study.
I finished my assignments on time.
I attended .. of the lectures (0 – 25%,. . .., 76 – 100%). .47

* these items have been reversed

 Explaining Achievement 243

and 10 an excellent grade. We computed the grade-point average for the seven
courses (GPA), which was used as the outcome measure. We preferred GPA to, for
example, number of credits obtained because GPA indicates passing the test as well
as the level of achievement.

Analysis
We analysed the different relationships between student pre-entry characteristics,
students’ perceptions of the learning environment, student behaviour and achieve-
ment using a path model. The theoretical model contained four exogenous variables:
gender, age, ability, and the-beginning-of-the-year’s low work discipline score.
Further, it contained six endogenous variables divided into three variables on the
Downloaded by [Staffordshire University] at 14:23 06 October 2014

perception of the learning environment: course/instructor, perceived difficulty, and

student involvement, two student behaviour characteristics: end-of-the-year work
discipline and deep information processing strategies used, and the outcome variable
overall grade-point average.
We analysed the above-mentioned theoretical model using LISREL 8.5. Missing
values were imputed using the ‘‘expectation-maximisation method’’ (EM, Dempster,
Laird, & Rubin, 1977). The model was analysed with the maximum-likelihood
method based on a covariance matrix. We used the root mean square error of
approximation (RMSEA) with a cut-off value of .06 and the non-normed fit index
(NNFI) with a cut-off value of .96 (Hu & Bentler, 1999) to indicate the fit between
the observed data and the hypothesised model.

Results
Zero-Order Correlations
Table 3 shows the zero-order correlation matrix used in the analyses. It shows that
deep information processing is not significantly correlated with the outcome variable.
Therefore, in the path model we expected that deep information processing strategies
would not show strong effects on study outcome. However, because deep information
processing is usually seen as a desired way of studying, we decided to incorporate this
variable in the model.
Table 3 shows strong correlations between ability (r = .64), gender (r = .23),
beginning-of-the-year work discipline (r = -.48), rating of instructor/course (r = .25),
involvement (r = .39), end-of-the-year work discipline (r = -.51), and GPA at the end
of the 1st year. Students with higher GPA in pre-university education, women,
students with higher beginning-of-the-year work discipline, students who attend
classes more often and do their assignments on time, and students with greater end-
of-the-year work discipline obtain higher grades in the courses. We also see moderate
correlations between ability and involvement (r = .27) and ability and work discipline
(r = -.37), between involvement and work discipline (begin r = -.54 and end r = -.57)
and between the ratings of the instructor/course and involvement (r = .34). Because
 244 E. P. W. A. Jansen and M. Bruinsma

Table 3. Zero order correlations of the variables used in the model

Gen Abi Age Wrkdsb Instr Dif Invol Wrkdse Dip

Abi .13
Age -.12 -.33
Wkdsb -.15 -.50 .14
Instr .11 .19 .05 -.27
Dif .06 -.04 -.12 .06 -.15
Invol .20 .27 .16 -.54 .34 .10
Wrkdse -.03 -.37 -.19 .83 -.19 .11 -.57
Dip -.17 -.01 .40 -.12 .10 -.01 .21 -.25
GPA .23 .64 -.09 -.48 .25 -.12 .39 -.51 .07

Note. italics: p 5 .05 (2-tailed), bold italics: p 5 .01 (2-tailed), Gen = gender, Abi = ability, grade-
Downloaded by [Staffordshire University] at 14:23 06 October 2014

point average secondary school, Age = age, Wrkdsb: score low work discipline at the beginning of
the year, Instr = rating instructor/course, Dif = rating perceived difficulty, Invol = rating involve-
ment, Wrkdse = score low work discipline at the end of the year, Dip = score deep information
processing, GPA = Grade-Point Average on the seven courses

the items on the scale work discipline were formulated negatively (see Table 1), the
correlations are negative as well.

The Path Model

Table 4 shows the fit measures, which indicate the fit between the theoretical model
and the observed model. Furthermore, it illustrates the total amount of explained
variance by the models. Although the theoretical model explains the same amount of
variance as the final model (54%), the fit measures do not indicate a good fit. The
RMSEA is .14 and the NNFI is .77.
Based on the modification indices that LISREL generated, the model was adjusted.
The model showed a sufficient fit after four modifications (see Table 4). We added
the following paths: from involvement to course/instructor, from instructor to
perceived difficulty, from gender to deep information processing strategies, and,
finally, from gender to involvement. Table 5 shows the significant standardised direct
and total effects.

Student pre-entry characteristics and students’ perceptions of the learning environment

user3/>In contrast to our expectations, we found total effects of gender on
involvement (.12), on deep information processing strategies (-.14), and on the
rating of the course/instructor (.05). That is, female students were more involved in
their studies than male students, they used less deep information processing
strategies, and they rated the course/instructor lower. Age had no effect on perceived
difficulty but in contrast with our expectations, age did have an effect on involvement
(.30) and on GPA (.18). Furthermore, Table 5 shows a total effect from age on the-
 Explaining Achievement 245

Table 4. Fit measures and explained variance for the theoretical and modified

Model RMSEA NNFI Explained variance

Theoretical .14 .77 54%

Modified .056 .96 54%

Note: RMSEA = root mean square error of approximation, NNFI = non normed fit index

end-of-the-year work discipline (-.32) and the use of deep information processing
strategies (.41). Ability revealed one significant effect on the perceptions of the
learning environment variables. Students with higher entry ability were more positive
on the course/instructor (.04). Students who enter the university with a high GPA in
Downloaded by [Staffordshire University] at 14:23 06 October 2014

pre-university education were more disciplined and achieved, as expected, higher

GPA at the end of the 1st year. Low work discipline at the beginning of the year
affected, as expected, involvement (-.51), work discipline at the end of the year (.86),
and GPA (-.19). The direct effect of beginning-of-the-year low work discipline on
GPA was positive, whereas the total effect, including all indirect effects, was negative
as expected. Furthermore, low work discipline at the beginning of the year did also
affect the rating of the course/instructor (-.20) and the use of deep information
processing strategies (-.09).

Students’ perceptions of the environment and general study behaviour. We expected the
instructor/course to have an effect on work discipline at the end of the year. The total
effect is .10. Besides this expected effect, we found effects of the rating of the course/
instructor on perceived difficulty (-.22) and on involvement (-.05). The expected effect
of perceived difficulty on involvement is .21. Besides this effect, we found an effect of the
perceived difficulty on the rating of the instructor/course (.08). As expected, there was
an effect of involvement on the end-of-the-year work discipline (-.07). This is not
surprising, because the involvement variable is based on study behaviour during the
specific courses and the work discipline variable is related to study behaviour in general.
Finally, we did find an effect of involvement on the rating of the course/instructor (.38),
on deep information processing strategies (.18), and on GPA (.14).

Students’ pre-entry characteristics, perceptions of the environment and achievement.

Considering the total model to explain academic achievement, we find that ability
had the strongest effect (.58). Gender and age contributed moderately to the
explanation of achievement (total effects of .15 and .18, respectively). In contrast to
expectations, we found a significant effect (-.14) of gender on deep information
processing strategies used. Female students used fewer deep information processing
strategies than male students.
At the rating level, only involvement affected achievement, that is, the total effect of
involvement is .14. Neither the rating of the instructor/course nor the perceived
difficulty had a significant effect on GPA. Beginning-of-the-year work discipline
shows an effect on achievement (-.19) but the effect of the end-of-the-year discipline
 246
E. P. W. A. Jansen and M. Bruinsma
Downloaded by [Staffordshire University] at 14:23 06 October 2014

Table 5. Significant standardised direct (D) and total (T) effects

Lwdb Gen Abi Age Instr Dif Invol Lwde Dip

D T D T D T D T D T D T D T D T D T

Instr -.20 .05 .04 .12 -.02 .08 .39 .38

Dif -.01 -.23 -.22 -.02 -.09
Invol -.52 -.51 .13 .12 .31 .30 -.05 .22 .21 -.02
Lwde .82 .86 .08 .07 -.07 -.30 -.32 .09 .10 -.11 -.07
Dip -.09 -.17 -.14 .35 .41 .04 .18 .18
GPA .25 -.19 .17 .15 .54 .58 .18 .14 -.45 -.45

Note: Lwdb = score low work discipline at the beginning of the year, Gen = gender, Abi = ability, grade-point average secondary school, Age = age,
Instr = rating course/instructor, Dif = rating perceived difficulty, Invol = rating involvement, Lwde = score low work discipline at the end of the year,
Dip = score deep information processing, GPA = Grade-Point Average on the seven courses.
 Explaining Achievement 247

is much greater (-.45). Strikingly, but to be expected based on the zero-order

correlations, we found no effect of deep information processing strategy use on
achievement.

Discussion and Conclusions

The purpose of this study was to analyse a model that included aspects concerning
students’ pre-entry characteristics, students’ perceptions of the learning environment,
study behaviour and academic achievement. With this model we tried to combine
views from Berger and Milem (1999), Jansen (1996, 2004), Prins (1997) and Tinto
(1975, 1987). In line with results from school effectiveness research and with the
earlier findings of, for example, Jansen (1996, 2004) and Van der Hulst and Jansen
Downloaded by [Staffordshire University] at 14:23 06 October 2014

(2002), we found a large effect of ability on achievement. However, Lizzio, Wilson,

and Simons (2002) reported that students’ school achievement was a positive but
weak predictor of their university achievement, although they relate it to the inherent
restriction of range. Because there is a difference in admissions policies between
Dutch and American universities – in The Netherlands universities are not allowed to
select students on their grades from secondary education – there is probably more
variation in grade obtained at secondary school in the Dutch situation than in the
American situation. According to Lizzio et al. (2002), students’ perceptions of their
current learning environment have a stronger influence on study outcome than their
mean grade in secondary education. However, our findings point to a strong effect of
ability on achievement. De Jong, Roeleveld, and Webbink (1997) reported a small
effect of ability on the probability to drop out.
We found effects of gender on achievement, rating of course/instructor,
involvement, end-of-the-year work discipline, and deep information processing
strategies. In accordance with other findings, for example those of Jansen (1996,
2004), Richardson and Woodley (2003), Shah and Burke (1999), Simonite (2003),
Van den Berg (2002) and, Van der Hulst and Jansen (2002), we found that female
students achieved better results than male students. This is in contrast with the
findings of Slotte, Lonka, and Lindblom-Ylänne (2001). They reported differences
between men and women in learning strategy but not in learning outcomes. The
effect of gender on work discipline was as expected and is in line with Macan et al.
(1990). They found that women are better in time management than men. The path
of gender on involvement was not specified in our theoretical model. Because
involvement and work discipline are correlated, this could explain the effect of gender
on involvement in the same way as the effect of gender on work discipline. In their
study on results in different courses in Dutch higher education, Beekhoven, De Jong,
and Van Hout (2003) expected women to study more efficiently.
Finally, the analyses revealed that older students had higher scores on deep
information processing strategy use than younger students. Deep information
processing strategies can also be seen as something that goes together with
maturation. In that way more life experience will work out more positively for older
students. In addition, we found an effect of age on work discipline and a positive
 248 E. P. W. A. Jansen and M. Bruinsma

effect on involvement. Older students showed better results, were more involved, and
showed higher work discipline. The effects of age on achievement are in contrast with
other studies, such as De Jong, Roeleveld, Webbink, and Verbeek (1997), Jansen
(1996, 2004), Shah and Burke (1999), Van den Berg (2002), and Van der Hulst and
Jansen (2002). These studies reported that younger students had better results.
According to Van der Hulst and Jansen (2002), age can be seen as a proxy for ability,
that is, older students needed more time for pre-university education, for example
because they had failed their examinations. Another explanation may be the fact that
older students often do not receive enough financial support for their studies, so they
have to work and can spend less time on study activities. In this study, we did not find
this effect, maybe due to less variation in age than in the other studies.
A main question in this study concerned the relationship between the perceptions
Downloaded by [Staffordshire University] at 14:23 06 October 2014

of the learning environment on the one hand, and work discipline and deep
information processing strategies on the other. Students who reported that they
perceived the course as less difficult were more satisfied with the course/instructor.
This minor effect is in line with Perry (1997), who states that students with high-
perceived control benefit more from effective instruction than students with low-
perceived control. A striking finding is that satisfaction with the course/instructor had
a negative effect on a student’s achievement as well as on perceived difficulty. This
contrasts with, for example, Feldman (1989), who showed significant positive
correlations between different teaching behaviours and achievement. The effects of
the course/instructor on involvement and end-of-the-year work discipline were
positive. That is, students who rated the course/instructor positively were more
involved but had lower work discipline. A last finding on the rating outcomes is that
those students who evaluated the course content and the assignments as difficult,
attended lectures more frequently and completed their assignments on time.
Furthermore, those students used more deep information processing strategies.
End-of-the-year work discipline showed a large effect on achievement. This is not
surprising because time on task and regular study have been proved to affect
achievement (Carrol, 1963; Creemers, 1994). Students with higher work discipline
obtained higher grades and students with higher work discipline at the end of the year
were more satisfied with the course/instructor.
Vermunt (1992) states that students who are using a meaning directed learning
style, that is students who are critical and who are relating various aspects of the
learning content, have been found to receive higher grades in higher education.
However, it is striking that deep information processing strategy use in our study does
not affect achievement. This may be a result of the way the students are tested: If the
test does not refer deep information processing strategies, students will not use deep
information processing strategies. Possibly the explanation by Slotte et al. (1999) is
applicable. They suggest that a high degree of deep-level learning is necessary
especially at the end of the study. Different phases in the study demand different
kinds of study strategy. Because in our study only 1st-year students were involved, it
might be the case that deep information processing strategies are required less in the
1st year than in the following years of study.
 Explaining Achievement 249

Evidently there are some limitations to this study. Firstly, these concern the use of
student ratings forms as indicators of instruction. Hinton (1993), for example,
contends that these ratings are poor measures of teaching effectiveness, because there
is no universally acceptable model of good teaching that can be used for the validation
of the constructs in the questionnaires. Other authors, for example McKeachie
(1997) and Centra (1993), are not satisfied with the evaluation forms used because
they focus mainly on traditional types of instruction, such as lectures and other
teacher-centred methods. Further, there is extensive debate about the assumption
that course grades are positively related to course evaluation (Stumpf & Freedman,
1979). However, analyses have shown that this assumption is not applicable to this
study (see Jansen & Bruinsma, 2002). Secondly, this study is limited to students
within one department and within one university. It seems obvious that this study
Downloaded by [Staffordshire University] at 14:23 06 October 2014

should be replicated at more departments and more universities. Several studies

indicate the effects of disciplinary differences (e.g., Becher, 1994; Kekäle, 1999;
Lindblom-Ylänne & Lonka, 1999). Therefore, it is necessary to look at the value of
the model within different departments. Perhaps it would be even better to replicate
this study separately for different disciplines and universities. Further, related to this,
departments and even disciplines within departments are free in their choice of
evaluations. Some departments do not use questionnaires at all, whereas other
departments use different ratings forms. Using standard questionnaires for different
departments, faculties, and universities is important to us as researchers, but this idea
has not been widely supported by the departments at our university.
Finally, there are many other student characteristics, for example motivation, and
course and curricular characteristics that will influence academic achievement.
However, in this study we have tried to give some insight into the relationship
between students’ characteristics, the perceived learning environment, and reported
study behaviour and academic achievement.

References
Bean, J.P. (1980). Dropouts and turnover: The synthesis and test of a causal model of student
attrition. Research in Higher Education, 12, 155 – 187.
Bean, J.P., & Metzner, B.S. (1985). A conceptual model of non-traditional undergraduate student
attrition. Review of Educational Research, 55, 485 – 540.
Becher, T. (1994). The significance of disciplinary differences. Studies in Higher Education, 19,
151 – 161.
Beck, H.P., & Davidson, W.D. (2001). Establishing an early warning system: Predicting low grades
in college students from survey of academic orientations scores. Research in Higher Education,
42, 709 – 723.
Beekhoven, S., De Jong, U., & Van Hout, H. (2003). Different courses, different students, same
results? An examination of differences in study progress of students in different courses.
Higher Education, 46, 37 – 59.
Berger, J.B., & Milem, J.F. (1999). The role of student involvement and perceptions of integration
in a causal model of student persistence. Research in Higher Education, 40, 641 – 664.
Bijleveld, R.J. (1993). Numeriek rendement en studiestaking [Numerical returns and dropout].
Doctoral dissertation, University of Twente. Utrecht: Lemma.
Carroll, J.B. (1963). A model of school learning. Teachers College Record, 64, 723 – 733.
 250 E. P. W. A. Jansen and M. Bruinsma

Centra, J. (1993). Reflective faculty evaluation. San Francisco: Jossey-Bass.

Creemers, B.P.M. (1994). The effective classroom. London: Cassell.
De Jong, U., Roeleveld, J., & Webbink, H.D. (1997). Verder studeren in de jaren negentig. Studiekeuze
en studieloopbaan over de periode 1991 – 1995 [Studying in the nineties. Choices and study
careers 1991 – 1995]. The Hague: Sdu.
De Jong, U., Roeleveld, J., Webbink, H.D., & Verbeek, A.E. (1997). Het Amsterdamse
schoolloopbaanmodel [The Amsterdam model for study careers]. The Hague: Sdu.
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood estimation from
incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1 –
38.
Feldman, K.A. (1989). The association between student ratings of specific instructional dimensions
and student achievement: Refining and extending the synthesis of data from multisection
validity studies. Research in Higher Education, 30, 583 – 645.
Hinton, H. (1993). Reliability and validity of student evaluations: Testing models versus survey
Downloaded by [Staffordshire University] at 14:23 06 October 2014

research models, PS: Political Science & Politics, 26, 562 – 569.
Hu, L., & Bentler, P.M. (1999). Cut-off criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural-Equation-Modeling, 6(1), 1 – 55.
Jansen, E.P.W.A. (1993a). Curriculum organization and study progress. In Th. H. Joostens,
G.W.H. Heijnen, & A. Heevel (Eds.), Doability of curricula (pp. 135 – 149). Lisse, The
Netherlands: Swets & Zeitlinger.
Jansen, E.P.W.A. (1993b). Educational programming related to study-progress. In J.K. Koppen &
W.D. Webler (Eds.), Strategies for increasing access and performance in higher education (pp.121 –
129). Amsterdam: Thesis.
Jansen, E.P.W.A. (1996). Curriculumorganisatie en studievoortgang [Curriculum organization and
study progress]. Doctoral Dissertation. Groningen, The Netherlands: GION.
Jansen, E.P.W.A. (2004). The influence of the curriculum organization on study progress in higher
education. Higher Education, 47(4), 411 – 435.
Jansen, E.P.W.A., & Bruinsma, M. (2002). Student ratings and measurement of study behaviour in a
model for explaining study progress. Paper presented at the Annual Meeting of the American
Educational Research Association, New Orleans [On-line]. Available: http://www.cowog.nl/
homepage/publications/aeraellenmarjon.PDF.
Kekäle, J. (1999). ‘‘Preferred’’ patterns of academic leadership in different disciplinary
(sub)cultures. Higher Education, 37, 217 – 238.
Lindblom-Ylänne, S., & Lonka, K. (1999). Individual ways of interacting with the learning
environment – are they related to study success? Learning and Instruction, 9, 1 – 18.
Lizzio, A.K., Wilson, K., & Simons, R. (2002). University students’ perceptions of the learning
environment and academic outcomes: Implications for theory and practice. Studies in Higher
Education, 27, 27 – 52.
Macan, T., Shahani, C., Dipboye, R.L., & Phillips, A. (1990). College students’ time management:
correlations with academic performance and stress. Journal of Educational Psychology, 82,
760 – 768.
Marton, F., & Säljö, R. (1997). Approaches to learning. In F. Marton, D. Hounsell, & N. Entwistle
(Eds.), The Experience of learning (pp. 39 – 58). Edinburgh: Scottish Academic press.
McKeachie, W. (1997). Student ratings, the validity of use, American Psychologist, 52, 1218 – 1225.
Need, A., & De Jong, U. (1998, April). De invloed van opleiding op studieresultaten en studieuitval na
één jaar studeren in het hoger onderwijs [The influence of the department on achievement en
drop out after one year of study in higher education]. Paper presented at the Sociaal
Wetenschappelijke Studiedagen, Amsterdam.
Pascarella, E.T. (1980). Student-faculty informal contact and college outcomes. Review of
Educational Research, 50 , 545 – 575.
 Explaining Achievement 251

Perry, R.P. (1997). Perceived control in college students: Implications for instruction in higher
education. In R.P. Perry & J.C. Smart (Eds.), Effective teaching in higher education: Research and
practice (pp. 11 – 60). New York: Agathon Press.
Prins, J. (1997). Studieuitval in het wetenschappelijk onderwijs. Studentkenmerken en opleidingskenmer-
ken als verklaring voor studieuitval [Drop out in university education. Student characteristics
and educational characteristics as explanatory factors]. Doctoral Dissertation. Nijmegen, The
Netherlands: University Press.
Ramsden, P. (2003). Learning to teach in higher education (2nd ed.). London: RoutledgeFalmer.
Ramsden, P., & Entwistle, N.J. (1981). Effects of academic departments on students’ approaches to
studying, British Journal of Educational Psychology, 51, 151 – 164.
Richardson, J.T.E. (2000). Researching student learning. Approaches to studying in campus-based and
distance education. Buckingham, UK: SRHE and Open University Press.
Richardson, J.T.E, & Woodley, A. (2003). Another look at the role of age, gender and subject as
predictors of academic attainment in higher education. Studies in Higher Education, 28, 475 –
Downloaded by [Staffordshire University] at 14:23 06 October 2014

493.
Schouwenburg, H.C. (1996). Handleiding bij de VSP’94 [Manual for the Test for Study Problems].
Groningen, The Netherlands: Studie Ondersteuning.
Schouwenburg, H.C., & Schilder, A.J.E. (1996). Handleiding bij de Test voor Diepgaande
LeerstofVerwerking DLV’95 [Manual for the Test for Deep Information Processing].
Groningen, The Netherlands: Studie Ondersteuning.
Schouwenburg, H.C., & Stevens, I.L. (1996). De Checklist StudieMotivatie CSM’96 [The Checklist
for Study Motivation]. Groningen, The Netherlands: Studie Ondersteuning.
Shah, C., & Burke, G. (1999). An undergraduate students’ flow model: Australian higher
education. Higher Education, 37, 359 – 375.
Simonite, V. (2003). A longitudinal study of achievement in a modular first degree course. Studies
in Higher Education, 28, 293 – 302.
Slotte, V., Lonka, K., & Lindblom-Ylänne, S. (2001). Study-strategy use in learning from text.
Does gender make a difference? Instructional Science, 29, 255 – 272.
Smith, J., & Naylor, R. (2001). Determinants of degree performance in UK universities: A
statistical analysis of the 1993 student cohort. Oxford Bulletin of Economics and Statistics, 63,
29 – 60.
Spady, W. (1971). Dropouts from higher education: Toward an empirical model. Interchange, 2(3),
38 – 62.
Stumpf, S.A., & Friedman, R.D. (1979). Expected grade covariation with student ratings of
instruction: Individual versus class effects. Journal of Educational Psychology, 71, 293 – 302.
Terenzini, P. (1999). Research and practice in undergraduate education: And never the twain shall
meet? Higher Education, 38, 33 – 48.
Tinto, V. (1975). Drop-out from higher education: A theoretical synthesis of recent research.
Review of Educational Research, 45(1), 89 – 125.
Tinto, V. (1987). Leaving college: Rethinking causes and cures of student attrition. Chicago: The
University of Chicago.
Trigwell, K., & Prosser, M. (1991). Improving the quality of student learning: The influences of
learning context and student approaches to learning on learning outcomes. Higher Education,
22, 251 – 266.
Trigwell, K., & Prosser, M. (2003). Qualitative difference in university teaching. In M. Tight (Ed.),
Access and exclusion international perspectives on higher education research (Vol. 2, pp. 185 – 216).
Oxford: Elseviers Science.
Van den Berg, M.N. (2002). Studeren? (g)een punt! Een kwantitatieve studie naar studievoortgang in het
Nederlandse wetenschappelijk onderwijs in de periode 1996 – 2000 [A quantitative study of study
progress in Dutch university education, 1996 – 2000]. Amsterdam: Thela Thesis.
 252 E. P. W. A. Jansen and M. Bruinsma

Van der Drift, K.D.J.M., & Vos, P. (1987). Anatomie van een leeromgeving. Een onderwijseconomische
analyse van het universitaire onderwijs [Anatomy of a learning environment. An educational
economic analysis of higher education]. Lisse, The Netherlands: Swets & Zeitlinger.
Van der Hulst, M., & Jansen, E.P.W.A. (2002). Effects of curriculum organisation on study
progress in engineering studies. Higher Education, 43, 489 – 506.
Vermunt, J.D.H.M. (1992). Leerstijlen en sturen van leerprocessen in hoger onderwijs. Naar procesgerichte
instructie en zelfstandig denken [Learning styles and regulation of the learning processes in
higher education]. Amsterdam/Lisse: Swets & Zeitlinger.
Downloaded by [Staffordshire University] at 14:23 06 October 2014