Victoria University First year

experience data 2017

what does it tell us and how can
we use it to improve student
transition experiences?
 Transition to university: Navigating
assessment practices and curriculum
knowledge
a pan-university project investigating students transition
from school to university, and in particular the relationships
between achievement in Level 3 of the National Certificate
of Educational Achievement (NCEA) and success in the first
year at university
 Three interrelated strands of work:
Perceptions of, and engagement in, learning and study
practices in NCEA and in the first year at university.
A broad analysis of the relationship between the content and
quality of first-year students NCEA qualifications and their
achievement in first-year degree programmes at VUW, and
the influence of social variables on this relationship.
A discipline-specific investigation of the relationship between
the structure and content of, and quality of performance in,
NCEA courses and assessment programmes, disciplinary
knowledge (epistemic, declarative and procedural), success
in first-year degree programmes, and students perceptions of
their transition experience.
 Methodology and data collection
Mixed methods
Student surveys:
February (before university started) (n=1656)
July (after trimester 1) (n=639)
November (after trimester 2) (in progress)
100-level course coordinator survey
(September) (n=75)
Secondary teachers survey (November, in progress)
Analysis of student NCEA data and university results for
2017 cohort of beginning students
Focus groups (planned for 2018)
 Barriers to Student Success anecdotal
evidence backed up from the literature
1. Academic Preparedness
Interpreting and meeting assessment criteria
Ghost of NCEA
Navigating Entry Criteria
Learning to Learn
Expectations
Quick feedback
Access to staff
How to study independently what is new at 3o level not clearly stated
2. Social Issues
Competing demands
Part-time study
Engagement and Identity
 The Ghost of NCEA
Credit grabbing the value of the intended learning is framed almost
exclusively in terms of credits to be gained
Issues that reduce motivation and achievement
o the ability to not do parts of a course that the student didnt like;
o being able to avoid subjects/standards seen as challenging to ones
learning;
o not completing assessments where the student expected to do poorly;
o not sitting external examinations (student has enough credits).
Issues related to learning
o School teachers prioritise the product over the process of learning -
much spoon feeding
o Intensively scaffolded learning experiences
o Students unaware of their learning styles
o Students unable to critically analyse information presented to them.
First year university students
perspectives on NCEA as a
preparation for university
Factors contributing to learning and results for Level
3 standards

Assessment schedules showed what was required for

each grade (A, M, E)

Not at all

Only a little

Somewhat

A great Deal

0 10 20 30 40 50
Factors contributing to learning and results for Level
3 standards

Re-submission helping to improve grades

Not at all

Only a little

Somewhat

A great Deal

0 5 10 15 20 25 30 35
Factors contributing to learning and results for Level
3 standards

Formative feedback on previous assessments

Not at all

Only a little

Somewhat

A great Deal

0 10 20 30 40 50 60
 Factors contributing to learning and results for Level
3 standards

Reducing workload by withdrawing from other

standards
Not at all

Only a little

Somewhat

A great Deal

0 5 10 15 20 25 30 35
 Reasons for not completing
standards
I chose not to attempt the assessment as I
I wanted to concentrate on doing better in other
There was not enough time in an exam session
There was not enough time in an exam session
I had left school
I was not interested in the focus of the standard
Personal reasons (e.g., sick, overseas)
I had already met the requirements for Level 3
I had already met the requirements for UE
0.0 5.0 10.0 15.0 20.0 25.0

External Assessment Internal assessment

What do students see as the merits and
limitations of the NCEA system?
Flexibility
Internal Assessments
External Assessments
Motivation
Structure of NCEA
Nature of Learning and Knowledge
 Flexibility of NCEA
Everyone could study something that suited their interests and
needs. And it progressively taught us to become flexible,
independent learners. NCEA is awesome for people from
households that don't exactly encourage academic success as
NCEA provided the means to tackle a very broad range of
problems that we could come across no matter where or who we
are.

I enjoyed the variety of topics which you could complete for one
subject and that there were a range of internal and external
opportunities to do well. This meant if you were better under
different test circumstances/environments you had different
opportunities to achieve better grades.
 Internal Assessments
I enjoyed the take-home internal assessments the most.
This was because with these type of assessments there was
plenty of time to thoroughly think through what the
assessment was asking for, how you were going to construct
your report, and time to proof read.

Internal assessments provided opportunities to complete

work to a higher standard than time pressured exams, as well
as the information being fresh in your mind, and teacher
feedback as to how it would score against the standard.
 Motivation
How it allowed you to progress throughout the year instead of
focusing all your effort towards the end of the year and how it
allowed you to resubmit your work so that you could present the
best piece of work you could.

That you were able to gain credits and achieve in courses that
interested you personally. You were able to succeed in subjects
that you wanted to do and were not forced to conform to the
same mandatory subjects, to a certain degree at least.
 External Assessments
The amount of exams at the end of the year . As there were so
many papers in one subject (eg chemistry with 3 externals and 2
internals) meant that we had only a small amount of time for end
of year exam prep as it takes a long time to teach the entirety of
each paper. So the pressure to do well in end of year exams is
immense.
I disliked how the external standards are more based on
memorisation rather than your actual knowledge or capability.
More often than not, after my external standards have been
tested I barely remember the information I had written - because
it was more about how much you could remember rather than how
much you actually understood/comprehended in-depth.
 Structure of NCEA
I like the style of grading, where failure in one standard doesn't
potentially impact on your overall grade for the year, allowing
opportunities to learn and grow throughout the year to achieve
greater results.
The difference between marks (achieved, merit and excellence)
was based on a checklist This meant that everything had to be
done exactly as the schedule said. The implication of this was all
students work ended up being similar and it was difficult to be
original because we were told exactly what to write and how. There
was little creativity . instead of learning and exploring possibilities
. I ended up like a machine just pumping out exactly what had to
be done and ONLY what had to be done. I was just reguritating
words I had read or essays I had memorised in order to do well.
 Critique of Learning and Knowledge
. Students have limited opportunities to explore what interests
them and genuinely engage in learning. NCEA is primarily about
passing/failing content in a bid to create the false perception of
educational advancement of the student. An NCEA qualification
has no legitimate value like a university degree does, and it does
not prepare the student with the necessary critical thought and
abilities needed to succeed at tertiary study or in the workforce.
The learning was too linear. It felt as though we were learning to
remember, not to actually become knowledgeable.
First year university students
perspectives on how well they
were prepared for university
after 1 trimester
 How well did you feel prepared for
university?
Before starting After T1

Not prepared Strongly disagree

Slightly Prepared Disagree

Prepared Somewhat agree

Extremely prepared Strongly agree

0 20 40 60
0 10 20 30 40 50 60
I feel that I was academically prepared for
Academically Emotionally University

I feel that I was socially and emotionally

prepared for University
Looking back, how well do you feel NCEA set you
up for university study?

Failed at least one course Passed all courses

50
40
30
20
10
0
Extremely Very well Moderately Slightly well Not well at all
well well
 I have difficulty coping with the mathematical
skills required for my university work

I have difficulty coping with the reading and/or

writing required for my university work

My time management skills need improvement

I have a hard time making new friends and feeling

like I belong

It seems like everyone else has university figured

out but me

I struggle to keep up with the amount of work my

courses requirements

I often prioritise my friends and social life over my

university work
0 20 40 60 80 100 120

Strongly disagree Disagree Somewhat agree Strongly agree

 Challenges
120

100

80

60

40

20

challenging
for > 75%
Not at all challenging Not very challenging Somewhat challenging Very challenging Extremely challenging
 Emotional State
Almost never Rarely Sometimes Most of the time Almost all of the time

Angry
Lonely
Anxious
Depressed
Stressed
Overwhelmed
In control
Confident
Happy
Optimistic
0 10 20 30 40 50 60 70 80 90 100
Cumualtive percentage of responses
Challenges - From the Student Comments
(after 1 trimester)
Time management

Differences in the assessment systems

Preparing for a new living situation

Navigating campus systems

What to expect in my programmes/courses

Being academically prepared

Variability across courses

Links to careers
What we wish we had known before we began
Differences in the assessment systems
That there aren't any re-do's of assessments once it's done, its done
The structure of how they examine students in the assessments. They are not as
flexible as teachers with marking,
School was easier by far, at University there are no second chances so if you mess
up say, in an assignment or mid-term test, you could potentially get a far lower
grade than what you were hoping for, with no option to better the grade
The amount of work, and I mean that in the way that the assessments are so
close together and are worth so much more than any assessment was at high
school
School made it almost impossible to fail as they gave you multiple times to resit
papers where as your lectures don't have as much of a connection to you passing
or failing
The marking at uni was hard to comprehend at first compared to NCEA.
Navigating campus systems
Note-taking is important and learning to do this quickly and coherently is
paraomount
I wish I knew the size of the lectures and how many people would be there,
and that it would be hard to instantly develop focus and routine.
There are lots of people at uni, no one tells you the things you need to know -
have to find the information yourself
The larger classes, the flexibility, the lack of accountability.
The lectures don't spoon feed you everything like they do at school
There is always help available to you at school and someone to help you along
the right track in academics. At university, however, I feel like there is no one
that can really help you out.
At school you have a lot more support and one on one
What to expect in my programmes/courses
Less help about how to do assessments is provided.
There's a lot more work at university and it's more difficult.
Lectures, you don't do any exercises in class so retaining information is/will be
a little harder .
School was easy to pass without doing work outside of class. Uni is
impossible to pass unless you put alot of work in outside of class.
Have to do work every day not just around times of tests/exams. Have to seek
outside help for content where high school content was very easy.
More work necessary now in the little time given compared to high school.
deadlines are not flexible.
At school, you had a whole year to learn the subject whereas at uni we have
12 weeks to learn the material and test for the course
Everything, expectations are completely different. Learning styles taught at
school are completely different
Hard but totally achievable
An impersonal place
That it's a massive place and you have to actively search for friends
During the day, interaction must be sought out rather than happening
automatically
School - I had lots of friends, Uni - I don't.
Less time to form friendships in a class. Less time spent attending classes
Uni is far more impersonal
There is always help available to you at school and someone to help you along
the right track in academics. At university, however, I feel like there is no one
that can really help you out.
.lack of contact and natural build up of relations between students and
lecturers due to the sizes and contact hours of the classes.
Time management
I wish that someone had told me the importance of a self-timetable and keeping
on top of things like BB announcements, due dates etc.
Self-driven, less class time and more personal study time required.
University is a lot more independent. In school teachers told you about your
progress and what you needed to work on but at university it is all up to you
Material is covered faster. also fewer exercises and worksheets than at school.
University involves more self-management, more exams and tests rather than
essays, more self-taught learning, more content to learn, more interesting content
At school, there were constant reminders about assessments, while at university
there aren't, meaning you have to be fairly organised and know what's coming up.
The lack of support and push from teachers at school compared to the very
independent university atmosphere and expectations
Course work is independently managed all of the time. There is no set class time to
work on assignments in university like there is at school.
I can choose what, and when, I want to study. I don't have to turn up and do
work.
 Course Coordinators comments
20% are organized/prepared students who take university work seriously ..the
other 80% are unprepared.

. I spend a lot of time discussing and making explicit the expectations of the
environment that students are transitioning TO (university) but have very little
knowledge of what they are transitioning FROM. I've known for some time that this
is a problem

The impression I have gotten from teaching 100 level students is that what they are
getting at school is conditioning them to being 'spoon fed'. They want exact road
maps for everything & don't seem to manage the transition to self-directed
learning that well.

Why are so many students, when they are just coming from NCEA, just interested
in how to pass a course or assignment, but not how to do really well?
 Students Academic Preparedness
Empirical approach
Measurement scales were calibrated for each of internal and external
assessment for NCEA in chemistry, mathematics and history using a
graded response model (Samejima).
Each student in a 100-level course of interest was located on the
measurement scales for NCEA internal and external assessment in the
same subject as the 100-level course. by prior performance in each of
internal and external assessment in NCEA.
Step-wise regression was used to estimate the proportions of variance
in 100-level course results accounted for
A qualitative analysis of the standards and assessment takes was
undertaken to establish the types of knowledge that might underpin the
statistical relationships between NCEA and 100-level performance.
Potential sources of association between NCEA
and 100-level performance
Specific knowledge

Facility with learning different types of knowledge (declarative, conceptual,

epistemic)

Facility with different assessment modalities and types of tasks.

Chemistry: Chem114 (Principles of
Chemistry)
Proportion of variance in 100-level performance

Internal Joint External

0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
 Flexibility of NCEA
Everyone could study something that suited their interests and
needs. And it progressively taught us to become flexible,
independent learners. NCEA is awesome for people from
households that don't exactly encourage academic success as
NCEA provided the means to tackle a very broad range of
problems that we could come across no matter where or who we
are.

I enjoyed the variety of topics which you could complete for one
subject and that there were a range of internal and external
opportunities to do well. This meant if you were better under
different test circumstances/environments you had different
opportunities to achieve better grades.
 Internal Assessments
I enjoyed the take-home internal assessments the most.
This was because with these type of assessments there was
plenty of time to thoroughly think through what the
assessment was asking for, how you were going to construct
your report, and time to proof read.

Internal assessments provided opportunities to complete

work to a higher standard than time pressured exams, as well
as the information being fresh in your mind, and teacher
feedback as to how it would score against the standard.
 Motivation
How it allowed you to progress throughout the year instead of
focusing all your effort towards the end of the year and how it
allowed you to resubmit your work so that you could present the
best piece of work you could.

That you were able to gain credits and achieve in courses that
interested you personally. You were able to succeed in subjects
that you wanted to do and were not forced to conform to the
same mandatory subjects, to a certain degree at least.
 External Assessments
The amount of exams at the end of the year . As there were so
many papers in one subject (eg chemistry with 3 externals and 2
internals) meant that we had only a small amount of time for end
of year exam prep as it takes a long time to teach the entirety of
each paper. So the pressure to do well in end of year exams is
immense.
I disliked how the external standards are more based on
memorisation rather than your actual knowledge or capability.
More often than not, after my external standards have been
tested I barely remember the information I had written - because
it was more about how much you could remember rather than how
much you actually understood/comprehended in-depth.
 Structure of NCEA
I like the style of grading, where failure in one standard doesn't
potentially impact on your overall grade for the year, allowing
opportunities to learn and grow throughout the year to achieve
greater results.
The difference between marks (achieved, merit and excellence)
was based on a checklist This meant that everything had to be
done exactly as the schedule said. The implication of this was all
students work ended up being similar and it was difficult to be
original because we were told exactly what to write and how. There
was little creativity . instead of learning and exploring possibilities
. I ended up like a machine just pumping out exactly what had to
be done and ONLY what had to be done. I was just reguritating
words I had read or essays I had memorised in order to do well.
 Critique of Learning and Knowledge
. Students have limited opportunities to explore what interests
them and genuinely engage in learning. NCEA is primarily about
passing/failing content in a bid to create the false perception of
educational advancement of the student. An NCEA qualification
has no legitimate value like a university degree does, and it does
not prepare the student with the necessary critical thought and
abilities needed to succeed at tertiary study or in the workforce.
The learning was too linear. It felt as though we were learning to
remember, not to actually become knowledgeable.