2015 IEEE 7th International Conference on Engineering Education (ICEED)
Evaluating Teaching Quality Using Data From
Student Online Feedback System
Rosni Abu Kassim, Norlida Buniyamin
Engineering & Technical Education Research Group (EnTER),
Faculty of Electrical Engineering, Universiti Teknologi MARA,
40450 Shah Alam, Selangor, Malaysia
rosni637@salam.uitm.edu.my, nbuniyamin@salam.uitm.edu.my
Abstract— This paper provides an overview of online student
feedback systems in tertiary institutions. It is then introduce a
student feedback online system (SuFO) developed and used in a
public university in Malaysia. Subsequently, this paper describes
how data from SuFO is used to observe the quality of teaching
between experience and inexperience lecturer. The evaluation is
based on students’ response in SUFO that measured the
lecturers’ ability based on the lecturer’s professionalism and
teaching methods. The results of this preliminary investigation to
compare the quality of teaching between experience and
inexperience lecturers are included. The results indicated from
the student feedback on teaching quality of experience and
inexperience lecturers is inconclusive since both categories of
lecturers obtained low and also high rating from students.
Keywords—Student feedback; online system; experience
lecturer, inexperience lecturer, academic achivement
I. INTRODUCTION
In the last decade, as the number of private and public
universities grew exponentially in Malaysia, there is a
shortage of experience and adequately qualified Science,
Technology, Engineering and Mathematics (STEM) lecturers
in Malaysia. Thus, the quality of teaching became an issue. To
evaluate teaching effectiveness, many Malaysian universities
introduced student feedback system. Feedback from students
when appropriate questions are asked can be used to evaluate
teaching quality. This feedback could be used to improve
teaching quality or as reported by Emery (Emery 2003), a
student’s evaluation of teaching quality has widely been used
as part of the faculty member’s performance evaluation.
The study presented in this paper, focusses on the use of
Student Feedback system that aims to improve teaching and
course content quality. This paper presents an overview of
student’s feedback system and concludes with a case study
that shows how feedback from engineering students obtained
through an online system was used to investigate whether
there is a lot of difference between the teaching qualities of a)
an experience lecturer (i.e. the lecturer have taught the subject
for three semesters or more) vs inexperience (first time
teaching the subject). The main aim is to test the usefulness
and validity of data obtained from the online system.
A. Types of Student feedback system
Student feedback systems is a tool that enables institution
of higher learning (IHL) to obtain data related to teaching and
978-1-4799-8810-5/15/$31.00 ©2015 IEEE 64
course content. The use of the large array of data about
students and courses collected by the institutions is termed
academic analytics and it is currently getting notable
attention, because it assists educational institutions in
improving teaching performance, student achievement and
success, increasing student retention, and reduce the load of
liability and accountability (Usamah Mat 2013)
Various types of student feedback systems have been
implemented in universities worldwide. In his review paper,
(Richardson 2005) found that feedback systems can be used as
a useful instrument to enable evaluation of teaching quality.
These feedback systems range from simple paper based
questionnaire given to student to complex computer-based
online feedback system that not only provides questionnaires
but also includes analysis of the data collected. Both paper-
based and online evaluations systems have advantage and
disadvantage such as response rate and data processing
capability.
Although many education institutions have spent
significant resources to design such instruments to obtain
feedback on quality of teaching so as to improve quality of
education, it is important to emphasize that student feedback is
only one possible source of information on teaching
effectiveness and impression on the subjects and the course
itself.
For the feedback to be effective, the collection of data
must be followed by analysis and interpretation. Many
institutions used the feedback to enhance teaching and
learning process, course revision and also used to provide
summative evidence for staff promotion and awards (Emery
2003).
II. SUFO STUDENT FEEDBACK ONLINE SYSTEM
This paper describes an online system that is use in a
Malaysian University. The system termed SuFO (Student
Feedback Online) was first introduced in the university in the
December 2010 academic semester with the main objective to
obtain feedback from students and use the data obtained to
carry out continuous improvement and quality assurance
processes with the objective to maintain and improve the
standard and quality of teaching and learning delivery (Abdul
Aziz 2010). Figure 1 is a snapshot of the SuFO online
interaction page.
 2015 IEEE 7th International Conference on Engineering Education (ICEED)

Fig. 1. SuFO webpage snapshot

A. SuFO Feedback Questions
To access SuFO, students need to log in through the SuFO obtains feedback from students in four categories of
student portal to access the feedback questionnaires. Then, statements as listed in Table 1. The students agree or disagree
they can access the respective subject which is made available to the statements in a scale of four sections which are strongly
from week 12 onwards of an academic semester. Lecturers agree, agree, disagree and strongly disagree (Refer to Table 1
with passwords have access to course evaluation divided into and Figure 1). In addition, the students are given
students groups. To ensure that the students give an honest opportunities to give their views in open ended questions. For
respond without any fear of discrimination from lecturers in this investigation, the focus is on teaching quality feedback
future semesters, respondent identity is not available to the obtained from the formal question. The objective is to see
lecturers. whether there is any room for improvement in teaching
When SuFO was first introduced, the respondent rate was quality.
extremely poor. To ensure that the university gets feedback
from all the students, a SuFO response is made compulsory for
every student before they can access their examination results
for that particular semester. Since there is an abundance of
data from SuFO, a simple investigation was carried out to test
the usefulness of data from SuFO.
TABLE I. CATEGORIES OF SUFO STATEMENTS
Categories of Statements Questions Example of questions
Section A: 1. The course content is related to my field of study.
Overall Impression about the course
2. The method of assessments in this course has enhanced my learning ability.
Section B: 1. The lecturer completes the scheduled hours of instruction.
Lecturer Professionalism 2. The lecturer is ever ready to provide academic guidance to students
3. The lecturer uses English as a medium of instruction
4. The lecturer is approachable
5. The lecturer is accessible for discussion
6. The lecturer monitors student attendance
7. Overall, the lecturer exhibits high professionalism
Section C : 1. The lecturer explains the course content
Teaching and Learning Activities 2. The lecturer explains the method of assessment
3. The lecturer teaches according to plan
4. The lecturer actively involves students in the learning process
5. The lecturer creates an environment for students to ask questions and offer opinion
6. The lecturer delivers the content interestingly
7. The lecturer’s delivery style challenges the mind
8. The lecturer provide feedback for each assessment/assignment
9. The lecturer helps students master the learning content
10. Overall, I enjoyed the teaching style of this lecturer
Section D : 1 The equipment space for teaching and learning is conducive.
Infrastructure 2. The teaching and learning equipment are adequate and functioning

978-1-4799-8810-5/15/$31.00 ©2015 IEEE 65

 2015 IEEE 7th International Conference on Engineering Education (ICEED)

III. ANALYSIS AND DISCUSSION ON SUFO DATA

B. Using SuFO data to analyse teaching quality
In the case study university, due to a high number of
students in a semester and with the requirement of EAC (EAC
2012) that classes must be kept to a maximum of 35 students,
a number of subject had to be offered in numerous groups to
cater to the large number of students. For example, in the
September 2014-Jan 2015 semester, Signal and Systems a
subject that is offered in the 5th semester has 238 students
divided into 9 groups (labelled G1 to G9 in Figure 2) that is
taught by 6 lecturers. The lecturers comprises of four (4)
experience and (2) inexperience lecturers. As previously
defined, an experience lecturer is a lecturer that have taught
the subject for at least three semesters and an inexperience
lecturer is a lecturer that is teaching the subject for the first
time.
Fig. 2 shows the students’ SuFO response from all the nine
groups to questions from section B and C which is the
lecturers’ professionalism and teaching & learning activities.
The average results from both sections are plotted as shown in
the graph labelled as teaching quality. In addition the average
results of the students respond to Section D which is related to
infrastructure facilities provided is also plotted. This was
added as a check to ensure that students responded honestly
since in section D it is not expected that students will be bias.
Thus, if both plots have the same trend line, then we can
assume that the students also responded honestly to sections B
and C. Section A is not used in this research as for this paper
we are only interested in looking at the teaching quality rather
than course content.
Fig 2 shows that the pattern for both infrastructure and
teaching quality are almost the same, that is, students’ rate the
two categories approximately alike. With assumption that
student rates infrastructure sincerely, then the student rating on
the teaching quality is assumed valid.
Student Feedback vs Teaching & Learning and
A. Evaluation of teaching quality based on same subject and
experience of lecturers
To compare SuFO rating of the lecturers in terms of
teaching experience, for lecturers teaching two groups, the
average SuFO rating from both groups is considered. Two of
the six lecturers, are inexperience and as shown in Fig 3 they
are labeled IE1 and IE2 whilst the rest are experience lecturers
labelled E1, E2, E3 and E4.
Fig. 3 below shows the feedback obtained for sections B
and C; which is lecturer professionalism and also teaching and
learning activities for the six lecturers. The students’ feedback
calculated has similar averages for both sections B and C in
each group, however the average for each group differs quite
significantly from one group to another. This thus indicates
that different lecturers have different rating for both teaching
and learning activities even when the course content and
lecturer presentation slides used by the lecturers are the same.
From Fig 3, as depicted, there is no clear correlation between
experience and non-experience lecturers with students’
satisfaction as measured using their teaching activity and
professionalism ratings. For example, an inexperience lecturer
IE2 obtains much better rating than an experience lecturer E4
which is unexpected. However, the inexperience lecturer IE1
performs badly when compared to experience lecturers E1, E2
and E3 which was expected. This indicates that, the findings
are inconclusive which validates the literature review finding
as discussed earlier that student feedback is only one of the
factors that affect quality of teaching.
Student feedback: Inexperience and Experience
lecturers for Section B and C
92
90
88
86
% SuFO

Infrastructure 84
100 82
90 80
% Student Response

80 78
70 76
60 IE1 IE2 E1 E2 E3 E4
50
Section B 82.43 90.02 85.82 89.93 87.14 81.61
40
30 Section C 81.90 90.46 85.84 89.69 86.39 81.48
20
10
0 Fig. 3. Comparisong of SuFO rating for inexperience and experience
G1 G2 G3 G4 G5 G6 G7 G8 G9
lecturers
Teaching quality 85.83 83.55 80.78 86.98 93.5 91.3 86.77 88.31 81.55
Infrastructure 84.38 79.83 70.63 84.63 91.75 90.75 86.13 83 78.75

Another phenomenon that can be seen is that inexperience

Fig. 2. Comparison of SuFO rating on infrastructure and Teaching and lecturers could also obtain either low or very high SuFO
Learning
percentage. For example, lecturer IE2 obtained the highest
percentage compared to all other lectures. These further shows
that the quality of teaching and learning is not only determined

978-1-4799-8810-5/15/$31.00 ©2015 IEEE 66

 2015 IEEE 7th International Conference on Engineering Education (ICEED)

by whether a person is an experienced lecturer or not but that
there are also other factors that affect the quality of teaching.
Other factors that may affect learning outcomes are
attitudes, student abilities, enthusiasm and dedication of
students in learning and lecturer in delivering the course
content. Looking at the students’ rating on professionalism
(Section B) and teaching and learning activities (Section C),
there is not much difference between the two.
B. Evalution of teaching quality based on student academic
achievement on the subject
Student achievement were analyse based on marks
obtained by students for this particular subject. From the
marks, they are divided into four categories which are
excellent, good, fair and weak. The categories and marks
relationship is as in Table 2.

Fig. 4 is obtained by averaging Section B and C and also TABLE II. CATEGORIES OF STUDENT ACHIEVEMENT
averaging students’ feedback for inexperience and experience
lecturers. Even though the difference is considered small, that GRADE STATUS
is, less than 2%, surprisingly it indicates that SuFO rating is A+, A Excellent
higher for inexperience lecturers compared to experience
lecturers. This could be that the first-time lecturer who is A-, B+, B Good
teaching a new subject for the first time is very excited to B-, C+, C Fair
share what they know with students while the lecturers who
have been teaching the same subject numerous time has been C-, F, D, D+, C- Weak
increasingly less enthusiastic and is now bored with the same
subject. Teaching the same subject numerous times using
same presentation slides and monograph may have affected Fig. 5 shows students’ academic achievement in groups
their teaching quality. taught by inexperience and experience lecturers. The graph
shows that student in the classes taught by inexperience
lecturer can achieve better grades than student in an
experience lecturer’s class.
SuFO (%) : Av. B and C for Experience
and inexperience lecturer
86.5 86.2 Academic achivement vs Group
86 60
Grade Percentage

50
85.5 40
85 30
84.69 SuFO (%) 20
84.5 10
0
84 IE1 IE2 E1 E2 E3 E4
Excellent 11.3 55.55 16 27.7 38.5 36.0
83.5
IE E Good 48.4 38.85 32 50.5 38.5 32.0
Fair 37.1 5.55 36 18.6 19.2 28.0

Fig. 4. SuFO rating for inexperience and experience lecturers Weak 3.25 0 16 3.4 3.8 4.0

This validates the findings of (Annie McKillop 2005) who
interviewed five faculty lecturers with at least five years
teaching experience and each of them taught between two and
five streams of the same paper, with some teaching two 2-hour
streams on the same day. They used the following words to
describe their feeling of having to teach the same thing over
and over again. “Dreadful, stultifying, mind-numbing, tedious,
infuriating, boring and frustrating “. The lecturers confessed
that they feel stale and jaded about their teaching. Several,
lecturers felt that there had to be some limits to repetition; it
was not tolerable and infinitum: “I will need to move on to
something else eventually.”
Fig. 5. Students’ Academic achiement for inexperience and experience
lecturers
In Fig. 6 the plot of students’ academic achivement is
based on average data of students with excellent and good
grades for a particular group. The academic achivement is
then compared to percentage of SuFO (average of B and C).
Except for group IE2, all groups exhibit higher percentage for
SuFO ratings than academic achivement. The highest gap
between academic achivement is for lecturer E1. Lecturer E1
is actually the most experience lecturer. The high SuFO rate
indicates that the lecturer had conducted the teaching and
learning clasess quite well however there could be other
factors that retards student learning in that class which led to
poor academic achivement. One of thefactors could be that
even if the lecturer deliverd the lecture well and all the
contents were explained in detail, the students still could not

978-1-4799-8810-5/15/$31.00 ©2015 IEEE 67

 2015 IEEE 7th International Conference on Engineering Education (ICEED)

understand. This could be due to several factors, one of which
could be due to the student’s English language proficiency
(ELP) as the course is taught in English and it could be the
student’s poor ELP rather than the teacher’s teaching ability
that is the cause of the students’ poor academic achievement.
Many research have been carried out in this area that indicates
better ELP will enable engineering student to obtain better
academic results. For example (Ghenghesh 2015) found that “
the higher the English proficiency of students on entry to the
university, the better they performed in their degree area
courses’. (Saquing-Guingab 2015) also found that competency
in English will affect a students’ academic achievement,
where a student who is good in English would in most cases
be good as well in Science and Mathematics. These two
subject plays a big role in an engineering program especialyy
in this subject, Signal and Systems and is a prerequisite to
many other courses.
Academic achivement and SuFO rating
100
Student Response & Grade (%)
IV. CONCLUSION
The investigation showed that there is no no direct corellation
between SuFO rating of a lecturer’s teaching quality and
whetehr the lecturer is experience or inexperience lecturer.
Thus this indicates that a students rating should not be used as
the only paramater to determine a lecturer’s quality of
teaching. There is a correlation between academic student
achievement and SuFO rating, however the grades achieved
by students based on experience and inxerepeiced lecturer is
inconclusive.
References
[1] Abdul Aziz, Azlan, Mohd Ali Mohd Isa (2010). SuFO Guidelines (For
students, Instructors And Sufo Administrator) University Teknologi
MARA. University Teknologi MARA, i-Learn.
[2] Annie McKillop, Robyn Ramage (2005). Time after time: getting some
zip back into your info lit. LIANZA Conference,. Christchurch, NZ: 1-
14.
[3] EAC, Engineering Accreditation Council, (2012). Engineering

90 Programme Accreditation Manual. Qualifying Requirements and

80 Accreditation Criteria, Engineering Accreditation Council | BEM.
70 [4] Emery, C., Tracy R. Kramer, Robert G. Tian (2003). "Return to
60 Academic standards: a critique of student evaluations of teaching
50 effectiveness." Quality Assurance in Education Vol. 11(No. 1): 37-46.
40
[5] Ghenghesh, P. (2015). "The Relationship Between English Language
30
Proficiency And Academic Performance of University Students –
20
Should Academic Institutions Really be Concerned?" International
10
Journal of Applied Linguistics & English Literature 4(2).
0
IE1 IE2 E1 E2 E3 E4 [6] Richardson, J. T. E. (2005). "Instruments for obtaining student feedback:
Academic achivement 59.7 94.4 48 78.2 77 68 a review of the literature." Assessment & Evaluation in Higher
Education Vol. 30(4): 387–415.
SuFO rating 82.16 90.24 85.83 89.81 81.55 81.55
[7] Usamah Mat, Norlida Buniyamin, P. Mohd Arshad, R. A. Kassim
(2013). An Overview of Using Academic Analytics to Predict and
Fig. 6. Students’ Academic achiement for inexperience and experience Improve Students' Achievement: A Proposed Proactive Intelligent
lecturers Sufo rating – average of B & C Intervention. IEEE 5th International Conference on Engineering
Education (2013 ICEED), Selangor, Malaysia, , IEEE.

978-1-4799-8810-5/15/$31.00 ©2015 IEEE 68