Professional Documents
Culture Documents
net/publication/263967125
CITATIONS READS
18 903
4 authors:
Some of the authors of this publication are also working on these related projects:
The development of valuable products from oil palm wastes View project
All content following this page was uploaded by Kim Yeow Tshai on 29 December 2014.
Corresponding author:
K.Y. Tshai, Department of Mechanical, Materials and Manufacturing Engineering, University of Nottingham
Malaysia Campus, Jalan Broga, 43500 Semenyih, Selangor, Malaysia
Email: Kim-Yeow.Tshai@nottingham.edu.my, Phone: +60 (3) 8924 8614
Abstract
Outcome-based Education (OBE) emphasises on two main components in terms of student
achievement in an academic programme. One is the Programme Outcomes (POs) which is
measured at the point of graduation, and the other, the Programme Educational Objectives
(PEOs) is assessed over a longer period of time (around 4–5 years) after graduation. This
study focuses on the establishment of a systematic procedure adopted by the Department
of Mechanical, Materials and Manufacturing Engineering at the University of Nottingham
Malaysia Campus (UNMC) to formulate PEOs assessment criteria with integration of
stakeholders' input, methodology for an unbiased measurement of graduates' long-term
attainment rate, as well as analysis and identification of a set of strategies for continuous
quality improvement (CQI). These PEOs are mapped using the guidelines set by the
Engineering Council to those required by the Engineering Accreditation Council (EAC),
Malaysia. The outcome of the mapping exercise was used to formulate an anonymous
online questionnaire survey as a measure of the PEOs' attainment. Key outcomes from this
study revealed that graduates are broadly satisfied with their achievement in all eight PEOs.
Strategies were also proposed to improve the attainment level in four PEOs with relatively
lower attainment rate, as part of the CQI process adopted in the department.
Introduction
Outcome-based Education (OBE) has become a central feature in the accreditation of
engineering degrees offered by institutions of higher learning worldwide. For example, the
assessment and evaluation of students' learning outcomes are the general criteria set by
the Accreditation Board for Engineering and Technology (ABET) (ABET 2012) in the United
States, as well as the Australian Graduate Attributes set by the Engineers Australia (2005).
In the case of institutions of higher learning in Malaysia, the accreditation guidelines are set
by the Engineering Accreditation Council (EAC) within the Board of Engineers Malaysia
(BEM), which is also moving towards the adoption of OBE, as pointed out by the study
of Aziz et al. (2005).The main reason for this high level of worldwide adoption is due to the
fact that OBE advocates a paradigm shift from traditional practises of course credit
accumulation to a focus on the students' achievement in high order learning and the
mastery of cognitive thinking skills. This is believed to be a better measurement of the
students' success in achieving the key necessary skills. Within an institution, OBE
implementation requires the restructuring of the educational programme, courses,
curriculum, assessment and reporting systems (Malan 2000).
In the 1950s, the work of Bloom (1956) and his colleagues in developing taxonomies for
educational objectives became important. The taxonomies eventually became benchmarks
which were then used in the formulation of specific objectives and the development of
targeted criteria to establish the learners' attainment of acceptable standards against the
desired learning outcomes. Bloom's work, particularly in addressing the cognitive domain,
remains instrumental in the assessment of OBE. In his work, Killen (2000) stressed the
importance of linking assessments to long-term and significant outcomes expected of the
learners, or to the short-term enabling outcomes which were derived from these long-term
outcomes. This is in accordance to the principles of ‘clarity of focus’ and ‘designing back’.
There are two major types of outcomes in OBE, as defined by Killen (2000). The first
emphasises on measurement of coursework, examination results, rates of course
completion and employment upon graduation, while the second performance indicator
is less tangible, commonly required the leaners to express what they have learned and
capable to perform as a result of completing their education. It places greater focus on
long-term outcomes leading to the future career success of the learner. OBE consists of
four main principles (Brandt 1992): clarity of focus, expanded opportunity, high
expectations and design down.
Clarity of focus requires gearing the development of curriculum towards the outcomes
expected of the students. These should not be short-term targets (weekly, semester or
yearly) but end results that students would achieve upon graduation and beyond. This
also emphasises on the need to pursue continuous development throughout the career.
The second principle, expanded opportunity, aims to provide learners with various ways to
achieve the learning outcomes. Approaches such as lectures, tutorials, assessments,
seminars and workshops, industrial visits and dialogues, industrial attachment,
departmental briefing, student sharing sessions and competitions are adopted within the
OBE framework.
The high expectations principle requires every learner to consistently achieve high level
of performance. Students must develop a structural mind set and a sharp mind to tackle
problems. They have to utilise the engineering principles and common sense efficiently to
track sequential thoughts. Other attributes such as fast learning by organising information
rather than just receiving information, holistic approach in handling engineering problems
with appropriate safety, sustainable and engineering awareness are highly desirable.
The final principle, design down, requires all curriculum design and teaching activities
to comply with the outcomes from top to bottom levels. Spady (1994) illustrates the
development of outcomes following the hierarchy of Exit Outcomes, Programme
Outcomes, Course Outcomes, Unit Outcomes and Lesson Outcomes. The assessment of
the overall achievement includes a linkage to long-term educational outcome of the learner,
e.g. the long-term Programme Educational Objective (PEO) must be measured for a period
of at least five years from graduation, as illustrated in Figure 1.
Another key element in OBE focuses on the quality assurance of the education system.
Quality measurement is important in engineering education (Owlia & Aspinwall 1996, 1998)
Figure 1 The proposed linkage of Spady’s (1994) top to bottom levels of outcomes development
with the incorporation of long-term PEOs.
Engineering at the UNMC (as indicated at the top level in the hierarchy shown in Figure 1).
The next section provides an overview of the OBE and the hierarchy of the measurable
outcomes, as well as the incorporation of CQI in the PEO formulation framework. The third
section illustrates the methodology used in the assessment of students' attainment of
PEOs. Results are discussed in the fourth section. The fifth section concludes this study and
makes appropriate recommendations for future works.
Assessment methodology
The main methodology used in measuring the attainment level of PEOs is through an
online anonymous questionnaire survey. Fifty-one MEng graduates from the Department of
Mechanical, Materials and Manufacturing Engineering at UNMC are selected as the target
population for this study, with a total of 24 respondents. The attainment level of the PEOs is
then analysed through the survey results. A dedicated set of survey questions must be
customised for individual courses. The set of questions were strengthened with a focus
group discussion consisting of members of staff in the department to formulate the survey.
The outcomes of the focus group discussion were translated into an anonymous online
questionnaire survey as a tool to measure the attainment of PEOs against a five discrete
level gauge (from very satisfied to very dissatisfied).
UK-SPEC statements
PEOs PEOs statement
adapted by the IMechE
PEO1 An understanding of the principles and techniques US1, US2, US3, US1m, US2m, M5, M6
of mechanical engineering science and of the
computational and mathematical tools used
in the solution of real problems
PEO2 The ability to apply these principles and methods US3m, US4m, E1, E2, E3, E4,
in the practice of mechanical engineering E1m, P3, P4, US2
PEO3 The ability to visualise, design and analyse D1, D2, D3, D4, D5, D6, M5, D4m
components and assemblies in
three dimensions
PEO4 A sound knowledge of material properties D1m, D4m, P1, P2, P7, P1m, P2m
and manufacturing methods and their cost
effective use in the design and improvement
of engineering components and equipment
PEO5 The ability to communicate ideas effectively in M1, M2, M3
written reports, verbally and by means
of presentations to groups
PEO6 The ability to work as a member of a team and M4, M7, US3m
to display initiative and enterprise in the
planning and realisation of engineering projects
PEO7 Confidence through practice in the use E2m, E3m
of experimental methods
PEO8 The appreciation of the responsibilities of S1, S2, S3, S4, S5, S1m, S2m,
Professional Mechanical Engineers and of P5, P6, P8, P8m, M7, US4m,
the social, management and financial D3, D6, P1m, P7
environment in which they work
The online questionnaire survey contains 34 questions from six groups consisting of
details, professional development, salary scale, job prospects, continuous professional
training and the PEOs, as shown in Table 3.
All alumni graduated in the last five years were invited to respond to the survey.
The questionnaire survey is designed to monitor the student's progress in their career
since graduation. The survey is relatively short, simple and straightforward to encourage
a higher level of participation. The purpose of the survey is also self-explanatory and
made available to the alumni at the onset of the session, emphasising on the anonymity of
the survey. Alumni who agreed to participate are given a web link to access the survey and
the questionnaire is made available in a predefined period of time. They are given the
option to withdraw at any time for the case where they do not feel comfortable to submit
their survey.
Upon obtaining the results, statistical analyses are conducted on the measured data.
Quantitative data are analysed and depicted in graphs and charts to determine the
percentage of a particular finding, with percentage of various elements of the PEOs being
rated as ‘high (above 70%)’, ‘medium (50–69%)’ and ‘low (49% and below)’ as a guideline to
identify the most crucial elements of the PEOs for continuous quality improvement in the
education system of the engineering programme.
Group Categories
1. Details Student's degree
Student's programme
Year of graduation/study
2. Professional development Board of Engineers Malaysia (BEM)
Institution of Engineers Malaysia (IEM)
Institution of Mechanical Engineers, UK (IMechE)
Other relevant professional institutions
Class of membership
3. Salary scale Monthly salary scale
4. Job prospect Area/industry
Job title
Number of staff under your leadership
5. Continuous professional training Job training, self-initiated professional training
Attendance of short course, seminar, workshop or
conference
6. Programme Educational Objectives PEO1 to PEO8
(PEOs)
PEO1, PEO2, PEO5 and PEO6 achieved attainment rate above 70%. These PEOs are related
to core principles of mechanical engineering and its application in practical problems, the
ability to communicate ideas effectively using written formats and verbal presentation,
working together as a team member displaying initiative and enterprise in engineering
projects. These PEOs are the basis of the core structure of the mechanical engineering
programme hence its coverage throughout the programme is eminent especially in the
compulsory elements of the programme e.g. Professional Studies, Solid Mechanics,
Materials and Manufacturing, Structural Vibration, Thermofluids, etc. Therefore, students
had vast opportunities to work in a team in several modules featuring group work e.g.
Design and Manufacture, Group Design Project and Group Design and Make. A 12 week
industrial internship is compulsory for students, giving them the required exposure in
applying their knowledge in a practical environment. Students are also strongly encouraged
to participate in team competitions such as Bosch Power Drill Race, Formula Varsity, the
James Dyson Award, etc. to cultivate the culture of team working and to build up the
experiences in applying these principles in their career upon graduation. This has
contributed to the graduates' confidence in the attainment of these PEOs.
PEO3 has the lowest attainment rate of 54%. This is concerned with the graduates' ability to
visualise, design and analyse components and assemblies in three dimensions. The
achievement of this PEO is mainly relied on the delivery, contents and assessment of
modules with design elements. The low attainment indicates that more emphasis should be
placed in guiding the students in constructing three-dimensional (3D) engineering models
in orthographic and auxiliary projections, isometric views, sectioning, dimensioning and
assembly drawings, as well as their perceptions in conceptual, embodiment and detailed
design. Strategies to improve the attainment of PEO3 formed part of a portfolio of
CQI processes to be implemented in the programme. These include introduction of a
comprehensive self-help guide on Moodle (the university's Virtual Learning Environment),
employing more research assistants with knowledge of 3D modelling, engaging industrial
lecturers to contribute to the teaching of design, continuously updating and improving the
video demonstrations used, improving the efficiency of the assessment of these elements
in the programme.
PEO4 is mostly linked to the graduates' understanding on materials properties and
manufacturing, as well as cost effective in design and system improvement. This is related
to the ‘design’ and ‘practice’ in the UK-SPEC Learning Outcomes. Only 63% of the
graduates felt that they have acquired this educational objective and applied it in their
career. A relatively low attainment rate indicates that the graduates need more practicing
Conclusions
Using a pre-determined set of PEOs complying with the UK-SPECS outcomes, this paper
reports a survey to measure the graduates' attainment level of PEOs from their degree
References
Accreditation Board for Engineering and Technology (ABET) (2012) Criteria for accrediting
engineering programs 2013–2014. Baltimore, MD: ABET.
Anuar, A., Shuaib, N., Sahari, K. and Abidin, I. (2009) Continual improvement and
assessment plan for mechanical engineering programme at UNITEN, pp19–24.
Kuala Lumpur: IEEE.
Aziz, A., Megat Mohd Noor, M., Abang Ali, A. and Jaafar, M. (2005) A Malaysian
outcome-based engineering education model. International Journal of Engineering
and Technology 2 (1), 14–21.
Bloom, B.S. (1956) Taxanomy of educational objective: handbook 1. Cognitive domain.
New York: McKay.
Brandt, R. (1992) On outcome-based education: a conversation with Bill Spady. Educational
Leadership 50 (4), 66–70.
Engineering Council, UK (2010) The accreditation of higher education programmes: UK
standard for professional engineering competence. London: Engineering Council, UK.
Engineers Australia 2005. Accreditation management system educational programs at the
level of professional engineer. Barton, ACT: Engineers Australia.
Ho, J.-H., Tshai, K.Y. and Yap, E.H. (2013) Students' Attainment of Programme Objectives
(POs) for the Mechanical Engineering Undergraduate Degrees at the University of
Nottingham Malaysia Campus (UNMC), International Symposium on Education, Psychology
and Social Sciences (ISEPSS 2013). Kuala Lumpur, Malaysia.
Killen, R. (2000) Standards-referenced assessment: linking outcomes, assessment and
reporting. Keynote address at the Annual Conference of the Association for the Study of
Evaluation in Education in Southern Africa, Port Elizabeth, South Africa, 26–29 September.
Magee, C.L. (2004) Needs and possibilities for engineering education: one industrial/
academic perspective. International Journal of Engineering Education 20 (3), 341–352.
Malan, S.P. (2000) The "New Paradigm" of outcome based education in perspective. Journal
of Family, Ecology and Consumer Sciences 28, 22–28.
May, E. and Strong, D. (2006) Is engineering education delivering what industry requires.
The Third CDEN Design Conference, Toronto, Ontario, Canada.
Owlia, M. and Aspinwall, E. (1996) A framework for the dimension of quality in higher
education. Quality Assurance in Education 4 (2), 12–20.
Owlia, M. and Aspinwall, E. (1998) A framework for measuring quality in engineering
education. Total Quality Management 9 (6), 501–518.
Rozeha, A.R., Mazlina, E., Sadruddin, M. and Abdullah, R. (2007) Continuous improvement
in engineering education through effective value for money audits. In Proceeding, Second
International Conference on Engineering Education & Training (ICEET-2), MOVENPICK Hotel
Kuwait, Kuwait City, Kuwait.
Sani, M. et al. (2008) Assesment of the mechanical engineering programs by exit surveys
at University Malaysia Pahang, pp317–323. Shah Alam, s.n.
Shekar, C.R., Farook, O. and Bouktache, E. (2008) Continuous improvement process based
on outcome based education. Nashville, TN: IAJC-IJME.
Spady, W. (1994) Choosing outcomes of significance. Educational Leadership 51 (6), 18–22.
Todd, R.H. and Magleby, S.P. (2005) Elements of a successful capstone course considering
the needs of stakeholders. European Journal of Engineering Education 30 (2), 203–214.
Waks, S. and Frank, M. (2002) Engineering curriculum versus industry needs: a case study.
IEEE Transactions on Education 43 (3), 349–352.