Adapting Malcolm Baldrige Quality Assessment into QMeasure

Instruments for Measuring Quality Standards of Educational

Organisations

Jamil Hj Hamali, (PhD)

Universiti Teknologi MARA Sarawak, Malaysia
jamil1@sarawak.uitm.edu.my
&
Iris Syawe Seh Ling ( PhD)
Universiti Teknologi MARA Sarawak, Malaysia
iris@sarawak.uitm.edu.my

ABSTRACT

Quality performance measurement and application of quality award schemes enable an

organization to identify its current quality standards and determine its direction. The
measurement itself provides information and data on whether the organisation is
achieving its goals and objectives while the award schemes determine its comparative
standard. The QMeasure Instruments have been designed to determine the QMeasure
Index of the quality standard of an educational organisation and to identify areas where
further improvements are required. The main instrument is an adaptation of Malcolm
Baldrige Quality Assessment into a framework for operationalising total quality which
consists of eight main criteria. For assessment, evidences of activities and performance
indicators are collected under various items in these criteria. In addition, two
instruments in the form of questionnaires are also designed to gauge the quality
measurement index on the standard of services rendered to the main customers: staff and
students. The radar chart which is generated from the measurement scores gives a clear
picture of the performance of the organisation. QMeasure Instruments are self-
assessment tools which allow an educational organisation to monitor its own
performance so that it can plan its future direction. The assessment can be used for any
quality award schemes. These instruments have been used to determine the quality
performance of Universiti Teknologi MARA Sarawak which led to its winning various
awards.

Key words: quality, performance measurement, self-assessment, quality award schemes

1. INTRODUCTION

The choice of educational organizations especially institutions of higher learning by stakeholders

depends on whether an organization has a quality delivery system which guarantees that a student will
receive quality qualification. In other words, the university is able to meet the requirement of
preparing the student for the job market. The basic quality of education is to develop programs and
services that meet the needs of customers, which are basically the students of the university. Meeting
needs of students is really getting a tertiary qualification that is relevant for securing employment.
Three elements for quality in higher education suggested by Green (1994) are meeting customers’
needs, fitness for purpose and effectiveness in achieving institutional goals.

Hamali & Hides (1999) quoted that performance measurement keeps tab of where an organization is
and where it is heading. It guides an organization as to whether it is en route to achieving its goal or
otherwise. While measurement is a key management activity that provides information necessary for
making decision, monitoring performance and effectively allocating resources (Webster and Hung,
1994), performance measurement has the basic functions of coordinating, monitoring and diagnosing
(Atkinson et al, 1997).

The need to consider various elements when designing any measurement system was suggested by
various authors on performance measurement (Oakland, 1996, Ovretveit, 1993, Edvardsson et al,
1994). The measurement system must be linked to the organization’s strategies, made up of multiple
measures, simple and easy to administer, inexpensive, and involve employees to ensure their support.
Edvardsson et al, (1994) believed that performance measurement should be covered by quantitative
and qualitative methods. This mix method allows for unambiguous measures such as waiting time,
service availability and precision as well as information that cannot be obtained through quantitative
method such as ‘information-rich’ citations of customers’ expectations and requirements.

Various authors have suggested a number of commonly used models for performance measurement
especially for quality award schemes. Amongst the models are the Balanced Scorecard, the European
Foundation for Quality Management (EFQM) Excellence Model and Malcolm Baldrige Quality
Assessment and ROKUM. The Balanced Scorecard is the most frequently quoted model in any
communication on quality, while the EFQM Excellence Model is widely used in Europe. Both the
Malaysian Federal and Sarawak State Governments have used Malcolm Baldrige Quality Assessment
in their quality awards such as the Prime Minister Quality Award, Productivity Award and the Quality
Management Excellence Award.

A performance measurement system especially for self-assessment is a major requirement for any
organization (especially educational organization) in providing quality services. It allows an
organization to keep in touch with its customers and enables an organization to compare actual results
with targets and goals. Together with quality award schemes, it enables an organization to identify
where it is and where it is going. While the measurement itself provides information and data on
whether the organisation is achieving its goals and objectives, the award schemes determine its
comparative standard with other organizations.

QMeasure Instruments which is an adaptation of Malcolm Baldrige Quality Assessment Model, have
been developed as self-assessment instruments. This paper covers a brief description of Malcom
Baldrige Quality Assessment Model according to Baldrige’s “2000 Education Criteria for
Performance Excellence” which was adapted for QMeasure Instruments in 2001. It also covers a
description of QMeasure Instruments and their application in Universiti Teknologi MARA (UiTM)
Sarawak.

2. MALCOLM BALDRIGE QUALITY ASSESSMENT MODEL

The Baldrige’s Education Criteria for Performance Excellence provides a measurement model for
organizations where education is their core business. The model emphasized the following seven
main criteria: Leadership, Strategic Planning, Student and Stakeholder Focus, Information and
Analysis, Faculty and Staff Focus, Educational and Support Process Management, and Organizational
Performance Results.

The two aspects of leadership addressed under the Leadership category are organizational leadership,
and public responsibility and citizenship. Strategic Planning covers strategic development and
deployment while Student and Stakeholder Focus include knowledge of student needs and
expectations, and student and stakeholder satisfaction and relationships. Information and Analysis
includes effective performance measurement system and analysis of performance data and
information. Faculty and Staff Focus looks at the work system, faculty and staff education,
training/development, well-being and satisfaction. Educational and Support Process Management
covers education design and delivery, education support processes and partnership processes, and
Organizational Performance Results focus on students’ performance results, student and stakeholder
2
focused results (including their satisfaction and dissatisfaction), key budgetary and financial
performance results and faculty and staff results.

Baldrige’s Education Criteria for Performance Excellence provides point values to each of the seven
categories with an overall total of 1000 points. 125 points are allocated to Leadership criterion, 85
points each to Strategic Planning, Student and Stakeholder Focus, Information and Analysis, Faculty
and Staff Focus, and Education and Support Process Management criteria while Educational
Performance Results criterion gets 450 points.

Baldrige’s Education Criteria for Performance Excellence also provides scoring guidelines and
percentage scores for each of the seven criteria on the aspects of approach/deployment and/or results.
With regard to scoring guidelines for results, it also awarded percentage scores based on the
description established during the audit process.

3. QMEASURE INSTRUMENTS

The QMeasure Instruments have been designed to determine the QMeasure Index of the quality
standard of an educational organisation and to identify areas where further improvements are required.
The main instrument (QMeasure Instrument III) is an adaptation from the ROKUM (Rangka Operasi
Kualiti Menyeluruh or A Framework for Operationalising Total Quality, 1996) and Malcolm Baldrige
Education Criteria for Performance Excellence. This instrument consists of eight main criteria similar
to that of seven main criteria in the Malcolm Baldrige Quality Assessment. The eight main criteria
are: Organizational Leadership; Information and Analysis; Strategic and Operational Planning;
Development and Management of Resources; Educational Management Process; Innovation;
Products/Performance Results; and Customer Satisfaction. The instrument assesses actual evidences
versus a set of expected evidences and based on these, scores are given for the various criteria and
sub-criteria measured.

The QMeasure Instrument I and QMeasure Instrument II in the form of questionnaires are designed to
assess the main customers (student and the staff) perceptions of the services provided by the
organization and ultimately to gauge the quality measurement index on the standard of services
rendered to them. This index contributes to the Customer Satisfaction criteria in QMeasure
Instrument III.

Similar to the Malcolm Baldrige Quality Assessment, QMeasure Instruments also allocates point
values for each of the eight perspectives of quality. The breakdown of point values are: 90 points for
Organizational Leadership; 75 points for Information and Analysis; 75 points for Strategic and
Operational Planning; 150 points for Development and Management of Resources; 150 points for
Educational Management Process; 75 points for Innovation; 155 points for Products / Performance
Results; and 230 points for Customer Satisfaction. The point values indicate that greater importance
or weighting is placed on Customer Satisfaction, Products, Educational Management Process, and
Development and Management of Resources.

A percentage score is given for each criterion based on evidences and data available. Then, the actual
number of points for each criterion is calculated. A perfect percentage score for each criterion should
be 100% to obtain the total value points for that criterion. This is shown in Figure 1 which is the
perfect radar chart for QMeasure.

The summation of all the total points obtained for all the criteria is then converted into the QMeasure
Index of the organization. A maximum index of 7 is for a total quality system which is continuously
enhanced with mature quality culture producing consistent and excellent results. In other words, this
index means world class quality. The lowest index of 1 is for a system where there is only some
awareness of quality with no co-ordination among departments, incongruence results and approach.

3
 Leadership (90)
100
80
Customer Satisfaction (230) Information & Analysis (75)
60
40
20
Products (155) 0 Strategic Planning & Operations (75)

Innovation (75) Development & Management of Resources (150)

Educational Management Process (150)

Note: A perfect radar chart for ROKUM with 100% score for each criterion and point values in brackets.

Figure 1: Perfect Radar Chart for QMeasure

4. APPLICATION OF QMEASURE IN UiTM SARAWAK

The QMeasure Instruments were first used in UiTM Sarawak in 2001 to determine its performance
and to identify areas for further improvement. This was necessary in the light of the Total Quality
Management initiatives undertaken by the university. A survey was conducted by using QMeasure
Instruments I and II on students and staff to gauge the standard of services rendered to its customers.
A total of 860 students and 195 staff responded. The quality measurement committee used QMeasure
Instrument III to obtain the QMeasure Index. Overall results showed the index to be at 5 which
indicated that quality initiatives were actively implemented in the campus although there was plenty
of room for further improvement.

However, the initial use of QMeasure Instrument I to obtain feedback from the students showed an
alarming dissatisfaction among them. This contributed to an indented corner for customer satisfaction
in the radar chart which was at a customer satisfaction index of less than 50%. Through the feedback
collected via the instrument, the leadership and management as well as the staff made drastic changes
and improvement which led to an improvement in the next application of the instrument in 2003.

Since 2001, the QMeasure Instruments have been improved and used every two years to assess the
university. Figure 2 shows the radar chart for QMeasure assessment of the university for 2001 and
2003. This self-assessment initiative has revealed that currently there is clear quality culture in all
units and divisions. This is especially reflected in students and staff feedback, collected via
QMeasure Instruments I and II, for improvements in services and facilities as well as the positive
attitude of staff in accepting the findings and their willingness to improve their services.

The application of QMeasure Instruments is very significant especially where the management of the
university incorporated the major and highly relevant findings into strategic plans and decisions. In
addition, after each application, a number of feedback sessions were held with staff from various units
and divisions to reveal and discuss critical findings. These sessions also helped in obtaining feedback
for developing strategies and policies on quality management.
4
Through the use of QMeasure Instruments for self-assessment, UiTM Sarawak has been able to
further determine the direction of its quality journey beyond awareness, quality activities, quality
system, documentation and self-assessment to cultivating quality, quality culture and hopefully
achieving world class status by 2015.

Leadership
100
80
Customer Satisfaction Information & Analysis
60
40
20
Products 0 Strategic Planning & Operations

Development & Management of

Innovation
Resources

Educational Management
Process

2001 2003

Source: Syawe et al, 2004, p. 90.

Figure 2: Radar Chart for QMeasure Assessment of UiTM Sarawak for 2001 and 2003

5. APPLICATION OF QMEASURE IN AWARD SCHEMES

The continuous self-assessment efforts with the application of the QMeasure Instruments in
determining the quality standard of UiTM Sarawak has enabled the campus to participate in various
award schemes. The Award Schemes like the Malaysian Prime Minister’s Quality Award, the
Sarawak Federal Secretary’s Quality Award and the Sarawak State Secretary’s Quality Award use
criteria similar to Malcolm Baldrige Quality Assessment. Hence, it is quite easy for data collected
using the QMeasure Instruments to be incorporated into the criteria of these awards.

Since the implementation of the QMeasure Instruments in UiTM Sarawak in 2001, the campus has
won the overall UiTM Vice Chancellor’s Quality Award for three consecutive years (2003, 2004,
2005) and the UiTM Vice Chancellor’s Excellent Quality Award in 2006. UiTM Sarawak also won
second place in the Sarawak Federal Secretary’s Quality Award for 2004 and 2006. In addition,
UiTM Sarawak won the Sarawak State Secretary’s Quality Award and also won fourth place in the
Malaysian Prime Minister’s Quality Award, both in 2006.

6. CONCLUDING REMARKS

5
The application of QMeasure for determining the QMeasure index for any educational organization
will reveal the strengths and weaknesses of the organisation where improvements need to be made.
Once the organization knows its quality performance position or index, it can easily plan its quality
journey to lead it towards a well-designed quality system with excellent integration, co-ordination,
and outstanding performance.

The QMeasure Instruments and Methodology has won two awards: Gold Medal Award at
“Inventions, Innovations and Designs (IID) 2007” organized by Institute of Research,
Development and Commercialisation (IRDC) of UiTM (24-25 January 2007), and Bronze
Medal Award at “Malaysia Technology Expo (MTE) 2007” organized by Malaysian
Association of Research Scientists (MARS) (29-31 March 2007).

Acknowledgement

The authors would like to extend their acknowledgement to the other members of the research
(QMeasure) team: Mary Gunjew Nee’ Gaik, Goy Siew Ching, Azlin Abdillah, Dorothy Kueh Chiok
Choo, Serah Haji Jaya, William Duncan, Ho Boon Heong, Dr. Nagarajah Lee and Kelvin Goh Tee
Hiong.

REFERENCES

Atkinson, A. A., Waterhouse, J. H. and Wells, R. B. (1997). A Stakeholder Approach to Strategic

Performance Measurement. Sloan Management Review. Vol. 38, No. 3, 1997. Pp 25-37.

Baldrige National Quality Program. 2000 Education Criteria for Performance Excellence. Available
on the Internet at: http://www.quality.nist.gov [June 2001]

Center for Total Quality Education. ITM. (1996). A Framework for Operationalising Total Quality.

Edvardsson, B., Thomasson, B., and Ovretveit, J. (1994). Quality of Service. London: McGraw-Hill.

Green, D. (1994). Society for Research into Higher Education and Open University Press. What is
Quality in Higher Education? Great Britain: St. Edmundsbury Press.

Hamali, J. and Hides, M. T. (1999). Performance Measurement. Unpublished paper.

Oakland, J. S. (1996). Total Quality Management. Oxford: Butterworth-Heinemann.

Ovretveit, J. (1993). Measuring Service Quality. Hertfordshire, England: Technical Communication

(Publishing) Ltd.

Syawe, I.S.L., Hamali, J., Gunjew, M.N.G., Duncan, W. & Ho, B.H. (2002). Quality Performance of
MARA University of Technology, Samarahan Campus 2001. Bureau of Research and
Consultancy, UiTM, Malaysia.

Syawe, I.S.L., Gunjew, M.N.G., Goy, S.C., Abdillah, A., Kueh, D.C.C., Jaya, S., Lee, N.H.L., & Goh,
K.T.H. (2002). Quality Performance of Universiti Teknologi MARA Sarawak 2003. UiTM
Sarawak, Malaysia.

Webster, C. and Hung, L. (1994). Measuring Service Quality and Promoting Decentring. The TQM
Magazine. Vol. 6, No. 5, 1994. Pp 50-55.