Evaluating Training and Learning Practices in Large Australian Firms

Susan Dawe

Susan Dawe
susandaw@ncver.edu.au
National Centre for Vocational Education Research Ltd.

Paper presented at the 2002 Australasian Evaluation Society International Conference October/November 2002
– Wollongong Australia www.aes.asn.au

Abstract
Australian studies of enterprise training during the mid-1990s found that while training practices
varied considerably between industry sectors and individual firms, formalised training and learning
practices were more likely to be found in larger organisations. These studies also found that the major
drivers of training in organisations were workplace change and innovation, and the need to
implement quality assurance processes and adapt to new technology. More recent studies of training
practices in organisations conducted by the National Centre of Vocational Education Research have
confirmed the importance of these factors. These studies have also found that the shift to self-
managing work teams, and the need for firms to maintain competitive advantage have required
training which is focussed on working closely with customers, suppliers and distributors. Also
highlighted has been widespread use of generic skills training focussed on communication,
negotiation, problem solving, mentoring, coaching, and skills required for innovation and the self-
management of learning and career development.

While evaluation of workplace training has generally been neglected, the implementation of
competency-based training approaches has allowed evaluation to occur more readily. For example,
as workplace training becomes more individualised and linked to individual performance reviews,
line managers and individuals will evaluate the training approaches. This paper reports on a study
that compares current and past practices and investigates the operation of other major drivers of
workplace training and evaluation of training approaches.

Key words
evaluation, training, enterprise, ‘workplace change’, ‘generic skills’, ‘competency-based’

Introduction
A recent study conducted by the National Centre for Vocational Education Research Ltd. (NCVER),
‘Determinants of successful training practices in large Australian firms’ by Susan Dawe (in press), aimed to
identify training and learning practices that demonstrated positive benefits over a period of time. It primarily
focussed on the integration of teaching and learning within the firm, factors that influence firms to recruit
existing skills as opposed to developing skills sets within the firm, the use of accredited training and training
packages, the development of career structures within firms, the impact of globalisation on training and learning
practices, and returns on their investment in training.

This paper reports on this study which examined the determinants of successful training and learning practices
and the factors that have influenced these practices in large Australian firms in recent times. It will also outline
how large firms undertake training evaluation and measure returns on investment in training.

Methodology
This study was conducted in three phases. Phase 1 comprised a review of the existing literature on training and
learning in the workplace, factors which influence enterprise training, the development of a training or learning
culture, and how the success of training is measured. Phase 2 comprised a meta-analysis of case studies that had
already been conducted. Forty-nine previous case studies of large Australian firms (those employing 100 or
more employees) from recent publications by Smith et al (1995), Noble et al (1996), Kearns and Papadopoulos

1
(2000), Johnston (2001), Figgis et al (2001) and Dawe (2002) were included. This analysis focussed on the
impact of globalisation, use of accredited training, demographic changes, new technology and changes in
industrial relations, which may influence the integration and development of training and learning practices,
career structures, the recruitment of skill sets required within these firms and evaluation of training practices.
Phase 3 comprised an update of these findings through five new case studies. Companies selected for this phase
included two large firms from the wine industry, a new unique tourism operator and a family-owned
manufacturing company. Also included for comparison was one large government organisation.

Determinants of successful training practices

Smith et al (1995) found that the major drivers of training from their study of forty-two firms were workplace
change and innovation. The need to provide quality assurance and adapt to new technology were also very
important. The development of a training culture within the organisation was found to be dependent on a number
of factors. These included management attitude to training, investment in training as part of the business
strategy, and management-employee relationships (Smith 1997). More recent studies of training practices in
organisations (Kearns and Papadopoulos 2000, Johnston et al 2001; Figgis et al 2001, Dawe 2002) also
highlighted the importance of ‘change and innovation’ as major factors driving workplace training.

The analysis of previous case studies found ten major elements which contributed to successful training
practices in large Australian firms. The new case studies also supported these findings. The three most important
elements included
® having in place an organisational culture that supports learning
® linking training to the major features of a business strategy
® responding to change within the organisation or external to it
The other seven elements included:
® increasing the diversity of training and learning approaches
® sourcing formal training from within the organisation itself
® adopting accredited training, often linked to the National Training Framework
® increasing the use of informal training
® decentralising the training within the organisation
® responding to the needs of the individual
® evaluating the training

An organisational culture in which there was respect for all individuals, a willingness to share knowledge and
expertise, and a positive attitude or ‘can do’ mentality among workers, was essential to successful training and
learning. ‘Open communication’ and, in particular, cross-functional team consultations, also stimulated learning
which led to innovative solutions.

A supportive learning culture also encouraged individual development through its human resource practices,
such as providing training and learning opportunities for all workers, career planning and promotion
opportunities within the organisation, and individual performance feedback in a performance management
system. Within such a culture, employee achievements were recognised through employee or team awards for
excellence, quality and consistency or goal-orientated bonus reward schemes. In addition, having a graded
competency system ensured that excellence in performance was recognised.

The linking of training to the business strategies, by conducting training needs analysis for the skills required to
implement these strategies, also led to ensuring its relevance to individuals and corporate objectives. Because
improving quality and consistency, and complying with occupational health and safety and environment
standards, were generally accepted as part of sound business practice, enterprise training now focussed on
maintaining a competitive advantage through increasing efficiency in production, developing innovative
solutions, products or services, and exceeding customer expectations. In the new case studies, all firms indicated
that they were market-driven and so listening to their customers was paramount. For example, engineers and
technicians were encouraged to talk with customers, such as the regular visits arranged for contractors, as the
manufacturing company believed that creativity and innovation came from listening to customers.

Responding to workplace change was another major element for ensuring the success of workplace training.
Whether the stimulus for workplace change was new technology, new management, new government legislation
or new competition, it remained the major driver of enterprise training.

At the strategic level, successful training was often part of the change management process in preparing staff to
accept or implement change. When training was related to current work practices, it became more relevant to

2
individual workers and this increased their motivation to learn. At the manufacturing plant, workplace trainers
believed that most operatives now enjoy change since the company was committed to open communication,
listened to employees and had supported their learning to prepare them for change.

The other elements which contribute to successful training practices highlight the increasing diversity of training
and learning approaches used in large firms. Large firms have more choices in how they resource their training
functions. They may source formal training from within the organisation itself or import customised training
from a range of external training providers. In addition, large firms are increasingly providing training towards
externally-recognised qualifications in order to attract and retain high quality staff for a competitive advantage in
a global market. Firms have been assisted in implementing training for national industry qualifications by the
industry-specific competency standards in training packages.

Large firms have found it cost effective to work closely with external training providers to develop customised
courses and learning materials to meet the needs of the company and their employees. These firms were able to
attract training providers to deliver the courses in the workplace or provide training facilitators on-site to support
employees’ learning. Other firms preferred to have their own staff qualified as workplace trainers and assessors
and be registered as a training organisation in their own right. For example, Coles Supermarkets had the Coles
Institute and the Accor hotel group had Académie Accor, which employed their own staff as qualified workplace
trainers and assessors, and developed customised learning and assessment materials to meet the national industry
competency standards (Dawe 2002).

Informal training also occurred in functional and cross-functional team meetings and project-based activities.
These large firms had in place ‘training coordinators’ who were responsible for developing the Group Training
Plan but, in general, training delivery was decentralised within the organisation. Supervisors were increasingly
taking on the role of workplace trainers and coaches for their teams. This was a response to the higher level
skills required of operatives and the expectation that they will be able to solve problems and make decisions.

Self-managing teams have become common practice in large firms and this has increased the importance of
learning within the team and the use of mentors in the workplace and professional or business networks. At
Aspect Computing in South Australia, information technology professionals have been able to improve their
communication skills, including negotiation, presentation, and business communication, by working on a
technical project for a client, and being guided through this by a trained mentor (Dawe 2002). This learning
strategy has enabled them to learn about communication skills and easily transfer them into practice in their
work activities.

At one firm, it was recognised that working in teams required a greater understanding of one’s own style of
working and that of other workers. The Training Plan for a particular department in this firm focussed on
enhancing teams by identifying personality types (for example, using Myer Briggs analysis) and preferred
learning styles of individual workers. The Learning Style questionnaire to be completed by all employees
explored the four Honey and Mumford learning styles (that is, activist, reflectors, theorists and pragmatist).
Knowing their learning style preference equipped workers to choose learning opportunities, and to expand their
repertoire and become all round learners. In addition, knowing the personality type and preferred learning styles
of team members assisted the team leader in the further development and motivation of individuals within the
team.

Increasingly, individuals have been able to help identify their own particular training needs through the
completion of individual training and development plans. While team-building, computer skills, workplace
safety and quality training were generally required to implement the firm’s business strategies, other training for
multi-skilling and personal development undertaken by employees was influenced by the individual’s career
goals and interests.

Another indication of successful training practices in large firms was the formal evaluation of training. In
reviewing training and assessing the feedback from trainees, customers and line managers, firms were able to
modify training and increase learning to ensure that they met the needs of the employees and the company. One
firm in this study had an evaluation model in which the value of training to the company’s profitability was
identified.

3
Evaluating training and measuring returns on investments in training
This study has found that the evaluation of training was most often based on feedback from trainees, employees,
line managers and customers. The human resource manager, or training coordinator, in these firms conducted
the evaluations and was responsible for recommendations for future training investment to senior managers. The
arguments for investment in training were based on evidence of long-term benefits derived from improved
employee relations, increased confidence and self-esteem of the workforce, and the retention of highly qualified
and experienced staff to maintain a competitive advantage in the market place. Other benefits of investment in
training included increased efficiency through improvement in the analysis of risk by operators and so
minimising wastage of products or workplace accidents.

In the main, the training coordinators sought verbal feedback from employees and line managers or collected
information from formal feedback sheets to evaluate the effectiveness of training. Formal structures such as
staff/management Joint Consultative Committees or Training Advisory Committees consisting of workplace
trainers and managers, were also used in some firms to evaluate training and make recommendations on future
training programs. In other firms, surveys of customers and/or staff members were regularly conducted to
measure satisfaction with staff training.

Increasingly, large firms had in place a performance management system where, at half-yearly or yearly
intervals, formal appraisals of individuals’ performance were conducted by line managers and individual
training plans were developed. In some firms, feedback on the success of on- and off-the-job training was
gathered by the human resource manager from appraisal forms after the staff performance appraisals had been
conducted by line managers. Training coordinators also obtained information following formal three-monthly
appraisals conducted for apprentices or trainees or formal appraisal of new employees after an induction period.

This study also supported the findings of Dockery (2001) which recommended that research into the benefits
and impacts of training should look at the specific purpose of the training rather than general performance
measures. Until recently, few Australian firms carried out systematic evaluation of their training and fewer
attempted to calculate the returns on their investment in training. However, four research projects managed by
NCVER examined the returns to enterprises of their investments in training using a variety of methodologies
(Blandy et al, 2000, Doucouliagos & Sgro, 2000, Maglen, Hopkins and Burke, 2001, Moy & McDonald, 2000).
One of the aims of these projects was to develop ways in which enterprises might be able to measure returns for
themselves. Another NCVER-managed project investigated the development of training and learning cultures in
Australian enterprises and the non-financial returns that enterprises can gain from their training investments
(Figgis et al, 2001). NCVER published the findings and outcomes of these returns on investment research
projects in its series ‘Research at a glance’ (NCVER, 2001) and ‘Research readings’ (Smith 2001).

Training represents an investment by firms in their employees. Like other investments undertaken by firms, a
cost is incurred in anticipation of a future return to the firm. Measuring returns to training investment was not
considered an easy task (NCVER, 2001). The lack of this type of evaluation of training in most firms was likely
to be related to accounting systems where training costs were treated as an inseparable part of labour costs and
not as an investment from which a return was expected. In addition, a range of measures needed to be developed
including measures in productivity, profitability, efficiency and value-added activities (NCVER 2001). Value-
added activities may occur as a result of greater levels of employee skills, increased flexibility amongst
employees, reduced overhead costs (more efficient use of existing facilities), and greater ability to innovate,
adopt new technology and change work practices.

The key objective of the returns on investment in training evaluation model developed by a particular
department of one of the firms in this study was to allow the company to identify the value of training to its
profitability. Data was collected for the key components of the department’s Training Plan including training to
meet industry, quality and environment standards. For example, detailed records of production, including the
causes of down-time, were used to identify opportunities for improvement in productivity. Causes related to
operator errors, adjustments and change-over times were identified and analysed for training implications. The
collection of this data allowed improvements in down-time to be attributed to operator training. The costs
attributable to training could be linked directly to improvements in productivity and returns on training
investment determined (that is, returns on training investment = benefits/training costs x 100%).

Conclusions
This study of large Australian firms found that ten elements were the foundations for best practice in enterprise
training and learning practices. The first three elements are the most important factors in contributing to
successful training practices. These include having in place:

4
® an organisation culture that supports learning
® mechanisms to link training to the business strategy
® mechanisms to link training to workplace change

The other elements contributing to successful training practices relate to the customisation of training for
enterprises and individual workers and include:
® increasing the diversity of training and learning approaches
® sourcing formal training from within the organisation itself
® adopting accredited training, often linked to the National Training Framework
® increasing the use of informal training
® decentralising the training within the organisation
® responding to the needs of the individual
® evaluating the training

Determining the returns on training investments requires careful consideration and the development of specific
and appropriate measures of productivity, profitability, efficiency and value added activities. The reports of the
NCVER-managed research projects on returns on investment in training note the ways in which enterprises
might be able to measure returns for themselves. NCVER reports1 are available from www.ncver.edu.au.

References
Blandy R, Dockery, M, Hawke, A and Webster, E 2002, ‘Does training pay? Evidence from Australian
enterprises’, NCVER, Adelaide.
Dawe, S 2002, ‘Focussing on generic skills in training packages’, NCVER, Adelaide
Doucouliagos, C and Sgro, P 2000, ‘Enterprise return on a training investment’, NCVER, Adelaide.
Figgis, J, Blackwell, A, Alderson, A, Butarac, A, Mitchell, K and Zubrick, A 2001, ‘What convinces enterprises
to value training and learning and what does not?’, NCVER, Adelaide.
Honey, P and Mumford, A, Self-assessment Learning Style questionnaire (http://www.peterhoney.co.uk
accessed 15 July 2002)
Johnston, R, Hawke, G, Harris, R, Bridge, K, Comyn, P, Thoneman, A and Willis, P 2001, ‘Case studies of
organisations with established learning cultures’, NCVER (NR9014), Adelaide.
Kearns, P and Papadopoulos, G 2000, ‘Building a learning and training culture: The experience of five OECD
countries’, NCVER, Adelaide.
Maglen, L, Hopkins, S and Burke, G 2001, ‘Training for productivity’, NCVER, Adelaide.
Moy, J and McDonald, R 2000, ‘Analysing enterprise returns on training’, NCVER, Adelaide.
NCVER 2001, ‘Research at a glance: returns on investment in training’, NCVER, Adelaide.
Noble, C, Smith, A and Gonci, A 1996, ‘Enterprise training in Australia: Industry profiles and case studies’,
Office of Training and Further Education, Melbourne.
Smith, A 2001, ‘Return on investment in training: research readings’, edited A. Smith, NCVER, Adelaide.
Smith, A, Roberts, P, Noble, C, Hayton, G and Thorne, E 1995, ‘Enterprise training: The Factors that Affect
Demand. Final report’ Volume 1, 2, 3 and 4, Office of Training and Further Education, Melbourne.
1
NCVER also produces the VOCED database which has been endorsed as the UNESCO/NCVER international
research database of abstracts for TVET research. VOCED is a unique and essential tool for finding current
information. VOCED is available free on the web at www.voced.edu.au