Marketing Intelligence & Planning

Measuring sales training effectiveness at the behavior and results levels using self- and
supervisor evaluations
Ashraf M. Attia Earl D. Honeycutt Jr
Article information:
To cite this document:
Ashraf M. Attia Earl D. Honeycutt Jr, (2012),"Measuring sales training effectiveness at the behavior and
results levels using self- and supervisor evaluations", Marketing Intelligence & Planning, Vol. 30 Iss 3 pp.
324 - 338
Permanent link to this document:
http://dx.doi.org/10.1108/02634501211226294
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Downloaded on: 28 February 2016, At: 21:15 (PT)

References: this document contains references to 48 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 1421 times since 2012*
Users who downloaded this article also downloaded:
Piyali Ghosh, Jagdamba Prasad Joshi, Rachita Satyawadi, Udita Mukherjee, Rashmi Ranjan,
(2011),"Evaluating effectiveness of a training programme with trainee reaction", Industrial and Commercial
Training, Vol. 43 Iss 4 pp. 247-255 http://dx.doi.org/10.1108/00197851111137861
Piyali Ghosh, Rachita Satyawadi, Jagdamba Prasad Joshi, Rashmi Ranjan, Priya Singh, (2012),"Towards
more effective training programmes: a study of trainer attributes", Industrial and Commercial Training, Vol.
44 Iss 4 pp. 194-202 http://dx.doi.org/10.1108/00197851211231469
Wei-Tao Tai, (2006),"Effects of training framing, general self-efficacy and training motivation
on trainees' training effectiveness", Personnel Review, Vol. 35 Iss 1 pp. 51-65 http://
dx.doi.org/10.1108/00483480610636786

Access to this document was granted through an Emerald subscription provided by emerald-srm:566188 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald for
Authors service information about how to choose which publication to write for and submission guidelines
are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as well as
providing an extensive range of online products and additional customer resources and services.
 Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the Committee
on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive
preservation.

*Related content and download information correct at time of download.

Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

MIP
30,3
324
Received 3 July 2011
Accepted 4 July 2011
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)
Marketing Intelligence & Planning
Vol. 30 No. 3, 2012
pp. 324-338
q Emerald Group Publishing Limited
0263-4503
DOI 10.1108/02634501211226294
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0263-4503.htm
Measuring sales training
effectiveness at the behavior and
results levels using self- and
supervisor evaluations
Ashraf M. Attia
Marketing Department, SUNY Oswego, Oswego, New York, USA, and
Earl D. Honeycutt Jr
Department of Marketing and Entrepreneurship, Elon University, Elon,
North Carolina, USA
Abstract
Purpose – The purpose of this article is to improve the sales training process by gaining a deeper
understanding of the first two levels – reaction and learning.
Design/methodology/approach – A total of 79 sales supervisors of a US firm operating in Egypt
participated in the study. They evaluated level 1 (reaction) by rating the design and operation of their
sales training programs. Level 2 measured the value of information gained from training topics.
Findings – Trainees stated that the training either helped or solved sales and non-sales problems.
Data from both training levels were factored analyzed and each resulted in a two-factor solution.
Research limitations/implications – This research offers advice for setting standards for
evaluating sales training programs. Knowledge gained is more important to determining if sales
training has been effective. Learning evaluation showed that trainees believed the value of selling
skills, company information, and behavioral training topics were valuable and the role of the instructor
was important.
Practical implications – This research offers advice for setting standards for evaluating sales
training programs. Knowledge gained is more important to determining if sales training has been
effective. Learning evaluation showed that trainees believed the value of selling skills company
information and behavioral training topics were valuable and the role of the instructor was important.
Originality/value – Since global firms devote significant sums of time and money training their
sales forces, the greater understanding of the process results in improved effectiveness of sales
training programs.
Keywords Sales management, Sales training, Evaluation, Reaction, Learning, Training effectiveness
Paper type Research paper
Introduction
Sales training is important because: there have been dramatic changes in the selling
and sales management environment over the last decade (Jones et al., 2005) to include
the nature of the sales position (Cron et al., 2005; Tanner et al., 2008); and sales training
contributes significantly to salesperson knowledge and skill levels, performance,
effectiveness (Lichtenthal and Tellefsen, 2001; Piercy et al., 1998), customer orientation
(Pettijohn et al., 2009), and overall firm performance (Pelham, 2002). Businesses invest
substantial time, money, and effort training their sales team with ultimate objectives of
increasing sales productivity and profitability (Wilson et al., 2002).
 When firms invest significant resources in sales training, they must assess their Measuring sales
sales training programs and outcomes to ensure that these investments help them training
reach their goals (Attia et al., 2002). However sales training evaluation, the most critical
issue facing sales force development efforts (Leach et al., 2005), globally receives a low effectiveness
priority (Galanou and Priporas, 2009).
The purpose of this research is to extend our understanding of sales training
evaluation by empirically assessing Kirkpatrick’s behavior and results assessment 325
levels of a sales training program utilizing self- and supervisor-evaluations. Assessing
higher levels of training effectiveness permits sales managers to plan and conduct
more effective future training programs.

Evaluating sales training: theoretical background and hypotheses

development
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

In the 1950s, sales managers and sales trainers recognized the importance of training
assessment but they were uncertain how to accomplish the procedure. To address this
lacuna, Kirkpatrick (1959a, b, 1960a, b) proposed a model that included four training
evaluation levels:
(1) reaction;
(2) learning;
(3) behaviour; and
(4) results.

These four evaluation levels progress in order from the least to the most complex to
accomplish (Honeycutt and Stevenson, 1989).
In the half-century since the introduction of Kirkpatrick’s model, a handful of other
training evaluation models have been proposed. While myriad articles relate to sales
training, only a few demonstrate how to empirically evaluate a sales training program
(e.g. Honeycutt et al., 2001). Extant literature suggests that additional study is needed
to empirically measure sales training effectiveness at the higher assessment levels. In
response, this article empirically assesses Kirkpatrick’s third (behavior) and fourth
(results) levels of training using self- and supervisor-evaluations.

Interactive evaluative levels

Sales training evaluation is the systematic collection of information necessary to
determine the effectiveness of sales training activities and the outcomes of those actions.
The first two training levels – reaction and knowledge – have both been criticized
(Summy, 2007). That is, reaction ratings that assess trainee satisfaction or perceptions
about training programs can be influenced by instructor personality or skewed by
learning environments that are beyond the instructor’s control. Reaction or “happiness
sheets” have been described as being worse than useless (Broadwell, 1989). Likewise, for
knowledge or level two, trainees can learn sales principles and techniques but be unable
or unwilling to employ them on the job. For example, buyer-seller simulations employed
to teach sales skills are often taught in a sterile classroom setting. For these reasons it is
recommended that higher-level measures be utilized (Summy, 2007) to evaluate trainee
outcomes in order to confirm that trainees are applying the skills and knowledge that
was transferred in training sessions (Kirkpatrick, 1960a).
 MIP Level 3 measures individual job behavior after receiving training and level 4
30,3 assesses individual and organizational results as an outcome of salesperson knowledge
being applied on-the-job. According to one study (Tyler, 2002), companies assess
training at the reaction level most often (78 percent), followed by learning (32 percent),
behavior change (9 percent), and results (ROI or financial value) (7 percent).

326 Measuring behavior

Identifying job behavior that is attributable to sales training is a more complex and
time-consuming assessment than measuring reaction and learning levels (Kirkpatrick,
1960a). Reaction and knowledge can be “evaluated” in a single event during or at the
end of a training program. Evaluating the third level or behavior requires:
.
sufficient time for change(s) to occur;
.
advanced experimental design and analytics be employed;
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

.
obtaining 100 percent response rates or employing a random sample;
.
surveying and/or interviewing the trainees, their immediate supervisors, their
subordinates, their peers, and/or customers who regularly interact with the
salesperson;
. repeating the evaluation(s) when necessary; and
.
comparing cost and benefits (Kirkpatrick, 1994).

A number of empirical studies (Leach and Liu, 2003; Jobber et al., 1993; Ingram et al.,
1992) record that managers believe that qualitative measures offer insight into training
effectiveness for levels three and four. Perhaps managers believe that
qualitative-oriented variables provide more useful information because positive
behavior leads to salesperson success (Morris et al., 1994). For example, when
organizations manage relationship selling in the form of CRM pipeline analysis,
behavioral variables (sales calls/proposals) are viewed as leading to outcome measures
like sales revenue (Tanner et al., 2009).

Measuring results
Measuring individual and organizational results attributable to sales training is the
most complex evaluation level to assess (Summy, 2007; Honeycutt, 1996).
Recommended objective measures of training program effectiveness include sales
per trainee or sales revenue to quota per trainee (Phillips, 1991). However, these
measures become unreliable when sales territories vary, extraneous factors influence
sales figures, or data are irregular or seasonal. Also performance assessment that
focuses on a single output measure may be inadequate for assessing the learning
process (Rich et al., 1999), which may partially explain why managers rely on
qualitative measures to assess training’s impact (Kumpikaite, 2007).
To date, three successful operational evaluations of sales training results have been
conducted. First, Meyer and Raich (1983) incorporated an experimental design that
matched 14 retail stores into seven groups, based upon market characteristics and
location. Average sales commission was utilized as the evaluation criterion and
employees who received training earned a statistically higher commission rate and
remained in their positions longer.
 Also within the UK retailing environment, Doyle and Cook (1984) employed a before Measuring sales
and after with control group experimental design study that paired store locations for a training
major UK chain of 263 fashion shops. One retail location was trained, while the other
served as a control store. The results showed that average weekly sales revenue at effectiveness
stores receiving training was significantly higher than for control group outlets. In
addition, trained salespersons accomplished multiple sales at a significantly higher
level. 327
Last, Cavusgil (1990) evaluated sales training for independent Caterpillar dealers in
Central America by matching locations and providing training to one of the locations.
As predicted, sales were higher at the geographical locations that received training. All
three studies used store or location as the unit of measure and confirm that it is
possible to measure the cumulative effect of training for a location rather than at the
individual level.
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Hypotheses development
In order to evaluate salesperson behavior (level 3), five different assessment methods are
possible: customer-evaluation (Lambert et al., 1997), self-evaluation, supervisor-evaluation
(Tziner and Falbe, 1993), peer-evaluation, and subordinate evaluation (Erffmeyer et al.,
1991). To measure the net effect of training and to ensure that the sales training program
yields positive behavior changes (level 3) and beneficial performance results (level 4), we
employ an experimental design approach (before and after measures with a control group)
(Kumpikaite, 2007) to assess self- and supervisor-evaluations.
The use of before and after measures with experimental and control groups is a
powerful way to evaluate sales training outcomes (Attia et al., 2002). Based upon earlier
studies regarding the significant positive impact of training on salesperson
effectiveness, sales, and performance (e.g. Honeycutt et al., 2001; Lichtenthal and
Tellefsen, 2001), the behavior and results improvements of the experimental group
(trainees) are expected to be significantly higher than those of the control group
(non-trainees) for both self- and supervisor-evaluations.
In this study trainees are compared to non-trainees to test the effectiveness of sales
training program outcomes across Kirkpatrick’s level 3 (behavior) and level 4 (results)
to test the following hypotheses. Therefore:
H1. Trainee behavior improvement will be significantly higher than for
non-trainees.
H1.1. The behavior improvement reported by trainee self-evaluation will be
significantly higher than that reported by non-trainees.
H1.2. The behavior improvement reported by trainee supervisor-evaluation will be
significantly higher than that achieved by non-trainees.
H2. Trainee results improvement will be significantly higher than for
non-trainees:
H2.1. The results improvement reported by trainee self-evaluation will be
significantly higher than that achieved by non-trainees.
H2.2. The results reported by trainee supervisor-evaluation will be significantly
higher than that achieved by non-trainees.
 MIP Methodology
30,3 Measurement instruments
In level 3, the behavior improvement of trainee performance attributable to the sales
training intervention is measured using 21 variables. In level 4, trainee results
improvement is assessed using four variables that are based on the efficiency and
effectiveness logic that sales training leads to an increase in meeting organizational
328 objectives and results using the same level of resources or achieving similar
organizational objectives and results with fewer resources (Bashaw et al., 2002).
To measure behavior and results improvement, a nine-point Likert scale was
employed, where “9” represents “Excellent,” “5” equates to “Average,” and “1” signifies
“Needs improvement.” Attitude change can be measured by collecting questionnaires
from supervisors, subordinates, and clients, or by observing the salesperson in action
(Honeycutt and Stevenson, 1989). This study utilized supervisor- and self-appraisal.
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Sales training program design

To identify training needs, training objectives, and topics/content, meetings were
conducted at two levels. First, a series of individual and group meetings were held
between company executives and the training consultant. Second, ten sales
supervisors and four section heads served as representatives of trainees and
management. A brainstorming technique was employed at both individual and group
meetings to generate the sales training program conducted for sales supervisors.

Sample
The data were collected from Egyptian sales supervisors from a single, anonymous
global company operating in the soft drink industry. In this firm, front-line sales
representatives are drivers, who are order takers and distributors that neither solve
problems nor open new sales accounts. The sales supervisor sells the product, solves
customer problems, negotiates, closes the sale, and opens new sales accounts. Based
upon the sales training needs analysis, the sales supervisors were the individuals that
were selected for training.
A single firm was employed in this study in order to control for extraneous market
forces. Previous studies employed a single company to evaluate training program
effectiveness at the results level (Cavusgil, 1990; Doyle and Cook, 1984; Meyer and
Raich, 1983). At the time of the needs analysis, the participating company employed
143 sales supervisors.
All sales supervisors were trained in two phases: five groups were trained in phase 1,
and four groups trained in phase 2 (seven months later). In phase 1, 79 sales supervisors
received training as an experimental group, while the remaining 64 sales supervisors
served as a control group. Shortly after completing the training program, 11 sales trainees
resigned and began working for the major competitor, four trainees were promoted, and a
number of non-trainees left the firm. As a result of these personnel changes, the final
sample size, decreased to 59 trainees and 42 non-trainees, for a total of 101 sales trainees.
The assignment of sales supervisors to experimental and control groups was based
upon decisions made by company executives and mirrored previous research studies
(Erffmeyer et al., 1991, 1993). Criteria used to assign sales supervisors to groups were
based upon the nature of the sales territory, competition, workload, and experience
levels. A comparison of pre-training evaluations by trainees and their supervisors
 confirmed no significant differences in ratings. This suggests that the assignment of Measuring sales
respondents, while not truly random, produced two relatively balanced samples. training
Data collection procedures
effectiveness
Both self-evaluation and supervisor-evaluation makeup the 25 variables measuring
behavior and results were completed by all members of the experimental and control
groups in order to compare scores (Mezoff, 1987). Because trainees tend to report a higher 329
evaluation than their superiors (Connolly, 1987), all members of both experimental and
control groups were evaluated on the identical by their supervisor. Salinger and Deming
(1987) recommend testing participants several times before and after training to ensure
that the knowledge or skill level gained through training is maintained. Accordingly, the
post-treatment evaluation of trainees and non-trainees occurred three (Post-test 1) and
four (Post-test 2) months after the treatment was administered.
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

The researchers requested such ratio data as sales revenue, sales-to-quota, and
current sales/last year sales for all members of experimental and control groups.
However, no data were provided. As a result, both self-evaluation and the
supervisor-evaluation forms were used to compare training program results for
members of experimental and control groups.

Data analysis
Within each category, three forms of analyses were performed:
(1) Self-evaluation and supervisor-evaluation (Post-test 1 – Pre-test) – emphasizes
the behavior or results improvement for both trainees and non-trainees that is
calculated by totalling 21 behavior measures (or four results measures),
separately, for each respondent for both the Pre-test and Post-test 1. Then, the
total score of the pre-test is deducted from the total score of Post-test 1 to
generate the behavior or results improvement score from the Pre-test period to
Post-test 1 (three months after training).
(2) Self-evaluation and supervisor-evaluation (Post-test 2 – Pre-test) – the behavior
improvement for both trainees and non-trainees is computed by adding the
scores of the 21 behavior measures (or four results measures) for all respondents
for both Pre-test and Post-test 2. Then, the total score of the Pre-test is deducted
from the total score of Post-test 2 to generate the behavior or results
improvement score from the Pre-test period to Post-test 2.
(3) Self-evaluation and supervisor-evaluation [(Post-test 1 þ Post-test 2)/2] –
Pre-test]. The average behavior/results improvement for both trainees and
non-trainees is calculated by adding all the scores of the 21 behavior measures (or
four results measures) for each respondent for Pre-test, Post-test 1, and Post-test
2, separately. Finally, the total score of the Pre-test is deducted from the average
score of post-test 1 and post-test 2 to generate the average behavior or results
improvement score from the Pre-test period to Post-test 1 and Post-test 2.

Reliability analysis
Cronbach’s alpha was employed to measure the reliability and consistency of the
instrument (see Table I). The Cronbach’s alpha scores for behavior ranged from 0.8650
to 0.9889. In measuring level 4 (results), Cronbach’s alpha ranges from 0.8568 to 0.9498.
 MIP
Total
30,3 (trainees
Number of Non- þ non-
Level Evaluation variables Trainees trainees trainees)

3 Self-evaluation (pre-test) 21 0.9640 0.9818 0.9741

330 3 Self-evaluation (post-test 1) 21 0.9526 0.9780 0.9707
3 Self-evaluation (post-test 2) 21 0.9528 0.9681 0.9653
3 Supervisor-evaluation (pre-test) 21 0.9817 0.9889 0.9863
3 Supervisor-evaluation (post-test 1) 21 0.9734 0.9746 0.9747
3 Supervisor-evaluation (post-test 2) 21 0.8650 0.9720 0.9262
4 Self -evaluation (pre-test) 4 0.8694 0.9527 0.9160
4 Self-evaluation (post-test 1) 4 0.8618 0.9182 0.9020
4 Self-evaluation (post-test 2) 4 0.8568 0.8516 0.8668
Table I. 4 Supervisor-evaluation (pre-test) 4 0.9479 0.9509 0.9498
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Cronbach’s Alpha for 4 Supervisor-evaluation (post-test 1) 4 0.9378 0.9028 0.9224

trainees and non-trainees 4 Supervisor-evaluation (post-test 2) 4 0.9340 0.8957 0.9207

The scales for behavior and results assessment exceed recommended standards of 0.80
(Carmines and Zeller, 1979).

Results and discussion

Measuring behavior
In order to examine level 3, we compared general results, covariates, and the
underlying dimensions of behavior for both self-evaluation and supervisor-evaluation.
From the findings shown in Table II, the behavior of trainees improved through
positive scores that ranged from 17.78 to 25.52 on the 21 criteria. That is, an average
behavior improvement of one point [(17.78 þ 25.52)/ 2 ¼ 21:65
occurred per criterion,
which means that the trainees’ behavior improved significantly from the pre-test
period to Post-test 1 and Post-test 2.
However, when using an experimental design approach (before-after with a control
group) and performing both ANOVA (Analysis of Variance) and MANOVA (Multivariate
Analysis of Variance), behavior improvement exhibited insignificant statistical differences
between the trainees and non-trainees for both self-evaluation and supervisor-evaluation.

Behavior Behavior
improvement improvement
for trainees for non-trainees
Type of evaluation n Meana n Meana

Self-evaluation (post-test 1 – pre-test) 19.88 19.64 18.09 17.38

Self-evaluation (post-test 2 – pre-test) 25.52 25.01 24.83 24.43
Self-evaluation [[(post-test 1 þ post-test 2)/2] – pre-test] 22.88 22.5 21.21 20.67
Supervisor-evaluation (post-test 1 – pre-test) 17.78 17.55 15.44 16.13
Supervisor-evaluation (post-test 2 – pre-test) 24.36 23.97 21.05 22.15
Table II. Supervisor-evaluation [[(post-test 1 þ post-test 2)/2] – before] 21.31 20.99 18.24 19.15
Measuring behavior
improvement for trainees Note: None of these differences are significant; aAfter controlling for the previous sales training
and non-traineesa experience covariate
 Therefore, H1.1 and H1.2 were not statistically supported; however, trainee behavior Measuring sales
improvement seen in Table II is uniformly higher than that of non-trainees. This training
suggests that the sales training program was beneficial at the behavior level since
changes are positive and in the proper direction. effectiveness
A potential reason for the insignificant statistical results associated with
self-evaluation of behavior is that self-report measures complicate the assessment of
change because of the definition of change (Arvey and Cole, 1989). Golembiewski et al. 331
(1976) and Zmud and Armenakis (1978) described three major kinds of change that can
occur with self-reported data:
(1) Alpha change: the true observed difference between the Pre-test and the
Post-test scores on the construct of interest attributable to sales training
intervention.
(2) Beta change, occurs when a true alpha change is confounded by recalibration of
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

the scale used to measure the construct of interest. Numerous researchers (Rahn,
1989; Preziosi and Legg, 1989; Mezoff, 1987) describe a common beta change,
called response-shift bias. When the trainee takes the pre-test, s/he probably feels
that s/he already is an expert. For example, a trainee scores 7 on a scale of one to
nine on a criterion assuming that s/he is knowledgeable about the subject. Then,
when the trainee takes the training program, s/he finds himself/herself already
lacking knowledge and skills, which caused an overestimation of 7 as a pre-test
score; that is, a lower score would have been more realistic. Consequently, after
training, the trainee again gives himself/herself a score of 7, giving the spurious
impression that the training had little or no effect.
(3) Gamma change, which is the subject’s re-conceptualization of the construct of
interest; that is, the subject’s personal understanding of time management for
example may change qualitatively as a result of training. Consequently, the
subject’s score on a questionnaire (designed to tap this construct) reflects
different criteria at pre-test in comparison to the post-test period.

Last, alpha, beta, and gamma changes are not mutually exclusive; that is, all three can
affect a single subject.

Underlying dimensions of behavior through self-evaluation

In the self-evaluation level, as shown in Table III, the 21 criteria of trainee behavior
grouped into two factors, with eigenvalues greater than 1 for both post-tests.
The first factor for both Post-test 1 and Post-test 2 includes the same variables that
load highly on the first factor labelled “Salesperson Capabilities & Skills.” Consistently,
the second factor includes identical variables for Post-test 1 and Post-test 2 that was
named “Work Environment & Culture.”
The cumulative percentage of variance explained by the first factor, “Salesperson
Capabilities & Skills,” is 64 percent for Post-test 1 and 67 percent for Post-test 2. The
second factor, “Work Environment & Culture,” explains 6 percent of the cumulative
percentage of variance for post-test 1 and 5 percent for post-test 2. To obtain this factor
solution, a varimax rotation method was used and 70 percent and 72 percent of
cumulative percentage of variance was explained, respectively, by both factors for
Post-test 1 and Post-test 2.
 MIP
The effective selling points Factor 1 Factor 2
30,3
1. Effective time management 0.72817 0.30033
2. Efficiency in closing 0.82333 0.30624
3. A better planning of sales routing 0.56848 0.51494
4. Negotiation effectiveness 0.82436 0.33945
332 5. Decrease in the rate of complaints 0.59668 0.47911
6. Increase the efficiency in handling objections 0.73591 0.43163
7. Relationship w/ customers 0.75699 0.33718
8. Relationship w/ key customer accounts 0.71685 0.28470
9. Relationship w/ peers 0.36516 0.78680
10. Relationship w/ subordinates 0.37697 0.79959
11. Relationship w/ supervisors 0.26801 0.80180
12. Building team spirit 0.33386 0.75839
13. Teamwork 0.42489 0.76902
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

14. Willingness to accept criticism & feedback 0.65219 0.41858

15. Maturity & bearing responsibilities 0.80065 0.32556
16. Hard worker 0.76297 0.39107
17. Initiation 0.78806 0.31976
18. Creativity and innovation 0.67353 0.49005
19. Challenging personality with the sincerity in succeeding and growing 0.76463 0.42315
Table III. 20. Credibility 0.59688 0.44057
Loadings for 21. Aggressive and a strong personality 0.72526 0.41257
self-evaluation of
behavior Note: Factor 1: “Sales Capabilities and Skills”; Factor 2: “Work Environment and Culture”

Underlying dimensions of behavior through supervisor-evaluation

In the supervisor-evaluation level, the 21 criteria of trainees’ behavior are also grouped
into two factors with eigenvalues greater than one, for Post-test 1 and Post-test 2.
Table IV shows the factor loadings for the 21 criteria for measuring behavior for
Post-test 1. The Bartlett test of sphericity is significant (Post-test 1: X2 ¼ 2386:3624; P
, 0.000; Post-test 2: X2 ¼ 2347:1135; P , 0.000). The first factor for both post-test 1
and post-test 2 includes almost the same variables of self-evaluation in Table IV that
load highly on the first factor, which is labelled as “Salesperson Capabilities & Skills.”
Likewise, the second factor includes nearly the same variables of self-evaluation seen
in Table IV. This factor is named “Work Environment & Culture.”
The cumulative percentage of variance explained by the first factor, “Salesperson
Capabilities & Skills,” is 67 percent for Post-test 1 and 64 percent for Post-test 2. The
second factor, “Work Environment & Culture,” explains 6 percent of the cumulative
percentage of variance for Post-test 1 and 7 percent for Post-test 2. To obtain this factor
solution, a varimax rotation method is used and 73 percent and 71 percent of
cumulative percentage of variance is explained by both factors for Post-test 1 and
Post-test 2, respectively. In summary, the variables that loaded highly on the first and
second factors for both the self- and supervisor-evaluations were nearly identical.

Measuring results
Table V presents the comparison of measurement improvement results of trainees vs.
non-trainees. For both self- and supervisor-evaluations, trainee results improved by
positive scores that ranged from 3.90 to 5.29 for the four criteria. That is, on average
 Measuring sales
The effective selling points Factor 1 Factor 2
training
1. Effective time management 0.70607 0.42129 effectiveness
2. Efficiency in closing 0.79272 0.43559
3. A better planning of sales routing 0.81383 0.29029
4. Negotiation effectiveness 0.77236 0.44064
5. Decrease in the rate of complaints 0.70674 0.38464 333
6. Increase the efficiency in handling objections 0.81446 0.31171
7. Relationship w/ customers 0.65319 0.56308
8. Relationship w/ key customer accounts 0.64783 0.38316
9. Relationship w/ peers 0.35902 0.80442
10. Relationship w/ subordinates 0.28817 0.87539
11. Relationship w/ supervisors 0.27728 0.87415
12. Building team spirit 0.51129 0.68983
13. Teamwork 0.59220 0.62116
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

14. Willingness to accept criticism & feedback 0.42868 0.61689

15. Maturity & bearing responsibilities 0.71055 0.44825
16. Hard worker 0.58171 0.57083
17. Initiation 0.79289 0.38727
18. Creativity and innovation 0.82140 0.30171
19. Challenging personality with the sincerity in succeeding and growing 0.82849 0.30575
20. Credibility 0.57437 0.54111 Table IV.
21. Aggressive and a strong personality 0.69865 0.46223 Loadings for
supervisor-evaluation of
Note: Factor 1: “Salesperson Capabilities and Skills”; Factor 2: “Work Environment and Culture” behavior

Results
Results improvement
improvement for non-
Type of evaluation for trainees trainees

Self-evaluation (post-test 1 – pre-test) * *, * 3.90 3.88; 3.94 3.03 3.20; 3.00

Supervisor-evaluation (post-test 1 – pre-test) * * * * 4.04 3.94; 3.99 2.54 2.65; 2.47
Self-evaluation (post-test 2 – pre-test) 5.00 4.94; 5.02 4.13 4.33; 4.07
Supervisor-evaluation (post-test 2 – pre-test) 5.29 5.10; 5.18 4.10 4.35; 4.03
Self-evaluation [[(post-test 1 þ post-test 2)/2] – pre-test] * 4.45 4.41; 4.48 3.58 3.77; 3.53
Supervisor-evaluation [[(post-test 1 þ post-test 2)/2] 2 pre-test] * * * 4.69 4.54; 4.61 3.32 3.50; 3.35
Notes: *Mean differences significant at the 10 percent level of significance between the results
improvement for trainees and non-trainees after controlling for the sales region covariate; * *Mean
differences significant at the 10 percent level of significance between the results improvement for Table V.
trainees and non-trainees; * * *Mean differences significant at the 5 percent level of significance Measuring results
between the results improvement for trainees and non-trainees; * * * *Mean differences significant at improvement for trainees
the 1 percent level of significance between the results improvement for trainees and non-trainees and non-trainees

results improved by more than one point (3.90 þ 5.29/2 ¼ 1:15Þ per criterion on the
nine-point scale, which means that the trainees’ results improved significantly from
Pre-test to Post-test 1 and Post-test 2.
To measure the effect of the current sales training program on both self- and
supervisor-evaluations, an experimental design approach was used and MANOVA
was employed to control the overall error rate. In this case, there are two dependent
 MIP variables (results improvement through self-evaluation and results improvement
30,3 through supervisor-evaluation) and one independent variable (the sales training
program). By using the F-test of significance with (1,86) degrees of freedom, the results
improvement shows significant differences between the trainees and non-trainees for
Post-test 1 self-evaluation when a 10 percent level of significance is considered
(F ¼ 3.13; p-value ¼ 0.08).
334 After controlling for sales region as a covariate and using the F-test of significance
with (1,84) degrees of freedom for post-test 1 and (1,83) degrees of freedom for Post-test
1 þ Post-test 2, the current sales training program has a significant effect on the results
improvement of sales supervisors generated by self-evaluation for both Post-test 1
ðF ¼ 3:44; p-value ¼ 0.06), and Post-test 1 þ Post-test 2 ðF ¼ 2:63; p-value ¼ 0.10)
analyses. Controlling the sales region covariate reduces the variability error, which
improves the ability to identify the effect of the current sales training programs on
results improvement. That is, H2.1 is supported. Consistently, by looking at the results
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

improvement for self-evaluation, the trainee results improvement is slightly higher than
the results improvement of non-trainees across all comparisons. This means that for
self-evaluation as a total, the sales training program is beneficial at the results level.
For supervisor-evaluation, both MANOVA and ANOVA were employed. In ANOVA,
there is only one dependent variable (results improvement through supervisor-evaluation)
and one independent variable (the training program); whereas in MANOVA, there are two
dependent variables (results improvement through self-evaluation and results
improvement through supervisor-evaluation) and one independent variable (the
training program). Both MANOVA and ANOVA generate consistent results.
The results improvement shows significant differences between the trainees and
non-trainees for supervisor-evaluation for Post-test 1 – Pre-test when using F-test with
(1,90) degrees of freedom ðF ¼ 8:39; p-value ¼ 0.00), and ([(Post-test 1 þ Post-test
2)/2] – Pre-test) when using F-test with (1,89) degrees of freedom ðF ¼ 4:74;
p-value ¼ 0.03). Therefore, H2.2 is supported. In regards to supervisor-evaluation, the
results improvement of non-trainees for Post-test 1 – Pre-test and [(Post-test 1 þ
Post-test 2)/2] – Pre-test). For the supervisor-evaluation, the sales training program is
significantly more beneficial for trainees than for non-trainees at the results level.

The underlying dimensions of results

Utilizing factor analysis, one factor was extracted for the four items that was labelled
“self- and supervisor-evaluation of results.” The Bartlett test of sphericity was significant
for both self-evaluation (Pre-test: X2 ¼ 278:5312; p , 0.000; Post-test 1: X2 ¼ 260:8102;
p , 0:000; Post-test 2: X2 ¼ 211:1932; p , 0.000) and supervisor evaluation (Pre-est:
X2 ¼ 382.8085, p , 0.000; Post-test 1: X2 ¼ 343:49437; p , 0.000; Post-test 2:
X2 ¼ 332.3273, p , 0.000). This factor solution explains 84 percent of the cumulative
variance for supervisor-evaluation and 77 percent of the cumulative variance for
self-evaluation. Thus, the supervisor-evaluation of results demonstrated more solid
outcomes and support for the hypotheses tested than the self-evaluation of results.

Managerial implications
This study offers important implications for sales executives, trainers, and field sales
managers about assessing higher levels of sales training. First, the level of difficulty in
the data collection process increases significantly when an experimental design
 approach is used to measure higher levels of sales training. Extraneous variables Measuring sales
influenced the sales training process when managers emphasized day-to-day business training
over conducting sales training. As a result sales trainees missed training sessions in
order to complete a sale or solve a customer problem. effectiveness
Second, as demonstrated in this study few companies comprehensively assess higher
sales training levels because of the problems managers encounter when conducting a
comprehensive evaluation (Attia et al., 2002). That said, sales training evaluations can be 335
performed as demonstrated in this study. Although not a simple process to complete, it is
possible for sales managers to assess sales training effectiveness to confirm they are
receiving a positive return on their investment in sales training.
The third implication is that comprehensive evaluations can be conducted by
internal staff members or training consultants using the scales employed in this study.
It is also imperative that firms evaluate multiple training levels that focus on different
dimensions in order to provide management with snapshots of the training process
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

that can be woven into a mosaic. Sales training is a complex process and a single
assessment measure does not provide a comprehensive picture of training’s effect on
sales trainees.
Fourth, companies should not overlook the self-evaluations performed by the
salesperson and the salesperson’s supervisor. In this study, supervisor-evaluations
documented more positive outcomes than did trainee self-evaluations of results. One
explanation for this finding exists within the boundaries of the alpha, beta, and gamma
changes that influence the self-evaluation of results by recalibrating the scale used to
measure the construct of interest and the subject’s re-conceptualization of that construct.
Halo-effect bias is another potential explanation since sales supervisors knew who
had attended the training program and they may have been biased toward the fact that
sales training programs should have more positive effects on trainee results. That said,
sales supervisors have a much broader understanding of the importance of training,
possess greater information, and are more capable of evaluating their subordinates’
performance than are trainees and non-trainees when using behaviorally anchored
scales and/or scales shown in this study (Tanner et al., 2009).
Finally, although collecting quantitative sales performance data to measure results
can prove difficult, additional effort may lead to the successful gathering and
assessment of objective measures. Quantitative measures are easier to analyze when
researchers compare the total sales figures between sales territories that received
training and territories that were not subject to training. However, when quantitative
data are unavailable, as occurred in this study, sales managers can evaluate behavior
and results changes attributable to sales training by utilizing self- and
supervisor-evaluations to more effectively plan future sales training programs.

References
Arvey, R. and Cole, D. (1989), “Evaluating change due to training”, in Goldstein, I.L. (Ed.),
Training and Development in Organizations, Jossey-Bass, San Francisco, CA, pp. 89-117.
Attia, A.M., Honeycutt, E.D. Jr. and Attia, M. (2002), “The difficulties of evaluating sales
training”, Industrial Marketing Management, Vol. 31, pp. 253-9.
Bashaw, R.E., Ingram, T.N. and Keillor, B.D. (2002), “Improving sales training cycle times for
new trainees: an exploratory study”, Industrial Marketing Management, Vol. 31 No. 4,
pp. 329-38.
 MIP Broadwell, M. (1989), “Why trainees should not evaluate trainers”, in Geber, B. (Ed.), Evaluating
Training, Lakewood Books, Minneapolis, MN, pp. 79-82.
30,3
Carmines, E. and Zeller, R. (1979), “Reliability and validity assessment”, Sage University Paper
Series on Quantitative Applications in Social Sciences, Vol. 17, Sage, Beverly Hills, CA.
Cavusgil, T. (1990), “The importance of distributor training at Caterpillar”, Industrial Marketing
Management, Vol. 19 No. 1, pp. 1-9.
336 Connolly, S. (1987), “Participant evaluation: finding out how well training worked”,
in Kirkpatrick, D. (Ed.), More Evaluating Training Programs, American Society For
Training & Development, Alexandria, VA, pp. 232-6.
Cron, W., Marshall, G., Singh, J., Spiro, R. and Sujan, H. (2005), “Salesperson selection, training,
and development: trends, implications, and research opportunities”, Journal of Personal
Selling & Sales Management, Vol. 25 No. 2, pp. 123-36.
Doyle, P. and Cook, D. (1984), “An evaluation of a sales training programme in a retailing
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

environment”, European Journal of Operational Research, Vol. 16 No. 3, pp. 308-13.

Erffmeyer, R., Russ, R. and Hair, J. (1991), “Needs assessment and evaluation in sales-training
programs”, Journal of Personal Selling & Sales Management, Vol. 11 No. 1, pp. 17-30.
Erffmeyer, R., Al-Khatib, J., Habib, M. and Hair, J. (1993), “Sales training practices:
a cross-national comparison”, International Marketing Review, Vol. 10 No. 1, pp. 45-9.
Galanou, E. and Priporas, C.-V. (2009), “A model for evaluating the effectiveness of middle
managers’ training courses: evidence from a major banking organization in Greece”,
International Journal of Training Development, Vol. 13 No. 4, pp. 221-46.
Golembiewski, R., Billingsley, K. and Yeager, S. (1976), “Measuring change and persistence in
human affairs: types of change generated by OD designs”, Journal of Applied Behavioral
Science, Vol. 12, pp. 133-57.
Honeycutt, E.D. Jr (1996), “Conducting a sales training audit”, Industrial Marketing Management,
Vol. 25 No. 2, pp. 105-13.
Honeycutt, E.D. Jr and Stevenson, T. (1989), “Evaluating sales training programs”, Industrial
Marketing Management, Vol. 18 No. 3, pp. 215-22.
Honeycutt, E.D. Jr, Karande, K., Attia, A. and Maurer, S. (2001), “A utility based framework for
evaluating the financial impact of sales force training programs”, Journal of Personal
Selling & Sales Management, Vol. 21 No. 3, pp. 229-38.
Ingram, T., Schwepker, C. and Hutson, D. (1992), “Why sales people fail”, Industrial Marketing
Management, Vol. 21, pp. 225-30.
Jobber, D., Hooley, G. and Shipley, D. (1993), “Organizational size and salesforce evaluation
practices”, Journal of Personal Selling & Sales Management, Vol. 13 No. 2, pp. 37-48.
Jones, E., Brown, S.P., Zoltners, A.A. and Weitz, B.A. (2005), “The changing environment of
selling and sales management”, Journal of Personal Selling & Sales Management, Vol. 25
No. 2, pp. 105-11.
Kirkpatrick, D.L. (1959a), “Techniques for evaluating training programs: reaction”, American
Society for Training and Development Journal, Vol. 18, pp. 3-9.
Kirkpatrick, D.L. (1959b), “Techniques for evaluating training programs: learning”, American
Society for Training and Development Journal, Vol. 18, pp. 21-6.
Kirkpatrick, D.L. (1960a), “Techniques for evaluating training programs: behavior”, American
Society for Training and Development Journal, Vol. 14, pp. 13-18.
Kirkpatrick, D.L. (1960b), “Techniques for evaluating training programs: results”, American
Society for Training and Development Journal, Vol. 14, pp. 8-12.
 Kirkpatrick, D.L. (1994), Evaluating Training Programs: The Four Levels, Berrett-Koehler Measuring sales
Publishers, San Francisco, CA.
Kumpikaite, V. (2007), “Human resource training evaluation”, Engineering Economics, Vol. 55
training
No. 5, pp. 29-36. effectiveness
Lambert, D., Sharma, A. and Levy, M. (1997), “What information can relationship marketers
obtain from customer evaluations of salespeople?”, Industrial Marketing Management,
Vol. 26, pp. 177-87. 337
Leach, M. and Liu, A. (2003), “Investigating interrelationships among sales training evaluation
methods”, Journal of Personal Selling & Sales Management, Vol. 23 No. 4, pp. 325-37.
Leach, M., Liu, A. and Johnston, W. (2005), “The role of self-regulation training in developing the
motivation management capabilities of salespeople”, Journal of Personal Selling & Sales
Management, Vol. 25 No. 3, pp. 261-81.
Lichtenthal, D. and Tellefsen, T. (2001), “Toward a theory of business buyer-seller similarity”,
Journal of Personal Selling & Sales Management, Vol. 21 No. 1, pp. 1-14.
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Mezoff, B. (1987), “How to get accurate self-reports of training outcomes”, in Kirkpatrick, D. (Ed.),
More Evaluating Training Programs, American Society for Training & Development,
Alexandria, VA, pp. 217-22.
Meyer, H. and Raich, M. (1983), “An objective evaluation of a behavior modeling training
program”, Personnel Psychology, Vol. 36, pp. 755-61.
Morris, M., LaForge, R. and Allen, J. (1994), “Salesperson failure: definition, determinants, and
outcomes”, Journal of Personal Selling & Sales Management, Vol. 14 No. 1, pp. 1-15.
Pelham, A. (2002), “An exploratory model and initial test of the influence of firm level
consulting-oriented sales force programs on sales force performance”, Journal of Personal
Selling & Sales Management, Vol. 22 No. 2, pp. 97-109.
Pettijohn, L., Pettijohn, C. and Taylor, A. (2009), “Retail sales training: activities and effects on
job satisfaction, organizational commitment, turnover and customer-orientation”,
The Marketing Management Journal, Vol. 19 No. 1, pp. 46-57.
Phillips, J.J. (1991), Handbook of Training Evaluation and Measurement Methods, Gulf
Publishing Company, Houston, TX.
Piercy, N.F., Craven, D.W. and Morgan, N.A. (1998), “Sales force performance and
behavior-based management processes in business to business sales organizations”,
European Journal of Marketing, Vol. 32, pp. 79-100.
Preziosi, R. and Legg, L. (1989), “Add ‘then’ testing to prove training’s effectiveness”, in Geber, B.
(Ed.), Evaluating Training, Lakewood Books, Minneapolis, MN, pp. 69-70.
Rahn, P. (1989), “One more time: test trainees before you train them”, in Geber, B. (Ed.),
Evaluating Training, Lakewood Books, Minneapolis, MN, pp. 65-6.
Rich, G.A., Bommer, W.H., McKenzie, S.B., Podsakoff, P.M. and Johnson, J.L. (1999), “Methods in
sales research: apples and apples or apples and oranges? A meta-analysis of objective and
subjective measures of salesperson performance”, Journal of Personal Selling & Sales
Management, Vol. 19 No. 4, pp. 41-52.
Salinger, R. and Deming, B. (1987), “Practical strategies for evaluating training”, in Kirkpatrick,
D. (Ed.), More Evaluating Training Programs, American Society for Training
& Development, Alexandria, VA, pp. 157-65.
Summy, G. (2007), “How to measure – or not measure – the effectiveness of sales training”,
Velocity Reprint, Vol. 9 No. 1, pp. 30-2.
Tanner, J.F. Jr, Honeycutt, E.D. Jr and Erffmeyer, R.C. (2009), Sales Management: Shaping Future
Sales Leaders, Pearson Prentice Hall, Upper Saddle River, NJ.
 MIP Tanner, J.F. Jr, Fournier, C., Wise, J., Hollet, S. and Poujol, J. (2008), “Executives’ perspectives of
the changing role of the sales profession: views from France, the United States, and
30,3 Mexico”, Journal of Business & Industrial Marketing, Vol. 23 No. 3, pp. 193-202.
Tyler, K. (2002), “Evaluating evaluations: asking the right questions is only the first step in
creating a good training evaluation”, HR Magazine, Vol. 47 No. 6, pp. 85-9.
Tziner, A. and Falbe, C. (1993), “Training-related variables, gender, and training outcomes: a field
338 investigation”, International Journal of Psychology, Vol. 28 No. 2, pp. 203-21.
Wilson, P., Strutton, D. and Farris, M.T. (2002), “Investigating the perceptual aspect of sales
training”, Journal of Personal Selling & Sales Management, Vol. 22 No. 2, pp. 77-86.
Zmud, R.W. and Armenakis, A. (1978), “Understanding the measurement of change”, Academy of
Management Review, Vol. 3, pp. 661-9.

Further reading
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)

Attia, A.M. and Honeycutt, E.D. Jr. (2005), “and Mark Leach A three-stage model for assessing
and improving sales force training and development”, Journal of Personal Selling & Sales
Management, Vol. 25 No. 3, pp. 249-64.

About the authors

Ashraf M. Attia, PhD (Old Dominion University), is Professor of Marketing at SUNY Oswego. He
publishes in the area of sales management and global marketing.
Earl D. Honeycutt, Jr, PhD (University of Georgia), is Martha and Spencer Love Professor of
Marketing and Director of the Chandler Family Professional Sales Center at Elon University. His
research has appeared in the areas of sales management, B2B, and global marketing. Earl
D. Honeycutt, Jr is the corresponding author and can be contacted at: ehoneycutt@elon.edu

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com

Or visit our web site for further details: www.emeraldinsight.com/reprints
 This article has been cited by:

1. William J. Rowe, Steven J. Skinner. 2016. Delegating slotting allowance authority to the sales force.
Industrial Marketing Management . [CrossRef]
2. Vijay Lakshmi Singh, Ajay K. Manrai, Lalita A. Manrai. 2015. Sales training: A state of the art and
contemporary review. Journal of Economics, Finance and Administrative Science . [CrossRef]
3. Victoria Bush, Alan J. Bush, Jared Oakley, John E. Cicala. 2015. The Sales Profession as a Subculture:
Implications for Ethical Decision Making. Journal of Business Ethics . [CrossRef]
4. Earl D. Honeycutt, Sharon K. Hodge, Ashraf M. Attia. 2015. Addressing Service Challenges to Improve
Sales Training. Services Marketing Quarterly 36:4, 352. [CrossRef]
5. Ashraf M. Attia, M. Asri Jantan, Nermine Atteya, Rana Fakhr. 2014. Sales training: comparing
multinational and domestic companies. Marketing Intelligence & Planning 32:1, 124-138. [Abstract] [Full
Text] [PDF]
Downloaded by La Trobe University At 21:15 28 February 2016 (PT)