See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/229487496

A Method of Assessing Leadership Effectiveness

Article  in  Performance Improvement Quarterly · October 2008

DOI: 10.1111/j.1937-8327.2006.tb00355.x

CITATIONS READS

9 9,968

1 author:

A. Olu Oyinlade
University of Nebraska at Omaha
12 PUBLICATIONS   59 CITATIONS   

SEE PROFILE

All content following this page was uploaded by A. Olu Oyinlade on 04 June 2018.

The user has requested enhancement of the downloaded file.

P e r f o r m a n c e I m p r o v e m e n t Q u a r t e r l y , 1 9 ( 1 ) pp . 2 5 - 4 0

A Method of Assessing
Leadership Effectiveness
Introducing the Essential Behavioral Leadership Qualities Approach

A. Olu Oyinlade

T
he purpose of this study is to present a new Assessing the effectiveness of a
method for assessing the effectiveness of a leader is often a difficult exercise for
leader. many organizations. This is usually
because most assessment procedures
Scholars (such as Stogdill, 1974; Katz & Kahn, 1978; are influenced by organizational poli-
Yukl, 1981) have defined and explained leadership in tics, they are not standard based, and
various ways based on their interests and concerns. the items on which a leader is assessed
Stogdill (1974), claimed that, perhaps, there were as are undefined or poorly defined.
This study presents the Essential
many definitions of leadership as there were scholars, Behavioral Leadership Qualities (EBLQ)
and Bennis and Nanus (1985) claimed that researchers approach for assessing leadership ef-
had defined leadership in over 350 different ways in the fectiveness as an alternative method
30 years prior to 1985. Also, Conger (1992) indicated to commonly used assessment pro-
cedures. Among other assumptions,
that “leadership is largely an intuitive concept for which the EBLQ method is built on the
there can never be a single agreed-upon definition” (p. assumptions that a leader should be
17). However, central to the various definitions is the evaluated on clearly defined behav-
recurring theme that leadership involves the use of non- ioral qualities and his/her effective-
ness rating should be standard based.
coercive influences to coordinate the activities of group Hence, the EBLQ method measures
members toward the accomplishment of group goals the effectiveness of a leader against
(Oyinlade, Gellhaus, & Darboe, 2003). the essentiality levels of behav-
Just as scholars have defined leadership in different iors deemed necessary for effective
leadership. Leadership effectiveness
ways, they equally vary in their perceptions of the fac- is determined for each leadership
tors of effective leadership. The leader traits theory that behavior and for overall leadership
dominated leadership literature in the 1930s explained performance.
leadership effectiveness by the natural characteris- The EBLQ method was demon-
strated in the assessment of the lead-
tics and abilities (such as superior intelligence, good ership effectiveness of the principals
memory, bountiful energy, persuasiveness, etc.) of the of schools for students who are blind
leader (Steers, Porter, & Bigley, 1996). Notable leaders or visually impaired.
that have been affiliated with this theory include “Attila
the Hun, Catherine the Great, Alexander the Great,
Winston Churchill, George Washington, Queen Elizabeth I, Jesus of Naza-
reth, Machiavelli, Moses, Napoleon, Nixon, Teddy and Franklin Roosevelt”
(Reeves, 2004, p. 8). It was, however, discovered that this theory lacked
predictive power in linking leadership traits to performance (see Stogdill,
1948), hence, by the late 1940s and early 1950s, the leader behavior theories
which explained leadership effectiveness by leader behaviors, instead of
natural traits, was introduced to provide a new perspective in understand-
ing leadership effectiveness (Steers et al., 1996). Now, post industrialization
perspective on leadership is primarily action oriented, placing leadership

Volume 19, Number 1 / 2006 25

 emphasis on the interaction between multiple behavioral factors that occur
within the leadership process (Rost, 1991).
Regardless of the perspective on leadership to which one may sub-
scribe, measuring leadership effectiveness in meaningful and useful ways
often prove to be a difficult exercise, and very often, the exercise is a re-
sounding failure (Schellhardt, 1996; Smith, Hornsby, & Shirmeyer, 1996).
Worse still, as indicated by Hogan and Hogan (2001) very few empirical
literature on how to measure leadership effectiveness exist, therefore
creating a dearth of empirical resources for scholars and leadership
practitioners interested in studying approaches to measuring leadership
effectiveness. Available studies on the assessment of leadership effective-
ness include, but are not limited to, Stogdill (1948), Bray and Howard
(1983), and Bentz (1985). One common factor among these studies is
that individual researcher’s definition or theory of leadership was the
most determining factor of what was assessed. When a researcher defines
leadership in terms of certain behaviors, only those behaviors are assessed
while other behaviors of the leader are typically not assessed (Hughes,
Ginnett, & Curphy, 1999). Examples of what had been assessed in the past
by other scholars include leader’s behaviors (Hazucha, 1992; Komacki,
1986), charisma and rhetorical skills (Shamir, Arthur, & House, 1994;
Bass & Yammarino, 1988) intelligence levels (Lord, DeVader, & Allinger,
1986), and personality traits (Hogan, Curphy, & Hogan, 1994). This shows
that leadership effectiveness is a relative decision based on definition and
assessed characteristics.
Another factor that makes the assessment of good leadership effective-
ness difficult is organizational politics and fear of potential backlash from
giving negative feedback to a superior (Heck, Johnsrad, & Roser, 2000).
As expressed by Guinn (1996), giving a superior, especially an executive,
negative feedback can be a career-limiting move. This is especially true
when the evaluation of the leader is tied to decisions for such things as
promotion, raise, contract renewal and other positive or negative job condi-
tions (Heck et al., 2000). In addition to organizational politics, leadership
expectations are often ambiguous, contradictory, and undefined or poorly
defined (Reeves, 2004). Given these conditions, most executives get vir-
tually no regular performance feedback, other than superficial praise or
criticism (Longenecker, 1992). They “seldom get the kind of feedback that
can enable them to change the behaviors that everyone whispers about
behind their backs…the result is that the organization limps on being less
than its potential, or the inevitable crisis arises and the executive derails”
(Guinn, 1996, p. 9).
In determining what is assessed, Hughes et al. (1999) indicated that
many organizations use a competence model whereby frequently assessed
behaviors crystallized into a set of skills and behaviors in which a leader
must excel to be effective. Usually, these behaviors are observable, they
require no references, assumptions or interpretations, and they include
definitions and behavioral indicators that describe each behavior in ac-
tion (Guinn, 1996). Literature on the competence model indicates that

26 Performance Improvement Quarterly

a competence is a way of behaving or thinking across situations (Guion,
1991), and it is an “underlying characteristic of an individual that is caus-
ally related to criterion-referenced effective and/or superior performance
in a job situation” (Spencer & Spencer, 1993, p. 9). Competence items for
effective leadership have included a leader’s ability to motivate others,
to provide support for subordinates, to listen well, to have knowledge of
his/her organization, to have vision, to have good interpersonal skills, to
be able to resolve conflicts, to have knowledge of the law, to be able to
establish directions for others, as well as align people toward common
directions (Spencer & Spencer, 1993; Conger, 1992; Kotter, 1990). Other
characteristics (for effective leadership) have included a leader’s ability to
be a strategic opportunist, being globally adept, being a keen data analyst,
having the ability to lead across organizational bound-
aries, being a community builder, having sensitivity to
…most executives get
diversity issues, having the ability to develop leaders
in his/her subordinates, and having good communica- virtually no regular
tion skills (Conger & Benjamin, 1999). performance feedback,
Competency models, hence, help to determine other than superficial
what is assessed, as well as carry significant implica- praise or criticism
tions for rewards and punishments within an organiza- (Longenecker, 1992).
tion (Hughes et al., 1999). The model can be used to
They “seldom get the
determine who is promoted, demoted, trained, and
so forth. One major advantage of the competence kind of feedback that can
model is that it helps to keep congruity between what enable them to change the
is assessed and the behaviors expected of a leader for behaviors that everyone
effectiveness, so that there is no misalignment between whispers about behind
skills and behaviors that are evaluated and those that their backs…”
are necessary for leadership effectiveness (Hughes et
al., 1999).
To arrest the problem of undefined or poorly defined leadership
standards, Reeves (2004) expressed that a good leadership effectiveness
method should be standard based—based on clear standards or speci-
fied objective standards. “Without standards, evaluations are constantly
subject to the shifting standards of relative performance rather than the
bedrock of clear, fair, and immutable standards” (p. 21). Standard based
evaluations avoid the fallacy of false complacence and inaccurate dispar-
agement found in non-standard based systems where performance is based
on relative comparison of one leader’s performance against another. In
false complacency, a non-effective leader is reassured because he is better
than another leader. Here ineffectiveness can be rewarded simply because
one is better than another. In inaccurate disparagement, an effective leader
can be (is) labeled as ineffective or insufficiently effective because another
leader is more effective. In addition to having clear standards, an evalu-
ation method for measuring leadership effectiveness should be designed
such that both the evaluator and the person being evaluated have a com-
mon and clear understanding of the difference between various levels of
performance (Reeves, 2004).

Volume 19, Number 1 / 2006 27

 The impact of leadership can be measured on both the organization
and on the leader’s subordinates. The impact of leadership on an orga-
nization mostly focuses on the organization’s bottom line, such as unit
sales, profit margins, number of students graduated, amount of money
raised, increase in productivity, and win-loss record, while the impact of
leadership on subordinates usually focuses on subordinates’ job satisfac-
tion (Hughes et al., 1999). Measuring leadership effectiveness through
unit performance and employee satisfaction, may, however, be riddled
with problems. For example, a leader’s unit performance can be affected
by many factors beyond the leader’s control. A period of economic reces-
sion, for example, can cause poor sales, while sales may be high during
periods of economic boom and lower interest rates without much stra-
tegic efforts by the leader (Hughes et al., 1999). The impact of leadership
on the subordinate is most commonly measured by the leader’s superior,
using a specified assessment instrument which often focuses on rating
the leader on several leadership domains such as integrity, administra-
tive skills and communication. However, such assessment often fails to
reflect the true impact of a leader on his/her followers. As indicated by
Hughes et al. (1999):

Sometimes, superiors do not take the time to provide accurate and

comprehensive performance appraisal ratings; at other times, superiors
may be largely unaware of, or unfamiliar with a target leader’s perfor-
mance...also…some superiors have difficulties dealing with conflict and
would rather give average ratings than have to deal with the emotions
and distress associated with unflattering rating. (p. 120)

In some situations, superior’s ratings may also reflect the impact of strategic
sucking-up by a leader to his/her superiors. In addition, subordinates may
rate a leader as effective simply because the leader did not make them work
hard, or the leader may be rated ineffective if he/she makes the subordinates
work too hard (Hughes et al., 1999).
Given the flaws in various assessment processes, Hughes et al. (1999)
stated that there was no one perfect or best method of assessing leadership
effectiveness. They (the authors), however, still proclaimed that, “if leader-
ship is defined partly in the eyes of the followers, then perhaps a better
way to judge leadership success is to ask subordinates to rate their level of
satisfaction or the effectiveness of their leader” (Hughes et al., 1999, p. 120).
Except for conducting a real-time assessment through direct observational
method, surveying subordinates’ opinion on leadership effectiveness is the
most capable way of telling the direct impact of leadership on subordinates’
levels of job satisfaction.

Purpose
The purpose of this present study is to develop a new method for mea-
suring leadership effectiveness, by building on some of the assessment char-

28 Performance Improvement Quarterly

acteristics (such as competence model and standards) already explained in
existing literature, to produce an evaluation system that will be meaningful,
constructive and instructive to both the evaluator and the evaluated. This
method, the Essential Behavioral Leadership Qualities (EBLQ) approach,
is rooted in the principles of the leader behavior theories which emphasize
action orientation to leadership, and on the competence model, which in-
dicates that for effective leadership, a leader would need to excel at certain
specific behaviors. These behaviors are systematically determined by people
who are familiar with the leader’s roles in his/her specific organization. Such
people may include the leader (and his/her peers) and the leader’s subordi-
nates who are most likely to be very familiar with the expected leadership
behaviors of the leader. Such people may or may not include the leader’s
supervisor, depending on the extent to which the su-
pervisor is accurately familiar with the details of the
…the Essential Behavioral
leader’s everyday activities. The specific assumptions
underpinning the EBLQ approach are as follows:
Leadership Qualities
(EBLQ) approach, is rooted
1. Leadership behaviors rather than natural traits in the principles of the
determine leadership outcomes. leader behavior theories
2. A good evaluation system should be effective which emphasize action
in coaching. The system should be capable,
orientation to leadership,
in systematic ways, of indicating the areas of
leadership in which a leader is doing well and
and on the competence
those areas where the leader may need improve- model, which indicates that
ment. for effective leadership,
3. Competency model: Relative to the job situa- a leader would need to
tion, a leader ought to be competent in certain excel at certain specific
skills and behaviors to be successful. Those skills
behaviors.
and behaviors are termed essential behavioral
leadership qualities (EBLQ), and the assessment
of a leader’s effectiveness should be based solely on these qualities.
4. Decisions about EBLQ to assess are to be made by a select group
(called judges) of the leader’s subordinates and others with adequate
knowledge of the leader’s roles.
5. The best way to assess the effectiveness of a leader is through the
perceptions of his/her followers. The most direct impact of leadership
behaviors is upon subordinates, hence, the subordinates are best posi-
tioned to indicate leadership impact on themselves. The subordinates
provide the most direct looking-glass for leadership feedback.
6. Effectiveness of a leader is best determined based on relative essenti-
ality of the EBLQ items being measured. That is, the evaluation of a
leader on EBLQ items must be compared to the extent to which the
items are assessed to be essential for effective leadership.

Details of the elements and procedure of the EBLQ method are con-
tained in the following case study of leadership effectiveness of the princi-
pals of the schools for students who are blind or visually impaired.

Volume 19, Number 1 / 2006 29

 Case Study
The Essential Behavioral Leadership Qualities (EBLQ) approach was
used to determine the effectiveness of the principals of the schools for stu-
dents who are blind and visually impaired. This population of school leaders
was chosen for this case study because effective leadership is important to
them as it is for other organizations, and leadership effectiveness in these
schools has not been widely studied. A sample of principals in several
schools, rather than just one school, was studied to conceal the identity of
the principals because there is only one school of the blind in each state in
the country. The following is a step-by-step description of the assessment
process, using the EBLQ method, in four stages.

Stage I: Instrument Design

This is the beginning stage of the competence model. At this stage,
judges were used to provide basic information that was used to construct
two leadership effectiveness evaluation scales. The following is a full de-
scription of this stage as conducted in the case study.
Borrowing from studies by Oyinlade et al. (2003) and Oyinlade and
Gellhaus (2005), ten professionals in the field of visual impairment (four
principals, four teachers and two superintendents) from ten different states
were selected, through availability sampling, to serve as judges for the con-
struction of two questionnaire scales used for data collection on leadership
effectiveness of the principals. Each judge described, in open-ended format
on separate forms, the behavioral qualities he/she perceived to be essential
for effective leadership for a principal at the schools for students with visual
impairments. Their responses were summarized, coded and analyzed for
frequency. All items with a minimum frequency of five (cited by at least
50% of the judges) were deemed acceptable for inclusion in the effective-
ness scales. This process yielded 18 essential behavioral items which were
operationalized (see below) from the general descriptions and narratives
used by the judges. These items and their accompanying definitions were
used to construct one seven-point (extremely unessential, very unessential,
unessential, somewhat essential, essential, very essential, extremely essen-
tial), Likert-type scale of essential characteristics for effective leadership.
The higher the value obtained on an item, the greater the essentiality of
the item for effective leadership of the principals. The 18 behavioral items
were also used to construct a second scale which requested respondents
to rate their principals on a range from 1 (low) to 7 (high) on how they
(respondents) perceived their principals to be performing effectively, on
each of the 18 items of essential leadership behaviors.

Validity and Reliability

Validity was obtained through two measures:
1. Face Validity—was used to ensure that all the EBLQ items were logi-
cal and conceptually valid in measuring effective leadership. Two conditions
sufficiently helped to establish face validity in this study: comments of the
judges who provided the EBLQ items (see Vogt, 1993), and the consistency

30 Performance Improvement Quarterly

 Table 1
Factor analysis using principal components factor extraction method, 75%
variance extraction rule and orthotran/varimax transformation method

Oblique Solution Reference Structure Sampling

EBLQ Items Factor 1 Factor 2 Factor 3 Factor 4 Adequacy

Good listening skills .822 -.166 -.240 .056 .982

Good presentation skills .732 -.074 .193 .528 .946
Participative decision-making style .813 -.275 .010 -.175 .963
Motivation .774 -.329 -.021 .156 .961
Honesty and good ethics .834 -.006 -.297 -.065 .969
Organizational Knowledge .764 -.368 -.186 .037 .961
Good interpersonal skills .808 -.350 -.166 .197 .938
Fiscal efficiency .789 -.332 .100 .017 .965
Knowledge of policies .687 -.542 -.132 -.044 .917
Vision for the future .841 -.221 .140 .020 .955
Delegating authority .705 -.208 .367 -.394 .951
Providing support .871 -.211 .082 -.160 .960
Fairness .868 -.176 -.247 -.090 .957
Courage and firmness .807 .117 .201 -.011 .974
Creativity .822 -.051 .298 .149 .962
Hardworking .807 .305 -.036 -.022 .960
Good prioritization skills .841 .001 .043 -.105 .961
Problem resolution skills .886 .044 .107 -.048 .971

Total matrix sampling adequacy .959

Factor Analysis Summary: Number of items=18

Number of factors=4
Number of cases=238
Number missing=23
Degree of Freedom=170
Bartlett’s Chi Square=4162.485
P-Value<.0001

of the EBLQ items with factors of effective leadership in the competence

models of Spencer and Spencer (1993), Conger (1992), and Conger and
Benjamin (1999).
2. Factor Analysis­—Given that the EBLQ items met the two conditions
of face validity (mentioned above), exploratory factor analysis using prin-
cipal component and oblique rotation was used to determine covariance
among all the items. Results (see Table 1) show that all the items loaded

Volume 19, Number 1 / 2006 31

 highly only on one factor, indicating the unidimensionality of the EBLQ
scale. All the items are determined to be measuring the same concept
(leadership effectiveness established through face validity).
Reliability—Cronbach’s alpha established reliability at alpha=.92.

Operational Definitions
Each of the eighteen items of essential behavioral leadership qualities
were operationalized on the questionnaire (Oyinlade et al., 2003; Oyinlade
& Gellhaus, 2005) as follows:

a. Good listening skills: Ability to listen carefully, without prejudg-

ment, empathize with the speaker and honestly try to understand
the speaker’s point of view.
b. Good presentation skills: Ability to communicate ideas and inten-
tions to others clearly without being misunderstood (good commu-
nication skills).
c. Participative decision-making style: Interest in soliciting and
using others’ input in decision-making; working with subordinates
through leadership by example.
d. Motivator: Ability to help create a work environment in which sub-
ordinates are happy and eager to work and to achieve needed goals.
e. Honest and Ethical: Being always truthful, trustworthy, and abiding
by a high standard of “right” and “wrong.”
f. Organizational knowledge: Knowledge of how the school system
for the schools for students with visual impairments works; knowl-
edge of the “hows” and “whys” of instructional curriculum; well
informed on current issues regarding schools for students with visual
impairments.
g. Good interpersonal skills: Being friendly, humorous, cordial,
polite and treating people with respect and dignity. Relating well to
others.
h. Fiscal efficiency: Ability to prepare good financial budgets and
spend wisely.
i. Knowledge of policies: Having a good knowledge of local, state and
federal laws, and policies regarding schools for students with visual
impairments.
j. Vision for the future: Having ideas, goals and objectives for the
future of the school for students with visual impairments; the abil-
ity to make long range planning to meet these goals and objectives;
ability to meet present needs.
k. Delegating authority: Ability to share responsibilities with and to
give adequate authority to subordinates to perform tasks.
l. Providing support: Ability to readily guide and support the activi-
ties of subordinates; helping subordinates grow and succeed in their
goals.
m. Fairness: Treating people equally and appropriating resources
evenly among various constituencies without bias or favoritism.

32 Performance Improvement Quarterly

 n. Courage and firmness: Willingness to make tough and unpopular
but necessary decisions and sticking to them.
o. Creativity: Openness to new ways of doing things; using new ideas
to do things differently.
p. Hardworking: Commitment to spending long hours; persisting
through the best possible efforts to accomplish goals efficiently and
effectively.
q. Good prioritizing skills: Ability to prioritize tasks; making sure the
most essential tasks are performed before the less essential ones.
r. Problem-solving skills: Having interests and capabilities in solving
a variety of problems, and being able to compromise as well as assist
others in solving problems.

Stage II: Data Collection

Data for this study were collected through random sampling of 53%
(N=25) of the 47 schools, in the 1997-1998 membership list of the Council
of Schools for the Blind. The 25 schools were located in 25 different states
across all regions in the country. A total survey of the entire population of
the teachers in each school was conducted due to a lack of direct access
to the names of the teachers for the purpose of random sampling. On the
questionnaire, each responding teacher indicated on the first scale, the
extent to which he/she believed each of the 18 EBLQ items was essential
for effective leadership of his/her principal. On the second scale, each
responding teacher gave a score for the extent to which he/she perceived
his/her principal to be effective on each of the 18 EBLQ items.
The sampling yielded a useful return rate of 32% (N=294) of the ap-
proximately 900 questionnaires that were mailed to the selected schools.
Sampling adequacy was determined through factor analysis which estab-
lished high sampling adequacy for each item with a total matrix sampling
adequacy of .959 (see Table 1 for details).

Stage III: Competence Model (EBLQ) Items

From the responses of the teachers to the questionnaire items, mean
scores and rankings were derived for the extent to which each of the 18
EBLQ items was deemed essential for the effectiveness of the principals of
the schools for the blind and visually impaired (see Table 2). To establish
legitimacy and validity for the ranking of the EBLQ items for essentiality,
both the t-test and Chi Square test of one sample analysis were conducted
to determine that significant difference did exist among the mean scores of
essentiality. The results of the t-test for essentiality show an overall mean
score of 6.267, DF=17, t-value=105.048, and P<.0001. Chi Square also estab-
lished significant difference with variance=.064, standard deviation=.253,
standard error=.060, x2=1.089, and P<.0001. The ranking of the EBLQ items
for essentiality shows that “good listening skills,” “honesty and good ethics,”
and “fairness,” respectively, were determined as the top three most essential
EBLQ items for effective leadership for the principals. The bottom ranking
items, that is, the items that were ranked to be least essential for effective

Volume 19, Number 1 / 2006 33

Table 2
Perceived levels of essentiality and levels of performance effectiveness
for principals on all 18 EBLQ items

MEss- MEff-
Essent Essential Behavioral MEss MEff Effectiveness Effect
Meff MEss
Rank Leadership Quality Score Score Description Rank
Diff Rate
1 Good listening skills 6.672 6.238 0.434 .93 Exemplary 4
2 Honesty and ethics 6.653 5.508 1.145 .83 Proficient 9
3 Fairness 6.544 4.971 1.583 .76 Competent 15
4 Motivator 6.506 4.916 1.590 .76 Competent 15
5 Providing support 6.440 4.761 1.979 .74 Competent 18
6 Participative decision 6.409 4.794 1.615 .75 Competent 17
making
7 Good interpersonal 6.376 4.958 1.418 .78 Competent 14
skills
8 Problem solving skills 6.347 5.147 1.200 .81 Proficient 12
9 Organizational 6.282 5.513 0.769 .88 Proficient 5
knowledge
10 Good presentation 6.270 5.238 1.032 .84 Proficient 7
skills
11 Courage and firmness 6.197 5.097 1.100 .82 Proficient 11
12 Good prioritization 6.135 5.105 1.030 .83 Proficient 9
skills
13 Vision for the future 6.151 5.349 0.802 .87 Proficient 6
14 Creativity 6.077 4.933 1.144 .81 Proficient 12
15 Delegating authority 6.077 5.088 0.989 .84 Proficient 7
16 Knowledge of policies 6.050 5.678 0.372 .94 Exemplary 2
17 Hardworking 5.865 5.832 0.033 .99 Distinguished 1
18 Fiscal efficiency 5.760 5.403 0.357 .94 Exemplary 2
Mean of all 18 items 112.83 93.57 19.26 .84 Proficient

Essent Rank: Essentiality Rank

MEss: Mean Score of Essentiality
MEff: Mean Score of Effectiveness
MEff-MEss Rate: Rate of mean score of effectiveness relative to mean score of essentiality
Effect Rank: Rank of Effectiveness relative to essentiality of an item.

leadership of the principals were “knowledge of policies,” “hardworking,”

and “fiscal efficiency” respectively (see Table 2 for details).

Stage IV: Effectiveness Ratings

The effectiveness rates of the principals, relative to the levels of es-
sentiality of each EBLQ item were determined through the use of a simple
ratio calculation termed the “rate of effectiveness” formula: MEff-MEss

34 Performance Improvement Quarterly

Rate=(MEff/MEss), where MEff=mean score of effectives, MEss=mean
score of essentiality, and MEff-MEss Rate=rate of effectiveness relative to
level (mean score) of essentiality. The MEff-MEss rates were calculated for
each item to derive item-by-item effectiveness rate, and for the aggregate
of all 18 items, to derive the overall effectiveness rate of the principals. The
MEff-MEss rates were ranked from 1 (highest score and most effective) to
18 (lowest score and least effective) after both the t-test and Chi Square
test of one sample analysis were conducted to determine that significant
difference did exist among the MEff- MEss scores. The t-test showed a

Table 3
Complete descriptions of scores of effectiveness and qualitative rankings

Rating Descriptions
MEff-MEss Rate Qualitative Ranking Acceptance Level
.95 and above Distinguished (D) Good
.90 to .94 Exemplary (E) Good
.80 to .89 Proficient (P) Good
.70 to .79 Competent (C) Average
.69 and below Ineffective Unacceptable

mean MEff-MEss rate of .840, DF=17, t-value=48.980, and P<.0001. The

Chi Square test showed a variance of .005, Std. Dev.=.073, Std. Error=.017,
DF=17, x2=.090, and P<.0001. See Table 2 for the ranking of effectiveness
(MEff-MEss) rates.
The MEff-MEss rates were also qualitatively ranked from distinguished
(highest level of acceptable performance) to ineffective (lowest level of per-
formance) as adapted from Reeves (2004). The cut-off points for the MEff-
MEss values used to determine the qualitative descriptions of a leader’s
effectiveness were based on the traditional American educational grading
system (A=90% and above, B=80%-89%, etc.) with a slight modification for
distinguished ranking. This method is used because it has been generally
accepted as a standard grading system and because of its simplicity. Table
3 shows the complete levels of qualitative rankings used in this study.
The ranking of effectiveness (MEff-MEss) rates for each of the 18 EBLQ
items showed that the principals were perceived to be most effective on the
17th most essential (that is, 2nd least essential) EBLQ item, “hardworking,”
for leadership effectiveness, at an effectiveness (MEff-MEss) rate of .99
(distinguished ranking). The next two items on which the principals were
perceived to be most effective were “knowledge of policy” and “fiscal effi-
ciency,” each at .94 effectiveness rate (exemplary), while the fourth item on
which they were most effective was “good listening skills” at .93 effectiveness
rate (exemplary). The two items on which the principals were perceived
to be least effective were “participative decision-making” and “providing

Volume 19, Number 1 / 2006 35

 support” at .75 and .74 (competent) effectiveness rates respectively. Overall
effectiveness rate (aggregate of all 18 EBLQ items) for the principals was
.84 (proficient). See Table 2 for details.

Discussion and Conclusion

Unlike most assessment tools which have no standards against which to
compare the assessment of a leader, the EBLQ method uniquely measures
a leader’s effectiveness against the level of essentiality of the leadership
behavioral characteristics that are measured. This method shows the ef-
fectiveness of a leader on each essential item of effectiveness, as well as the
leader’s overall level of effectiveness on the aggregate of all the essential
items for effectiveness.
In the above case study, the principals in the schools for students with
visual impairments were assessed by their faculty against the qualities
that the faculty perceived to be essential for effective leadership. That is,
rather than being assessed by an administrative assessment tool which the
subordinates typically do not have input in designing, the design of the
EBLQ assessment instrument includes the opinion of the subordinates
(teachers in the case study) who served as judges. This shows that the EBLQ
method of assessing leadership effectiveness is bottom-up, as the teachers
constituted 40% of the judges, 100% of the opinions that determined the
levels of essentiality of all the EBLQ items, and 100% of the opinions of the
effectiveness of their school principals. This method of assessment allows
the source of information for the leadership qualities on the evaluation
instrument to conduct the actual evaluation. This is consistent with the
claim by Beckahard (1969) that decision-making is best delegated to the
source of information, rather than being made a function of a set of rigid
hierarchy.
The substantive findings of this study provide opportunities for the
principals to see how they were perceived by their teachers on the items the
teachers perceived to be essential qualities to effectively lead them (teach-
ers). The overall effectiveness rating is qualitatively described as “compe-
tent,” showing that the performance of the principals in this study needed
significant improvements, from the perspectives of their subordinates.
In the item-by-item analysis of effectiveness, the EBLQ method shows
differences in the extent to which each of the assessed leadership behaviors
is perceived to be essential for leadership effectiveness. That is, the method
shows that the EBLQ items are not equally important for effective leader-
ship. For example, the case study shows that “good listening skills” was
deemed the most important (essential) factor of leadership effectiveness,
while “fiscal efficiency” was deemed least important. The EBLQ method is
also effective in showing patterns of incongruence (and any congruence)
between the essentiality of assessed items and the level of effectiveness of
a leader on the items. In the case study, a pattern of incongruence was pre-
dominant between the levels of essentiality of each EBLQ item (depicted by
mean scores and ranks of essentiality) and the principals’ levels of effective-

36 Performance Improvement Quarterly

ness on each item (depicted by Meff-MEss rates). No congruence existed be-
tween perceived essentiality levels and perceived effectiveness rates on any
item. For example, item-by-item analysis showed that the principals were
rated highly effective on behaviors such as “hardworking” (effectiveness
rank=1) and “knowledge of policies” and “fiscal efficiency”(effectiveness
rank=2; tied), but the rankings of essentiality for these three items were 17th,
16th and 18th respectively. In the reverse, the performance of the principals
on the 1st, 2nd and 3rd most essential leadership behaviors (“good listening
skills,” “honesty and ethics,” and “fairness” respectively) were ranked 4th,
9th, and 15th respectively, for leadership effectiveness of the principals.
Reeves (2004) criticized many evaluation methods for using meaning-
less descriptions such as “meets expectations,” “above average,” “superior,”
“growth needed,” and so on to indicate a leader’s level
of effectiveness. According to Reeves, these descrip-
…the EBLQ method
tions are meaningless because they are based mainly
on subjective judgments of the evaluator, rather than uniquely measures a
from objective or systematically derived standards. leader’s effectiveness
Such descriptions are doomed to be inconsistent in against the level of
their application, and they are of little use for coach- essentiality of the
ing (Reeves, 2004). To avoid this situation in the leadership behavioral
EBLQ method, the qualitative description of a leader’s
characteristics that are
level of effectiveness is derived from a systematically
constructed scale (see Table 3) using values derived measured.
through the MEff-MEss formula and the standard
American educational grading system. In the case study of this research,
for example, the rating of “proficient” and higher for the effectiveness of the
principals (MEff-MEss rate ≥ .80) were recorded on 70% of the EBLQ items,
while exemplary effectiveness rating (MEff-MEss rate .90-.94) was obtained
on only one item (good listening skills). Distinguished rating (MEff-MEss
rate ≥.95) was recorded for the principals on only one item, “hardworking,”
and the ranking of ineffective (MEff-Mess rate ≥.69) performance was not
recorded on any item. The qualitative ranking of the principals on each of the
five items in which they (principals) were perceived least effective was “com-
petent,” ranging from .74 (providing support) to .78 (good interpersonal
skills) effectiveness ratings. From these findings, the EBLQ areas in which
the principals needed to improve their effectiveness were revealed. While
improvements could be useful in many areas of the EBLQ items, (especially
all items below .90 MEff-MEss rate), the principals would need to more ur-
gently improve their leadership effectiveness on such variables as “providing
support,” “participative decision-making,” “motivator,” “fairness” and “good
interpersonal skills”. These were the areas where the effectiveness rates of
the principals fell below .80 (proficient), akin to scoring les than the grade
of “B” in traditional American educational grading system.
While this case study measured the effectiveness of a collection of
principals across the country, the EBLQ approach can similarly be used to
measure the effectiveness of a single leader in one unit or one organization.
When used to assess leadership effectiveness of one leader, judges can be

Volume 19, Number 1 / 2006 37

 systematically or randomly selected among a leader’s subordinates and
others determined to have adequate knowledge of the leader’s roles. Also,
a sampling of subordinates will not be necessary if all the subordinates are
to evaluate the leader. In this case, every subordinate of the leader being
assessed will be requested to complete the assessment questionnaire.
Since the determination of the EBLQ items used to evaluate leadership
effectiveness in this study is based on the perceptions of the leader’s subor-
dinates in a specific occupation, subordinates of other leaders in different
occupations or organizations may perceive the EBLQ items for effective
leadership of their particular leaders to be different from those outlined
for the leaders studied in this research. That is, just as there is no one best
definition of leadership, there will be no one-size-fits-all set of EBLQ items
for effective leadership in all situations. Organizations that use the EBLQ
method will be required to use different assessment qualities for each leader
in the organization based on the perceptions of each leader’s subordinates
on what constitutes the EBLQ items for their leader.
Lastly, as indicated under validity, the majority of the EBLQ items are
the same as the items in the competence models of Spencer and Spencer
(1993), Conger (1992), and Conger and Benjamin (1999) collectively. Hence,
this study can be said to equally validate the items in the competence models
of these authors.

Limitations
Due to limited budget, and given that factor analysis established ad-
equate sampling for each EBLQ item, no efforts were made to elicit par-
ticipation from more potential respondents. This, however, means that the
extent of sampling bias due to non-response is unknown. It is therefore wise
to exercise caution in interpreting the result of the case study.
It is also worth mentioning that while this study was built on the prin-
ciples of the competence model, it is not a direct study of the model. A
competence model is criterion-referenced. That is, a competence model
is designed to predict something in the real world. “A characteristic is not
a competent unless it predicts something meaningful in the real world”
(Spencer & Spencer, 1993, p. 13). This study was designed for the sole
purpose of assessing leadership effectiveness. The purpose in this study is
neither to develop a competence model nor to predict standard qualities
that would determine leadership effectiveness in all situations. Rather, this
study is designed to give organizations a new perspective on how to assess
the effectiveness of their leaders relative to the perceptions of subordinates
on what constitutes essential leadership qualities for their specific leaders.
This assessment can be used by organizations to determine the reward
system for their leaders as well as to determine leadership areas where a
leader may need improvements.

38 Performance Improvement Quarterly

References
Bass, B. M., & Yammarino, F. J. (1998). Long term forecasting of transformational leadership
and its effects among Naval officers: Some preliminary findings. (Technical Report No.
ONR-TR-2). Arlington, VA: Office of Naval Research.
Beckahard, R. (1969). Organization development: Strategies and methods. Reading, MA:
Addision-Wesley.
Bennis, W. G., & Nanus, B. (1985). Leaders: The strategies of taking charge. San Francisco:
Harper Collins.
Bentz, V. J. (1985, August). A view from the top: A thirty year perspective of research devoted
to the discovery, description and prediction of executive behavior. Paper presented at
the 93rd Annual Convention of the American Psychological Association, Los Angeles,
CA.
Bray, D. W., & Howard, A. (1983). The AT&T longitudinal study of managers. In K. W. Schaie
(Ed.), Longitudinal studies of adult psychological development (pp. 112-146). New York:
Guilford.
Conger, J. (1992). Learning to lead. San Francisco: Jossey-Bass.
Conger, J., & Benjamin, B. (1999). Building leaders. San Francisco: Jossey-Bass.
Guion, R. M. (1991). Personnel assessment selection and placement. In M. D. Dunnette &
L. M. Hough (Eds.), Handbook of industrial and organizational psychology (p. 335). Palo
Alto, CA: Consulting Psychology Press.
Guinn, K. A. (1996). Assessment techniques for top executives. Career Development Inter-
national, 1(3), 9-14.
Hazucha, J. F. (1992, September). PDI indicator: Competence, potential and jeopardy. What
gets managers ahead may not keep them out of trouble. Minneapolis, MN: Personnel
Decisions, Inc.
Heck, R. H., Johnsrad L., & Rosser, V. J. (2000). Administrative effectiveness in higher
education: Improving assessment procedures. Research in Higher Education, 14(6),
663-684.
Hogan, R. J., Curphy, G. J., & Hogan, J. (1994). What we know about personality: Leadership
and effectiveness? American Psychologist, 49, 494-504.
Hogan, R., & Hogan, J. (2001). Assessing Leadership: A View from the Dark Side. Interna-
tional Journal of Selection and Assessment, 9(1 & 2), 40-51.
Hughes, R. L., Ginnett, R. C., & Curphy, G. J. (1999). Leadership: Enhancing the lessons of
experience. Boston: McGraw-Hill.
Katz, D., & Kahn, R. L. (1978). The social psychology of organizations (2nd ed.). New York:
John Wiley.
Komacki, J. L. (1986). Toward effective supervision: An operant analysis and comparison
of managers at work. Journal of Applied Psychology, 71, 270-279.
Kotter, J. (1990). A force for change: How leadership differs from management. New York:
Free Press.
Longenecker, C. O., & Gioia, D. A. (1992, May). The executive appraisal paradox. Academy
of Management Executive, 18-28.
Lord, R. G., DeVader, C. L., & Allinger, G. M. (1986). A meta-analysis of the relationship
between personality traits and leadership perceptions: An application of validity
generalization procedures. Journal of Applied Psychology, 71, 402-410.
Oyinlade, A. O., & Gellhaus, M. (2005). Perceptions of effective leadership in schools for
students with visual impairments: A national study. Journal of Visual Impairment &
Blindness, 99(5), 261-275.
Oyinlade, A. O., Gellhaus, M., & Darboe, K. (2003). Essential leadership qualities for effective
leadership in schools for students who are visually impaired: A national study. Journal
of Visual impairment & Blindness, 97(7), 389-402.
Reeves, D. B. (2004). Assessing educational leaders. Thousand Oaks, CA: Corwin Press.
Rost, J. C. (1991). Leadership for the twenty-first century. New York: Pager.
Schellhardt, T. (1996, November 19). Annual agony: It’s time to evaluate your work and all
involved are groaning. Wall Street Journal, A1.
Shamir, B., Arthur, M. B., & House, R. J. (1994). The rhetoric of charismatic leadership: A
theoretical extension, a case study, and implications for research. Leadership Quar-
terly, 5, 25-42.

Volume 19, Number 1 / 2006 39

 Smith, B., Hornsby, J. S., & Shirmeyer, R. (1996, Summer). Current trends in performance ap-
praisal: An examination of managerial practice. SAM Advanced Management, 10-15.
Spencer, L., & Spencer, S. (1993). Competence at work. New York: John Wiley & Sons.
Steers, R. M., Porter, L., & Bigley, G. (1996). Motivation and leadership at work (6th ed.). New
York: McGraw-Hill.
Stogdill, R.M. (1948). Personal factors associated with leadership: A survey of literature.
Journal of Personality, 25, 35-71.
Stogdill, R. M. (1974). Handbook of leadership. New York: The Free Press.
Vogt, P. W. (1993). Dictionary of statistics and methodology. Newbury Park, CA: Sage.
Yukl, G. A. (1981). Leadership in organizations. Englewood Cliffs, NJ: Prentice Hall.

A. OLU OYINLADE
Dr. A. Olu Oyinlade is professor and organizational sociologist at
the Department of Sociology and Anthropology, University of Nebraska,
Omaha. His research and consulting interests are in leadership effective-
ness, employee motivation and alienation, job satisfaction, diversity plan-
ning and management in organizations, and organizational power. E-mail:
aoyinlade@mail.unomaha.edu

40 Performance Improvement Quarterly

View publication stats